2024-04-26T20:43:36.673 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-26T20:43:36.673 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675187 --owner scheduled_rishabh@teuthology --verbose --name rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/auto-repair} -- /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675187/orig.config.yaml 2024-04-26T20:43:36.716 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T20:43:37.063 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675187 branch: wip-rishabh-testing-20240426.111959 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/auto-repair} email: null first_in_suite: false job_id: '7675187' kernel: client: sha1: distro flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240426.111959 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - force file system read-only - bad backtrace - MDS in read-only mode - \(MDS_READ_ONLY\) sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240426.111959 sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 816 sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 sleep_before_teardown: 0 subset: 166/512 suite: fs suite_branch: wip-rishabh-testing-20240426.111959 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi073.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPuP5xc655KyTC67uHuRxnDpDvFCni0MBRVVT7rV83pMbuacfY62UQuLfldLSMdgrbW6XfVZ9os4+Dw/sX7BmWs= smithi120.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAInL6Cd1zlE5BV47vgZj307nYru1k87T263QM4ue067QJH8zg6FZQxg32GhaY25U32cUojUHfXAG7kOUdzMuuM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_auto_repair teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-26_19:30:57 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-26T20:43:37.063 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa; will attempt to use it 2024-04-26T20:43:37.064 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks 2024-04-26T20:43:37.064 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-26T20:43:37.066 INFO:teuthology.task.internal:Checking packages... 2024-04-26T20:43:37.087 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'bc612ffb9c1d5bc27536bc917a705262a36d4387' 2024-04-26T20:43:37.087 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-26T20:43:37.087 INFO:teuthology.packaging:ref: None 2024-04-26T20:43:37.087 INFO:teuthology.packaging:tag: None 2024-04-26T20:43:37.088 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240426.111959 2024-04-26T20:43:37.088 INFO:teuthology.packaging:sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T20:43:37.088 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240426.111959 2024-04-26T20:43:37.307 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3306.gbc612ffb 2024-04-26T20:43:37.309 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-26T20:43:37.314 INFO:teuthology.task.internal:no buildpackages task found 2024-04-26T20:43:37.314 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-26T20:43:37.319 INFO:teuthology.task.internal:Saving configuration 2024-04-26T20:43:37.335 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-26T20:43:37.340 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-26T20:43:37.367 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi073.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675187', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 20:38:50.032382', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPuP5xc655KyTC67uHuRxnDpDvFCni0MBRVVT7rV83pMbuacfY62UQuLfldLSMdgrbW6XfVZ9os4+Dw/sX7BmWs='} 2024-04-26T20:43:37.389 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi120.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675187', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 20:38:50.030457', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAInL6Cd1zlE5BV47vgZj307nYru1k87T263QM4ue067QJH8zg6FZQxg32GhaY25U32cUojUHfXAG7kOUdzMuuM='} 2024-04-26T20:43:37.389 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-26T20:43:37.395 INFO:teuthology.task.internal:roles: ubuntu@smithi073.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-26T20:43:37.396 INFO:teuthology.task.internal:roles: ubuntu@smithi120.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-26T20:43:37.396 INFO:teuthology.run_tasks:Running task console_log... 2024-04-26T20:43:37.466 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fadf4cf2160>, signals=[15]) 2024-04-26T20:43:37.467 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-26T20:43:37.473 INFO:teuthology.task.internal:Opening connections... 2024-04-26T20:43:37.473 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi073.front.sepia.ceph.com 2024-04-26T20:43:37.475 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T20:43:37.554 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi120.front.sepia.ceph.com 2024-04-26T20:43:37.555 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T20:43:37.630 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-26T20:43:37.636 DEBUG:teuthology.orchestra.run.smithi073:> uname -m 2024-04-26T20:43:37.653 INFO:teuthology.orchestra.run.smithi073.stdout:x86_64 2024-04-26T20:43:37.653 DEBUG:teuthology.orchestra.run.smithi073:> cat /etc/os-release 2024-04-26T20:43:37.710 INFO:teuthology.orchestra.run.smithi073.stdout:NAME="CentOS Stream" 2024-04-26T20:43:37.710 INFO:teuthology.orchestra.run.smithi073.stdout:VERSION="9" 2024-04-26T20:43:37.710 INFO:teuthology.orchestra.run.smithi073.stdout:ID="centos" 2024-04-26T20:43:37.710 INFO:teuthology.orchestra.run.smithi073.stdout:ID_LIKE="rhel fedora" 2024-04-26T20:43:37.710 INFO:teuthology.orchestra.run.smithi073.stdout:VERSION_ID="9" 2024-04-26T20:43:37.710 INFO:teuthology.orchestra.run.smithi073.stdout:PLATFORM_ID="platform:el9" 2024-04-26T20:43:37.710 INFO:teuthology.orchestra.run.smithi073.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T20:43:37.710 INFO:teuthology.orchestra.run.smithi073.stdout:ANSI_COLOR="0;31" 2024-04-26T20:43:37.710 INFO:teuthology.orchestra.run.smithi073.stdout:LOGO="fedora-logo-icon" 2024-04-26T20:43:37.710 INFO:teuthology.orchestra.run.smithi073.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T20:43:37.710 INFO:teuthology.orchestra.run.smithi073.stdout:HOME_URL="https://centos.org/" 2024-04-26T20:43:37.711 INFO:teuthology.orchestra.run.smithi073.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T20:43:37.711 INFO:teuthology.orchestra.run.smithi073.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T20:43:37.711 INFO:teuthology.orchestra.run.smithi073.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T20:43:37.711 INFO:teuthology.lock.ops:Updating smithi073.front.sepia.ceph.com on lock server 2024-04-26T20:43:37.739 DEBUG:teuthology.orchestra.run.smithi120:> uname -m 2024-04-26T20:43:37.757 INFO:teuthology.orchestra.run.smithi120.stdout:x86_64 2024-04-26T20:43:37.757 DEBUG:teuthology.orchestra.run.smithi120:> cat /etc/os-release 2024-04-26T20:43:37.814 INFO:teuthology.orchestra.run.smithi120.stdout:NAME="CentOS Stream" 2024-04-26T20:43:37.815 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION="9" 2024-04-26T20:43:37.815 INFO:teuthology.orchestra.run.smithi120.stdout:ID="centos" 2024-04-26T20:43:37.815 INFO:teuthology.orchestra.run.smithi120.stdout:ID_LIKE="rhel fedora" 2024-04-26T20:43:37.815 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION_ID="9" 2024-04-26T20:43:37.815 INFO:teuthology.orchestra.run.smithi120.stdout:PLATFORM_ID="platform:el9" 2024-04-26T20:43:37.815 INFO:teuthology.orchestra.run.smithi120.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T20:43:37.815 INFO:teuthology.orchestra.run.smithi120.stdout:ANSI_COLOR="0;31" 2024-04-26T20:43:37.815 INFO:teuthology.orchestra.run.smithi120.stdout:LOGO="fedora-logo-icon" 2024-04-26T20:43:37.815 INFO:teuthology.orchestra.run.smithi120.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T20:43:37.815 INFO:teuthology.orchestra.run.smithi120.stdout:HOME_URL="https://centos.org/" 2024-04-26T20:43:37.815 INFO:teuthology.orchestra.run.smithi120.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T20:43:37.815 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T20:43:37.816 INFO:teuthology.orchestra.run.smithi120.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T20:43:37.816 INFO:teuthology.lock.ops:Updating smithi120.front.sepia.ceph.com on lock server 2024-04-26T20:43:37.839 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-26T20:43:37.848 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-26T20:43:37.852 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-26T20:43:37.853 DEBUG:teuthology.orchestra.run.smithi073:> test '!' -e /home/ubuntu/cephtest 2024-04-26T20:43:37.855 DEBUG:teuthology.orchestra.run.smithi120:> test '!' -e /home/ubuntu/cephtest 2024-04-26T20:43:37.871 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-26T20:43:37.877 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-26T20:43:37.877 DEBUG:teuthology.orchestra.run.smithi073:> test -z $(ls -A /var/lib/ceph) 2024-04-26T20:43:37.912 DEBUG:teuthology.orchestra.run.smithi120:> test -z $(ls -A /var/lib/ceph) 2024-04-26T20:43:37.931 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-26T20:43:37.982 INFO:teuthology.run_tasks:Running task kernel... 2024-04-26T20:43:37.996 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-04-26T20:43:37.996 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2024-04-26T20:43:37.996 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-04-26T20:43:37.996 DEBUG:teuthology.orchestra.run.smithi073:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T20:43:37.997 DEBUG:teuthology.orchestra.run.smithi120:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T20:43:38.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:43:38.012 DEBUG:teuthology.orchestra.run.smithi120:> uname -r 2024-04-26T20:43:38.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:43:38.013 DEBUG:teuthology.orchestra.run.smithi073:> uname -r 2024-04-26T20:43:38.068 INFO:teuthology.orchestra.run.smithi120.stdout:5.14.0-437.el9.x86_64 2024-04-26T20:43:38.068 INFO:teuthology.task.kernel:Running kernel on smithi120: 5.14.0-437.el9.x86_64 2024-04-26T20:43:38.069 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum install -y kernel 2024-04-26T20:43:38.070 INFO:teuthology.orchestra.run.smithi073.stdout:5.14.0-437.el9.x86_64 2024-04-26T20:43:38.070 INFO:teuthology.task.kernel:Running kernel on smithi073: 5.14.0-437.el9.x86_64 2024-04-26T20:43:38.070 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum install -y kernel 2024-04-26T20:43:40.278 INFO:teuthology.orchestra.run.smithi073.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-04-26T20:43:41.143 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - BaseOS 5.8 MB/s | 8.0 MB 00:01 2024-04-26T20:43:43.153 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - AppStream 20 MB/s | 19 MB 00:00 2024-04-26T20:43:44.095 INFO:teuthology.orchestra.run.smithi073.stdout:CentOS Stream 9 - AppStream 6.9 MB/s | 19 MB 00:02 2024-04-26T20:43:47.396 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - CRB 8.9 MB/s | 6.2 MB 00:00 2024-04-26T20:43:48.213 INFO:teuthology.orchestra.run.smithi073.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-04-26T20:43:49.171 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - Extras packages 42 kB/s | 16 kB 00:00 2024-04-26T20:43:49.630 INFO:teuthology.orchestra.run.smithi073.stdout:CentOS Stream 9 - Extras packages 60 kB/s | 16 kB 00:00 2024-04-26T20:43:50.892 INFO:teuthology.orchestra.run.smithi073.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-04-26T20:43:51.325 INFO:teuthology.orchestra.run.smithi120.stdout:Extra Packages for Enterprise Linux 10 MB/s | 21 MB 00:02 2024-04-26T20:43:55.948 INFO:teuthology.orchestra.run.smithi073.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-04-26T20:43:56.439 INFO:teuthology.orchestra.run.smithi120.stdout:lab-extras 11 kB/s | 1.7 kB 00:00 2024-04-26T20:43:57.862 INFO:teuthology.orchestra.run.smithi073.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-26T20:43:57.862 INFO:teuthology.orchestra.run.smithi073.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-26T20:43:57.967 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-04-26T20:43:57.969 INFO:teuthology.orchestra.run.smithi073.stdout:Nothing to do. 2024-04-26T20:43:57.969 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-04-26T20:43:58.071 DEBUG:teuthology.orchestra.run.smithi073:> echo no | sudo yum reinstall kernel || true 2024-04-26T20:43:58.321 INFO:teuthology.orchestra.run.smithi120.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-26T20:43:58.321 INFO:teuthology.orchestra.run.smithi120.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-26T20:43:58.414 INFO:teuthology.orchestra.run.smithi120.stdout:Dependencies resolved. 2024-04-26T20:43:58.415 INFO:teuthology.orchestra.run.smithi120.stdout:Nothing to do. 2024-04-26T20:43:58.415 INFO:teuthology.orchestra.run.smithi120.stdout:Complete! 2024-04-26T20:43:58.528 DEBUG:teuthology.orchestra.run.smithi120:> echo no | sudo yum reinstall kernel || true 2024-04-26T20:43:58.570 INFO:teuthology.orchestra.run.smithi073.stdout:Last metadata expiration check: 0:00:03 ago on Fri 26 Apr 2024 08:43:55 PM UTC. 2024-04-26T20:43:58.689 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-04-26T20:43:58.697 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-04-26T20:43:58.697 INFO:teuthology.orchestra.run.smithi073.stdout: Package Architecture Version Repository Size 2024-04-26T20:43:58.697 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-04-26T20:43:58.697 INFO:teuthology.orchestra.run.smithi073.stdout:Reinstalling: 2024-04-26T20:43:58.697 INFO:teuthology.orchestra.run.smithi073.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T20:43:58.697 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:43:58.697 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction Summary 2024-04-26T20:43:58.697 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-04-26T20:43:58.697 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:43:58.697 INFO:teuthology.orchestra.run.smithi073.stdout:Total download size: 255 k 2024-04-26T20:43:58.697 INFO:teuthology.orchestra.run.smithi073.stdout:Installed size: 0 2024-04-26T20:43:58.698 INFO:teuthology.orchestra.run.smithi073.stderr:Operation aborted. 2024-04-26T20:43:58.759 INFO:teuthology.orchestra.run.smithi073.stdout:Is this ok [y/N]: 2024-04-26T20:43:58.760 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum reinstall -y kernel || true 2024-04-26T20:43:59.026 INFO:teuthology.orchestra.run.smithi120.stdout:Last metadata expiration check: 0:00:03 ago on Fri 26 Apr 2024 08:43:56 PM UTC. 2024-04-26T20:43:59.151 INFO:teuthology.orchestra.run.smithi120.stdout:Dependencies resolved. 2024-04-26T20:43:59.162 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-04-26T20:43:59.163 INFO:teuthology.orchestra.run.smithi120.stdout: Package Architecture Version Repository Size 2024-04-26T20:43:59.163 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-04-26T20:43:59.163 INFO:teuthology.orchestra.run.smithi120.stdout:Reinstalling: 2024-04-26T20:43:59.163 INFO:teuthology.orchestra.run.smithi120.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T20:43:59.163 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T20:43:59.163 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction Summary 2024-04-26T20:43:59.163 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-04-26T20:43:59.163 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T20:43:59.163 INFO:teuthology.orchestra.run.smithi120.stdout:Total download size: 255 k 2024-04-26T20:43:59.163 INFO:teuthology.orchestra.run.smithi120.stdout:Installed size: 0 2024-04-26T20:43:59.163 INFO:teuthology.orchestra.run.smithi120.stderr:Operation aborted. 2024-04-26T20:43:59.220 INFO:teuthology.orchestra.run.smithi120.stdout:Is this ok [y/N]: 2024-04-26T20:43:59.220 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum reinstall -y kernel || true 2024-04-26T20:43:59.249 INFO:teuthology.orchestra.run.smithi073.stdout:Last metadata expiration check: 0:00:04 ago on Fri 26 Apr 2024 08:43:55 PM UTC. 2024-04-26T20:43:59.365 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-04-26T20:43:59.366 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-04-26T20:43:59.366 INFO:teuthology.orchestra.run.smithi073.stdout: Package Architecture Version Repository Size 2024-04-26T20:43:59.366 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-04-26T20:43:59.366 INFO:teuthology.orchestra.run.smithi073.stdout:Reinstalling: 2024-04-26T20:43:59.366 INFO:teuthology.orchestra.run.smithi073.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T20:43:59.366 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:43:59.366 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction Summary 2024-04-26T20:43:59.366 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-04-26T20:43:59.366 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:43:59.366 INFO:teuthology.orchestra.run.smithi073.stdout:Total download size: 255 k 2024-04-26T20:43:59.367 INFO:teuthology.orchestra.run.smithi073.stdout:Installed size: 0 2024-04-26T20:43:59.367 INFO:teuthology.orchestra.run.smithi073.stdout:Downloading Packages: 2024-04-26T20:43:59.592 INFO:teuthology.orchestra.run.smithi073.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.5 MB/s | 255 kB 00:00 2024-04-26T20:43:59.592 INFO:teuthology.orchestra.run.smithi073.stdout:-------------------------------------------------------------------------------- 2024-04-26T20:43:59.592 INFO:teuthology.orchestra.run.smithi073.stdout:Total 1.1 MB/s | 255 kB 00:00 2024-04-26T20:43:59.610 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction check 2024-04-26T20:43:59.711 INFO:teuthology.orchestra.run.smithi120.stdout:Last metadata expiration check: 0:00:03 ago on Fri 26 Apr 2024 08:43:56 PM UTC. 2024-04-26T20:43:59.809 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction check succeeded. 2024-04-26T20:43:59.809 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction test 2024-04-26T20:43:59.814 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction test succeeded. 2024-04-26T20:43:59.814 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction 2024-04-26T20:43:59.826 INFO:teuthology.orchestra.run.smithi120.stdout:Dependencies resolved. 2024-04-26T20:43:59.826 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-04-26T20:43:59.826 INFO:teuthology.orchestra.run.smithi120.stdout: Package Architecture Version Repository Size 2024-04-26T20:43:59.826 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-04-26T20:43:59.826 INFO:teuthology.orchestra.run.smithi120.stdout:Reinstalling: 2024-04-26T20:43:59.827 INFO:teuthology.orchestra.run.smithi120.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T20:43:59.827 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T20:43:59.827 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction Summary 2024-04-26T20:43:59.827 INFO:teuthology.orchestra.run.smithi120.stdout:================================================================================ 2024-04-26T20:43:59.827 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T20:43:59.827 INFO:teuthology.orchestra.run.smithi120.stdout:Total download size: 255 k 2024-04-26T20:43:59.828 INFO:teuthology.orchestra.run.smithi120.stdout:Installed size: 0 2024-04-26T20:43:59.828 INFO:teuthology.orchestra.run.smithi120.stdout:Downloading Packages: 2024-04-26T20:43:59.936 INFO:teuthology.orchestra.run.smithi073.stdout: Preparing : 1/1 2024-04-26T20:44:00.018 INFO:teuthology.orchestra.run.smithi073.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T20:44:00.018 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T20:44:00.133 INFO:teuthology.orchestra.run.smithi120.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.5 MB/s | 255 kB 00:00 2024-04-26T20:44:00.134 INFO:teuthology.orchestra.run.smithi120.stdout:-------------------------------------------------------------------------------- 2024-04-26T20:44:00.134 INFO:teuthology.orchestra.run.smithi120.stdout:Total 831 kB/s | 255 kB 00:00 2024-04-26T20:44:00.151 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction check 2024-04-26T20:44:00.335 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction check succeeded. 2024-04-26T20:44:00.335 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction test 2024-04-26T20:44:00.341 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction test succeeded. 2024-04-26T20:44:00.341 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction 2024-04-26T20:44:00.494 INFO:teuthology.orchestra.run.smithi120.stdout: Preparing : 1/1 2024-04-26T20:44:00.551 INFO:teuthology.orchestra.run.smithi120.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T20:44:00.551 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T20:44:00.771 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T20:44:00.771 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T20:44:01.291 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T20:44:01.291 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T20:44:01.429 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T20:44:01.429 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:44:01.429 INFO:teuthology.orchestra.run.smithi073.stdout:Reinstalled: 2024-04-26T20:44:01.429 INFO:teuthology.orchestra.run.smithi073.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-26T20:44:01.429 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:44:01.429 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-04-26T20:44:01.602 DEBUG:teuthology.orchestra.run.smithi073:> rpm -q kernel | sort -rV | head -n 1 2024-04-26T20:44:01.648 INFO:teuthology.orchestra.run.smithi073.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-26T20:44:01.649 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-26T20:44:01.649 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-26T20:44:01.650 DEBUG:teuthology.parallel:result is None 2024-04-26T20:44:01.993 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T20:44:01.993 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T20:44:01.993 INFO:teuthology.orchestra.run.smithi120.stdout:Reinstalled: 2024-04-26T20:44:01.993 INFO:teuthology.orchestra.run.smithi120.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-26T20:44:01.993 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T20:44:01.993 INFO:teuthology.orchestra.run.smithi120.stdout:Complete! 2024-04-26T20:44:02.180 DEBUG:teuthology.orchestra.run.smithi120:> rpm -q kernel | sort -rV | head -n 1 2024-04-26T20:44:02.269 INFO:teuthology.orchestra.run.smithi120.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-26T20:44:02.269 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-26T20:44:02.269 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-26T20:44:02.269 DEBUG:teuthology.parallel:result is None 2024-04-26T20:44:02.270 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-26T20:44:02.277 INFO:teuthology.task.internal:Creating test directory... 2024-04-26T20:44:02.277 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T20:44:02.280 DEBUG:teuthology.orchestra.run.smithi120:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T20:44:02.294 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-26T20:44:02.300 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-26T20:44:02.309 INFO:teuthology.task.internal:Creating archive directory... 2024-04-26T20:44:02.309 DEBUG:teuthology.orchestra.run.smithi073:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T20:44:02.336 DEBUG:teuthology.orchestra.run.smithi120:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T20:44:02.365 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-26T20:44:02.404 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-26T20:44:02.404 DEBUG:teuthology.orchestra.run.smithi073:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T20:44:02.407 DEBUG:teuthology.orchestra.run.smithi120:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T20:44:02.437 INFO:teuthology.orchestra.run.smithi073.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T20:44:02.438 INFO:teuthology.orchestra.run.smithi120.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T20:44:02.450 INFO:teuthology.orchestra.run.smithi120.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T20:44:02.454 INFO:teuthology.orchestra.run.smithi073.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T20:44:02.456 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-26T20:44:02.462 INFO:teuthology.task.internal:Configuring sudo... 2024-04-26T20:44:02.462 DEBUG:teuthology.orchestra.run.smithi073:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T20:44:02.498 DEBUG:teuthology.orchestra.run.smithi120:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T20:44:02.520 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-26T20:44:02.530 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-26T20:44:02.530 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T20:44:02.562 DEBUG:teuthology.orchestra.run.smithi120:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T20:44:02.576 DEBUG:teuthology.orchestra.run.smithi073:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T20:44:02.657 DEBUG:teuthology.orchestra.run.smithi073:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T20:44:02.712 DEBUG:teuthology.orchestra.run.smithi073:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T20:44:02.792 DEBUG:teuthology.orchestra.run.smithi073:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T20:44:02.833 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-26T20:44:02.833 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T20:44:02.896 DEBUG:teuthology.orchestra.run.smithi120:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T20:44:02.939 DEBUG:teuthology.orchestra.run.smithi120:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T20:44:02.990 DEBUG:teuthology.orchestra.run.smithi120:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T20:44:03.072 DEBUG:teuthology.orchestra.run.smithi120:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T20:44:03.110 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-26T20:44:03.110 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T20:44:03.173 DEBUG:teuthology.orchestra.run.smithi073:> sudo service rsyslog restart 2024-04-26T20:44:03.176 DEBUG:teuthology.orchestra.run.smithi120:> sudo service rsyslog restart 2024-04-26T20:44:03.207 INFO:teuthology.orchestra.run.smithi073.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T20:44:03.246 INFO:teuthology.orchestra.run.smithi120.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T20:44:03.682 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-26T20:44:03.689 INFO:teuthology.task.internal:Starting timer... 2024-04-26T20:44:03.690 INFO:teuthology.run_tasks:Running task pcp... 2024-04-26T20:44:03.699 INFO:teuthology.run_tasks:Running task selinux... 2024-04-26T20:44:03.713 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-26T20:44:03.713 DEBUG:teuthology.orchestra.run.smithi073:> sudo service auditd rotate 2024-04-26T20:44:03.766 INFO:teuthology.orchestra.run.smithi073.stdout:Rotating logs: 2024-04-26T20:44:03.767 DEBUG:teuthology.orchestra.run.smithi120:> sudo service auditd rotate 2024-04-26T20:44:03.808 INFO:teuthology.orchestra.run.smithi120.stdout:Rotating logs: 2024-04-26T20:44:03.809 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-26T20:44:03.810 DEBUG:teuthology.orchestra.run.smithi073:> /usr/sbin/getenforce 2024-04-26T20:44:03.828 INFO:teuthology.orchestra.run.smithi073.stdout:Permissive 2024-04-26T20:44:03.829 DEBUG:teuthology.orchestra.run.smithi120:> /usr/sbin/getenforce 2024-04-26T20:44:03.869 INFO:teuthology.orchestra.run.smithi120.stdout:Permissive 2024-04-26T20:44:03.869 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi073.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi120.front.sepia.ceph.com': 'permissive'} 2024-04-26T20:44:03.869 DEBUG:teuthology.orchestra.run.smithi073:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T20:44:03.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:44:03.893 DEBUG:teuthology.orchestra.run.smithi120:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T20:44:03.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:44:03.933 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-26T20:44:03.933 DEBUG:teuthology.orchestra.run.smithi073:> sudo /usr/sbin/setenforce permissive 2024-04-26T20:44:03.957 DEBUG:teuthology.orchestra.run.smithi120:> sudo /usr/sbin/setenforce permissive 2024-04-26T20:44:03.996 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-26T20:44:04.004 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-26T20:44:04.021 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-26T20:44:04.022 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi073.front.sepia.ceph.com,smithi120.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-26T20:47:31.280 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi073.front.sepia.ceph.com'), Remote(name='ubuntu@smithi120.front.sepia.ceph.com')] 2024-04-26T20:47:31.282 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi073.front.sepia.ceph.com' 2024-04-26T20:47:31.283 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T20:47:31.369 DEBUG:teuthology.orchestra.run.smithi073:> true 2024-04-26T20:47:31.448 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi073.front.sepia.ceph.com' 2024-04-26T20:47:31.449 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-04-26T20:47:31.449 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T20:47:31.524 DEBUG:teuthology.orchestra.run.smithi120:> true 2024-04-26T20:47:31.609 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-04-26T20:47:31.609 INFO:teuthology.run_tasks:Running task clock... 2024-04-26T20:47:31.623 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-26T20:47:31.623 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T20:47:31.624 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T20:47:31.626 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T20:47:31.626 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T20:47:31.651 INFO:teuthology.orchestra.run.smithi073.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T20:47:31.664 INFO:teuthology.orchestra.run.smithi073.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T20:47:31.676 INFO:teuthology.orchestra.run.smithi120.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T20:47:31.686 INFO:teuthology.orchestra.run.smithi073.stderr:sudo: ntpd: command not found 2024-04-26T20:47:31.689 INFO:teuthology.orchestra.run.smithi120.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T20:47:31.697 INFO:teuthology.orchestra.run.smithi073.stdout:506 Cannot talk to daemon 2024-04-26T20:47:31.709 INFO:teuthology.orchestra.run.smithi073.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T20:47:31.712 INFO:teuthology.orchestra.run.smithi120.stderr:sudo: ntpd: command not found 2024-04-26T20:47:31.722 INFO:teuthology.orchestra.run.smithi073.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T20:47:31.722 INFO:teuthology.orchestra.run.smithi120.stdout:506 Cannot talk to daemon 2024-04-26T20:47:31.735 INFO:teuthology.orchestra.run.smithi120.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T20:47:31.747 INFO:teuthology.orchestra.run.smithi120.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T20:47:31.778 INFO:teuthology.orchestra.run.smithi073.stderr:bash: line 1: ntpq: command not found 2024-04-26T20:47:31.782 INFO:teuthology.orchestra.run.smithi073.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T20:47:31.782 INFO:teuthology.orchestra.run.smithi073.stdout:=============================================================================== 2024-04-26T20:47:31.782 INFO:teuthology.orchestra.run.smithi073.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T20:47:31.782 INFO:teuthology.orchestra.run.smithi073.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T20:47:31.782 INFO:teuthology.orchestra.run.smithi073.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T20:47:31.782 INFO:teuthology.orchestra.run.smithi073.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T20:47:31.802 INFO:teuthology.orchestra.run.smithi120.stderr:bash: line 1: ntpq: command not found 2024-04-26T20:47:31.806 INFO:teuthology.orchestra.run.smithi120.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T20:47:31.806 INFO:teuthology.orchestra.run.smithi120.stdout:=============================================================================== 2024-04-26T20:47:31.806 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T20:47:31.806 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T20:47:31.806 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T20:47:31.806 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T20:47:31.806 INFO:teuthology.run_tasks:Running task install... 2024-04-26T20:47:31.817 DEBUG:teuthology.task.install:project ceph 2024-04-26T20:47:31.818 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}} 2024-04-26T20:47:31.818 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'} 2024-04-26T20:47:31.818 INFO:teuthology.task.install:Using flavor: default 2024-04-26T20:47:31.823 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-26T20:47:31.824 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-26T20:47:31.824 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-26T20:47:31.824 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T20:47:31.826 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-26T20:47:31.827 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T20:47:31.979 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-26T20:47:31.979 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-26T20:47:32.037 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-26T20:47:32.038 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-26T20:47:32.150 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T20:47:32.150 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-26T20:47:32.151 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T20:47:32.157 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T20:47:32.157 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-26T20:47:32.157 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T20:47:32.175 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T20:47:32.175 DEBUG:teuthology.orchestra.run.smithi120:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T20:47:32.182 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T20:47:32.182 DEBUG:teuthology.orchestra.run.smithi073:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T20:47:32.239 DEBUG:teuthology.orchestra.run.smithi120:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T20:47:32.247 DEBUG:teuthology.orchestra.run.smithi073:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T20:47:32.312 DEBUG:teuthology.orchestra.run.smithi120:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T20:47:32.320 DEBUG:teuthology.orchestra.run.smithi073:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T20:47:32.375 INFO:teuthology.orchestra.run.smithi120.stdout:check_obsoletes = 1 2024-04-26T20:47:32.376 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum clean all 2024-04-26T20:47:32.382 INFO:teuthology.orchestra.run.smithi073.stdout:check_obsoletes = 1 2024-04-26T20:47:32.384 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum clean all 2024-04-26T20:47:32.600 INFO:teuthology.orchestra.run.smithi120.stdout:48 files removed 2024-04-26T20:47:32.606 INFO:teuthology.orchestra.run.smithi073.stdout:48 files removed 2024-04-26T20:47:32.626 DEBUG:teuthology.orchestra.run.smithi120:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T20:47:32.633 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T20:47:33.239 INFO:teuthology.orchestra.run.smithi073.stdout:ceph packages for x86_64 240 kB/s | 84 kB 00:00 2024-04-26T20:47:33.297 INFO:teuthology.orchestra.run.smithi120.stdout:ceph packages for x86_64 202 kB/s | 84 kB 00:00 2024-04-26T20:47:33.690 INFO:teuthology.orchestra.run.smithi073.stdout:ceph noarch packages 29 kB/s | 12 kB 00:00 2024-04-26T20:47:33.723 INFO:teuthology.orchestra.run.smithi120.stdout:ceph noarch packages 30 kB/s | 12 kB 00:00 2024-04-26T20:47:34.073 INFO:teuthology.orchestra.run.smithi120.stdout:ceph source packages 6.0 kB/s | 1.9 kB 00:00 2024-04-26T20:47:34.107 INFO:teuthology.orchestra.run.smithi073.stdout:ceph source packages 5.0 kB/s | 1.9 kB 00:00 2024-04-26T20:47:34.757 INFO:teuthology.orchestra.run.smithi073.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.0 MB 00:00 2024-04-26T20:47:34.859 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-04-26T20:47:41.708 INFO:teuthology.orchestra.run.smithi073.stdout:CentOS Stream 9 - AppStream 3.2 MB/s | 19 MB 00:05 2024-04-26T20:47:46.408 INFO:teuthology.orchestra.run.smithi073.stdout:CentOS Stream 9 - CRB 5.4 MB/s | 6.2 MB 00:01 2024-04-26T20:47:48.059 INFO:teuthology.orchestra.run.smithi073.stdout:CentOS Stream 9 - Extras packages 32 kB/s | 16 kB 00:00 2024-04-26T20:47:49.088 INFO:teuthology.orchestra.run.smithi073.stdout:Extra Packages for Enterprise Linux 23 MB/s | 21 MB 00:00 2024-04-26T20:47:54.170 INFO:teuthology.orchestra.run.smithi073.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-04-26T20:47:54.699 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - AppStream 1.0 MB/s | 19 MB 00:18 2024-04-26T20:47:56.337 INFO:teuthology.orchestra.run.smithi073.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T20:47:56.338 INFO:teuthology.orchestra.run.smithi073.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T20:47:56.349 INFO:teuthology.orchestra.run.smithi073.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T20:47:56.349 INFO:teuthology.orchestra.run.smithi073.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T20:47:56.350 INFO:teuthology.orchestra.run.smithi073.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T20:47:56.351 INFO:teuthology.orchestra.run.smithi073.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T20:47:56.353 INFO:teuthology.orchestra.run.smithi073.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T20:47:56.355 INFO:teuthology.orchestra.run.smithi073.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T20:47:56.355 INFO:teuthology.orchestra.run.smithi073.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T20:47:56.416 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-04-26T20:47:56.425 INFO:teuthology.orchestra.run.smithi073.stdout:======================================================================================= 2024-04-26T20:47:56.425 INFO:teuthology.orchestra.run.smithi073.stdout: Package Arch Version Repository Size 2024-04-26T20:47:56.425 INFO:teuthology.orchestra.run.smithi073.stdout:======================================================================================= 2024-04-26T20:47:56.425 INFO:teuthology.orchestra.run.smithi073.stdout:Installing: 2024-04-26T20:47:56.425 INFO:teuthology.orchestra.run.smithi073.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T20:47:56.426 INFO:teuthology.orchestra.run.smithi073.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-26T20:47:56.426 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T20:47:56.426 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-26T20:47:56.426 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-26T20:47:56.426 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-26T20:47:56.426 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-26T20:47:56.426 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-26T20:47:56.426 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-26T20:47:56.426 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-26T20:47:56.426 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-26T20:47:56.426 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-26T20:47:56.426 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-26T20:47:56.427 INFO:teuthology.orchestra.run.smithi073.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-26T20:47:56.427 INFO:teuthology.orchestra.run.smithi073.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-26T20:47:56.427 INFO:teuthology.orchestra.run.smithi073.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-26T20:47:56.427 INFO:teuthology.orchestra.run.smithi073.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-26T20:47:56.427 INFO:teuthology.orchestra.run.smithi073.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T20:47:56.427 INFO:teuthology.orchestra.run.smithi073.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T20:47:56.427 INFO:teuthology.orchestra.run.smithi073.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T20:47:56.427 INFO:teuthology.orchestra.run.smithi073.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-26T20:47:56.427 INFO:teuthology.orchestra.run.smithi073.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-26T20:47:56.427 INFO:teuthology.orchestra.run.smithi073.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-26T20:47:56.427 INFO:teuthology.orchestra.run.smithi073.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T20:47:56.427 INFO:teuthology.orchestra.run.smithi073.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T20:47:56.427 INFO:teuthology.orchestra.run.smithi073.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T20:47:56.427 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T20:47:56.428 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T20:47:56.428 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T20:47:56.428 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-26T20:47:56.428 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-26T20:47:56.428 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-26T20:47:56.428 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-26T20:47:56.428 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-26T20:47:56.428 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T20:47:56.428 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-26T20:47:56.428 INFO:teuthology.orchestra.run.smithi073.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T20:47:56.428 INFO:teuthology.orchestra.run.smithi073.stdout:Upgrading: 2024-04-26T20:47:56.428 INFO:teuthology.orchestra.run.smithi073.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-26T20:47:56.428 INFO:teuthology.orchestra.run.smithi073.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T20:47:56.428 INFO:teuthology.orchestra.run.smithi073.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T20:47:56.428 INFO:teuthology.orchestra.run.smithi073.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T20:47:56.428 INFO:teuthology.orchestra.run.smithi073.stdout:Installing dependencies: 2024-04-26T20:47:56.429 INFO:teuthology.orchestra.run.smithi073.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T20:47:56.429 INFO:teuthology.orchestra.run.smithi073.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T20:47:56.429 INFO:teuthology.orchestra.run.smithi073.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T20:47:56.429 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-26T20:47:56.429 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-26T20:47:56.429 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-26T20:47:56.429 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-26T20:47:56.429 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-26T20:47:56.429 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-26T20:47:56.429 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-26T20:47:56.429 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-26T20:47:56.429 INFO:teuthology.orchestra.run.smithi073.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T20:47:56.429 INFO:teuthology.orchestra.run.smithi073.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T20:47:56.429 INFO:teuthology.orchestra.run.smithi073.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T20:47:56.430 INFO:teuthology.orchestra.run.smithi073.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T20:47:56.430 INFO:teuthology.orchestra.run.smithi073.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T20:47:56.430 INFO:teuthology.orchestra.run.smithi073.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T20:47:56.430 INFO:teuthology.orchestra.run.smithi073.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T20:47:56.430 INFO:teuthology.orchestra.run.smithi073.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T20:47:56.430 INFO:teuthology.orchestra.run.smithi073.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-26T20:47:56.430 INFO:teuthology.orchestra.run.smithi073.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T20:47:56.430 INFO:teuthology.orchestra.run.smithi073.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T20:47:56.430 INFO:teuthology.orchestra.run.smithi073.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T20:47:56.430 INFO:teuthology.orchestra.run.smithi073.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T20:47:56.430 INFO:teuthology.orchestra.run.smithi073.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T20:47:56.430 INFO:teuthology.orchestra.run.smithi073.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-26T20:47:56.430 INFO:teuthology.orchestra.run.smithi073.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T20:47:56.431 INFO:teuthology.orchestra.run.smithi073.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T20:47:56.431 INFO:teuthology.orchestra.run.smithi073.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T20:47:56.431 INFO:teuthology.orchestra.run.smithi073.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T20:47:56.431 INFO:teuthology.orchestra.run.smithi073.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T20:47:56.431 INFO:teuthology.orchestra.run.smithi073.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T20:47:56.431 INFO:teuthology.orchestra.run.smithi073.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T20:47:56.431 INFO:teuthology.orchestra.run.smithi073.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T20:47:56.431 INFO:teuthology.orchestra.run.smithi073.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T20:47:56.431 INFO:teuthology.orchestra.run.smithi073.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T20:47:56.431 INFO:teuthology.orchestra.run.smithi073.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T20:47:56.431 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T20:47:56.431 INFO:teuthology.orchestra.run.smithi073.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T20:47:56.431 INFO:teuthology.orchestra.run.smithi073.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T20:47:56.431 INFO:teuthology.orchestra.run.smithi073.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T20:47:56.431 INFO:teuthology.orchestra.run.smithi073.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T20:47:56.432 INFO:teuthology.orchestra.run.smithi073.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T20:47:56.432 INFO:teuthology.orchestra.run.smithi073.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T20:47:56.432 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T20:47:56.432 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-26T20:47:56.432 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-26T20:47:56.432 INFO:teuthology.orchestra.run.smithi073.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T20:47:56.432 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T20:47:56.432 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T20:47:56.432 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T20:47:56.432 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T20:47:56.432 INFO:teuthology.orchestra.run.smithi073.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T20:47:56.432 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T20:47:56.432 INFO:teuthology.orchestra.run.smithi073.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T20:47:56.432 INFO:teuthology.orchestra.run.smithi073.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T20:47:56.433 INFO:teuthology.orchestra.run.smithi073.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T20:47:56.433 INFO:teuthology.orchestra.run.smithi073.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T20:47:56.433 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T20:47:56.433 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T20:47:56.433 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T20:47:56.433 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T20:47:56.433 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T20:47:56.433 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T20:47:56.433 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T20:47:56.433 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T20:47:56.433 INFO:teuthology.orchestra.run.smithi073.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T20:47:56.433 INFO:teuthology.orchestra.run.smithi073.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T20:47:56.433 INFO:teuthology.orchestra.run.smithi073.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T20:47:56.433 INFO:teuthology.orchestra.run.smithi073.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T20:47:56.434 INFO:teuthology.orchestra.run.smithi073.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T20:47:56.434 INFO:teuthology.orchestra.run.smithi073.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T20:47:56.434 INFO:teuthology.orchestra.run.smithi073.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T20:47:56.434 INFO:teuthology.orchestra.run.smithi073.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T20:47:56.434 INFO:teuthology.orchestra.run.smithi073.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T20:47:56.434 INFO:teuthology.orchestra.run.smithi073.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T20:47:56.434 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T20:47:56.434 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T20:47:56.434 INFO:teuthology.orchestra.run.smithi073.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T20:47:56.434 INFO:teuthology.orchestra.run.smithi073.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T20:47:56.434 INFO:teuthology.orchestra.run.smithi073.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T20:47:56.434 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T20:47:56.434 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T20:47:56.434 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T20:47:56.434 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T20:47:56.435 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T20:47:56.435 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T20:47:56.435 INFO:teuthology.orchestra.run.smithi073.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T20:47:56.435 INFO:teuthology.orchestra.run.smithi073.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T20:47:56.435 INFO:teuthology.orchestra.run.smithi073.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T20:47:56.435 INFO:teuthology.orchestra.run.smithi073.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T20:47:56.435 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T20:47:56.435 INFO:teuthology.orchestra.run.smithi073.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T20:47:56.435 INFO:teuthology.orchestra.run.smithi073.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T20:47:56.435 INFO:teuthology.orchestra.run.smithi073.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T20:47:56.435 INFO:teuthology.orchestra.run.smithi073.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T20:47:56.435 INFO:teuthology.orchestra.run.smithi073.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T20:47:56.435 INFO:teuthology.orchestra.run.smithi073.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T20:47:56.435 INFO:teuthology.orchestra.run.smithi073.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T20:47:56.435 INFO:teuthology.orchestra.run.smithi073.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T20:47:56.436 INFO:teuthology.orchestra.run.smithi073.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T20:47:56.436 INFO:teuthology.orchestra.run.smithi073.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T20:47:56.436 INFO:teuthology.orchestra.run.smithi073.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T20:47:56.436 INFO:teuthology.orchestra.run.smithi073.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T20:47:56.436 INFO:teuthology.orchestra.run.smithi073.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T20:47:56.436 INFO:teuthology.orchestra.run.smithi073.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T20:47:56.436 INFO:teuthology.orchestra.run.smithi073.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T20:47:56.436 INFO:teuthology.orchestra.run.smithi073.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T20:47:56.436 INFO:teuthology.orchestra.run.smithi073.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T20:47:56.436 INFO:teuthology.orchestra.run.smithi073.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T20:47:56.436 INFO:teuthology.orchestra.run.smithi073.stdout:Installing weak dependencies: 2024-04-26T20:47:56.436 INFO:teuthology.orchestra.run.smithi073.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T20:47:56.436 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:47:56.436 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction Summary 2024-04-26T20:47:56.436 INFO:teuthology.orchestra.run.smithi073.stdout:======================================================================================= 2024-04-26T20:47:56.437 INFO:teuthology.orchestra.run.smithi073.stdout:Install 148 Packages 2024-04-26T20:47:56.437 INFO:teuthology.orchestra.run.smithi073.stdout:Upgrade 4 Packages 2024-04-26T20:47:56.437 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:47:56.437 INFO:teuthology.orchestra.run.smithi073.stdout:Total download size: 222 M 2024-04-26T20:47:56.437 INFO:teuthology.orchestra.run.smithi073.stdout:Downloading Packages: 2024-04-26T20:47:57.279 INFO:teuthology.orchestra.run.smithi073.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 35 kB/s | 6.6 kB 00:00 2024-04-26T20:47:57.479 INFO:teuthology.orchestra.run.smithi073.stdout:(2/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 4.3 MB/s | 889 kB 00:00 2024-04-26T20:47:57.529 INFO:teuthology.orchestra.run.smithi073.stdout:(3/152): ceph-immutable-object-cache-19.0.0-330 2.9 MB/s | 145 kB 00:00 2024-04-26T20:47:57.830 INFO:teuthology.orchestra.run.smithi073.stdout:(4/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 7.9 MB/s | 2.4 MB 00:00 2024-04-26T20:47:57.955 INFO:teuthology.orchestra.run.smithi073.stdout:(5/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 6.2 MB/s | 5.4 MB 00:00 2024-04-26T20:47:58.013 INFO:teuthology.orchestra.run.smithi073.stdout:(6/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 8.3 MB/s | 1.5 MB 00:00 2024-04-26T20:47:58.489 INFO:teuthology.orchestra.run.smithi073.stdout:(7/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 8.8 MB/s | 4.7 MB 00:00 2024-04-26T20:47:58.724 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.2 MB 00:00 2024-04-26T20:47:59.880 INFO:teuthology.orchestra.run.smithi073.stdout:(8/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el9 7.6 MB/s | 11 MB 00:01 2024-04-26T20:48:00.097 INFO:teuthology.orchestra.run.smithi073.stdout:(9/152): ceph-common-19.0.0-3306.gbc612ffb.el9. 7.0 MB/s | 21 MB 00:03 2024-04-26T20:48:00.122 INFO:teuthology.orchestra.run.smithi073.stdout:(10/152): ceph-selinux-19.0.0-3306.gbc612ffb.el 104 kB/s | 25 kB 00:00 2024-04-26T20:48:00.242 INFO:teuthology.orchestra.run.smithi120.stdout:CentOS Stream 9 - Extras packages 43 kB/s | 16 kB 00:00 2024-04-26T20:48:00.323 INFO:teuthology.orchestra.run.smithi073.stdout:(11/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x8 7.3 MB/s | 17 MB 00:02 2024-04-26T20:48:00.348 INFO:teuthology.orchestra.run.smithi073.stdout:(12/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 999 kB/s | 225 kB 00:00 2024-04-26T20:48:00.373 INFO:teuthology.orchestra.run.smithi073.stdout:(13/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 539 kB/s | 27 kB 00:00 2024-04-26T20:48:00.415 INFO:teuthology.orchestra.run.smithi073.stdout:(14/152): libcephfs-devel-19.0.0-3306.gbc612ffb 474 kB/s | 32 kB 00:00 2024-04-26T20:48:00.490 INFO:teuthology.orchestra.run.smithi073.stdout:(15/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 6.2 MB/s | 747 kB 00:00 2024-04-26T20:48:00.515 INFO:teuthology.orchestra.run.smithi073.stdout:(16/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 1.6 MB/s | 163 kB 00:00 2024-04-26T20:48:00.541 INFO:teuthology.orchestra.run.smithi073.stdout:(17/152): librados-devel-19.0.0-3306.gbc612ffb. 2.5 MB/s | 127 kB 00:00 2024-04-26T20:48:00.650 INFO:teuthology.orchestra.run.smithi073.stdout:(18/152): libradosstriper1-19.0.0-3306.gbc612ff 3.7 MB/s | 501 kB 00:00 2024-04-26T20:48:00.742 INFO:teuthology.orchestra.run.smithi073.stdout:(19/152): python3-ceph-argparse-19.0.0-3306.gbc 490 kB/s | 45 kB 00:00 2024-04-26T20:48:00.834 INFO:teuthology.orchestra.run.smithi073.stdout:(20/152): python3-ceph-common-19.0.0-3306.gbc61 1.4 MB/s | 135 kB 00:00 2024-04-26T20:48:00.934 INFO:teuthology.orchestra.run.smithi073.stdout:(21/152): python3-cephfs-19.0.0-3306.gbc612ffb. 1.6 MB/s | 161 kB 00:00 2024-04-26T20:48:01.102 INFO:teuthology.orchestra.run.smithi073.stdout:(22/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 9.6 MB/s | 5.4 MB 00:00 2024-04-26T20:48:01.135 INFO:teuthology.orchestra.run.smithi073.stdout:(23/152): python3-rados-19.0.0-3306.gbc612ffb.e 1.6 MB/s | 321 kB 00:00 2024-04-26T20:48:01.177 INFO:teuthology.orchestra.run.smithi073.stdout:(24/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 3.9 MB/s | 298 kB 00:00 2024-04-26T20:48:01.219 INFO:teuthology.orchestra.run.smithi073.stdout:(25/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 1.2 MB/s | 99 kB 00:00 2024-04-26T20:48:01.244 INFO:teuthology.orchestra.run.smithi073.stdout:(26/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 1.2 MB/s | 85 kB 00:00 2024-04-26T20:48:01.311 INFO:teuthology.orchestra.run.smithi073.stdout:(27/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 2.5 MB/s | 171 kB 00:00 2024-04-26T20:48:01.369 INFO:teuthology.orchestra.run.smithi120.stdout:Extra Packages for Enterprise Linux 21 MB/s | 21 MB 00:01 2024-04-26T20:48:01.386 INFO:teuthology.orchestra.run.smithi073.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3306.g 458 kB/s | 34 kB 00:00 2024-04-26T20:48:01.470 INFO:teuthology.orchestra.run.smithi073.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 1.7 MB/s | 142 kB 00:00 2024-04-26T20:48:01.570 INFO:teuthology.orchestra.run.smithi073.stdout:(30/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 8.8 MB/s | 3.1 MB 00:00 2024-04-26T20:48:01.987 INFO:teuthology.orchestra.run.smithi073.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 6.8 MB/s | 3.5 MB 00:00 2024-04-26T20:48:02.071 INFO:teuthology.orchestra.run.smithi073.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3306.gbc 2.9 MB/s | 249 kB 00:00 2024-04-26T20:48:02.138 INFO:teuthology.orchestra.run.smithi073.stdout:(33/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 741 kB/s | 49 kB 00:00 2024-04-26T20:48:02.246 INFO:teuthology.orchestra.run.smithi073.stdout:(34/152): ceph-mgr-diskprediction-local-19.0.0- 11 MB/s | 7.4 MB 00:00 2024-04-26T20:48:02.271 INFO:teuthology.orchestra.run.smithi073.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3306.gb 123 kB/s | 16 kB 00:00 2024-04-26T20:48:02.347 INFO:teuthology.orchestra.run.smithi073.stdout:(36/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 2.7 MB/s | 273 kB 00:00 2024-04-26T20:48:02.389 INFO:teuthology.orchestra.run.smithi073.stdout:(37/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 6.4 MB/s | 762 kB 00:00 2024-04-26T20:48:02.431 INFO:teuthology.orchestra.run.smithi073.stdout:(38/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 218 kB/s | 18 kB 00:00 2024-04-26T20:48:02.457 INFO:teuthology.orchestra.run.smithi073.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 1.2 MB/s | 80 kB 00:00 2024-04-26T20:48:02.556 INFO:teuthology.orchestra.run.smithi073.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.9 MB/s | 195 kB 00:00 2024-04-26T20:48:02.581 INFO:teuthology.orchestra.run.smithi073.stdout:(41/152): libgfortran-11.4.1-3.el9.x86_64.rpm 5.2 MB/s | 803 kB 00:00 2024-04-26T20:48:02.724 INFO:teuthology.orchestra.run.smithi073.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.5 MB/s | 253 kB 00:00 2024-04-26T20:48:02.790 INFO:teuthology.orchestra.run.smithi073.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 5.9 MB/s | 1.2 MB 00:00 2024-04-26T20:48:02.816 INFO:teuthology.orchestra.run.smithi073.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 106 kB 00:00 2024-04-26T20:48:02.874 INFO:teuthology.orchestra.run.smithi073.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-04-26T20:48:02.908 INFO:teuthology.orchestra.run.smithi073.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.3 MB/s | 125 kB 00:00 2024-04-26T20:48:02.975 INFO:teuthology.orchestra.run.smithi073.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.1 MB/s | 215 kB 00:00 2024-04-26T20:48:03.325 INFO:teuthology.orchestra.run.smithi073.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 1.6 MB/s | 689 kB 00:00 2024-04-26T20:48:03.392 INFO:teuthology.orchestra.run.smithi073.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-04-26T20:48:03.467 INFO:teuthology.orchestra.run.smithi073.stdout:(50/152): bison-3.7.4-5.el9.x86_64.rpm 1.9 MB/s | 943 kB 00:00 2024-04-26T20:48:03.493 INFO:teuthology.orchestra.run.smithi073.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 214 kB/s | 21 kB 00:00 2024-04-26T20:48:03.943 INFO:teuthology.orchestra.run.smithi073.stdout:(52/152): ceph-test-19.0.0-3306.gbc612ffb.el9.x 13 MB/s | 49 MB 00:03 2024-04-26T20:48:03.968 INFO:teuthology.orchestra.run.smithi073.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 69 kB/s | 33 kB 00:00 2024-04-26T20:48:03.993 INFO:teuthology.orchestra.run.smithi073.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 596 kB/s | 313 kB 00:00 2024-04-26T20:48:04.019 INFO:teuthology.orchestra.run.smithi073.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 362 kB/s | 18 kB 00:00 2024-04-26T20:48:04.044 INFO:teuthology.orchestra.run.smithi073.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-04-26T20:48:04.094 INFO:teuthology.orchestra.run.smithi073.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 908 kB/s | 45 kB 00:00 2024-04-26T20:48:04.120 INFO:teuthology.orchestra.run.smithi073.stdout:(58/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.6 MB/s | 160 kB 00:00 2024-04-26T20:48:04.278 INFO:teuthology.orchestra.run.smithi073.stdout:(59/152): libtool-2.4.6-45.el9.x86_64.rpm 3.6 MB/s | 582 kB 00:00 2024-04-26T20:48:04.304 INFO:teuthology.orchestra.run.smithi073.stdout:(60/152): librdkafka-1.6.1-102.el9.x86_64.rpm 3.1 MB/s | 662 kB 00:00 2024-04-26T20:48:04.337 INFO:teuthology.orchestra.run.smithi073.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 436 kB/s | 25 kB 00:00 2024-04-26T20:48:04.362 INFO:teuthology.orchestra.run.smithi073.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 811 kB/s | 47 kB 00:00 2024-04-26T20:48:04.421 INFO:teuthology.orchestra.run.smithi073.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-04-26T20:48:04.446 INFO:teuthology.orchestra.run.smithi073.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-04-26T20:48:04.471 INFO:teuthology.orchestra.run.smithi073.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 701 kB/s | 35 kB 00:00 2024-04-26T20:48:04.730 INFO:teuthology.orchestra.run.smithi073.stdout:(66/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.8 MB/s | 3.0 MB 00:00 2024-04-26T20:48:05.056 INFO:teuthology.orchestra.run.smithi073.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 4.9 MB/s | 1.6 MB 00:00 2024-04-26T20:48:05.247 INFO:teuthology.orchestra.run.smithi073.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 4.6 MB/s | 911 kB 00:00 2024-04-26T20:48:05.314 INFO:teuthology.orchestra.run.smithi073.stdout:(69/152): openblas-openmp-0.3.21-2.el9.x86_64.r 5.5 MB/s | 4.7 MB 00:00 2024-04-26T20:48:05.373 INFO:teuthology.orchestra.run.smithi073.stdout:(70/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 4.9 MB/s | 4.4 MB 00:00 2024-04-26T20:48:05.398 INFO:teuthology.orchestra.run.smithi073.stdout:(71/152): postgresql-private-libs-13.14-1.el9.x 887 kB/s | 133 kB 00:00 2024-04-26T20:48:05.448 INFO:teuthology.orchestra.run.smithi073.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-04-26T20:48:05.590 INFO:teuthology.orchestra.run.smithi073.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 4.7 MB/s | 1.0 MB 00:00 2024-04-26T20:48:05.649 INFO:teuthology.orchestra.run.smithi073.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 4.1 MB/s | 244 kB 00:00 2024-04-26T20:48:05.707 INFO:teuthology.orchestra.run.smithi073.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 4.1 MB/s | 248 kB 00:00 2024-04-26T20:48:05.749 INFO:teuthology.orchestra.run.smithi073.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 618 kB/s | 26 kB 00:00 2024-04-26T20:48:05.799 INFO:teuthology.orchestra.run.smithi073.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-04-26T20:48:05.858 INFO:teuthology.orchestra.run.smithi073.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-04-26T20:48:05.900 INFO:teuthology.orchestra.run.smithi073.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 837 kB/s | 35 kB 00:00 2024-04-26T20:48:06.192 INFO:teuthology.orchestra.run.smithi073.stdout:(80/152): postgresql-server-13.14-1.el9.x86_64. 7.1 MB/s | 5.8 MB 00:00 2024-04-26T20:48:06.276 INFO:teuthology.orchestra.run.smithi073.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.9 MB/s | 250 kB 00:00 2024-04-26T20:48:06.360 INFO:teuthology.orchestra.run.smithi073.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 222 kB 00:00 2024-04-26T20:48:06.435 INFO:teuthology.orchestra.run.smithi073.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 77 kB 00:00 2024-04-26T20:48:06.526 INFO:teuthology.orchestra.run.smithi120.stdout:lab-extras 8.0 kB/s | 1.7 kB 00:00 2024-04-26T20:48:06.527 INFO:teuthology.orchestra.run.smithi073.stdout:(84/152): python3-prettytable-0.7.2-27.el9.noar 461 kB/s | 42 kB 00:00 2024-04-26T20:48:06.602 INFO:teuthology.orchestra.run.smithi073.stdout:(85/152): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 269 kB 00:00 2024-04-26T20:48:06.686 INFO:teuthology.orchestra.run.smithi073.stdout:(86/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 159 kB 00:00 2024-04-26T20:48:06.769 INFO:teuthology.orchestra.run.smithi073.stdout:(87/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.3 MB/s | 279 kB 00:00 2024-04-26T20:48:06.844 INFO:teuthology.orchestra.run.smithi073.stdout:(88/152): python3-pytz-2021.1-5.el9.noarch.rpm 681 kB/s | 51 kB 00:00 2024-04-26T20:48:06.911 INFO:teuthology.orchestra.run.smithi073.stdout:(89/152): python3-requests-oauthlib-1.3.0-12.el 805 kB/s | 54 kB 00:00 2024-04-26T20:48:07.003 INFO:teuthology.orchestra.run.smithi073.stdout:(90/152): python3-babel-2.9.1-2.el9.noarch.rpm 3.8 MB/s | 6.0 MB 00:01 2024-04-26T20:48:07.087 INFO:teuthology.orchestra.run.smithi073.stdout:(91/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.3 MB/s | 5.1 MB 00:01 2024-04-26T20:48:07.112 INFO:teuthology.orchestra.run.smithi073.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 384 kB/s | 42 kB 00:00 2024-04-26T20:48:07.179 INFO:teuthology.orchestra.run.smithi073.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 481 kB/s | 44 kB 00:00 2024-04-26T20:48:07.271 INFO:teuthology.orchestra.run.smithi073.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 1.4 MB/s | 221 kB 00:00 2024-04-26T20:48:07.296 INFO:teuthology.orchestra.run.smithi073.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 307 kB/s | 36 kB 00:00 2024-04-26T20:48:07.355 INFO:teuthology.orchestra.run.smithi073.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 566 kB/s | 47 kB 00:00 2024-04-26T20:48:07.422 INFO:teuthology.orchestra.run.smithi073.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 868 kB/s | 58 kB 00:00 2024-04-26T20:48:07.447 INFO:teuthology.orchestra.run.smithi073.stdout:(98/152): socat-1.7.4.1-5.el9.x86_64.rpm 2.0 MB/s | 305 kB 00:00 2024-04-26T20:48:07.522 INFO:teuthology.orchestra.run.smithi073.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 935 kB/s | 94 kB 00:00 2024-04-26T20:48:07.548 INFO:teuthology.orchestra.run.smithi073.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 635 kB/s | 64 kB 00:00 2024-04-26T20:48:07.598 INFO:teuthology.orchestra.run.smithi073.stdout:(101/152): lua-devel-5.4.4-4.el9.x86_64.rpm 447 kB/s | 22 kB 00:00 2024-04-26T20:48:07.623 INFO:teuthology.orchestra.run.smithi073.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 445 kB/s | 45 kB 00:00 2024-04-26T20:48:07.782 INFO:teuthology.orchestra.run.smithi073.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 4.6 MB/s | 863 kB 00:00 2024-04-26T20:48:08.240 INFO:teuthology.orchestra.run.smithi073.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 889 kB/s | 548 kB 00:00 2024-04-26T20:48:08.316 INFO:teuthology.orchestra.run.smithi073.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 259 kB/s | 19 kB 00:00 2024-04-26T20:48:08.366 INFO:teuthology.orchestra.run.smithi073.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 529 kB/s | 309 kB 00:00 2024-04-26T20:48:08.449 INFO:teuthology.orchestra.run.smithi073.stdout:(107/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 312 kB/s | 26 kB 00:00 2024-04-26T20:48:08.558 INFO:teuthology.orchestra.run.smithi073.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 448 kB/s | 49 kB 00:00 2024-04-26T20:48:08.653 INFO:teuthology.orchestra.run.smithi120.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T20:48:08.653 INFO:teuthology.orchestra.run.smithi120.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T20:48:08.664 INFO:teuthology.orchestra.run.smithi120.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T20:48:08.665 INFO:teuthology.orchestra.run.smithi120.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T20:48:08.666 INFO:teuthology.orchestra.run.smithi120.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T20:48:08.667 INFO:teuthology.orchestra.run.smithi120.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T20:48:08.668 INFO:teuthology.orchestra.run.smithi120.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T20:48:08.671 INFO:teuthology.orchestra.run.smithi120.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T20:48:08.672 INFO:teuthology.orchestra.run.smithi120.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T20:48:08.700 INFO:teuthology.orchestra.run.smithi073.stdout:(109/152): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-04-26T20:48:08.725 INFO:teuthology.orchestra.run.smithi073.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 404 kB/s | 67 kB 00:00 2024-04-26T20:48:08.732 INFO:teuthology.orchestra.run.smithi120.stdout:Dependencies resolved. 2024-04-26T20:48:08.741 INFO:teuthology.orchestra.run.smithi120.stdout:======================================================================================= 2024-04-26T20:48:08.741 INFO:teuthology.orchestra.run.smithi120.stdout: Package Arch Version Repository Size 2024-04-26T20:48:08.741 INFO:teuthology.orchestra.run.smithi120.stdout:======================================================================================= 2024-04-26T20:48:08.741 INFO:teuthology.orchestra.run.smithi120.stdout:Installing: 2024-04-26T20:48:08.741 INFO:teuthology.orchestra.run.smithi120.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T20:48:08.741 INFO:teuthology.orchestra.run.smithi120.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-26T20:48:08.741 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T20:48:08.741 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-26T20:48:08.741 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-26T20:48:08.741 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-26T20:48:08.742 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-26T20:48:08.742 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-26T20:48:08.742 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-26T20:48:08.742 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-26T20:48:08.742 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-26T20:48:08.742 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-26T20:48:08.742 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-26T20:48:08.742 INFO:teuthology.orchestra.run.smithi120.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-26T20:48:08.742 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-26T20:48:08.742 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-26T20:48:08.742 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-26T20:48:08.742 INFO:teuthology.orchestra.run.smithi120.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T20:48:08.742 INFO:teuthology.orchestra.run.smithi120.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T20:48:08.743 INFO:teuthology.orchestra.run.smithi120.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T20:48:08.743 INFO:teuthology.orchestra.run.smithi120.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-26T20:48:08.743 INFO:teuthology.orchestra.run.smithi120.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-26T20:48:08.743 INFO:teuthology.orchestra.run.smithi120.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-26T20:48:08.743 INFO:teuthology.orchestra.run.smithi120.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T20:48:08.743 INFO:teuthology.orchestra.run.smithi120.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T20:48:08.743 INFO:teuthology.orchestra.run.smithi120.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T20:48:08.743 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T20:48:08.743 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T20:48:08.743 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T20:48:08.743 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-26T20:48:08.743 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-26T20:48:08.743 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-26T20:48:08.743 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-26T20:48:08.743 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-26T20:48:08.744 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T20:48:08.744 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-26T20:48:08.744 INFO:teuthology.orchestra.run.smithi120.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T20:48:08.744 INFO:teuthology.orchestra.run.smithi120.stdout:Upgrading: 2024-04-26T20:48:08.744 INFO:teuthology.orchestra.run.smithi120.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-26T20:48:08.744 INFO:teuthology.orchestra.run.smithi120.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T20:48:08.744 INFO:teuthology.orchestra.run.smithi120.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T20:48:08.744 INFO:teuthology.orchestra.run.smithi120.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T20:48:08.744 INFO:teuthology.orchestra.run.smithi120.stdout:Installing dependencies: 2024-04-26T20:48:08.744 INFO:teuthology.orchestra.run.smithi120.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T20:48:08.744 INFO:teuthology.orchestra.run.smithi120.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T20:48:08.744 INFO:teuthology.orchestra.run.smithi120.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T20:48:08.744 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-26T20:48:08.744 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-26T20:48:08.744 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-26T20:48:08.745 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-26T20:48:08.745 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-26T20:48:08.745 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-26T20:48:08.745 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-26T20:48:08.745 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-26T20:48:08.745 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T20:48:08.745 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T20:48:08.745 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T20:48:08.745 INFO:teuthology.orchestra.run.smithi120.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T20:48:08.745 INFO:teuthology.orchestra.run.smithi120.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T20:48:08.745 INFO:teuthology.orchestra.run.smithi120.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T20:48:08.745 INFO:teuthology.orchestra.run.smithi120.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T20:48:08.745 INFO:teuthology.orchestra.run.smithi120.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T20:48:08.746 INFO:teuthology.orchestra.run.smithi120.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-26T20:48:08.746 INFO:teuthology.orchestra.run.smithi120.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T20:48:08.746 INFO:teuthology.orchestra.run.smithi120.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T20:48:08.746 INFO:teuthology.orchestra.run.smithi120.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T20:48:08.746 INFO:teuthology.orchestra.run.smithi120.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T20:48:08.746 INFO:teuthology.orchestra.run.smithi120.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T20:48:08.746 INFO:teuthology.orchestra.run.smithi120.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-26T20:48:08.746 INFO:teuthology.orchestra.run.smithi120.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T20:48:08.746 INFO:teuthology.orchestra.run.smithi120.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T20:48:08.746 INFO:teuthology.orchestra.run.smithi120.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T20:48:08.746 INFO:teuthology.orchestra.run.smithi120.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T20:48:08.746 INFO:teuthology.orchestra.run.smithi120.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T20:48:08.746 INFO:teuthology.orchestra.run.smithi120.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T20:48:08.747 INFO:teuthology.orchestra.run.smithi120.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T20:48:08.747 INFO:teuthology.orchestra.run.smithi120.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T20:48:08.747 INFO:teuthology.orchestra.run.smithi120.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T20:48:08.747 INFO:teuthology.orchestra.run.smithi120.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T20:48:08.747 INFO:teuthology.orchestra.run.smithi120.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T20:48:08.747 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T20:48:08.747 INFO:teuthology.orchestra.run.smithi120.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T20:48:08.747 INFO:teuthology.orchestra.run.smithi120.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T20:48:08.747 INFO:teuthology.orchestra.run.smithi120.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T20:48:08.747 INFO:teuthology.orchestra.run.smithi120.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T20:48:08.747 INFO:teuthology.orchestra.run.smithi120.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T20:48:08.747 INFO:teuthology.orchestra.run.smithi120.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T20:48:08.747 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T20:48:08.747 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-26T20:48:08.747 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-26T20:48:08.747 INFO:teuthology.orchestra.run.smithi120.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T20:48:08.748 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T20:48:08.748 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T20:48:08.748 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T20:48:08.748 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T20:48:08.748 INFO:teuthology.orchestra.run.smithi120.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T20:48:08.748 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T20:48:08.748 INFO:teuthology.orchestra.run.smithi120.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T20:48:08.748 INFO:teuthology.orchestra.run.smithi120.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T20:48:08.748 INFO:teuthology.orchestra.run.smithi120.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T20:48:08.748 INFO:teuthology.orchestra.run.smithi120.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T20:48:08.748 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T20:48:08.748 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T20:48:08.748 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T20:48:08.748 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T20:48:08.748 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T20:48:08.749 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T20:48:08.749 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T20:48:08.749 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T20:48:08.749 INFO:teuthology.orchestra.run.smithi120.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T20:48:08.749 INFO:teuthology.orchestra.run.smithi120.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T20:48:08.749 INFO:teuthology.orchestra.run.smithi120.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T20:48:08.749 INFO:teuthology.orchestra.run.smithi120.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T20:48:08.749 INFO:teuthology.orchestra.run.smithi120.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T20:48:08.749 INFO:teuthology.orchestra.run.smithi120.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T20:48:08.749 INFO:teuthology.orchestra.run.smithi120.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T20:48:08.749 INFO:teuthology.orchestra.run.smithi120.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T20:48:08.749 INFO:teuthology.orchestra.run.smithi120.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T20:48:08.749 INFO:teuthology.orchestra.run.smithi120.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T20:48:08.749 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T20:48:08.749 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T20:48:08.750 INFO:teuthology.orchestra.run.smithi120.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T20:48:08.750 INFO:teuthology.orchestra.run.smithi120.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T20:48:08.750 INFO:teuthology.orchestra.run.smithi120.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T20:48:08.750 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T20:48:08.750 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T20:48:08.750 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T20:48:08.750 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T20:48:08.750 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T20:48:08.750 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T20:48:08.750 INFO:teuthology.orchestra.run.smithi120.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T20:48:08.750 INFO:teuthology.orchestra.run.smithi120.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T20:48:08.750 INFO:teuthology.orchestra.run.smithi120.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T20:48:08.750 INFO:teuthology.orchestra.run.smithi120.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T20:48:08.751 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T20:48:08.751 INFO:teuthology.orchestra.run.smithi120.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T20:48:08.751 INFO:teuthology.orchestra.run.smithi120.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T20:48:08.751 INFO:teuthology.orchestra.run.smithi120.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T20:48:08.751 INFO:teuthology.orchestra.run.smithi120.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T20:48:08.751 INFO:teuthology.orchestra.run.smithi120.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T20:48:08.751 INFO:teuthology.orchestra.run.smithi120.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T20:48:08.751 INFO:teuthology.orchestra.run.smithi120.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T20:48:08.751 INFO:teuthology.orchestra.run.smithi120.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T20:48:08.751 INFO:teuthology.orchestra.run.smithi120.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T20:48:08.751 INFO:teuthology.orchestra.run.smithi120.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T20:48:08.751 INFO:teuthology.orchestra.run.smithi120.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T20:48:08.751 INFO:teuthology.orchestra.run.smithi120.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T20:48:08.751 INFO:teuthology.orchestra.run.smithi120.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T20:48:08.751 INFO:teuthology.orchestra.run.smithi120.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T20:48:08.752 INFO:teuthology.orchestra.run.smithi120.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T20:48:08.752 INFO:teuthology.orchestra.run.smithi120.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T20:48:08.752 INFO:teuthology.orchestra.run.smithi120.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T20:48:08.752 INFO:teuthology.orchestra.run.smithi120.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T20:48:08.752 INFO:teuthology.orchestra.run.smithi120.stdout:Installing weak dependencies: 2024-04-26T20:48:08.752 INFO:teuthology.orchestra.run.smithi120.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T20:48:08.752 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T20:48:08.752 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction Summary 2024-04-26T20:48:08.752 INFO:teuthology.orchestra.run.smithi120.stdout:======================================================================================= 2024-04-26T20:48:08.752 INFO:teuthology.orchestra.run.smithi120.stdout:Install 148 Packages 2024-04-26T20:48:08.752 INFO:teuthology.orchestra.run.smithi120.stdout:Upgrade 4 Packages 2024-04-26T20:48:08.752 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T20:48:08.753 INFO:teuthology.orchestra.run.smithi120.stdout:Total download size: 222 M 2024-04-26T20:48:08.753 INFO:teuthology.orchestra.run.smithi120.stdout:Downloading Packages: 2024-04-26T20:48:08.784 INFO:teuthology.orchestra.run.smithi073.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-04-26T20:48:08.876 INFO:teuthology.orchestra.run.smithi073.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 5.8 MB/s | 548 kB 00:00 2024-04-26T20:48:08.935 INFO:teuthology.orchestra.run.smithi073.stdout:(113/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.9 MB/s | 839 kB 00:00 2024-04-26T20:48:08.960 INFO:teuthology.orchestra.run.smithi073.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 521 kB/s | 43 kB 00:00 2024-04-26T20:48:09.002 INFO:teuthology.orchestra.run.smithi073.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 480 kB/s | 32 kB 00:00 2024-04-26T20:48:09.027 INFO:teuthology.orchestra.run.smithi073.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 224 kB/s | 15 kB 00:00 2024-04-26T20:48:09.078 INFO:teuthology.orchestra.run.smithi073.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-04-26T20:48:09.103 INFO:teuthology.orchestra.run.smithi073.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 4.7 MB/s | 358 kB 00:00 2024-04-26T20:48:09.145 INFO:teuthology.orchestra.run.smithi073.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-04-26T20:48:09.161 INFO:teuthology.orchestra.run.smithi073.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 604 kB/s | 35 kB 00:00 2024-04-26T20:48:09.337 INFO:teuthology.orchestra.run.smithi073.stdout:(121/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 6.7 MB/s | 16 MB 00:02 2024-04-26T20:48:09.362 INFO:teuthology.orchestra.run.smithi073.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 947 kB/s | 206 kB 00:00 2024-04-26T20:48:09.421 INFO:teuthology.orchestra.run.smithi073.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 7.9 MB/s | 2.0 MB 00:00 2024-04-26T20:48:09.446 INFO:teuthology.orchestra.run.smithi073.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 128 kB/s | 11 kB 00:00 2024-04-26T20:48:09.479 INFO:teuthology.orchestra.run.smithi073.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 302 kB/s | 18 kB 00:00 2024-04-26T20:48:09.505 INFO:teuthology.orchestra.run.smithi073.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 395 kB/s | 23 kB 00:00 2024-04-26T20:48:09.538 INFO:teuthology.orchestra.run.smithi073.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 332 kB/s | 19 kB 00:00 2024-04-26T20:48:09.564 INFO:teuthology.orchestra.run.smithi073.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 336 kB/s | 20 kB 00:00 2024-04-26T20:48:09.606 INFO:teuthology.orchestra.run.smithi073.stdout:(129/152): python3-grpcio-tools-1.46.7-10.el9.x 539 kB/s | 144 kB 00:00 2024-04-26T20:48:09.639 INFO:teuthology.orchestra.run.smithi073.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 10 MB/s | 1.0 MB 00:00 2024-04-26T20:48:09.664 INFO:teuthology.orchestra.run.smithi073.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 461 kB/s | 46 kB 00:00 2024-04-26T20:48:09.672 INFO:teuthology.orchestra.run.smithi120.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 37 kB/s | 6.6 kB 00:00 2024-04-26T20:48:09.698 INFO:teuthology.orchestra.run.smithi073.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 856 kB/s | 79 kB 00:00 2024-04-26T20:48:09.724 INFO:teuthology.orchestra.run.smithi073.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 688 kB/s | 58 kB 00:00 2024-04-26T20:48:09.749 INFO:teuthology.orchestra.run.smithi073.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-04-26T20:48:09.782 INFO:teuthology.orchestra.run.smithi073.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-04-26T20:48:09.858 INFO:teuthology.orchestra.run.smithi073.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 260 kB/s | 28 kB 00:00 2024-04-26T20:48:09.883 INFO:teuthology.orchestra.run.smithi073.stdout:(137/152): python3-portend-3.1.0-2.el9.noarch.r 89 kB/s | 16 kB 00:00 2024-04-26T20:48:09.908 INFO:teuthology.orchestra.run.smithi073.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 245 kB/s | 31 kB 00:00 2024-04-26T20:48:09.933 INFO:teuthology.orchestra.run.smithi073.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-04-26T20:48:09.959 INFO:teuthology.orchestra.run.smithi073.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 783 kB/s | 59 kB 00:00 2024-04-26T20:48:09.980 INFO:teuthology.orchestra.run.smithi120.stdout:(2/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 2.8 MB/s | 889 kB 00:00 2024-04-26T20:48:10.011 INFO:teuthology.orchestra.run.smithi073.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 659 kB/s | 51 kB 00:00 2024-04-26T20:48:10.036 INFO:teuthology.orchestra.run.smithi073.stdout:(142/152): python3-tempora-5.0.0-2.el9.noarch.r 279 kB/s | 36 kB 00:00 2024-04-26T20:48:10.062 INFO:teuthology.orchestra.run.smithi073.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-04-26T20:48:10.087 INFO:teuthology.orchestra.run.smithi073.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-26T20:48:10.089 INFO:teuthology.orchestra.run.smithi120.stdout:(3/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 9.0 MB/s | 5.4 MB 00:00 2024-04-26T20:48:10.114 INFO:teuthology.orchestra.run.smithi120.stdout:(4/152): ceph-immutable-object-cache-19.0.0-330 1.1 MB/s | 145 kB 00:00 2024-04-26T20:48:10.129 INFO:teuthology.orchestra.run.smithi073.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 299 kB/s | 20 kB 00:00 2024-04-26T20:48:10.154 INFO:teuthology.orchestra.run.smithi073.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-04-26T20:48:10.179 INFO:teuthology.orchestra.run.smithi073.stdout:(147/152): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-04-26T20:48:10.221 INFO:teuthology.orchestra.run.smithi073.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-04-26T20:48:10.330 INFO:teuthology.orchestra.run.smithi073.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 12 MB/s | 1.3 MB 00:00 2024-04-26T20:48:10.331 INFO:teuthology.orchestra.run.smithi120.stdout:(5/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 9.8 MB/s | 2.4 MB 00:00 2024-04-26T20:48:10.464 INFO:teuthology.orchestra.run.smithi073.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 18 MB/s | 2.4 MB 00:00 2024-04-26T20:48:10.490 INFO:teuthology.orchestra.run.smithi120.stdout:(6/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 4.0 MB/s | 1.5 MB 00:00 2024-04-26T20:48:10.824 INFO:teuthology.orchestra.run.smithi120.stdout:(7/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 9.6 MB/s | 4.7 MB 00:00 2024-04-26T20:48:11.130 INFO:teuthology.orchestra.run.smithi073.stdout:(151/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 3.3 MB/s | 3.1 MB 00:00 2024-04-26T20:48:11.330 INFO:teuthology.orchestra.run.smithi073.stdout:(152/152): librados2-19.0.0-3306.gbc612ffb.el9. 2.9 MB/s | 3.4 MB 00:01 2024-04-26T20:48:11.333 INFO:teuthology.orchestra.run.smithi073.stdout:-------------------------------------------------------------------------------- 2024-04-26T20:48:11.333 INFO:teuthology.orchestra.run.smithi073.stdout:Total 15 MB/s | 222 MB 00:14 2024-04-26T20:48:11.807 INFO:teuthology.orchestra.run.smithi120.stdout:(8/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el9 11 MB/s | 11 MB 00:00 2024-04-26T20:48:11.899 INFO:teuthology.orchestra.run.smithi120.stdout:(9/152): ceph-selinux-19.0.0-3306.gbc612ffb.el9 274 kB/s | 25 kB 00:00 2024-04-26T20:48:12.116 INFO:teuthology.orchestra.run.smithi120.stdout:(10/152): ceph-common-19.0.0-3306.gbc612ffb.el9 8.1 MB/s | 21 MB 00:02 2024-04-26T20:48:12.191 INFO:teuthology.orchestra.run.smithi120.stdout:(11/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 2.9 MB/s | 225 kB 00:00 2024-04-26T20:48:12.258 INFO:teuthology.orchestra.run.smithi120.stdout:(12/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 407 kB/s | 27 kB 00:00 2024-04-26T20:48:12.325 INFO:teuthology.orchestra.run.smithi120.stdout:(13/152): libcephfs-devel-19.0.0-3306.gbc612ffb 475 kB/s | 32 kB 00:00 2024-04-26T20:48:12.444 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction check 2024-04-26T20:48:12.459 INFO:teuthology.orchestra.run.smithi120.stdout:(14/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 5.5 MB/s | 747 kB 00:00 2024-04-26T20:48:12.542 INFO:teuthology.orchestra.run.smithi120.stdout:(15/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 1.9 MB/s | 163 kB 00:00 2024-04-26T20:48:12.586 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction check succeeded. 2024-04-26T20:48:12.586 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction test 2024-04-26T20:48:12.618 INFO:teuthology.orchestra.run.smithi120.stdout:(16/152): librados-devel-19.0.0-3306.gbc612ffb. 1.7 MB/s | 127 kB 00:00 2024-04-26T20:48:12.701 INFO:teuthology.orchestra.run.smithi120.stdout:(17/152): libradosstriper1-19.0.0-3306.gbc612ff 5.9 MB/s | 501 kB 00:00 2024-04-26T20:48:13.277 INFO:teuthology.orchestra.run.smithi120.stdout:(18/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x8 6.0 MB/s | 17 MB 00:02 2024-04-26T20:48:13.410 INFO:teuthology.orchestra.run.smithi120.stdout:(19/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 7.6 MB/s | 5.4 MB 00:00 2024-04-26T20:48:13.427 INFO:teuthology.orchestra.run.smithi120.stdout:(20/152): python3-ceph-argparse-19.0.0-3306.gbc 300 kB/s | 45 kB 00:00 2024-04-26T20:48:13.478 INFO:teuthology.orchestra.run.smithi120.stdout:(21/152): python3-ceph-common-19.0.0-3306.gbc61 2.0 MB/s | 135 kB 00:00 2024-04-26T20:48:13.503 INFO:teuthology.orchestra.run.smithi120.stdout:(22/152): python3-cephfs-19.0.0-3306.gbc612ffb. 2.1 MB/s | 161 kB 00:00 2024-04-26T20:48:13.545 INFO:teuthology.orchestra.run.smithi120.stdout:(23/152): python3-rados-19.0.0-3306.gbc612ffb.e 4.7 MB/s | 321 kB 00:00 2024-04-26T20:48:13.588 INFO:teuthology.orchestra.run.smithi120.stdout:(24/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 3.4 MB/s | 298 kB 00:00 2024-04-26T20:48:13.613 INFO:teuthology.orchestra.run.smithi120.stdout:(25/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 1.4 MB/s | 99 kB 00:00 2024-04-26T20:48:13.655 INFO:teuthology.orchestra.run.smithi120.stdout:(26/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 1.2 MB/s | 85 kB 00:00 2024-04-26T20:48:13.730 INFO:teuthology.orchestra.run.smithi120.stdout:(27/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 2.2 MB/s | 171 kB 00:00 2024-04-26T20:48:13.806 INFO:teuthology.orchestra.run.smithi120.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3306.g 455 kB/s | 34 kB 00:00 2024-04-26T20:48:13.881 INFO:teuthology.orchestra.run.smithi120.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 1.9 MB/s | 142 kB 00:00 2024-04-26T20:48:13.998 INFO:teuthology.orchestra.run.smithi120.stdout:(30/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 8.0 MB/s | 3.1 MB 00:00 2024-04-26T20:48:14.368 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction test succeeded. 2024-04-26T20:48:14.369 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction 2024-04-26T20:48:14.498 INFO:teuthology.orchestra.run.smithi120.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 5.7 MB/s | 3.5 MB 00:00 2024-04-26T20:48:14.590 INFO:teuthology.orchestra.run.smithi120.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3306.gbc 2.7 MB/s | 249 kB 00:00 2024-04-26T20:48:14.665 INFO:teuthology.orchestra.run.smithi120.stdout:(33/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 657 kB/s | 49 kB 00:00 2024-04-26T20:48:14.732 INFO:teuthology.orchestra.run.smithi120.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3306.gb 247 kB/s | 16 kB 00:00 2024-04-26T20:48:14.808 INFO:teuthology.orchestra.run.smithi120.stdout:(35/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 3.6 MB/s | 273 kB 00:00 2024-04-26T20:48:14.900 INFO:teuthology.orchestra.run.smithi120.stdout:(36/152): ceph-mgr-diskprediction-local-19.0.0- 8.2 MB/s | 7.4 MB 00:00 2024-04-26T20:48:15.367 INFO:teuthology.orchestra.run.smithi120.stdout:(37/152): ceph-test-19.0.0-3306.gbc612ffb.el9.x 14 MB/s | 49 MB 00:03 2024-04-26T20:48:15.392 INFO:teuthology.orchestra.run.smithi120.stdout:(38/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 1.3 MB/s | 762 kB 00:00 2024-04-26T20:48:15.417 INFO:teuthology.orchestra.run.smithi120.stdout:(39/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 35 kB/s | 18 kB 00:00 2024-04-26T20:48:15.526 INFO:teuthology.orchestra.run.smithi120.stdout:(40/152): fuse-2.9.9-15.el9.x86_64.rpm 507 kB/s | 80 kB 00:00 2024-04-26T20:48:15.609 INFO:teuthology.orchestra.run.smithi120.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-04-26T20:48:15.635 INFO:teuthology.orchestra.run.smithi120.stdout:(42/152): libquadmath-11.4.1-3.el9.x86_64.rpm 899 kB/s | 195 kB 00:00 2024-04-26T20:48:15.669 INFO:teuthology.orchestra.run.smithi120.stdout:(43/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.8 MB/s | 803 kB 00:00 2024-04-26T20:48:15.685 INFO:teuthology.orchestra.run.smithi120.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-04-26T20:48:15.719 INFO:teuthology.orchestra.run.smithi120.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-04-26T20:48:15.744 INFO:teuthology.orchestra.run.smithi120.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-04-26T20:48:15.778 INFO:teuthology.orchestra.run.smithi120.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-04-26T20:48:15.895 INFO:teuthology.orchestra.run.smithi120.stdout:(48/152): python3-cryptography-36.0.1-4.el9.x86 4.3 MB/s | 1.2 MB 00:00 2024-04-26T20:48:16.020 INFO:teuthology.orchestra.run.smithi120.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 689 kB 00:00 2024-04-26T20:48:16.079 INFO:teuthology.orchestra.run.smithi120.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 366 kB/s | 21 kB 00:00 2024-04-26T20:48:16.096 INFO:teuthology.orchestra.run.smithi120.stdout:(51/152): boost-program-options-1.75.0-8.el9.x8 532 kB/s | 107 kB 00:00 2024-04-26T20:48:16.138 INFO:teuthology.orchestra.run.smithi120.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-04-26T20:48:16.163 INFO:teuthology.orchestra.run.smithi120.stdout:(53/152): bison-3.7.4-5.el9.x86_64.rpm 2.4 MB/s | 943 kB 00:00 2024-04-26T20:48:16.189 INFO:teuthology.orchestra.run.smithi120.stdout:(54/152): flexiblas-3.0.4-8.el9.x86_64.rpm 355 kB/s | 33 kB 00:00 2024-04-26T20:48:16.222 INFO:teuthology.orchestra.run.smithi120.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 311 kB/s | 18 kB 00:00 2024-04-26T20:48:16.248 INFO:teuthology.orchestra.run.smithi120.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 190 kB/s | 11 kB 00:00 2024-04-26T20:48:16.281 INFO:teuthology.orchestra.run.smithi120.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-04-26T20:48:16.307 INFO:teuthology.orchestra.run.smithi120.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 771 kB/s | 45 kB 00:00 2024-04-26T20:48:16.398 INFO:teuthology.orchestra.run.smithi120.stdout:(59/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 11 MB/s | 3.0 MB 00:00 2024-04-26T20:48:16.432 INFO:teuthology.orchestra.run.smithi120.stdout:(60/152): librdkafka-1.6.1-102.el9.x86_64.rpm 4.3 MB/s | 662 kB 00:00 2024-04-26T20:48:16.457 INFO:teuthology.orchestra.run.smithi120.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 435 kB/s | 25 kB 00:00 2024-04-26T20:48:16.491 INFO:teuthology.orchestra.run.smithi120.stdout:(62/152): libtool-2.4.6-45.el9.x86_64.rpm 3.1 MB/s | 582 kB 00:00 2024-04-26T20:48:16.533 INFO:teuthology.orchestra.run.smithi120.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 471 kB/s | 47 kB 00:00 2024-04-26T20:48:16.558 INFO:teuthology.orchestra.run.smithi120.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 2.7 MB/s | 188 kB 00:00 2024-04-26T20:48:16.592 INFO:teuthology.orchestra.run.smithi120.stdout:(65/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 292 kB 00:00 2024-04-26T20:48:16.609 INFO:teuthology.orchestra.run.smithi120.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 466 kB/s | 35 kB 00:00 2024-04-26T20:48:16.742 INFO:teuthology.orchestra.run.smithi120.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-04-26T20:48:16.868 INFO:teuthology.orchestra.run.smithi120.stdout:(68/152): openblas-openmp-0.3.21-2.el9.x86_64.r 15 MB/s | 4.7 MB 00:00 2024-04-26T20:48:16.910 INFO:teuthology.orchestra.run.smithi120.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 5.3 MB/s | 911 kB 00:00 2024-04-26T20:48:16.935 INFO:teuthology.orchestra.run.smithi120.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.9 MB/s | 133 kB 00:00 2024-04-26T20:48:17.035 INFO:teuthology.orchestra.run.smithi120.stdout:(71/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-04-26T20:48:17.077 INFO:teuthology.orchestra.run.smithi120.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 7.1 MB/s | 1.0 MB 00:00 2024-04-26T20:48:17.103 INFO:teuthology.orchestra.run.smithi120.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-04-26T20:48:17.161 INFO:teuthology.orchestra.run.smithi120.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 4.1 MB/s | 244 kB 00:00 2024-04-26T20:48:17.228 INFO:teuthology.orchestra.run.smithi120.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 3.6 MB/s | 248 kB 00:00 2024-04-26T20:48:17.287 INFO:teuthology.orchestra.run.smithi120.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 438 kB/s | 26 kB 00:00 2024-04-26T20:48:17.354 INFO:teuthology.orchestra.run.smithi120.stdout:(77/152): postgresql-server-13.14-1.el9.x86_64. 13 MB/s | 5.8 MB 00:00 2024-04-26T20:48:17.380 INFO:teuthology.orchestra.run.smithi120.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 208 kB/s | 19 kB 00:00 2024-04-26T20:48:17.455 INFO:teuthology.orchestra.run.smithi120.stdout:(79/152): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-04-26T20:48:17.480 INFO:teuthology.orchestra.run.smithi120.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 347 kB/s | 35 kB 00:00 2024-04-26T20:48:17.505 INFO:teuthology.orchestra.run.smithi120.stdout:(81/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-04-26T20:48:17.539 INFO:teuthology.orchestra.run.smithi120.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-04-26T20:48:17.572 INFO:teuthology.orchestra.run.smithi120.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.3 MB/s | 222 kB 00:00 2024-04-26T20:48:17.598 INFO:teuthology.orchestra.run.smithi120.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-04-26T20:48:17.623 INFO:teuthology.orchestra.run.smithi120.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 839 kB/s | 42 kB 00:00 2024-04-26T20:48:17.665 INFO:teuthology.orchestra.run.smithi120.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 3.9 MB/s | 269 kB 00:00 2024-04-26T20:48:17.691 INFO:teuthology.orchestra.run.smithi120.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-04-26T20:48:17.766 INFO:teuthology.orchestra.run.smithi120.stdout:(88/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 16 MB/s | 5.1 MB 00:00 2024-04-26T20:48:17.783 INFO:teuthology.orchestra.run.smithi120.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 555 kB/s | 51 kB 00:00 2024-04-26T20:48:17.808 INFO:teuthology.orchestra.run.smithi120.stdout:(90/152): python3-pyasn1-modules-0.4.8-6.el9.no 1.9 MB/s | 279 kB 00:00 2024-04-26T20:48:17.825 INFO:teuthology.orchestra.run.smithi120.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 912 kB/s | 54 kB 00:00 2024-04-26T20:48:17.858 INFO:teuthology.orchestra.run.smithi120.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 830 kB/s | 42 kB 00:00 2024-04-26T20:48:17.876 INFO:teuthology.orchestra.run.smithi120.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 876 kB/s | 44 kB 00:00 2024-04-26T20:48:17.918 INFO:teuthology.orchestra.run.smithi120.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-04-26T20:48:17.943 INFO:teuthology.orchestra.run.smithi120.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 536 kB/s | 36 kB 00:00 2024-04-26T20:48:17.985 INFO:teuthology.orchestra.run.smithi120.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 709 kB/s | 47 kB 00:00 2024-04-26T20:48:18.018 INFO:teuthology.orchestra.run.smithi120.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 4.0 MB/s | 305 kB 00:00 2024-04-26T20:48:18.044 INFO:teuthology.orchestra.run.smithi120.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 986 kB/s | 58 kB 00:00 2024-04-26T20:48:18.077 INFO:teuthology.orchestra.run.smithi120.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-04-26T20:48:18.102 INFO:teuthology.orchestra.run.smithi120.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-26T20:48:18.136 INFO:teuthology.orchestra.run.smithi120.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 764 kB/s | 45 kB 00:00 2024-04-26T20:48:18.445 INFO:teuthology.orchestra.run.smithi120.stdout:(102/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 25 MB/s | 16 MB 00:00 2024-04-26T20:48:18.795 INFO:teuthology.orchestra.run.smithi120.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 32 kB/s | 22 kB 00:00 2024-04-26T20:48:19.237 INFO:teuthology.orchestra.run.smithi120.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 738 kB/s | 548 kB 00:00 2024-04-26T20:48:19.346 INFO:teuthology.orchestra.run.smithi120.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 179 kB/s | 19 kB 00:00 2024-04-26T20:48:19.455 INFO:teuthology.orchestra.run.smithi120.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 469 kB/s | 309 kB 00:00 2024-04-26T20:48:19.563 INFO:teuthology.orchestra.run.smithi120.stdout:(107/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 240 kB/s | 26 kB 00:00 2024-04-26T20:48:19.672 INFO:teuthology.orchestra.run.smithi120.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 448 kB/s | 49 kB 00:00 2024-04-26T20:48:19.772 INFO:teuthology.orchestra.run.smithi120.stdout:(109/152): libarrow-9.0.0-11.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-04-26T20:48:19.806 INFO:teuthology.orchestra.run.smithi120.stdout:(110/152): protobuf-compiler-3.14.0-13.el9.x86_ 517 kB/s | 863 kB 00:01 2024-04-26T20:48:19.831 INFO:teuthology.orchestra.run.smithi120.stdout:(111/152): libunwind-1.6.2-1.el9.x86_64.rpm 424 kB/s | 67 kB 00:00 2024-04-26T20:48:19.881 INFO:teuthology.orchestra.run.smithi120.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-04-26T20:48:19.990 INFO:teuthology.orchestra.run.smithi120.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 401 kB/s | 43 kB 00:00 2024-04-26T20:48:20.074 INFO:teuthology.orchestra.run.smithi120.stdout:(114/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.1 MB/s | 839 kB 00:00 2024-04-26T20:48:20.099 INFO:teuthology.orchestra.run.smithi120.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 296 kB/s | 32 kB 00:00 2024-04-26T20:48:20.182 INFO:teuthology.orchestra.run.smithi120.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 138 kB/s | 15 kB 00:00 2024-04-26T20:48:20.208 INFO:teuthology.orchestra.run.smithi120.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.5 MB/s | 172 kB 00:00 2024-04-26T20:48:20.308 INFO:teuthology.orchestra.run.smithi120.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-04-26T20:48:20.333 INFO:teuthology.orchestra.run.smithi120.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.4 MB/s | 303 kB 00:00 2024-04-26T20:48:20.417 INFO:teuthology.orchestra.run.smithi120.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 326 kB/s | 35 kB 00:00 2024-04-26T20:48:20.442 INFO:teuthology.orchestra.run.smithi120.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 1.8 MB/s | 206 kB 00:00 2024-04-26T20:48:20.511 INFO:teuthology.orchestra.run.smithi073.stdout: Preparing : 1/1 2024-04-26T20:48:20.568 INFO:teuthology.orchestra.run.smithi120.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-04-26T20:48:20.593 INFO:teuthology.orchestra.run.smithi120.stdout:(123/152): python3-asyncssh-2.13.2-5.el9.noarch 720 kB/s | 548 kB 00:00 2024-04-26T20:48:20.617 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T20:48:20.655 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T20:48:20.668 INFO:teuthology.orchestra.run.smithi120.stdout:(124/152): python3-grpcio-1.46.7-10.el9.x86_64. 8.1 MB/s | 2.0 MB 00:00 2024-04-26T20:48:20.685 INFO:teuthology.orchestra.run.smithi120.stdout:(125/152): python3-jaraco-8.2.1-3.el9.noarch.rp 91 kB/s | 11 kB 00:00 2024-04-26T20:48:20.709 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T20:48:20.710 INFO:teuthology.orchestra.run.smithi120.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 151 kB/s | 18 kB 00:00 2024-04-26T20:48:20.743 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T20:48:20.769 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T20:48:20.777 INFO:teuthology.orchestra.run.smithi120.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 213 kB/s | 23 kB 00:00 2024-04-26T20:48:20.803 INFO:teuthology.orchestra.run.smithi120.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 166 kB/s | 19 kB 00:00 2024-04-26T20:48:20.810 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T20:48:20.828 INFO:teuthology.orchestra.run.smithi120.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 168 kB/s | 20 kB 00:00 2024-04-26T20:48:20.904 INFO:teuthology.orchestra.run.smithi120.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 8.1 MB/s | 1.0 MB 00:00 2024-04-26T20:48:20.929 INFO:teuthology.orchestra.run.smithi120.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 368 kB/s | 46 kB 00:00 2024-04-26T20:48:20.962 INFO:teuthology.orchestra.run.smithi120.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 587 kB/s | 79 kB 00:00 2024-04-26T20:48:21.021 INFO:teuthology.orchestra.run.smithi120.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 492 kB/s | 58 kB 00:00 2024-04-26T20:48:21.046 INFO:teuthology.orchestra.run.smithi120.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-04-26T20:48:21.072 INFO:teuthology.orchestra.run.smithi120.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 150 kB/s | 16 kB 00:00 2024-04-26T20:48:21.130 INFO:teuthology.orchestra.run.smithi120.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 825 kB/s | 90 kB 00:00 2024-04-26T20:48:21.138 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T20:48:21.155 INFO:teuthology.orchestra.run.smithi120.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 259 kB/s | 28 kB 00:00 2024-04-26T20:48:21.181 INFO:teuthology.orchestra.run.smithi120.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 283 kB/s | 31 kB 00:00 2024-04-26T20:48:21.239 INFO:teuthology.orchestra.run.smithi120.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-04-26T20:48:21.253 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T20:48:21.265 INFO:teuthology.orchestra.run.smithi120.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 542 kB/s | 59 kB 00:00 2024-04-26T20:48:21.290 INFO:teuthology.orchestra.run.smithi120.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 328 kB/s | 36 kB 00:00 2024-04-26T20:48:21.349 INFO:teuthology.orchestra.run.smithi120.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 468 kB/s | 51 kB 00:00 2024-04-26T20:48:21.374 INFO:teuthology.orchestra.run.smithi120.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-04-26T20:48:21.374 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T20:48:21.399 INFO:teuthology.orchestra.run.smithi120.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 823 kB/s | 90 kB 00:00 2024-04-26T20:48:21.403 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T20:48:21.458 INFO:teuthology.orchestra.run.smithi120.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-04-26T20:48:21.483 INFO:teuthology.orchestra.run.smithi120.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 183 kB/s | 20 kB 00:00 2024-04-26T20:48:21.498 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T20:48:21.525 INFO:teuthology.orchestra.run.smithi120.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.5 MB/s | 192 kB 00:00 2024-04-26T20:48:21.533 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T20:48:21.592 INFO:teuthology.orchestra.run.smithi120.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-04-26T20:48:21.681 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T20:48:21.738 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T20:48:21.766 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T20:48:21.834 INFO:teuthology.orchestra.run.smithi120.stdout:(149/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 10 MB/s | 3.1 MB 00:00 2024-04-26T20:48:21.869 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T20:48:21.900 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T20:48:21.926 INFO:teuthology.orchestra.run.smithi120.stdout:(150/152): librados2-19.0.0-3306.gbc612ffb.el9. 7.7 MB/s | 3.4 MB 00:00 2024-04-26T20:48:21.946 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T20:48:21.968 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T20:48:21.987 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T20:48:22.026 INFO:teuthology.orchestra.run.smithi120.stdout:(151/152): openssl-3.2.1-1.el9.x86_64.rpm 3.1 MB/s | 1.3 MB 00:00 2024-04-26T20:48:22.142 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T20:48:22.176 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T20:48:22.284 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T20:48:22.335 INFO:teuthology.orchestra.run.smithi120.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 4.8 MB/s | 2.4 MB 00:00 2024-04-26T20:48:22.341 INFO:teuthology.orchestra.run.smithi120.stdout:-------------------------------------------------------------------------------- 2024-04-26T20:48:22.341 INFO:teuthology.orchestra.run.smithi120.stdout:Total 16 MB/s | 222 MB 00:13 2024-04-26T20:48:22.386 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T20:48:22.425 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T20:48:22.460 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T20:48:22.533 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T20:48:22.567 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T20:48:22.611 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T20:48:22.643 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T20:48:22.672 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T20:48:22.741 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T20:48:22.794 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T20:48:22.824 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T20:48:22.868 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T20:48:22.945 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T20:48:22.987 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-26T20:48:23.030 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-26T20:48:23.059 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-26T20:48:23.118 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T20:48:23.173 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T20:48:23.247 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T20:48:23.379 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T20:48:23.424 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction check 2024-04-26T20:48:23.433 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T20:48:23.567 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction check succeeded. 2024-04-26T20:48:23.567 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction test 2024-04-26T20:48:23.951 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T20:48:24.029 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T20:48:24.058 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T20:48:24.101 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T20:48:24.147 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-26T20:48:24.442 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T20:48:24.645 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T20:48:24.701 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T20:48:24.756 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T20:48:24.828 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T20:48:24.859 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T20:48:24.896 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T20:48:24.942 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T20:48:24.981 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T20:48:25.021 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T20:48:25.063 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T20:48:25.199 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T20:48:25.253 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T20:48:25.283 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T20:48:25.324 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T20:48:25.344 INFO:teuthology.orchestra.run.smithi120.stdout:Transaction test succeeded. 2024-04-26T20:48:25.344 INFO:teuthology.orchestra.run.smithi120.stdout:Running transaction 2024-04-26T20:48:25.418 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T20:48:25.466 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T20:48:25.506 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T20:48:25.532 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T20:48:25.556 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T20:48:25.631 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T20:48:25.758 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T20:48:25.797 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T20:48:25.825 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T20:48:25.859 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T20:48:25.898 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T20:48:25.930 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T20:48:25.960 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T20:48:25.994 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T20:48:26.028 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T20:48:26.069 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T20:48:26.104 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T20:48:26.190 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T20:48:26.234 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T20:48:26.265 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T20:48:26.312 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T20:48:26.411 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T20:48:26.459 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T20:48:27.182 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T20:48:27.230 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T20:48:27.264 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T20:48:27.362 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T20:48:27.413 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T20:48:27.500 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T20:48:27.558 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T20:48:27.735 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T20:48:27.764 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T20:48:27.842 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T20:48:28.484 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T20:48:28.586 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T20:48:29.800 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T20:48:29.910 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T20:48:29.938 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T20:48:29.971 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T20:48:30.286 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T20:48:30.325 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T20:48:30.416 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T20:48:30.447 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-26T20:48:30.483 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T20:48:30.964 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T20:48:30.995 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T20:48:31.066 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T20:48:31.097 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-26T20:48:31.469 INFO:teuthology.orchestra.run.smithi120.stdout: Preparing : 1/1 2024-04-26T20:48:31.556 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T20:48:31.594 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T20:48:31.648 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T20:48:31.682 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T20:48:31.707 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T20:48:31.750 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T20:48:32.086 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T20:48:32.192 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T20:48:32.314 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T20:48:32.343 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T20:48:32.434 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T20:48:32.472 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T20:48:32.622 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T20:48:32.686 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T20:48:32.714 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T20:48:32.822 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T20:48:33.017 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T20:48:33.103 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T20:48:33.125 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T20:48:33.144 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T20:48:33.325 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T20:48:33.358 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T20:48:33.424 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T20:48:33.466 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T20:48:33.562 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T20:48:33.599 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T20:48:33.634 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T20:48:33.660 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T20:48:33.699 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T20:48:33.733 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T20:48:33.777 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T20:48:33.809 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T20:48:33.838 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T20:48:33.906 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T20:48:33.967 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T20:48:33.999 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T20:48:34.034 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T20:48:34.110 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T20:48:34.145 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-26T20:48:34.189 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-26T20:48:34.202 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T20:48:34.218 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-26T20:48:34.246 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T20:48:34.287 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T20:48:34.322 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T20:48:34.323 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T20:48:34.323 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T20:48:34.323 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:48:34.341 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T20:48:34.356 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T20:48:34.415 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T20:48:34.558 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T20:48:34.610 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T20:48:35.141 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T20:48:35.223 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T20:48:35.252 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T20:48:35.286 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T20:48:35.324 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-26T20:48:35.366 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T20:48:35.525 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T20:48:35.595 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T20:48:35.651 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T20:48:35.724 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T20:48:35.754 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T20:48:35.791 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T20:48:35.837 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T20:48:35.874 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T20:48:35.926 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T20:48:35.976 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T20:48:36.114 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T20:48:36.165 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T20:48:36.195 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T20:48:36.237 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T20:48:36.333 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T20:48:36.362 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T20:48:36.402 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T20:48:36.428 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T20:48:36.453 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T20:48:36.528 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T20:48:36.656 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T20:48:36.711 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T20:48:36.739 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T20:48:36.773 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T20:48:36.812 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T20:48:36.844 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T20:48:36.874 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T20:48:36.909 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T20:48:37.192 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T20:48:37.286 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T20:48:37.319 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T20:48:37.410 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T20:48:37.458 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T20:48:37.489 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T20:48:37.537 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T20:48:37.663 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T20:48:37.732 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T20:48:38.464 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T20:48:38.513 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T20:48:38.547 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T20:48:38.649 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T20:48:38.706 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T20:48:38.794 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T20:48:38.833 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T20:48:39.009 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T20:48:39.039 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T20:48:39.117 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T20:48:39.766 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T20:48:39.889 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T20:48:41.127 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T20:48:41.204 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T20:48:41.232 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T20:48:41.265 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T20:48:41.576 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T20:48:41.611 INFO:teuthology.orchestra.run.smithi120.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T20:48:41.702 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T20:48:41.733 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-26T20:48:41.769 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T20:48:42.248 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T20:48:42.273 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T20:48:42.344 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T20:48:42.366 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-26T20:48:44.608 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T20:48:44.638 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T20:48:45.175 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T20:48:45.226 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T20:48:45.309 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T20:48:45.310 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T20:48:45.310 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T20:48:45.310 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T20:48:45.343 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T20:48:45.881 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T20:48:45.881 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /sys 2024-04-26T20:48:45.881 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /proc 2024-04-26T20:48:45.881 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /mnt 2024-04-26T20:48:45.882 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /var/tmp 2024-04-26T20:48:45.882 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /home 2024-04-26T20:48:45.882 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /root 2024-04-26T20:48:45.882 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /tmp 2024-04-26T20:48:45.882 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:48:46.051 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T20:48:46.067 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T20:48:46.067 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:48:46.067 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T20:48:46.067 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:48:47.538 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T20:48:47.563 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T20:48:47.563 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:48:47.563 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T20:48:47.563 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T20:48:47.564 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T20:48:47.564 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:48:47.797 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T20:48:47.819 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T20:48:47.819 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:48:47.819 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T20:48:47.819 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T20:48:47.819 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T20:48:47.819 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:48:48.264 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T20:48:48.286 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T20:48:48.287 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:48:48.287 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T20:48:48.287 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T20:48:48.287 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T20:48:48.287 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:48:48.393 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T20:48:48.442 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T20:48:48.497 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T20:48:48.681 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T20:48:48.731 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T20:48:48.808 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T20:48:48.896 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T20:48:49.028 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T20:48:49.073 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T20:48:49.105 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T20:48:49.132 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T20:48:49.551 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T20:48:49.581 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T20:48:49.605 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-26T20:48:49.666 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-26T20:48:49.696 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T20:48:50.733 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T20:48:50.776 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T20:48:51.738 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T20:48:51.764 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T20:48:51.865 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T20:48:52.026 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-26T20:48:52.066 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T20:48:52.089 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T20:48:52.090 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:48:52.090 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T20:48:52.090 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T20:48:52.090 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T20:48:52.090 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:48:52.142 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T20:48:52.153 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T20:48:52.251 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-26T20:48:52.298 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-26T20:48:53.222 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T20:48:53.253 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T20:48:53.276 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T20:48:53.277 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:48:53.277 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T20:48:53.277 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T20:48:53.277 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T20:48:53.277 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:48:53.320 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T20:48:53.342 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T20:48:53.342 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:48:53.342 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T20:48:53.343 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:48:53.388 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T20:48:53.410 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T20:48:53.410 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:48:53.410 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T20:48:53.410 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T20:48:53.410 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T20:48:53.411 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:48:53.713 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T20:48:53.736 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T20:48:53.736 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:48:53.736 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T20:48:53.736 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T20:48:53.736 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T20:48:53.736 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:48:56.799 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T20:48:56.799 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /sys 2024-04-26T20:48:56.799 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /proc 2024-04-26T20:48:56.799 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /mnt 2024-04-26T20:48:56.799 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /var/tmp 2024-04-26T20:48:56.799 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /home 2024-04-26T20:48:56.799 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /root 2024-04-26T20:48:56.799 INFO:teuthology.orchestra.run.smithi120.stdout:skipping the directory /tmp 2024-04-26T20:48:56.799 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T20:48:56.968 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T20:48:56.983 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T20:48:56.983 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:48:56.983 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T20:48:56.983 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T20:48:58.333 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-26T20:48:58.374 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-26T20:48:58.416 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-26T20:48:58.448 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T20:48:58.475 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T20:48:58.487 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T20:48:58.512 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T20:48:58.512 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:48:58.512 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T20:48:58.512 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T20:48:58.512 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T20:48:58.513 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T20:48:58.656 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-26T20:48:58.696 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T20:48:58.726 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-26T20:48:58.739 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T20:48:58.765 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T20:48:58.765 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:48:58.765 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T20:48:58.765 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T20:48:58.765 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T20:48:58.765 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T20:48:59.208 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T20:48:59.230 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T20:48:59.230 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:48:59.230 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T20:48:59.231 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T20:48:59.231 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T20:48:59.231 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T20:48:59.329 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T20:48:59.350 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T20:48:59.378 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T20:48:59.390 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T20:48:59.410 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T20:48:59.434 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T20:48:59.620 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T20:48:59.676 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T20:48:59.746 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T20:48:59.834 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T20:48:59.975 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T20:49:00.018 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T20:49:00.049 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T20:49:00.077 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T20:49:00.232 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-26T20:49:00.319 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T20:49:00.360 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T20:49:00.442 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T20:49:00.475 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T20:49:00.475 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T20:49:00.529 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T20:49:00.560 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T20:49:00.566 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T20:49:00.592 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-26T20:49:00.599 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T20:49:00.599 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T20:49:00.662 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-26T20:49:00.688 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T20:49:00.691 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T20:49:00.755 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T20:49:01.734 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T20:49:01.768 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T20:49:02.730 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T20:49:02.768 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T20:49:02.871 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T20:49:03.030 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-26T20:49:03.055 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T20:49:03.077 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T20:49:03.078 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:49:03.078 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T20:49:03.078 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T20:49:03.078 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T20:49:03.078 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T20:49:03.122 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T20:49:03.133 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T20:49:03.231 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-26T20:49:03.287 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-26T20:49:04.218 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T20:49:04.250 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T20:49:04.272 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T20:49:04.273 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:49:04.273 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T20:49:04.273 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T20:49:04.273 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T20:49:04.273 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T20:49:04.317 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T20:49:04.338 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T20:49:04.338 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:49:04.339 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T20:49:04.339 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T20:49:04.384 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T20:49:04.406 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T20:49:04.407 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:49:04.407 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T20:49:04.407 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T20:49:04.407 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T20:49:04.407 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T20:49:04.710 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T20:49:04.733 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T20:49:04.733 INFO:teuthology.orchestra.run.smithi120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:49:04.733 INFO:teuthology.orchestra.run.smithi120.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T20:49:04.733 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T20:49:04.734 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T20:49:04.734 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T20:49:06.702 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T20:49:06.702 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-26T20:49:06.702 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-26T20:49:06.703 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-26T20:49:06.703 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-26T20:49:06.703 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-26T20:49:06.703 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-26T20:49:06.703 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-26T20:49:06.703 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T20:49:06.703 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T20:49:06.703 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T20:49:06.703 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-26T20:49:06.703 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-26T20:49:06.703 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T20:49:06.703 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-26T20:49:06.703 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-26T20:49:06.703 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-26T20:49:06.704 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-26T20:49:06.704 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-26T20:49:06.704 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-26T20:49:06.704 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-26T20:49:06.704 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-26T20:49:06.704 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-26T20:49:06.704 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-26T20:49:06.704 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-26T20:49:06.704 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-26T20:49:06.704 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-26T20:49:06.704 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-26T20:49:06.704 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-26T20:49:06.704 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-26T20:49:06.704 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-26T20:49:06.704 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-26T20:49:06.704 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-26T20:49:06.705 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-26T20:49:06.705 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-26T20:49:06.705 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-26T20:49:06.705 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-26T20:49:06.705 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-26T20:49:06.705 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-26T20:49:06.705 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-26T20:49:06.705 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T20:49:06.705 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T20:49:06.705 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T20:49:06.705 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T20:49:06.705 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T20:49:06.705 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T20:49:06.705 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T20:49:06.705 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T20:49:06.705 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T20:49:06.706 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T20:49:06.706 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T20:49:06.706 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T20:49:06.706 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T20:49:06.706 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T20:49:06.706 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T20:49:06.706 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T20:49:06.706 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T20:49:06.706 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T20:49:06.706 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T20:49:06.706 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T20:49:06.706 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T20:49:06.706 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T20:49:06.706 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T20:49:06.706 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T20:49:06.707 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T20:49:06.707 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T20:49:06.707 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T20:49:06.707 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T20:49:06.707 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T20:49:06.707 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T20:49:06.707 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T20:49:06.707 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T20:49:06.707 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T20:49:06.707 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T20:49:06.707 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T20:49:06.707 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T20:49:06.707 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T20:49:06.707 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T20:49:06.707 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T20:49:06.708 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T20:49:06.708 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T20:49:06.708 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T20:49:06.708 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T20:49:06.708 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T20:49:06.708 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T20:49:06.708 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T20:49:06.708 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T20:49:06.708 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T20:49:06.708 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T20:49:06.708 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T20:49:06.708 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T20:49:06.708 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T20:49:06.708 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T20:49:06.708 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T20:49:06.708 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T20:49:06.709 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T20:49:06.709 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T20:49:06.709 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T20:49:06.709 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T20:49:06.709 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T20:49:06.709 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T20:49:06.709 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T20:49:06.709 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T20:49:06.709 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T20:49:06.709 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T20:49:06.709 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T20:49:06.709 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T20:49:06.709 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T20:49:06.709 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T20:49:06.709 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T20:49:06.710 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T20:49:06.710 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T20:49:06.710 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T20:49:06.710 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T20:49:06.710 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T20:49:06.710 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T20:49:06.710 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T20:49:06.710 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T20:49:06.710 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T20:49:06.710 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T20:49:06.710 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T20:49:06.710 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T20:49:06.710 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T20:49:06.710 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T20:49:06.710 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T20:49:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T20:49:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T20:49:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T20:49:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T20:49:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T20:49:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T20:49:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T20:49:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T20:49:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T20:49:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T20:49:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T20:49:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T20:49:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T20:49:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T20:49:06.712 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T20:49:06.712 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T20:49:06.712 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T20:49:06.712 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T20:49:06.712 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T20:49:06.712 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T20:49:06.712 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T20:49:06.712 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T20:49:06.712 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T20:49:06.712 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T20:49:06.712 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-26T20:49:06.712 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T20:49:06.712 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-26T20:49:06.712 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T20:49:06.712 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T20:49:06.712 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T20:49:06.713 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T20:49:08.802 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T20:49:08.803 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:49:08.803 INFO:teuthology.orchestra.run.smithi073.stdout:Upgraded: 2024-04-26T20:49:08.803 INFO:teuthology.orchestra.run.smithi073.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:08.803 INFO:teuthology.orchestra.run.smithi073.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:08.803 INFO:teuthology.orchestra.run.smithi073.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T20:49:08.803 INFO:teuthology.orchestra.run.smithi073.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T20:49:08.804 INFO:teuthology.orchestra.run.smithi073.stdout:Installed: 2024-04-26T20:49:08.804 INFO:teuthology.orchestra.run.smithi073.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T20:49:08.804 INFO:teuthology.orchestra.run.smithi073.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T20:49:08.804 INFO:teuthology.orchestra.run.smithi073.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T20:49:08.804 INFO:teuthology.orchestra.run.smithi073.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T20:49:08.804 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:08.804 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:08.805 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:08.805 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:08.805 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:49:08.805 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:08.805 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:08.805 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:08.805 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:49:08.805 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:49:08.805 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:49:08.805 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:49:08.805 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:49:08.806 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:08.806 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:08.806 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:49:08.806 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:08.806 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:08.806 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:08.807 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:49:08.807 INFO:teuthology.orchestra.run.smithi073.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:49:08.807 INFO:teuthology.orchestra.run.smithi073.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:08.807 INFO:teuthology.orchestra.run.smithi073.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:08.807 INFO:teuthology.orchestra.run.smithi073.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:49:08.807 INFO:teuthology.orchestra.run.smithi073.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T20:49:08.807 INFO:teuthology.orchestra.run.smithi073.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T20:49:08.807 INFO:teuthology.orchestra.run.smithi073.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T20:49:08.807 INFO:teuthology.orchestra.run.smithi073.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T20:49:08.807 INFO:teuthology.orchestra.run.smithi073.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T20:49:08.807 INFO:teuthology.orchestra.run.smithi073.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T20:49:08.808 INFO:teuthology.orchestra.run.smithi073.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T20:49:08.808 INFO:teuthology.orchestra.run.smithi073.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T20:49:08.808 INFO:teuthology.orchestra.run.smithi073.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T20:49:08.808 INFO:teuthology.orchestra.run.smithi073.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T20:49:08.808 INFO:teuthology.orchestra.run.smithi073.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T20:49:08.808 INFO:teuthology.orchestra.run.smithi073.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:08.808 INFO:teuthology.orchestra.run.smithi073.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:08.808 INFO:teuthology.orchestra.run.smithi073.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:08.808 INFO:teuthology.orchestra.run.smithi073.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T20:49:08.808 INFO:teuthology.orchestra.run.smithi073.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T20:49:08.808 INFO:teuthology.orchestra.run.smithi073.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T20:49:08.808 INFO:teuthology.orchestra.run.smithi073.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T20:49:08.808 INFO:teuthology.orchestra.run.smithi073.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T20:49:08.808 INFO:teuthology.orchestra.run.smithi073.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:08.809 INFO:teuthology.orchestra.run.smithi073.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:08.809 INFO:teuthology.orchestra.run.smithi073.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T20:49:08.809 INFO:teuthology.orchestra.run.smithi073.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:08.809 INFO:teuthology.orchestra.run.smithi073.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T20:49:08.809 INFO:teuthology.orchestra.run.smithi073.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T20:49:08.809 INFO:teuthology.orchestra.run.smithi073.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T20:49:08.809 INFO:teuthology.orchestra.run.smithi073.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T20:49:08.809 INFO:teuthology.orchestra.run.smithi073.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T20:49:08.809 INFO:teuthology.orchestra.run.smithi073.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T20:49:08.809 INFO:teuthology.orchestra.run.smithi073.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T20:49:08.809 INFO:teuthology.orchestra.run.smithi073.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T20:49:08.809 INFO:teuthology.orchestra.run.smithi073.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T20:49:08.809 INFO:teuthology.orchestra.run.smithi073.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T20:49:08.809 INFO:teuthology.orchestra.run.smithi073.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T20:49:08.809 INFO:teuthology.orchestra.run.smithi073.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T20:49:08.810 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T20:49:08.810 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T20:49:08.810 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T20:49:08.810 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T20:49:08.810 INFO:teuthology.orchestra.run.smithi073.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T20:49:08.810 INFO:teuthology.orchestra.run.smithi073.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T20:49:08.810 INFO:teuthology.orchestra.run.smithi073.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T20:49:08.810 INFO:teuthology.orchestra.run.smithi073.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T20:49:08.810 INFO:teuthology.orchestra.run.smithi073.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T20:49:08.810 INFO:teuthology.orchestra.run.smithi073.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T20:49:08.810 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T20:49:08.810 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:08.811 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:08.811 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:08.811 INFO:teuthology.orchestra.run.smithi073.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T20:49:08.811 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T20:49:08.811 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T20:49:08.811 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T20:49:08.811 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T20:49:08.811 INFO:teuthology.orchestra.run.smithi073.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T20:49:08.811 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T20:49:08.811 INFO:teuthology.orchestra.run.smithi073.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T20:49:08.811 INFO:teuthology.orchestra.run.smithi073.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T20:49:08.811 INFO:teuthology.orchestra.run.smithi073.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T20:49:08.811 INFO:teuthology.orchestra.run.smithi073.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T20:49:08.811 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T20:49:08.811 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T20:49:08.812 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T20:49:08.812 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T20:49:08.812 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T20:49:08.812 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T20:49:08.812 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T20:49:08.812 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T20:49:08.812 INFO:teuthology.orchestra.run.smithi073.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T20:49:08.812 INFO:teuthology.orchestra.run.smithi073.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T20:49:08.812 INFO:teuthology.orchestra.run.smithi073.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T20:49:08.813 INFO:teuthology.orchestra.run.smithi073.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T20:49:08.813 INFO:teuthology.orchestra.run.smithi073.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T20:49:08.813 INFO:teuthology.orchestra.run.smithi073.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T20:49:08.813 INFO:teuthology.orchestra.run.smithi073.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T20:49:08.813 INFO:teuthology.orchestra.run.smithi073.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T20:49:08.813 INFO:teuthology.orchestra.run.smithi073.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T20:49:08.813 INFO:teuthology.orchestra.run.smithi073.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T20:49:08.813 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T20:49:08.813 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T20:49:08.813 INFO:teuthology.orchestra.run.smithi073.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T20:49:08.813 INFO:teuthology.orchestra.run.smithi073.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T20:49:08.813 INFO:teuthology.orchestra.run.smithi073.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T20:49:08.813 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T20:49:08.813 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T20:49:08.814 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T20:49:08.814 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T20:49:08.814 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T20:49:08.814 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T20:49:08.814 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:08.814 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:08.814 INFO:teuthology.orchestra.run.smithi073.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T20:49:08.814 INFO:teuthology.orchestra.run.smithi073.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T20:49:08.814 INFO:teuthology.orchestra.run.smithi073.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T20:49:08.814 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:08.814 INFO:teuthology.orchestra.run.smithi073.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T20:49:08.814 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T20:49:08.815 INFO:teuthology.orchestra.run.smithi073.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T20:49:08.815 INFO:teuthology.orchestra.run.smithi073.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T20:49:08.815 INFO:teuthology.orchestra.run.smithi073.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T20:49:08.815 INFO:teuthology.orchestra.run.smithi073.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T20:49:08.815 INFO:teuthology.orchestra.run.smithi073.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T20:49:08.815 INFO:teuthology.orchestra.run.smithi073.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T20:49:08.816 INFO:teuthology.orchestra.run.smithi073.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T20:49:08.816 INFO:teuthology.orchestra.run.smithi073.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T20:49:08.817 INFO:teuthology.orchestra.run.smithi073.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T20:49:08.817 INFO:teuthology.orchestra.run.smithi073.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T20:49:08.817 INFO:teuthology.orchestra.run.smithi073.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T20:49:08.817 INFO:teuthology.orchestra.run.smithi073.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T20:49:08.817 INFO:teuthology.orchestra.run.smithi073.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T20:49:08.817 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:08.817 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:08.817 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:08.817 INFO:teuthology.orchestra.run.smithi073.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T20:49:08.818 INFO:teuthology.orchestra.run.smithi073.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T20:49:08.818 INFO:teuthology.orchestra.run.smithi073.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T20:49:08.818 INFO:teuthology.orchestra.run.smithi073.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T20:49:08.818 INFO:teuthology.orchestra.run.smithi073.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T20:49:08.818 INFO:teuthology.orchestra.run.smithi073.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T20:49:08.818 INFO:teuthology.orchestra.run.smithi073.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T20:49:08.818 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:49:08.818 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-04-26T20:49:09.240 DEBUG:teuthology.parallel:result is None 2024-04-26T20:49:09.323 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-26T20:49:09.373 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-26T20:49:09.414 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-26T20:49:09.446 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T20:49:09.473 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T20:49:09.662 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-26T20:49:09.712 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T20:49:09.742 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-26T20:49:10.333 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T20:49:10.365 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T20:49:10.384 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T20:49:11.214 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-26T20:49:11.305 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T20:49:11.345 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T20:49:11.429 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T20:49:11.461 INFO:teuthology.orchestra.run.smithi120.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T20:49:11.461 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T20:49:11.543 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T20:49:11.576 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T20:49:11.576 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T20:49:11.665 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T20:49:11.734 INFO:teuthology.orchestra.run.smithi120.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T20:49:17.782 INFO:teuthology.orchestra.run.smithi120.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T20:49:17.782 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-26T20:49:17.782 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-26T20:49:17.783 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-26T20:49:17.783 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-26T20:49:17.783 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-26T20:49:17.783 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-26T20:49:17.783 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-26T20:49:17.783 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T20:49:17.783 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T20:49:17.783 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T20:49:17.783 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-26T20:49:17.783 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-26T20:49:17.783 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T20:49:17.783 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-26T20:49:17.783 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-26T20:49:17.784 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-26T20:49:17.784 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-26T20:49:17.784 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-26T20:49:17.784 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-26T20:49:17.784 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-26T20:49:17.784 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-26T20:49:17.784 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-26T20:49:17.784 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-26T20:49:17.784 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-26T20:49:17.784 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-26T20:49:17.784 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-26T20:49:17.784 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-26T20:49:17.784 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-26T20:49:17.784 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-26T20:49:17.785 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-26T20:49:17.785 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-26T20:49:17.785 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-26T20:49:17.785 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-26T20:49:17.785 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-26T20:49:17.785 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-26T20:49:17.785 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-26T20:49:17.785 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-26T20:49:17.785 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-26T20:49:17.785 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-26T20:49:17.785 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T20:49:17.785 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T20:49:17.785 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T20:49:17.786 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T20:49:17.786 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T20:49:17.786 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T20:49:17.786 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T20:49:17.786 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T20:49:17.786 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T20:49:17.786 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T20:49:17.786 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T20:49:17.786 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T20:49:17.786 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T20:49:17.786 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T20:49:17.786 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T20:49:17.786 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T20:49:17.787 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T20:49:17.787 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T20:49:17.787 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T20:49:17.787 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T20:49:17.787 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T20:49:17.787 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T20:49:17.787 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T20:49:17.787 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T20:49:17.787 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T20:49:17.787 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T20:49:17.787 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T20:49:17.787 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T20:49:17.787 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T20:49:17.788 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T20:49:17.788 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T20:49:17.788 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T20:49:17.788 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T20:49:17.788 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T20:49:17.788 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T20:49:17.788 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T20:49:17.788 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T20:49:17.788 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T20:49:17.788 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T20:49:17.788 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T20:49:17.788 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T20:49:17.788 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T20:49:17.788 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T20:49:17.788 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T20:49:17.789 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T20:49:17.789 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T20:49:17.789 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T20:49:17.789 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T20:49:17.789 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T20:49:17.789 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T20:49:17.789 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T20:49:17.789 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T20:49:17.789 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T20:49:17.789 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T20:49:17.789 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T20:49:17.789 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T20:49:17.789 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T20:49:17.789 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T20:49:17.790 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T20:49:17.790 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T20:49:17.790 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T20:49:17.790 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T20:49:17.790 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T20:49:17.790 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T20:49:17.790 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T20:49:17.790 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T20:49:17.790 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T20:49:17.790 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T20:49:17.790 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T20:49:17.790 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T20:49:17.790 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T20:49:17.791 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T20:49:17.791 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T20:49:17.791 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T20:49:17.791 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T20:49:17.791 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T20:49:17.791 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T20:49:17.791 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T20:49:17.791 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T20:49:17.791 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T20:49:17.792 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T20:49:17.792 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T20:49:17.792 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T20:49:17.792 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T20:49:17.792 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T20:49:17.792 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T20:49:17.792 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T20:49:17.792 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T20:49:17.792 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T20:49:17.792 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T20:49:17.792 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T20:49:17.792 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T20:49:17.792 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T20:49:17.792 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T20:49:17.792 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T20:49:17.792 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T20:49:17.793 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T20:49:17.793 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T20:49:17.793 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T20:49:17.793 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T20:49:17.793 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T20:49:17.793 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T20:49:17.793 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T20:49:17.793 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T20:49:17.793 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T20:49:17.793 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T20:49:17.793 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T20:49:17.793 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T20:49:17.793 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T20:49:17.793 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-26T20:49:17.794 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T20:49:17.794 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-26T20:49:17.794 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T20:49:17.794 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T20:49:17.794 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T20:49:17.794 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T20:49:19.817 INFO:teuthology.orchestra.run.smithi120.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T20:49:19.817 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T20:49:19.817 INFO:teuthology.orchestra.run.smithi120.stdout:Upgraded: 2024-04-26T20:49:19.818 INFO:teuthology.orchestra.run.smithi120.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:19.818 INFO:teuthology.orchestra.run.smithi120.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:19.818 INFO:teuthology.orchestra.run.smithi120.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T20:49:19.818 INFO:teuthology.orchestra.run.smithi120.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T20:49:19.818 INFO:teuthology.orchestra.run.smithi120.stdout:Installed: 2024-04-26T20:49:19.818 INFO:teuthology.orchestra.run.smithi120.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T20:49:19.818 INFO:teuthology.orchestra.run.smithi120.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T20:49:19.818 INFO:teuthology.orchestra.run.smithi120.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T20:49:19.818 INFO:teuthology.orchestra.run.smithi120.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T20:49:19.818 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:19.818 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:19.818 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:19.818 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:19.819 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:49:19.819 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:19.819 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:19.819 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:19.819 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:49:19.819 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:49:19.819 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:49:19.819 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:49:19.819 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:49:19.819 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:19.819 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:19.819 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:49:19.819 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:19.819 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:19.819 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:19.820 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:49:19.820 INFO:teuthology.orchestra.run.smithi120.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:49:19.820 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:19.820 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:19.820 INFO:teuthology.orchestra.run.smithi120.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:49:19.820 INFO:teuthology.orchestra.run.smithi120.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T20:49:19.820 INFO:teuthology.orchestra.run.smithi120.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T20:49:19.820 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T20:49:19.820 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T20:49:19.820 INFO:teuthology.orchestra.run.smithi120.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T20:49:19.820 INFO:teuthology.orchestra.run.smithi120.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T20:49:19.820 INFO:teuthology.orchestra.run.smithi120.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T20:49:19.820 INFO:teuthology.orchestra.run.smithi120.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T20:49:19.820 INFO:teuthology.orchestra.run.smithi120.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T20:49:19.820 INFO:teuthology.orchestra.run.smithi120.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T20:49:19.821 INFO:teuthology.orchestra.run.smithi120.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T20:49:19.821 INFO:teuthology.orchestra.run.smithi120.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:19.821 INFO:teuthology.orchestra.run.smithi120.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:19.821 INFO:teuthology.orchestra.run.smithi120.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:19.821 INFO:teuthology.orchestra.run.smithi120.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T20:49:19.821 INFO:teuthology.orchestra.run.smithi120.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T20:49:19.821 INFO:teuthology.orchestra.run.smithi120.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T20:49:19.821 INFO:teuthology.orchestra.run.smithi120.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T20:49:19.821 INFO:teuthology.orchestra.run.smithi120.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T20:49:19.821 INFO:teuthology.orchestra.run.smithi120.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:19.821 INFO:teuthology.orchestra.run.smithi120.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:19.821 INFO:teuthology.orchestra.run.smithi120.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T20:49:19.821 INFO:teuthology.orchestra.run.smithi120.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:19.821 INFO:teuthology.orchestra.run.smithi120.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T20:49:19.821 INFO:teuthology.orchestra.run.smithi120.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T20:49:19.821 INFO:teuthology.orchestra.run.smithi120.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T20:49:19.822 INFO:teuthology.orchestra.run.smithi120.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T20:49:19.822 INFO:teuthology.orchestra.run.smithi120.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T20:49:19.822 INFO:teuthology.orchestra.run.smithi120.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T20:49:19.822 INFO:teuthology.orchestra.run.smithi120.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T20:49:19.822 INFO:teuthology.orchestra.run.smithi120.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T20:49:19.822 INFO:teuthology.orchestra.run.smithi120.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T20:49:19.822 INFO:teuthology.orchestra.run.smithi120.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T20:49:19.822 INFO:teuthology.orchestra.run.smithi120.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T20:49:19.822 INFO:teuthology.orchestra.run.smithi120.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T20:49:19.822 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T20:49:19.822 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T20:49:19.822 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T20:49:19.822 INFO:teuthology.orchestra.run.smithi120.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T20:49:19.822 INFO:teuthology.orchestra.run.smithi120.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T20:49:19.823 INFO:teuthology.orchestra.run.smithi120.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T20:49:19.823 INFO:teuthology.orchestra.run.smithi120.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T20:49:19.823 INFO:teuthology.orchestra.run.smithi120.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T20:49:19.823 INFO:teuthology.orchestra.run.smithi120.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T20:49:19.823 INFO:teuthology.orchestra.run.smithi120.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T20:49:19.823 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T20:49:19.823 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:19.823 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:19.823 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:19.823 INFO:teuthology.orchestra.run.smithi120.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T20:49:19.823 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T20:49:19.823 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T20:49:19.823 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T20:49:19.823 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T20:49:19.823 INFO:teuthology.orchestra.run.smithi120.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T20:49:19.823 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T20:49:19.824 INFO:teuthology.orchestra.run.smithi120.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T20:49:19.824 INFO:teuthology.orchestra.run.smithi120.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T20:49:19.824 INFO:teuthology.orchestra.run.smithi120.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T20:49:19.824 INFO:teuthology.orchestra.run.smithi120.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T20:49:19.824 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T20:49:19.824 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T20:49:19.824 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T20:49:19.824 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T20:49:19.824 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T20:49:19.824 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T20:49:19.824 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T20:49:19.824 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T20:49:19.824 INFO:teuthology.orchestra.run.smithi120.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T20:49:19.824 INFO:teuthology.orchestra.run.smithi120.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T20:49:19.824 INFO:teuthology.orchestra.run.smithi120.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T20:49:19.825 INFO:teuthology.orchestra.run.smithi120.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T20:49:19.825 INFO:teuthology.orchestra.run.smithi120.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T20:49:19.825 INFO:teuthology.orchestra.run.smithi120.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T20:49:19.825 INFO:teuthology.orchestra.run.smithi120.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T20:49:19.825 INFO:teuthology.orchestra.run.smithi120.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T20:49:19.825 INFO:teuthology.orchestra.run.smithi120.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T20:49:19.825 INFO:teuthology.orchestra.run.smithi120.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T20:49:19.825 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T20:49:19.825 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T20:49:19.825 INFO:teuthology.orchestra.run.smithi120.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T20:49:19.825 INFO:teuthology.orchestra.run.smithi120.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T20:49:19.825 INFO:teuthology.orchestra.run.smithi120.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T20:49:19.825 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T20:49:19.826 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T20:49:19.826 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T20:49:19.826 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T20:49:19.826 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T20:49:19.826 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T20:49:19.826 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:19.826 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:19.826 INFO:teuthology.orchestra.run.smithi120.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T20:49:19.826 INFO:teuthology.orchestra.run.smithi120.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T20:49:19.826 INFO:teuthology.orchestra.run.smithi120.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T20:49:19.826 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:19.826 INFO:teuthology.orchestra.run.smithi120.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T20:49:19.826 INFO:teuthology.orchestra.run.smithi120.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T20:49:19.826 INFO:teuthology.orchestra.run.smithi120.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T20:49:19.827 INFO:teuthology.orchestra.run.smithi120.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T20:49:19.827 INFO:teuthology.orchestra.run.smithi120.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T20:49:19.827 INFO:teuthology.orchestra.run.smithi120.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T20:49:19.827 INFO:teuthology.orchestra.run.smithi120.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T20:49:19.827 INFO:teuthology.orchestra.run.smithi120.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T20:49:19.827 INFO:teuthology.orchestra.run.smithi120.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T20:49:19.827 INFO:teuthology.orchestra.run.smithi120.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T20:49:19.827 INFO:teuthology.orchestra.run.smithi120.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T20:49:19.827 INFO:teuthology.orchestra.run.smithi120.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T20:49:19.827 INFO:teuthology.orchestra.run.smithi120.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T20:49:19.827 INFO:teuthology.orchestra.run.smithi120.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T20:49:19.827 INFO:teuthology.orchestra.run.smithi120.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T20:49:19.827 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:19.827 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:19.827 INFO:teuthology.orchestra.run.smithi120.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:49:19.827 INFO:teuthology.orchestra.run.smithi120.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T20:49:19.828 INFO:teuthology.orchestra.run.smithi120.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T20:49:19.828 INFO:teuthology.orchestra.run.smithi120.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T20:49:19.828 INFO:teuthology.orchestra.run.smithi120.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T20:49:19.828 INFO:teuthology.orchestra.run.smithi120.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T20:49:19.828 INFO:teuthology.orchestra.run.smithi120.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T20:49:19.828 INFO:teuthology.orchestra.run.smithi120.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T20:49:19.828 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T20:49:19.828 INFO:teuthology.orchestra.run.smithi120.stdout:Complete! 2024-04-26T20:49:20.268 DEBUG:teuthology.parallel:result is None 2024-04-26T20:49:20.268 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T20:49:20.489 DEBUG:teuthology.orchestra.run.smithi073:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T20:49:20.514 INFO:teuthology.orchestra.run.smithi073.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-26T20:49:20.515 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-26T20:49:20.515 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-26T20:49:20.517 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T20:49:20.659 DEBUG:teuthology.orchestra.run.smithi120:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T20:49:20.680 INFO:teuthology.orchestra.run.smithi120.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-26T20:49:20.680 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-26T20:49:20.680 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-26T20:49:20.682 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-26T20:49:20.682 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-26T20:49:20.682 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T20:49:20.712 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-26T20:49:20.712 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T20:49:20.748 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-26T20:49:20.748 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-26T20:49:20.748 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T20:49:20.781 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T20:49:20.847 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-26T20:49:20.847 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T20:49:20.870 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T20:49:20.933 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-26T20:49:20.933 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-26T20:49:20.933 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T20:49:20.959 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T20:49:21.025 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-26T20:49:21.025 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T20:49:21.048 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T20:49:21.112 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-26T20:49:21.112 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-26T20:49:21.112 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T20:49:21.138 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T20:49:21.202 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-26T20:49:21.203 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T20:49:21.226 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T20:49:21.292 INFO:teuthology.run_tasks:Running task ceph... 2024-04-26T20:49:21.377 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-26T20:49:21.377 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 777 /var/log/ceph 2024-04-26T20:49:21.380 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod 777 /var/log/ceph 2024-04-26T20:49:21.403 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-26T20:49:21.404 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-26T20:49:21.444 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-26T20:49:21.468 INFO:tasks.ceph:Creating extra log directories... 2024-04-26T20:49:21.469 DEBUG:teuthology.orchestra.run.smithi073:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-26T20:49:21.508 DEBUG:teuthology.orchestra.run.smithi120:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-26T20:49:21.540 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-26T20:49:21.540 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-26T20:49:21.540 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-26T20:49:21.541 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-26T20:49:21.541 DEBUG:teuthology.orchestra.run.smithi073:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-26T20:49:21.584 DEBUG:teuthology.orchestra.run.smithi073:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-26T20:49:21.691 DEBUG:teuthology.orchestra.run.smithi073:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-26T20:49:21.731 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-26T20:49:21.731 DEBUG:teuthology.orchestra.run.smithi120:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-26T20:49:21.745 DEBUG:teuthology.orchestra.run.smithi120:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-26T20:49:21.853 DEBUG:teuthology.orchestra.run.smithi120:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-26T20:49:21.899 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-26T20:49:21.899 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'force file system read-only', 'bad backtrace', 'MDS in read-only mode', '\\(MDS_READ_ONLY\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-26T20:49:21.899 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675187', 'branch': 'wip-rishabh-testing-20240426.111959', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/auto-repair}', 'email': None, 'first_in_suite': False, 'job_id': '7675187', 'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-testing-20240426.111959'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'force file system read-only', 'bad backtrace', 'MDS in read-only mode', '\\(MDS_READ_ONLY\\)'], 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-testing-20240426.111959', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 816, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'sleep_before_teardown': 0, 'subset': '166/512', 'suite': 'fs', 'suite_branch': 'wip-rishabh-testing-20240426.111959', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi073.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPuP5xc655KyTC67uHuRxnDpDvFCni0MBRVVT7rV83pMbuacfY62UQuLfldLSMdgrbW6XfVZ9os4+Dw/sX7BmWs=', 'smithi120.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAInL6Cd1zlE5BV47vgZj307nYru1k87T263QM4ue067QJH8zg6FZQxg32GhaY25U32cUojUHfXAG7kOUdzMuuM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'force file system read-only', 'bad backtrace', 'MDS in read-only mode', '\\(MDS_READ_ONLY\\)'], 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_auto_repair']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-26_19:30:57', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-26T20:49:21.900 DEBUG:teuthology.orchestra.run.smithi073:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-26T20:49:21.902 DEBUG:teuthology.orchestra.run.smithi120:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-26T20:49:21.947 DEBUG:teuthology.orchestra.run.smithi073:> sudo install -d -m0777 -- /var/run/ceph 2024-04-26T20:49:21.958 DEBUG:teuthology.orchestra.run.smithi120:> sudo install -d -m0777 -- /var/run/ceph 2024-04-26T20:49:22.011 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-26T20:49:22.011 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T20:49:22.033 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T20:49:22.034 DEBUG:teuthology.orchestra.run.smithi073:> stat /dev/vg_nvme/lv_1 2024-04-26T20:49:22.088 INFO:teuthology.orchestra.run.smithi073.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T20:49:22.088 INFO:teuthology.orchestra.run.smithi073.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T20:49:22.088 INFO:teuthology.orchestra.run.smithi073.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-26T20:49:22.088 INFO:teuthology.orchestra.run.smithi073.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T20:49:22.089 INFO:teuthology.orchestra.run.smithi073.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T20:49:22.089 INFO:teuthology.orchestra.run.smithi073.stdout:Access: 2024-04-26 20:49:01.109819740 +0000 2024-04-26T20:49:22.089 INFO:teuthology.orchestra.run.smithi073.stdout:Modify: 2024-04-26 20:47:12.708919733 +0000 2024-04-26T20:49:22.089 INFO:teuthology.orchestra.run.smithi073.stdout:Change: 2024-04-26 20:47:12.708919733 +0000 2024-04-26T20:49:22.089 INFO:teuthology.orchestra.run.smithi073.stdout: Birth: 2024-04-26 20:47:12.708919733 +0000 2024-04-26T20:49:22.089 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T20:49:22.151 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2024-04-26T20:49:22.152 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2024-04-26T20:49:22.152 INFO:teuthology.orchestra.run.smithi073.stderr:512 bytes copied, 0.000243776 s, 2.1 MB/s 2024-04-26T20:49:22.153 DEBUG:teuthology.orchestra.run.smithi073:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T20:49:22.209 DEBUG:teuthology.orchestra.run.smithi073:> stat /dev/vg_nvme/lv_2 2024-04-26T20:49:22.264 INFO:teuthology.orchestra.run.smithi073.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T20:49:22.264 INFO:teuthology.orchestra.run.smithi073.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T20:49:22.264 INFO:teuthology.orchestra.run.smithi073.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-26T20:49:22.264 INFO:teuthology.orchestra.run.smithi073.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T20:49:22.264 INFO:teuthology.orchestra.run.smithi073.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T20:49:22.264 INFO:teuthology.orchestra.run.smithi073.stdout:Access: 2024-04-26 20:49:01.110819735 +0000 2024-04-26T20:49:22.264 INFO:teuthology.orchestra.run.smithi073.stdout:Modify: 2024-04-26 20:47:13.029918741 +0000 2024-04-26T20:49:22.264 INFO:teuthology.orchestra.run.smithi073.stdout:Change: 2024-04-26 20:47:13.029918741 +0000 2024-04-26T20:49:22.264 INFO:teuthology.orchestra.run.smithi073.stdout: Birth: 2024-04-26 20:47:13.029918741 +0000 2024-04-26T20:49:22.265 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T20:49:22.325 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2024-04-26T20:49:22.325 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2024-04-26T20:49:22.325 INFO:teuthology.orchestra.run.smithi073.stderr:512 bytes copied, 0.000193838 s, 2.6 MB/s 2024-04-26T20:49:22.326 DEBUG:teuthology.orchestra.run.smithi073:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T20:49:22.382 DEBUG:teuthology.orchestra.run.smithi073:> stat /dev/vg_nvme/lv_3 2024-04-26T20:49:22.438 INFO:teuthology.orchestra.run.smithi073.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T20:49:22.438 INFO:teuthology.orchestra.run.smithi073.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T20:49:22.438 INFO:teuthology.orchestra.run.smithi073.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-04-26T20:49:22.438 INFO:teuthology.orchestra.run.smithi073.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T20:49:22.438 INFO:teuthology.orchestra.run.smithi073.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T20:49:22.439 INFO:teuthology.orchestra.run.smithi073.stdout:Access: 2024-04-26 20:49:01.110819735 +0000 2024-04-26T20:49:22.439 INFO:teuthology.orchestra.run.smithi073.stdout:Modify: 2024-04-26 20:47:13.373917679 +0000 2024-04-26T20:49:22.439 INFO:teuthology.orchestra.run.smithi073.stdout:Change: 2024-04-26 20:47:13.373917679 +0000 2024-04-26T20:49:22.439 INFO:teuthology.orchestra.run.smithi073.stdout: Birth: 2024-04-26 20:47:13.373917679 +0000 2024-04-26T20:49:22.439 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T20:49:22.501 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2024-04-26T20:49:22.501 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2024-04-26T20:49:22.501 INFO:teuthology.orchestra.run.smithi073.stderr:512 bytes copied, 0.000187772 s, 2.7 MB/s 2024-04-26T20:49:22.502 DEBUG:teuthology.orchestra.run.smithi073:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T20:49:22.558 DEBUG:teuthology.orchestra.run.smithi073:> stat /dev/vg_nvme/lv_4 2024-04-26T20:49:22.614 INFO:teuthology.orchestra.run.smithi073.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T20:49:22.614 INFO:teuthology.orchestra.run.smithi073.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T20:49:22.614 INFO:teuthology.orchestra.run.smithi073.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-26T20:49:22.615 INFO:teuthology.orchestra.run.smithi073.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T20:49:22.615 INFO:teuthology.orchestra.run.smithi073.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T20:49:22.615 INFO:teuthology.orchestra.run.smithi073.stdout:Access: 2024-04-26 20:49:01.110819735 +0000 2024-04-26T20:49:22.615 INFO:teuthology.orchestra.run.smithi073.stdout:Modify: 2024-04-26 20:47:13.727916585 +0000 2024-04-26T20:49:22.615 INFO:teuthology.orchestra.run.smithi073.stdout:Change: 2024-04-26 20:47:13.727916585 +0000 2024-04-26T20:49:22.615 INFO:teuthology.orchestra.run.smithi073.stdout: Birth: 2024-04-26 20:47:13.727916585 +0000 2024-04-26T20:49:22.616 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T20:49:22.677 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2024-04-26T20:49:22.677 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2024-04-26T20:49:22.677 INFO:teuthology.orchestra.run.smithi073.stderr:512 bytes copied, 0.000163258 s, 3.1 MB/s 2024-04-26T20:49:22.678 DEBUG:teuthology.orchestra.run.smithi073:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T20:49:22.734 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T20:49:22.788 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-26T20:49:22.788 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T20:49:22.803 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T20:49:22.804 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_1 2024-04-26T20:49:22.859 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T20:49:22.859 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T20:49:22.859 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-04-26T20:49:22.859 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T20:49:22.859 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T20:49:22.859 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-04-26 20:49:12.088276935 +0000 2024-04-26T20:49:22.859 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-04-26 20:47:12.704220003 +0000 2024-04-26T20:49:22.859 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-04-26 20:47:12.704220003 +0000 2024-04-26T20:49:22.859 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: 2024-04-26 20:47:12.704220003 +0000 2024-04-26T20:49:22.859 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T20:49:22.921 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-04-26T20:49:22.921 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-04-26T20:49:22.921 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000289052 s, 1.8 MB/s 2024-04-26T20:49:22.922 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T20:49:22.977 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_2 2024-04-26T20:49:23.032 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T20:49:23.033 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T20:49:23.033 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-04-26T20:49:23.033 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T20:49:23.033 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T20:49:23.033 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-04-26 20:49:12.088276935 +0000 2024-04-26T20:49:23.033 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-04-26 20:47:13.054211356 +0000 2024-04-26T20:49:23.033 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-04-26 20:47:13.054211356 +0000 2024-04-26T20:49:23.033 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: 2024-04-26 20:47:13.054211356 +0000 2024-04-26T20:49:23.033 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T20:49:23.094 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-04-26T20:49:23.094 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-04-26T20:49:23.094 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000188827 s, 2.7 MB/s 2024-04-26T20:49:23.095 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T20:49:23.151 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_3 2024-04-26T20:49:23.206 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T20:49:23.206 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T20:49:23.206 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-26T20:49:23.206 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T20:49:23.206 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T20:49:23.206 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-04-26 20:49:12.089276912 +0000 2024-04-26T20:49:23.206 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-04-26 20:47:13.391203031 +0000 2024-04-26T20:49:23.206 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-04-26 20:47:13.391203031 +0000 2024-04-26T20:49:23.206 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: 2024-04-26 20:47:13.391203031 +0000 2024-04-26T20:49:23.207 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T20:49:23.268 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-04-26T20:49:23.268 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-04-26T20:49:23.268 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000248407 s, 2.1 MB/s 2024-04-26T20:49:23.269 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T20:49:23.324 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_4 2024-04-26T20:49:23.378 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T20:49:23.378 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T20:49:23.378 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-26T20:49:23.379 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T20:49:23.379 INFO:teuthology.orchestra.run.smithi120.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T20:49:23.379 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-04-26 20:49:12.089276912 +0000 2024-04-26T20:49:23.379 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-04-26 20:47:13.716195002 +0000 2024-04-26T20:49:23.379 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-04-26 20:47:13.716195002 +0000 2024-04-26T20:49:23.379 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: 2024-04-26 20:47:13.716195002 +0000 2024-04-26T20:49:23.379 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T20:49:23.440 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-04-26T20:49:23.440 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-04-26T20:49:23.440 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000220663 s, 2.3 MB/s 2024-04-26T20:49:23.441 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T20:49:23.497 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T20:49:23.497 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi073.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi120.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-26T20:49:23.497 INFO:tasks.ceph:Generating config... 2024-04-26T20:49:23.498 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-26T20:49:23.498 INFO:tasks.ceph:[client] debug client = 20 2024-04-26T20:49:23.499 INFO:tasks.ceph:[client] debug ms = 1 2024-04-26T20:49:23.499 INFO:tasks.ceph:[client] pid file = 2024-04-26T20:49:23.499 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-26T20:49:23.499 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-26T20:49:23.499 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-04-26T20:49:23.499 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-26T20:49:23.499 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-26T20:49:23.499 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-26T20:49:23.499 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-26T20:49:23.499 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-26T20:49:23.499 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-26T20:49:23.499 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-26T20:49:23.499 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-26T20:49:23.499 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-26T20:49:23.499 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-26T20:49:23.500 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-26T20:49:23.500 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-26T20:49:23.500 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-26T20:49:23.500 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-26T20:49:23.500 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-26T20:49:23.500 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-26T20:49:23.500 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-26T20:49:23.500 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-26T20:49:23.500 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-26T20:49:23.500 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-26T20:49:23.500 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-04-26T20:49:23.500 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-04-26T20:49:23.500 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-26T20:49:23.500 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-26T20:49:23.500 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-04-26T20:49:23.500 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-26T20:49:23.501 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-26T20:49:23.501 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-26T20:49:23.501 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-26T20:49:23.501 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-26T20:49:23.501 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-26T20:49:23.501 INFO:tasks.ceph:Setting up mon.a... 2024-04-26T20:49:23.501 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-26T20:49:23.559 INFO:teuthology.orchestra.run.smithi073.stdout:creating /etc/ceph/ceph.keyring 2024-04-26T20:49:23.562 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-26T20:49:23.601 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-26T20:49:23.665 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.73'), ('mon.b', '172.21.15.120'), ('mon.c', '[v2:172.21.15.120:3301,v1:172.21.15.120:6790]')] 2024-04-26T20:49:23.665 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.73,172.21.15.120,[v2:172.21.15.120:3301,v1:172.21.15.120:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-26T20:49:23.666 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-26T20:49:23.666 DEBUG:teuthology.orchestra.run.smithi073:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-26T20:49:23.719 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.73 --add b 172.21.15.120 --addv c '[v2:172.21.15.120:3301,v1:172.21.15.120:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-26T20:49:23.804 INFO:teuthology.orchestra.run.smithi073.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-26T20:49:23.804 INFO:teuthology.orchestra.run.smithi073.stdout:monmaptool: generated fsid fbd3f548-b547-4713-856b-09f98f5d1764 2024-04-26T20:49:23.804 INFO:teuthology.orchestra.run.smithi073.stdout:setting min_mon_release = quincy 2024-04-26T20:49:23.804 INFO:teuthology.orchestra.run.smithi073.stdout:epoch 0 2024-04-26T20:49:23.804 INFO:teuthology.orchestra.run.smithi073.stdout:fsid fbd3f548-b547-4713-856b-09f98f5d1764 2024-04-26T20:49:23.805 INFO:teuthology.orchestra.run.smithi073.stdout:last_changed 2024-04-26T20:49:23.804620+0000 2024-04-26T20:49:23.805 INFO:teuthology.orchestra.run.smithi073.stdout:created 2024-04-26T20:49:23.804620+0000 2024-04-26T20:49:23.805 INFO:teuthology.orchestra.run.smithi073.stdout:min_mon_release 17 (quincy) 2024-04-26T20:49:23.805 INFO:teuthology.orchestra.run.smithi073.stdout:election_strategy: 1 2024-04-26T20:49:23.805 INFO:teuthology.orchestra.run.smithi073.stdout:0: [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] mon.a 2024-04-26T20:49:23.805 INFO:teuthology.orchestra.run.smithi073.stdout:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-04-26T20:49:23.805 INFO:teuthology.orchestra.run.smithi073.stdout:2: [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] mon.c 2024-04-26T20:49:23.805 INFO:teuthology.orchestra.run.smithi073.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-26T20:49:23.806 DEBUG:teuthology.orchestra.run.smithi073:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-26T20:49:23.821 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID fbd3f548-b547-4713-856b-09f98f5d1764... 2024-04-26T20:49:23.821 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-26T20:49:23.863 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-26T20:49:23.900 INFO:teuthology.orchestra.run.smithi073.stdout:[global] 2024-04-26T20:49:23.900 INFO:teuthology.orchestra.run.smithi073.stdout: chdir = "" 2024-04-26T20:49:23.900 INFO:teuthology.orchestra.run.smithi073.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-26T20:49:23.901 INFO:teuthology.orchestra.run.smithi073.stdout: auth supported = cephx 2024-04-26T20:49:23.901 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:49:23.901 INFO:teuthology.orchestra.run.smithi073.stdout: filestore xattr use omap = true 2024-04-26T20:49:23.901 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:49:23.901 INFO:teuthology.orchestra.run.smithi073.stdout: mon clock drift allowed = 1.000 2024-04-26T20:49:23.901 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:49:23.901 INFO:teuthology.orchestra.run.smithi073.stdout: osd crush chooseleaf type = 0 2024-04-26T20:49:23.901 INFO:teuthology.orchestra.run.smithi073.stdout: auth debug = true 2024-04-26T20:49:23.901 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:49:23.901 INFO:teuthology.orchestra.run.smithi073.stdout: ms die on old message = true 2024-04-26T20:49:23.901 INFO:teuthology.orchestra.run.smithi073.stdout: ms die on bug = true 2024-04-26T20:49:23.901 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:49:23.901 INFO:teuthology.orchestra.run.smithi073.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-26T20:49:23.901 INFO:teuthology.orchestra.run.smithi073.stdout: mon pg warn max object skew = 0 2024-04-26T20:49:23.901 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:49:23.902 INFO:teuthology.orchestra.run.smithi073.stdout: # disable pg_autoscaler by default for new pools 2024-04-26T20:49:23.902 INFO:teuthology.orchestra.run.smithi073.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-26T20:49:23.902 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:49:23.902 INFO:teuthology.orchestra.run.smithi073.stdout: osd pool default size = 2 2024-04-26T20:49:23.902 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:49:23.902 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd allow primary affinity = true 2024-04-26T20:49:23.902 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd allow pg remap = true 2024-04-26T20:49:23.902 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on legacy crush tunables = false 2024-04-26T20:49:23.902 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on crush straw calc version zero = false 2024-04-26T20:49:23.902 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on no sortbitwise = false 2024-04-26T20:49:23.902 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on osd down out interval zero = false 2024-04-26T20:49:23.902 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on too few osds = false 2024-04-26T20:49:23.902 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-26T20:49:23.902 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-26T20:49:23.902 INFO:teuthology.orchestra.run.smithi073.stdout: mon_allow_pool_size_one = true 2024-04-26T20:49:23.903 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:49:23.903 INFO:teuthology.orchestra.run.smithi073.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-26T20:49:23.903 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:49:23.903 INFO:teuthology.orchestra.run.smithi073.stdout: osd default data pool replay window = 5 2024-04-26T20:49:23.903 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:49:23.903 INFO:teuthology.orchestra.run.smithi073.stdout: mon allow pool delete = true 2024-04-26T20:49:23.903 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:49:23.903 INFO:teuthology.orchestra.run.smithi073.stdout: mon cluster log file level = debug 2024-04-26T20:49:23.903 INFO:teuthology.orchestra.run.smithi073.stdout: debug asserts on shutdown = true 2024-04-26T20:49:23.903 INFO:teuthology.orchestra.run.smithi073.stdout: mon health detail to clog = false 2024-04-26T20:49:23.903 INFO:teuthology.orchestra.run.smithi073.stdout: mon host = "172.21.15.73,172.21.15.120,[v2:172.21.15.120:3301,v1:172.21.15.120:6790]" 2024-04-26T20:49:23.903 INFO:teuthology.orchestra.run.smithi073.stdout: ms die on skipped message = False 2024-04-26T20:49:23.903 INFO:teuthology.orchestra.run.smithi073.stdout: fsid = fbd3f548-b547-4713-856b-09f98f5d1764 2024-04-26T20:49:23.903 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:49:23.903 INFO:teuthology.orchestra.run.smithi073.stdout:[osd] 2024-04-26T20:49:23.903 INFO:teuthology.orchestra.run.smithi073.stdout: osd journal size = 100 2024-04-26T20:49:23.903 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:49:23.904 INFO:teuthology.orchestra.run.smithi073.stdout: osd scrub load threshold = 5.0 2024-04-26T20:49:23.904 INFO:teuthology.orchestra.run.smithi073.stdout: osd scrub max interval = 600 2024-04-26T20:49:23.904 INFO:teuthology.orchestra.run.smithi073.stdout: osd mclock profile = high_recovery_ops 2024-04-26T20:49:23.904 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:49:23.904 INFO:teuthology.orchestra.run.smithi073.stdout: osd recover clone overlap = true 2024-04-26T20:49:23.904 INFO:teuthology.orchestra.run.smithi073.stdout: osd recovery max chunk = 1048576 2024-04-26T20:49:23.904 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:49:23.904 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug shutdown = true 2024-04-26T20:49:23.904 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug op order = true 2024-04-26T20:49:23.904 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug verify stray on activate = true 2024-04-26T20:49:23.904 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug trim objects = true 2024-04-26T20:49:23.904 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:49:23.904 INFO:teuthology.orchestra.run.smithi073.stdout: osd open classes on start = true 2024-04-26T20:49:23.904 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug pg log writeout = true 2024-04-26T20:49:23.904 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:49:23.904 INFO:teuthology.orchestra.run.smithi073.stdout: osd deep scrub update digest min age = 30 2024-04-26T20:49:23.905 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:49:23.905 INFO:teuthology.orchestra.run.smithi073.stdout: osd map max advance = 10 2024-04-26T20:49:23.905 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:49:23.905 INFO:teuthology.orchestra.run.smithi073.stdout: journal zero on create = true 2024-04-26T20:49:23.905 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:49:23.905 INFO:teuthology.orchestra.run.smithi073.stdout: filestore ondisk finisher threads = 3 2024-04-26T20:49:23.905 INFO:teuthology.orchestra.run.smithi073.stdout: filestore apply finisher threads = 3 2024-04-26T20:49:23.905 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:49:23.905 INFO:teuthology.orchestra.run.smithi073.stdout: bdev debug aio = true 2024-04-26T20:49:23.905 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug misdirected ops = true 2024-04-26T20:49:23.905 INFO:teuthology.orchestra.run.smithi073.stdout: bluestore block size = 96636764160 2024-04-26T20:49:23.905 INFO:teuthology.orchestra.run.smithi073.stdout: bluestore fsck on mount = True 2024-04-26T20:49:23.905 INFO:teuthology.orchestra.run.smithi073.stdout: debug bluefs = 20 2024-04-26T20:49:23.905 INFO:teuthology.orchestra.run.smithi073.stdout: debug bluestore = 20 2024-04-26T20:49:23.905 INFO:teuthology.orchestra.run.smithi073.stdout: debug ms = 1 2024-04-26T20:49:23.905 INFO:teuthology.orchestra.run.smithi073.stdout: debug osd = 20 2024-04-26T20:49:23.905 INFO:teuthology.orchestra.run.smithi073.stdout: debug rocksdb = 10 2024-04-26T20:49:23.905 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd backfillfull_ratio = 0.85 2024-04-26T20:49:23.906 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd full ratio = 0.9 2024-04-26T20:49:23.906 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd nearfull ratio = 0.8 2024-04-26T20:49:23.906 INFO:teuthology.orchestra.run.smithi073.stdout: osd failsafe full ratio = 0.95 2024-04-26T20:49:23.906 INFO:teuthology.orchestra.run.smithi073.stdout: osd objectstore = bluestore 2024-04-26T20:49:23.906 INFO:teuthology.orchestra.run.smithi073.stdout: osd op complaint time = 180 2024-04-26T20:49:23.906 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:49:23.906 INFO:teuthology.orchestra.run.smithi073.stdout:[mgr] 2024-04-26T20:49:23.906 INFO:teuthology.orchestra.run.smithi073.stdout: debug ms = 1 2024-04-26T20:49:23.906 INFO:teuthology.orchestra.run.smithi073.stdout: debug mgr = 20 2024-04-26T20:49:23.906 INFO:teuthology.orchestra.run.smithi073.stdout: debug mon = 20 2024-04-26T20:49:23.906 INFO:teuthology.orchestra.run.smithi073.stdout: debug auth = 20 2024-04-26T20:49:23.906 INFO:teuthology.orchestra.run.smithi073.stdout: mon reweight min pgs per osd = 4 2024-04-26T20:49:23.906 INFO:teuthology.orchestra.run.smithi073.stdout: mon reweight min bytes per osd = 10 2024-04-26T20:49:23.906 INFO:teuthology.orchestra.run.smithi073.stdout: mgr/telemetry/nag = false 2024-04-26T20:49:23.906 INFO:teuthology.orchestra.run.smithi073.stdout: debug client = 20 2024-04-26T20:49:23.907 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:49:23.907 INFO:teuthology.orchestra.run.smithi073.stdout:[mon] 2024-04-26T20:49:23.907 INFO:teuthology.orchestra.run.smithi073.stdout: debug ms = 1 2024-04-26T20:49:23.907 INFO:teuthology.orchestra.run.smithi073.stdout: debug mon = 20 2024-04-26T20:49:23.907 INFO:teuthology.orchestra.run.smithi073.stdout: debug paxos = 20 2024-04-26T20:49:23.907 INFO:teuthology.orchestra.run.smithi073.stdout: debug auth = 20 2024-04-26T20:49:23.907 INFO:teuthology.orchestra.run.smithi073.stdout: mon data avail warn = 5 2024-04-26T20:49:23.907 INFO:teuthology.orchestra.run.smithi073.stdout: mon mgr mkfs grace = 240 2024-04-26T20:49:23.907 INFO:teuthology.orchestra.run.smithi073.stdout: mon reweight min pgs per osd = 4 2024-04-26T20:49:23.907 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd reporter subtree level = osd 2024-04-26T20:49:23.907 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd prime pg temp = true 2024-04-26T20:49:23.907 INFO:teuthology.orchestra.run.smithi073.stdout: mon reweight min bytes per osd = 10 2024-04-26T20:49:23.907 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:49:23.907 INFO:teuthology.orchestra.run.smithi073.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-26T20:49:23.907 INFO:teuthology.orchestra.run.smithi073.stdout: auth mon ticket ttl = 660 # 11m 2024-04-26T20:49:23.908 INFO:teuthology.orchestra.run.smithi073.stdout: auth service ticket ttl = 240 # 4m 2024-04-26T20:49:23.908 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:49:23.908 INFO:teuthology.orchestra.run.smithi073.stdout: # don't complain about insecure global_id in the test suite 2024-04-26T20:49:23.908 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-26T20:49:23.908 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-26T20:49:23.908 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:49:23.908 INFO:teuthology.orchestra.run.smithi073.stdout: # 1m isn't quite enough 2024-04-26T20:49:23.908 INFO:teuthology.orchestra.run.smithi073.stdout: mon_down_mkfs_grace = 2m 2024-04-26T20:49:23.908 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:49:23.908 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_filestore_osds = false 2024-04-26T20:49:23.908 INFO:teuthology.orchestra.run.smithi073.stdout: mon down mkfs grace = 300 2024-04-26T20:49:23.908 INFO:teuthology.orchestra.run.smithi073.stdout: mon op complaint time = 120 2024-04-26T20:49:23.908 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:49:23.908 INFO:teuthology.orchestra.run.smithi073.stdout:[client] 2024-04-26T20:49:23.908 INFO:teuthology.orchestra.run.smithi073.stdout: rgw cache enabled = true 2024-04-26T20:49:23.908 INFO:teuthology.orchestra.run.smithi073.stdout: rgw enable ops log = true 2024-04-26T20:49:23.909 INFO:teuthology.orchestra.run.smithi073.stdout: rgw enable usage log = true 2024-04-26T20:49:23.909 INFO:teuthology.orchestra.run.smithi073.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-26T20:49:23.909 INFO:teuthology.orchestra.run.smithi073.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-26T20:49:23.909 INFO:teuthology.orchestra.run.smithi073.stdout: client mount timeout = 600 2024-04-26T20:49:23.909 INFO:teuthology.orchestra.run.smithi073.stdout: debug client = 20 2024-04-26T20:49:23.909 INFO:teuthology.orchestra.run.smithi073.stdout: debug ms = 1 2024-04-26T20:49:23.909 INFO:teuthology.orchestra.run.smithi073.stdout: pid file = "" 2024-04-26T20:49:23.909 INFO:teuthology.orchestra.run.smithi073.stdout: rados mon op timeout = 900 2024-04-26T20:49:23.909 INFO:teuthology.orchestra.run.smithi073.stdout: rados osd op timeout = 900 2024-04-26T20:49:23.909 INFO:teuthology.orchestra.run.smithi073.stdout:[mon.a] 2024-04-26T20:49:23.909 INFO:teuthology.orchestra.run.smithi073.stdout:[mon.b] 2024-04-26T20:49:23.909 INFO:teuthology.orchestra.run.smithi073.stdout:[mon.c] 2024-04-26T20:49:23.909 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.a] 2024-04-26T20:49:23.909 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.c] 2024-04-26T20:49:23.909 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.b] 2024-04-26T20:49:23.909 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.d] 2024-04-26T20:49:23.909 INFO:teuthology.orchestra.run.smithi073.stdout:[mds] 2024-04-26T20:49:23.910 INFO:teuthology.orchestra.run.smithi073.stdout: debug mds = 20 2024-04-26T20:49:23.910 INFO:teuthology.orchestra.run.smithi073.stdout: debug mds balancer = 20 2024-04-26T20:49:23.910 INFO:teuthology.orchestra.run.smithi073.stdout: debug ms = 1 2024-04-26T20:49:23.910 INFO:teuthology.orchestra.run.smithi073.stdout: mds debug frag = True 2024-04-26T20:49:23.910 INFO:teuthology.orchestra.run.smithi073.stdout: mds debug scatterstat = True 2024-04-26T20:49:23.910 INFO:teuthology.orchestra.run.smithi073.stdout: mds op complaint time = 180 2024-04-26T20:49:23.910 INFO:teuthology.orchestra.run.smithi073.stdout: mds verify scatter = True 2024-04-26T20:49:23.910 INFO:teuthology.orchestra.run.smithi073.stdout: osd op complaint time = 180 2024-04-26T20:49:23.910 INFO:teuthology.orchestra.run.smithi073.stdout: rados mon op timeout = 900 2024-04-26T20:49:23.910 INFO:teuthology.orchestra.run.smithi073.stdout: rados osd op timeout = 900 2024-04-26T20:49:23.944 INFO:teuthology.orchestra.run.smithi120.stdout:[global] 2024-04-26T20:49:23.944 INFO:teuthology.orchestra.run.smithi120.stdout: chdir = "" 2024-04-26T20:49:23.944 INFO:teuthology.orchestra.run.smithi120.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-26T20:49:23.944 INFO:teuthology.orchestra.run.smithi120.stdout: auth supported = cephx 2024-04-26T20:49:23.944 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T20:49:23.945 INFO:teuthology.orchestra.run.smithi120.stdout: filestore xattr use omap = true 2024-04-26T20:49:23.945 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T20:49:23.945 INFO:teuthology.orchestra.run.smithi120.stdout: mon clock drift allowed = 1.000 2024-04-26T20:49:23.945 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T20:49:23.945 INFO:teuthology.orchestra.run.smithi120.stdout: osd crush chooseleaf type = 0 2024-04-26T20:49:23.945 INFO:teuthology.orchestra.run.smithi120.stdout: auth debug = true 2024-04-26T20:49:23.945 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T20:49:23.945 INFO:teuthology.orchestra.run.smithi120.stdout: ms die on old message = true 2024-04-26T20:49:23.945 INFO:teuthology.orchestra.run.smithi120.stdout: ms die on bug = true 2024-04-26T20:49:23.945 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T20:49:23.945 INFO:teuthology.orchestra.run.smithi120.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-26T20:49:23.945 INFO:teuthology.orchestra.run.smithi120.stdout: mon pg warn max object skew = 0 2024-04-26T20:49:23.945 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T20:49:23.945 INFO:teuthology.orchestra.run.smithi120.stdout: # disable pg_autoscaler by default for new pools 2024-04-26T20:49:23.945 INFO:teuthology.orchestra.run.smithi120.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-26T20:49:23.945 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T20:49:23.946 INFO:teuthology.orchestra.run.smithi120.stdout: osd pool default size = 2 2024-04-26T20:49:23.946 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T20:49:23.946 INFO:teuthology.orchestra.run.smithi120.stdout: mon osd allow primary affinity = true 2024-04-26T20:49:23.946 INFO:teuthology.orchestra.run.smithi120.stdout: mon osd allow pg remap = true 2024-04-26T20:49:23.946 INFO:teuthology.orchestra.run.smithi120.stdout: mon warn on legacy crush tunables = false 2024-04-26T20:49:23.946 INFO:teuthology.orchestra.run.smithi120.stdout: mon warn on crush straw calc version zero = false 2024-04-26T20:49:23.946 INFO:teuthology.orchestra.run.smithi120.stdout: mon warn on no sortbitwise = false 2024-04-26T20:49:23.946 INFO:teuthology.orchestra.run.smithi120.stdout: mon warn on osd down out interval zero = false 2024-04-26T20:49:23.946 INFO:teuthology.orchestra.run.smithi120.stdout: mon warn on too few osds = false 2024-04-26T20:49:23.946 INFO:teuthology.orchestra.run.smithi120.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-26T20:49:23.946 INFO:teuthology.orchestra.run.smithi120.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-26T20:49:23.946 INFO:teuthology.orchestra.run.smithi120.stdout: mon_allow_pool_size_one = true 2024-04-26T20:49:23.946 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T20:49:23.946 INFO:teuthology.orchestra.run.smithi120.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-26T20:49:23.946 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T20:49:23.947 INFO:teuthology.orchestra.run.smithi120.stdout: osd default data pool replay window = 5 2024-04-26T20:49:23.947 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T20:49:23.947 INFO:teuthology.orchestra.run.smithi120.stdout: mon allow pool delete = true 2024-04-26T20:49:23.947 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T20:49:23.947 INFO:teuthology.orchestra.run.smithi120.stdout: mon cluster log file level = debug 2024-04-26T20:49:23.947 INFO:teuthology.orchestra.run.smithi120.stdout: debug asserts on shutdown = true 2024-04-26T20:49:23.947 INFO:teuthology.orchestra.run.smithi120.stdout: mon health detail to clog = false 2024-04-26T20:49:23.947 INFO:teuthology.orchestra.run.smithi120.stdout: mon host = "172.21.15.73,172.21.15.120,[v2:172.21.15.120:3301,v1:172.21.15.120:6790]" 2024-04-26T20:49:23.947 INFO:teuthology.orchestra.run.smithi120.stdout: ms die on skipped message = False 2024-04-26T20:49:23.947 INFO:teuthology.orchestra.run.smithi120.stdout: fsid = fbd3f548-b547-4713-856b-09f98f5d1764 2024-04-26T20:49:23.947 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T20:49:23.947 INFO:teuthology.orchestra.run.smithi120.stdout:[osd] 2024-04-26T20:49:23.947 INFO:teuthology.orchestra.run.smithi120.stdout: osd journal size = 100 2024-04-26T20:49:23.947 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T20:49:23.947 INFO:teuthology.orchestra.run.smithi120.stdout: osd scrub load threshold = 5.0 2024-04-26T20:49:23.947 INFO:teuthology.orchestra.run.smithi120.stdout: osd scrub max interval = 600 2024-04-26T20:49:23.948 INFO:teuthology.orchestra.run.smithi120.stdout: osd mclock profile = high_recovery_ops 2024-04-26T20:49:23.948 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T20:49:23.948 INFO:teuthology.orchestra.run.smithi120.stdout: osd recover clone overlap = true 2024-04-26T20:49:23.948 INFO:teuthology.orchestra.run.smithi120.stdout: osd recovery max chunk = 1048576 2024-04-26T20:49:23.948 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T20:49:23.948 INFO:teuthology.orchestra.run.smithi120.stdout: osd debug shutdown = true 2024-04-26T20:49:23.948 INFO:teuthology.orchestra.run.smithi120.stdout: osd debug op order = true 2024-04-26T20:49:23.948 INFO:teuthology.orchestra.run.smithi120.stdout: osd debug verify stray on activate = true 2024-04-26T20:49:23.948 INFO:teuthology.orchestra.run.smithi120.stdout: osd debug trim objects = true 2024-04-26T20:49:23.948 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T20:49:23.948 INFO:teuthology.orchestra.run.smithi120.stdout: osd open classes on start = true 2024-04-26T20:49:23.948 INFO:teuthology.orchestra.run.smithi120.stdout: osd debug pg log writeout = true 2024-04-26T20:49:23.948 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T20:49:23.948 INFO:teuthology.orchestra.run.smithi120.stdout: osd deep scrub update digest min age = 30 2024-04-26T20:49:23.948 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T20:49:23.948 INFO:teuthology.orchestra.run.smithi120.stdout: osd map max advance = 10 2024-04-26T20:49:23.949 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T20:49:23.949 INFO:teuthology.orchestra.run.smithi120.stdout: journal zero on create = true 2024-04-26T20:49:23.949 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T20:49:23.949 INFO:teuthology.orchestra.run.smithi120.stdout: filestore ondisk finisher threads = 3 2024-04-26T20:49:23.949 INFO:teuthology.orchestra.run.smithi120.stdout: filestore apply finisher threads = 3 2024-04-26T20:49:23.949 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T20:49:23.949 INFO:teuthology.orchestra.run.smithi120.stdout: bdev debug aio = true 2024-04-26T20:49:23.949 INFO:teuthology.orchestra.run.smithi120.stdout: osd debug misdirected ops = true 2024-04-26T20:49:23.949 INFO:teuthology.orchestra.run.smithi120.stdout: bluestore block size = 96636764160 2024-04-26T20:49:23.949 INFO:teuthology.orchestra.run.smithi120.stdout: bluestore fsck on mount = True 2024-04-26T20:49:23.949 INFO:teuthology.orchestra.run.smithi120.stdout: debug bluefs = 20 2024-04-26T20:49:23.949 INFO:teuthology.orchestra.run.smithi120.stdout: debug bluestore = 20 2024-04-26T20:49:23.949 INFO:teuthology.orchestra.run.smithi120.stdout: debug ms = 1 2024-04-26T20:49:23.949 INFO:teuthology.orchestra.run.smithi120.stdout: debug osd = 20 2024-04-26T20:49:23.949 INFO:teuthology.orchestra.run.smithi120.stdout: debug rocksdb = 10 2024-04-26T20:49:23.949 INFO:teuthology.orchestra.run.smithi120.stdout: mon osd backfillfull_ratio = 0.85 2024-04-26T20:49:23.950 INFO:teuthology.orchestra.run.smithi120.stdout: mon osd full ratio = 0.9 2024-04-26T20:49:23.950 INFO:teuthology.orchestra.run.smithi120.stdout: mon osd nearfull ratio = 0.8 2024-04-26T20:49:23.950 INFO:teuthology.orchestra.run.smithi120.stdout: osd failsafe full ratio = 0.95 2024-04-26T20:49:23.950 INFO:teuthology.orchestra.run.smithi120.stdout: osd objectstore = bluestore 2024-04-26T20:49:23.950 INFO:teuthology.orchestra.run.smithi120.stdout: osd op complaint time = 180 2024-04-26T20:49:23.950 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T20:49:23.950 INFO:teuthology.orchestra.run.smithi120.stdout:[mgr] 2024-04-26T20:49:23.950 INFO:teuthology.orchestra.run.smithi120.stdout: debug ms = 1 2024-04-26T20:49:23.950 INFO:teuthology.orchestra.run.smithi120.stdout: debug mgr = 20 2024-04-26T20:49:23.950 INFO:teuthology.orchestra.run.smithi120.stdout: debug mon = 20 2024-04-26T20:49:23.950 INFO:teuthology.orchestra.run.smithi120.stdout: debug auth = 20 2024-04-26T20:49:23.950 INFO:teuthology.orchestra.run.smithi120.stdout: mon reweight min pgs per osd = 4 2024-04-26T20:49:23.950 INFO:teuthology.orchestra.run.smithi120.stdout: mon reweight min bytes per osd = 10 2024-04-26T20:49:23.950 INFO:teuthology.orchestra.run.smithi120.stdout: mgr/telemetry/nag = false 2024-04-26T20:49:23.950 INFO:teuthology.orchestra.run.smithi120.stdout: debug client = 20 2024-04-26T20:49:23.951 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T20:49:23.951 INFO:teuthology.orchestra.run.smithi120.stdout:[mon] 2024-04-26T20:49:23.951 INFO:teuthology.orchestra.run.smithi120.stdout: debug ms = 1 2024-04-26T20:49:23.951 INFO:teuthology.orchestra.run.smithi120.stdout: debug mon = 20 2024-04-26T20:49:23.951 INFO:teuthology.orchestra.run.smithi120.stdout: debug paxos = 20 2024-04-26T20:49:23.951 INFO:teuthology.orchestra.run.smithi120.stdout: debug auth = 20 2024-04-26T20:49:23.951 INFO:teuthology.orchestra.run.smithi120.stdout: mon data avail warn = 5 2024-04-26T20:49:23.951 INFO:teuthology.orchestra.run.smithi120.stdout: mon mgr mkfs grace = 240 2024-04-26T20:49:23.951 INFO:teuthology.orchestra.run.smithi120.stdout: mon reweight min pgs per osd = 4 2024-04-26T20:49:23.951 INFO:teuthology.orchestra.run.smithi120.stdout: mon osd reporter subtree level = osd 2024-04-26T20:49:23.951 INFO:teuthology.orchestra.run.smithi120.stdout: mon osd prime pg temp = true 2024-04-26T20:49:23.951 INFO:teuthology.orchestra.run.smithi120.stdout: mon reweight min bytes per osd = 10 2024-04-26T20:49:23.951 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T20:49:23.951 INFO:teuthology.orchestra.run.smithi120.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-26T20:49:23.951 INFO:teuthology.orchestra.run.smithi120.stdout: auth mon ticket ttl = 660 # 11m 2024-04-26T20:49:23.951 INFO:teuthology.orchestra.run.smithi120.stdout: auth service ticket ttl = 240 # 4m 2024-04-26T20:49:23.952 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T20:49:23.952 INFO:teuthology.orchestra.run.smithi120.stdout: # don't complain about insecure global_id in the test suite 2024-04-26T20:49:23.952 INFO:teuthology.orchestra.run.smithi120.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-26T20:49:23.952 INFO:teuthology.orchestra.run.smithi120.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-26T20:49:23.952 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T20:49:23.952 INFO:teuthology.orchestra.run.smithi120.stdout: # 1m isn't quite enough 2024-04-26T20:49:23.952 INFO:teuthology.orchestra.run.smithi120.stdout: mon_down_mkfs_grace = 2m 2024-04-26T20:49:23.952 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T20:49:23.952 INFO:teuthology.orchestra.run.smithi120.stdout: mon_warn_on_filestore_osds = false 2024-04-26T20:49:23.952 INFO:teuthology.orchestra.run.smithi120.stdout: mon down mkfs grace = 300 2024-04-26T20:49:23.952 INFO:teuthology.orchestra.run.smithi120.stdout: mon op complaint time = 120 2024-04-26T20:49:23.952 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-26T20:49:23.952 INFO:teuthology.orchestra.run.smithi120.stdout:[client] 2024-04-26T20:49:23.952 INFO:teuthology.orchestra.run.smithi120.stdout: rgw cache enabled = true 2024-04-26T20:49:23.952 INFO:teuthology.orchestra.run.smithi120.stdout: rgw enable ops log = true 2024-04-26T20:49:23.953 INFO:teuthology.orchestra.run.smithi120.stdout: rgw enable usage log = true 2024-04-26T20:49:23.953 INFO:teuthology.orchestra.run.smithi120.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-26T20:49:23.953 INFO:teuthology.orchestra.run.smithi120.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-26T20:49:23.953 INFO:teuthology.orchestra.run.smithi120.stdout: client mount timeout = 600 2024-04-26T20:49:23.953 INFO:teuthology.orchestra.run.smithi120.stdout: debug client = 20 2024-04-26T20:49:23.953 INFO:teuthology.orchestra.run.smithi120.stdout: debug ms = 1 2024-04-26T20:49:23.953 INFO:teuthology.orchestra.run.smithi120.stdout: pid file = "" 2024-04-26T20:49:23.953 INFO:teuthology.orchestra.run.smithi120.stdout: rados mon op timeout = 900 2024-04-26T20:49:23.953 INFO:teuthology.orchestra.run.smithi120.stdout: rados osd op timeout = 900 2024-04-26T20:49:23.953 INFO:teuthology.orchestra.run.smithi120.stdout:[mon.a] 2024-04-26T20:49:23.953 INFO:teuthology.orchestra.run.smithi120.stdout:[mon.b] 2024-04-26T20:49:23.953 INFO:teuthology.orchestra.run.smithi120.stdout:[mon.c] 2024-04-26T20:49:23.953 INFO:teuthology.orchestra.run.smithi120.stdout:[mds.a] 2024-04-26T20:49:23.953 INFO:teuthology.orchestra.run.smithi120.stdout:[mds.c] 2024-04-26T20:49:23.953 INFO:teuthology.orchestra.run.smithi120.stdout:[mds.b] 2024-04-26T20:49:23.954 INFO:teuthology.orchestra.run.smithi120.stdout:[mds.d] 2024-04-26T20:49:23.954 INFO:teuthology.orchestra.run.smithi120.stdout:[mds] 2024-04-26T20:49:23.954 INFO:teuthology.orchestra.run.smithi120.stdout: debug mds = 20 2024-04-26T20:49:23.954 INFO:teuthology.orchestra.run.smithi120.stdout: debug mds balancer = 20 2024-04-26T20:49:23.954 INFO:teuthology.orchestra.run.smithi120.stdout: debug ms = 1 2024-04-26T20:49:23.954 INFO:teuthology.orchestra.run.smithi120.stdout: mds debug frag = True 2024-04-26T20:49:23.954 INFO:teuthology.orchestra.run.smithi120.stdout: mds debug scatterstat = True 2024-04-26T20:49:23.954 INFO:teuthology.orchestra.run.smithi120.stdout: mds op complaint time = 180 2024-04-26T20:49:23.954 INFO:teuthology.orchestra.run.smithi120.stdout: mds verify scatter = True 2024-04-26T20:49:23.954 INFO:teuthology.orchestra.run.smithi120.stdout: osd op complaint time = 180 2024-04-26T20:49:23.954 INFO:teuthology.orchestra.run.smithi120.stdout: rados mon op timeout = 900 2024-04-26T20:49:23.954 INFO:teuthology.orchestra.run.smithi120.stdout: rados osd op timeout = 900 2024-04-26T20:49:23.955 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-26T20:49:23.955 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-26T20:49:24.007 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-26T20:49:24.007 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-26T20:49:24.007 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-26T20:49:24.022 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-26T20:49:24.022 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-26T20:49:24.076 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi073.front.sepia.ceph.com 2024-04-26T20:49:24.076 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-26T20:49:24.076 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-26T20:49:24.076 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-26T20:49:24.149 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-26T20:49:24.149 DEBUG:teuthology.orchestra.run.smithi073:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-26T20:49:24.203 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi120.front.sepia.ceph.com 2024-04-26T20:49:24.204 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-26T20:49:24.204 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-26T20:49:24.204 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-26T20:49:24.237 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-26T20:49:24.238 DEBUG:teuthology.orchestra.run.smithi120:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-26T20:49:24.292 INFO:tasks.ceph:Setting up mon nodes... 2024-04-26T20:49:24.293 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-26T20:49:24.293 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-26T20:49:24.340 INFO:teuthology.orchestra.run.smithi073.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-26T20:49:24.343 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-26T20:49:24.396 INFO:teuthology.orchestra.run.smithi120.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-26T20:49:24.398 INFO:tasks.ceph:Setting up mds nodes... 2024-04-26T20:49:24.399 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-26T20:49:24.446 INFO:teuthology.orchestra.run.smithi073.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-26T20:49:24.449 DEBUG:teuthology.orchestra.run.smithi073:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-26T20:49:24.472 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-26T20:49:24.562 INFO:teuthology.orchestra.run.smithi073.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-26T20:49:24.563 DEBUG:teuthology.orchestra.run.smithi073:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-26T20:49:24.586 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-26T20:49:24.634 INFO:teuthology.orchestra.run.smithi120.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-26T20:49:24.637 DEBUG:teuthology.orchestra.run.smithi120:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-26T20:49:24.660 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-26T20:49:24.748 INFO:teuthology.orchestra.run.smithi120.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-26T20:49:24.750 DEBUG:teuthology.orchestra.run.smithi120:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-26T20:49:24.773 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-26T20:49:24.773 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-26T20:49:24.813 INFO:teuthology.orchestra.run.smithi073.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-26T20:49:24.825 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-26T20:49:24.903 INFO:teuthology.orchestra.run.smithi073.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-26T20:49:24.915 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-26T20:49:24.953 INFO:teuthology.orchestra.run.smithi120.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-26T20:49:24.965 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-26T20:49:25.043 INFO:teuthology.orchestra.run.smithi120.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-26T20:49:25.055 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-26T20:49:25.055 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi073.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi120.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-26T20:49:25.055 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-26T20:49:25.078 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T20:49:25.078 INFO:tasks.ceph:role: osd.0 2024-04-26T20:49:25.078 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi073.front.sepia.ceph.com 2024-04-26T20:49:25.078 DEBUG:teuthology.orchestra.run.smithi073:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-26T20:49:25.149 INFO:teuthology.orchestra.run.smithi073.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T20:49:25.149 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T20:49:25.149 INFO:teuthology.orchestra.run.smithi073.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T20:49:25.149 INFO:teuthology.orchestra.run.smithi073.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T20:49:25.149 INFO:teuthology.orchestra.run.smithi073.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T20:49:25.149 INFO:teuthology.orchestra.run.smithi073.stdout: = sunit=0 swidth=0 blks 2024-04-26T20:49:25.149 INFO:teuthology.orchestra.run.smithi073.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T20:49:25.149 INFO:teuthology.orchestra.run.smithi073.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T20:49:25.149 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T20:49:25.149 INFO:teuthology.orchestra.run.smithi073.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T20:49:25.634 INFO:teuthology.orchestra.run.smithi073.stdout:Discarding blocks...Done. 2024-04-26T20:49:25.635 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi073.front.sepia.ceph.com -o noatime 2024-04-26T20:49:25.636 DEBUG:teuthology.orchestra.run.smithi073:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-26T20:49:25.747 DEBUG:teuthology.orchestra.run.smithi073:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-26T20:49:25.777 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-26T20:49:25.841 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T20:49:25.841 INFO:tasks.ceph:role: osd.1 2024-04-26T20:49:25.841 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi073.front.sepia.ceph.com 2024-04-26T20:49:25.841 DEBUG:teuthology.orchestra.run.smithi073:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-26T20:49:25.912 INFO:teuthology.orchestra.run.smithi073.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T20:49:25.912 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T20:49:25.912 INFO:teuthology.orchestra.run.smithi073.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T20:49:25.912 INFO:teuthology.orchestra.run.smithi073.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T20:49:25.912 INFO:teuthology.orchestra.run.smithi073.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T20:49:25.912 INFO:teuthology.orchestra.run.smithi073.stdout: = sunit=0 swidth=0 blks 2024-04-26T20:49:25.912 INFO:teuthology.orchestra.run.smithi073.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T20:49:25.912 INFO:teuthology.orchestra.run.smithi073.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T20:49:25.912 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T20:49:25.912 INFO:teuthology.orchestra.run.smithi073.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T20:49:26.380 INFO:teuthology.orchestra.run.smithi073.stdout:Discarding blocks...Done. 2024-04-26T20:49:26.382 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi073.front.sepia.ceph.com -o noatime 2024-04-26T20:49:26.382 DEBUG:teuthology.orchestra.run.smithi073:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-26T20:49:26.453 DEBUG:teuthology.orchestra.run.smithi073:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-26T20:49:26.521 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-26T20:49:26.585 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T20:49:26.585 INFO:tasks.ceph:role: osd.2 2024-04-26T20:49:26.585 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi073.front.sepia.ceph.com 2024-04-26T20:49:26.585 DEBUG:teuthology.orchestra.run.smithi073:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-26T20:49:26.656 INFO:teuthology.orchestra.run.smithi073.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T20:49:26.656 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T20:49:26.656 INFO:teuthology.orchestra.run.smithi073.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T20:49:26.656 INFO:teuthology.orchestra.run.smithi073.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T20:49:26.656 INFO:teuthology.orchestra.run.smithi073.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T20:49:26.656 INFO:teuthology.orchestra.run.smithi073.stdout: = sunit=0 swidth=0 blks 2024-04-26T20:49:26.656 INFO:teuthology.orchestra.run.smithi073.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T20:49:26.656 INFO:teuthology.orchestra.run.smithi073.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T20:49:26.656 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T20:49:26.657 INFO:teuthology.orchestra.run.smithi073.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T20:49:27.139 INFO:teuthology.orchestra.run.smithi073.stdout:Discarding blocks...Done. 2024-04-26T20:49:27.141 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi073.front.sepia.ceph.com -o noatime 2024-04-26T20:49:27.141 DEBUG:teuthology.orchestra.run.smithi073:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-26T20:49:27.235 DEBUG:teuthology.orchestra.run.smithi073:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-26T20:49:27.264 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-26T20:49:27.327 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T20:49:27.327 INFO:tasks.ceph:role: osd.3 2024-04-26T20:49:27.327 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi073.front.sepia.ceph.com 2024-04-26T20:49:27.327 DEBUG:teuthology.orchestra.run.smithi073:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-26T20:49:27.398 INFO:teuthology.orchestra.run.smithi073.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T20:49:27.398 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T20:49:27.398 INFO:teuthology.orchestra.run.smithi073.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T20:49:27.398 INFO:teuthology.orchestra.run.smithi073.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T20:49:27.398 INFO:teuthology.orchestra.run.smithi073.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T20:49:27.398 INFO:teuthology.orchestra.run.smithi073.stdout: = sunit=0 swidth=0 blks 2024-04-26T20:49:27.399 INFO:teuthology.orchestra.run.smithi073.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T20:49:27.399 INFO:teuthology.orchestra.run.smithi073.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T20:49:27.399 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T20:49:27.399 INFO:teuthology.orchestra.run.smithi073.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T20:49:27.888 INFO:teuthology.orchestra.run.smithi073.stdout:Discarding blocks...Done. 2024-04-26T20:49:27.890 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi073.front.sepia.ceph.com -o noatime 2024-04-26T20:49:27.890 DEBUG:teuthology.orchestra.run.smithi073:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-26T20:49:28.036 DEBUG:teuthology.orchestra.run.smithi073:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-26T20:49:28.067 DEBUG:teuthology.orchestra.run.smithi073:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T20:49:28.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:49:28.159+0000 7fe1eebbe640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-26T20:49:28.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:49:28.159+0000 7fe1eebbe640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-26T20:49:28.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:49:28.159+0000 7fe1eebbe640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-26T20:49:28.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:49:28.160+0000 7fe1eebbe640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-26T20:49:28.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:49:28.160+0000 7fe1eebbe640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-26T20:49:28.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:49:28.167+0000 7fe1eebbe640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-26T20:49:31.017 DEBUG:teuthology.orchestra.run.smithi073:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-26T20:49:31.044 DEBUG:teuthology.orchestra.run.smithi073:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T20:49:31.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:49:31.131+0000 7f80f7cb5640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-26T20:49:31.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:49:31.131+0000 7f80f7cb5640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-26T20:49:31.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:49:31.131+0000 7f80f7cb5640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-26T20:49:31.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:49:31.131+0000 7f80f7cb5640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-26T20:49:31.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:49:31.131+0000 7f80f7cb5640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-26T20:49:31.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:49:31.131+0000 7f80f7cb5640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-26T20:49:33.976 DEBUG:teuthology.orchestra.run.smithi073:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-26T20:49:34.005 DEBUG:teuthology.orchestra.run.smithi073:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T20:49:34.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:49:34.094+0000 7fac0fb33640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-26T20:49:34.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:49:34.095+0000 7fac0fb33640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-26T20:49:34.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:49:34.095+0000 7fac0fb33640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-26T20:49:34.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:49:34.095+0000 7fac0fb33640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-26T20:49:34.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:49:34.095+0000 7fac0fb33640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-26T20:49:34.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:49:34.095+0000 7fac0fb33640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-26T20:49:36.942 DEBUG:teuthology.orchestra.run.smithi073:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-26T20:49:36.968 DEBUG:teuthology.orchestra.run.smithi073:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T20:49:37.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:49:37.056+0000 7faab7b31640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-26T20:49:37.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:49:37.056+0000 7faab7b31640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-26T20:49:37.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:49:37.056+0000 7faab7b31640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-26T20:49:37.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:49:37.056+0000 7faab7b31640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-26T20:49:37.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:49:37.056+0000 7faab7b31640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-26T20:49:37.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:49:37.056+0000 7faab7b31640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-26T20:49:39.916 DEBUG:teuthology.orchestra.run.smithi073:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-26T20:49:39.945 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-26T20:49:39.975 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T20:49:39.975 INFO:tasks.ceph:role: osd.4 2024-04-26T20:49:39.975 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi120.front.sepia.ceph.com 2024-04-26T20:49:39.975 DEBUG:teuthology.orchestra.run.smithi120:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-26T20:49:40.049 INFO:teuthology.orchestra.run.smithi120.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T20:49:40.049 INFO:teuthology.orchestra.run.smithi120.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T20:49:40.049 INFO:teuthology.orchestra.run.smithi120.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T20:49:40.049 INFO:teuthology.orchestra.run.smithi120.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T20:49:40.049 INFO:teuthology.orchestra.run.smithi120.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T20:49:40.049 INFO:teuthology.orchestra.run.smithi120.stdout: = sunit=0 swidth=0 blks 2024-04-26T20:49:40.049 INFO:teuthology.orchestra.run.smithi120.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T20:49:40.049 INFO:teuthology.orchestra.run.smithi120.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T20:49:40.049 INFO:teuthology.orchestra.run.smithi120.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T20:49:40.049 INFO:teuthology.orchestra.run.smithi120.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T20:49:40.557 INFO:teuthology.orchestra.run.smithi120.stdout:Discarding blocks...Done. 2024-04-26T20:49:40.559 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi120.front.sepia.ceph.com -o noatime 2024-04-26T20:49:40.559 DEBUG:teuthology.orchestra.run.smithi120:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-26T20:49:40.667 DEBUG:teuthology.orchestra.run.smithi120:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-26T20:49:40.696 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-26T20:49:40.761 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T20:49:40.761 INFO:tasks.ceph:role: osd.5 2024-04-26T20:49:40.761 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi120.front.sepia.ceph.com 2024-04-26T20:49:40.761 DEBUG:teuthology.orchestra.run.smithi120:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-26T20:49:40.832 INFO:teuthology.orchestra.run.smithi120.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T20:49:40.832 INFO:teuthology.orchestra.run.smithi120.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T20:49:40.833 INFO:teuthology.orchestra.run.smithi120.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T20:49:40.833 INFO:teuthology.orchestra.run.smithi120.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T20:49:40.833 INFO:teuthology.orchestra.run.smithi120.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T20:49:40.833 INFO:teuthology.orchestra.run.smithi120.stdout: = sunit=0 swidth=0 blks 2024-04-26T20:49:40.833 INFO:teuthology.orchestra.run.smithi120.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T20:49:40.833 INFO:teuthology.orchestra.run.smithi120.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T20:49:40.833 INFO:teuthology.orchestra.run.smithi120.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T20:49:40.833 INFO:teuthology.orchestra.run.smithi120.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T20:49:41.305 INFO:teuthology.orchestra.run.smithi120.stdout:Discarding blocks...Done. 2024-04-26T20:49:41.307 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi120.front.sepia.ceph.com -o noatime 2024-04-26T20:49:41.307 DEBUG:teuthology.orchestra.run.smithi120:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-26T20:49:41.388 DEBUG:teuthology.orchestra.run.smithi120:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-26T20:49:41.458 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-26T20:49:41.522 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T20:49:41.522 INFO:tasks.ceph:role: osd.6 2024-04-26T20:49:41.522 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi120.front.sepia.ceph.com 2024-04-26T20:49:41.523 DEBUG:teuthology.orchestra.run.smithi120:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-26T20:49:41.593 INFO:teuthology.orchestra.run.smithi120.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T20:49:41.594 INFO:teuthology.orchestra.run.smithi120.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T20:49:41.594 INFO:teuthology.orchestra.run.smithi120.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T20:49:41.594 INFO:teuthology.orchestra.run.smithi120.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T20:49:41.594 INFO:teuthology.orchestra.run.smithi120.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T20:49:41.594 INFO:teuthology.orchestra.run.smithi120.stdout: = sunit=0 swidth=0 blks 2024-04-26T20:49:41.594 INFO:teuthology.orchestra.run.smithi120.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T20:49:41.594 INFO:teuthology.orchestra.run.smithi120.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T20:49:41.594 INFO:teuthology.orchestra.run.smithi120.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T20:49:41.594 INFO:teuthology.orchestra.run.smithi120.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T20:49:42.101 INFO:teuthology.orchestra.run.smithi120.stdout:Discarding blocks...Done. 2024-04-26T20:49:42.103 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi120.front.sepia.ceph.com -o noatime 2024-04-26T20:49:42.103 DEBUG:teuthology.orchestra.run.smithi120:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-26T20:49:42.244 DEBUG:teuthology.orchestra.run.smithi120:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-26T20:49:42.273 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-26T20:49:42.336 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T20:49:42.337 INFO:tasks.ceph:role: osd.7 2024-04-26T20:49:42.337 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi120.front.sepia.ceph.com 2024-04-26T20:49:42.337 DEBUG:teuthology.orchestra.run.smithi120:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-26T20:49:42.408 INFO:teuthology.orchestra.run.smithi120.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T20:49:42.408 INFO:teuthology.orchestra.run.smithi120.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T20:49:42.408 INFO:teuthology.orchestra.run.smithi120.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T20:49:42.408 INFO:teuthology.orchestra.run.smithi120.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T20:49:42.408 INFO:teuthology.orchestra.run.smithi120.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T20:49:42.408 INFO:teuthology.orchestra.run.smithi120.stdout: = sunit=0 swidth=0 blks 2024-04-26T20:49:42.409 INFO:teuthology.orchestra.run.smithi120.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T20:49:42.409 INFO:teuthology.orchestra.run.smithi120.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T20:49:42.409 INFO:teuthology.orchestra.run.smithi120.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T20:49:42.409 INFO:teuthology.orchestra.run.smithi120.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T20:49:42.919 INFO:teuthology.orchestra.run.smithi120.stdout:Discarding blocks...Done. 2024-04-26T20:49:42.921 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi120.front.sepia.ceph.com -o noatime 2024-04-26T20:49:42.921 DEBUG:teuthology.orchestra.run.smithi120:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-26T20:49:43.040 DEBUG:teuthology.orchestra.run.smithi120:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-26T20:49:43.070 DEBUG:teuthology.orchestra.run.smithi120:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T20:49:43.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:49:43.174+0000 7fe03d1b6640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-26T20:49:43.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:49:43.174+0000 7fe03d1b6640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-26T20:49:43.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:49:43.174+0000 7fe03d1b6640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-26T20:49:43.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:49:43.175+0000 7fe03d1b6640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-26T20:49:43.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:49:43.175+0000 7fe03d1b6640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-26T20:49:43.176 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:49:43.175+0000 7fe03d1b6640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-26T20:49:46.021 DEBUG:teuthology.orchestra.run.smithi120:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-26T20:49:46.049 DEBUG:teuthology.orchestra.run.smithi120:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T20:49:46.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:49:46.136+0000 7f5eeee16640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-26T20:49:46.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:49:46.136+0000 7f5eeee16640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-26T20:49:46.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:49:46.136+0000 7f5eeee16640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-26T20:49:46.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:49:46.136+0000 7f5eeee16640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-26T20:49:46.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:49:46.136+0000 7f5eeee16640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-26T20:49:46.138 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:49:46.137+0000 7f5eeee16640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-26T20:49:48.991 DEBUG:teuthology.orchestra.run.smithi120:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-26T20:49:49.019 DEBUG:teuthology.orchestra.run.smithi120:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T20:49:49.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:49:49.107+0000 7f8f47623640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-26T20:49:49.108 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:49:49.107+0000 7f8f47623640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-26T20:49:49.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:49:49.107+0000 7f8f47623640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-26T20:49:49.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:49:49.107+0000 7f8f47623640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-26T20:49:49.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:49:49.107+0000 7f8f47623640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-26T20:49:49.109 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:49:49.108+0000 7f8f47623640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-26T20:49:51.901 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:49:51.905 DEBUG:teuthology.orchestra.run.smithi120:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:49:51.968 DEBUG:teuthology.orchestra.run.smithi120:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-26T20:49:52.032 DEBUG:teuthology.orchestra.run.smithi120:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T20:49:52.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:49:52.114+0000 7f3f5a410640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-26T20:49:52.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:49:52.114+0000 7f3f5a410640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-26T20:49:52.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:49:52.114+0000 7f3f5a410640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-26T20:49:52.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:49:52.114+0000 7f3f5a410640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-26T20:49:52.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:49:52.114+0000 7f3f5a410640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-26T20:49:52.116 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:49:52.115+0000 7f3f5a410640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-26T20:49:54.962 DEBUG:teuthology.orchestra.run.smithi120:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-26T20:49:55.029 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-26T20:49:55.030 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-26T20:49:55.030 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-26T20:49:55.058 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-26T20:49:55.058 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-26T20:49:55.125 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-26T20:49:55.125 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-26T20:49:55.191 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-26T20:49:55.191 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-26T20:49:55.258 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-26T20:49:55.258 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-26T20:49:55.324 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-26T20:49:55.324 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-26T20:49:55.390 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-26T20:49:55.390 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-26T20:49:55.455 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-26T20:49:55.456 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-26T20:49:55.484 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-26T20:49:55.484 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-26T20:49:55.550 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-26T20:49:55.551 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-26T20:49:55.616 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-26T20:49:55.616 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-26T20:49:55.682 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-26T20:49:55.683 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-26T20:49:55.747 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-26T20:49:55.747 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-26T20:49:55.811 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-26T20:49:55.811 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-26T20:49:55.874 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-26T20:49:55.875 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-26T20:49:55.893 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-26T20:49:55.893 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-26T20:49:55.951 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-26T20:49:55.952 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-26T20:49:55.966 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-26T20:49:55.967 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-26T20:49:56.021 INFO:tasks.ceph:Adding keys to all mons... 2024-04-26T20:49:56.021 DEBUG:teuthology.orchestra.run.smithi073:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-26T20:49:56.024 DEBUG:teuthology.orchestra.run.smithi120:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-26T20:49:56.065 INFO:teuthology.orchestra.run.smithi073.stdout:[mgr.y] 2024-04-26T20:49:56.065 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQBUEyxmZnhRFBAAB2V6HQEbpoUNUbAflZok8A== 2024-04-26T20:49:56.065 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.a] 2024-04-26T20:49:56.065 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQBUEyxmAE6mGhAAe52HRhOCVM94+tYM6+D8CQ== 2024-04-26T20:49:56.065 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.c] 2024-04-26T20:49:56.065 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQBUEyxmpyRtIRAAfZQyLswSVyCCJ/G/i2R+XQ== 2024-04-26T20:49:56.066 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.0] 2024-04-26T20:49:56.066 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQBYEyxmGSyTCRAAHyan+CyQu5SN2n3m7nvHKg== 2024-04-26T20:49:56.066 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.1] 2024-04-26T20:49:56.066 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQBbEyxmD2XdBxAA80mAzCXbmZGi80w3Dx6CHg== 2024-04-26T20:49:56.066 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.2] 2024-04-26T20:49:56.066 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQBeEyxmxfWzBRAAufFGKiy5pPW++6QRW3PHgQ== 2024-04-26T20:49:56.066 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.3] 2024-04-26T20:49:56.066 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQBhEyxmT1dmAxAANZmJAcTSTsZUotOhTQiTKA== 2024-04-26T20:49:56.066 INFO:teuthology.orchestra.run.smithi073.stdout:[mgr.x] 2024-04-26T20:49:56.066 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQBUEyxmLbWjFxAAo2evZCpMUZUHkU6N+eDa4A== 2024-04-26T20:49:56.066 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.b] 2024-04-26T20:49:56.066 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQBUEyxmS7XbJRAAqlMo8v1BqQcHUrDIScQh/Q== 2024-04-26T20:49:56.066 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.d] 2024-04-26T20:49:56.067 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQBUEyxmzMqgLBAAF33EqesRSqAWg392qc9ZIQ== 2024-04-26T20:49:56.067 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.4] 2024-04-26T20:49:56.067 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQBnEyxmjwN8ChAA4uHxtDWrxi+NX5t0VCNGag== 2024-04-26T20:49:56.067 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.5] 2024-04-26T20:49:56.067 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQBqEyxmnCEwCBAAGsOlPpl0/bxbbcDlxufpBw== 2024-04-26T20:49:56.067 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.6] 2024-04-26T20:49:56.067 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQBtEyxmRmZ5BhAAG/JdOd2r48xb8wHeXJNK0A== 2024-04-26T20:49:56.067 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.7] 2024-04-26T20:49:56.067 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQBwEyxmtK3mBhAAEe5+8WbXSYIWx2YstWwE6g== 2024-04-26T20:49:56.067 INFO:teuthology.orchestra.run.smithi073.stdout:[client.0] 2024-04-26T20:49:56.067 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQBUEyxmch+BMBAAap3kvW3+/FlCuAkhRtbWxQ== 2024-04-26T20:49:56.067 INFO:teuthology.orchestra.run.smithi073.stdout:[client.2] 2024-04-26T20:49:56.067 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQBUEyxmHJ7fNRAAbjIeTP+R5KsjQ5IYlmpqFg== 2024-04-26T20:49:56.067 INFO:teuthology.orchestra.run.smithi073.stdout:[client.1] 2024-04-26T20:49:56.067 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQBUEyxmg2vgOBAAPAq4hzJxsRywJoOawl7r7g== 2024-04-26T20:49:56.068 INFO:teuthology.orchestra.run.smithi073.stdout:[client.3] 2024-04-26T20:49:56.068 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQBVEyxmt+GfAhAA5qvKp2+NlXcGj975bUQS9Q== 2024-04-26T20:49:56.083 INFO:teuthology.orchestra.run.smithi120.stdout:[mgr.y] 2024-04-26T20:49:56.084 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQBUEyxmZnhRFBAAB2V6HQEbpoUNUbAflZok8A== 2024-04-26T20:49:56.084 INFO:teuthology.orchestra.run.smithi120.stdout:[mds.a] 2024-04-26T20:49:56.084 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQBUEyxmAE6mGhAAe52HRhOCVM94+tYM6+D8CQ== 2024-04-26T20:49:56.084 INFO:teuthology.orchestra.run.smithi120.stdout:[mds.c] 2024-04-26T20:49:56.084 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQBUEyxmpyRtIRAAfZQyLswSVyCCJ/G/i2R+XQ== 2024-04-26T20:49:56.084 INFO:teuthology.orchestra.run.smithi120.stdout:[osd.0] 2024-04-26T20:49:56.084 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQBYEyxmGSyTCRAAHyan+CyQu5SN2n3m7nvHKg== 2024-04-26T20:49:56.084 INFO:teuthology.orchestra.run.smithi120.stdout:[osd.1] 2024-04-26T20:49:56.084 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQBbEyxmD2XdBxAA80mAzCXbmZGi80w3Dx6CHg== 2024-04-26T20:49:56.084 INFO:teuthology.orchestra.run.smithi120.stdout:[osd.2] 2024-04-26T20:49:56.084 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQBeEyxmxfWzBRAAufFGKiy5pPW++6QRW3PHgQ== 2024-04-26T20:49:56.084 INFO:teuthology.orchestra.run.smithi120.stdout:[osd.3] 2024-04-26T20:49:56.085 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQBhEyxmT1dmAxAANZmJAcTSTsZUotOhTQiTKA== 2024-04-26T20:49:56.085 INFO:teuthology.orchestra.run.smithi120.stdout:[mgr.x] 2024-04-26T20:49:56.085 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQBUEyxmLbWjFxAAo2evZCpMUZUHkU6N+eDa4A== 2024-04-26T20:49:56.085 INFO:teuthology.orchestra.run.smithi120.stdout:[mds.b] 2024-04-26T20:49:56.085 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQBUEyxmS7XbJRAAqlMo8v1BqQcHUrDIScQh/Q== 2024-04-26T20:49:56.085 INFO:teuthology.orchestra.run.smithi120.stdout:[mds.d] 2024-04-26T20:49:56.085 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQBUEyxmzMqgLBAAF33EqesRSqAWg392qc9ZIQ== 2024-04-26T20:49:56.085 INFO:teuthology.orchestra.run.smithi120.stdout:[osd.4] 2024-04-26T20:49:56.085 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQBnEyxmjwN8ChAA4uHxtDWrxi+NX5t0VCNGag== 2024-04-26T20:49:56.085 INFO:teuthology.orchestra.run.smithi120.stdout:[osd.5] 2024-04-26T20:49:56.085 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQBqEyxmnCEwCBAAGsOlPpl0/bxbbcDlxufpBw== 2024-04-26T20:49:56.085 INFO:teuthology.orchestra.run.smithi120.stdout:[osd.6] 2024-04-26T20:49:56.085 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQBtEyxmRmZ5BhAAG/JdOd2r48xb8wHeXJNK0A== 2024-04-26T20:49:56.085 INFO:teuthology.orchestra.run.smithi120.stdout:[osd.7] 2024-04-26T20:49:56.085 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQBwEyxmtK3mBhAAEe5+8WbXSYIWx2YstWwE6g== 2024-04-26T20:49:56.086 INFO:teuthology.orchestra.run.smithi120.stdout:[client.0] 2024-04-26T20:49:56.086 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQBUEyxmch+BMBAAap3kvW3+/FlCuAkhRtbWxQ== 2024-04-26T20:49:56.086 INFO:teuthology.orchestra.run.smithi120.stdout:[client.2] 2024-04-26T20:49:56.086 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQBUEyxmHJ7fNRAAbjIeTP+R5KsjQ5IYlmpqFg== 2024-04-26T20:49:56.086 INFO:teuthology.orchestra.run.smithi120.stdout:[client.1] 2024-04-26T20:49:56.086 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQBUEyxmg2vgOBAAPAq4hzJxsRywJoOawl7r7g== 2024-04-26T20:49:56.086 INFO:teuthology.orchestra.run.smithi120.stdout:[client.3] 2024-04-26T20:49:56.086 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQBVEyxmt+GfAhAA5qvKp2+NlXcGj975bUQS9Q== 2024-04-26T20:49:56.087 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T20:49:56.111 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T20:49:56.176 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T20:49:56.178 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T20:49:56.230 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T20:49:56.233 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T20:49:56.284 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:49:56.287 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:49:56.335 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:49:56.337 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:49:56.385 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:49:56.387 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:49:56.435 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:49:56.437 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:49:56.486 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T20:49:56.489 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T20:49:56.537 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T20:49:56.540 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T20:49:56.591 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T20:49:56.593 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T20:49:56.644 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:49:56.647 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:49:56.695 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:49:56.698 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:49:56.746 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:49:56.749 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:49:56.796 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:49:56.799 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:49:56.847 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T20:49:56.849 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T20:49:56.899 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T20:49:56.902 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T20:49:56.953 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T20:49:56.955 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T20:49:57.006 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T20:49:57.008 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T20:49:57.059 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-26T20:49:57.060 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-26T20:49:57.083 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-26T20:49:57.177 DEBUG:teuthology.orchestra.run.smithi073:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-26T20:49:57.201 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-26T20:49:57.224 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-26T20:49:57.319 DEBUG:teuthology.orchestra.run.smithi120:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-26T20:49:57.342 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-26T20:49:57.407 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-26T20:49:57.500 DEBUG:teuthology.orchestra.run.smithi120:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-26T20:49:57.522 DEBUG:teuthology.orchestra.run.smithi073:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-26T20:49:57.525 DEBUG:teuthology.orchestra.run.smithi120:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-26T20:49:57.577 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-26T20:49:57.578 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-26T20:49:57.578 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-26T20:49:57.580 INFO:tasks.ceph.mon.a:Started 2024-04-26T20:49:57.580 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-26T20:49:57.581 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-26T20:49:57.621 INFO:tasks.ceph.mon.b:Started 2024-04-26T20:49:57.621 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-26T20:49:57.621 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-26T20:49:57.623 INFO:tasks.ceph.mon.c:Started 2024-04-26T20:49:57.623 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-26T20:49:57.624 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-26T20:49:57.624 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-26T20:49:57.626 INFO:tasks.ceph.mgr.y:Started 2024-04-26T20:49:57.626 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-26T20:49:57.626 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-26T20:49:57.628 INFO:tasks.ceph.mgr.x:Started 2024-04-26T20:49:57.628 INFO:tasks.ceph:Setting crush tunables to default 2024-04-26T20:49:57.628 DEBUG:teuthology.orchestra.run.smithi073:> sudo ceph --cluster ceph osd crush tunables default 2024-04-26T20:49:57.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:49:57.724+0000 7f31ca978640 1 Processor -- start 2024-04-26T20:49:57.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:49:57.724+0000 7f31ca978640 1 -- start start 2024-04-26T20:49:57.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:49:57.724+0000 7f31ca978640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f31c40b3860 0x7f31c416a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:49:57.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:49:57.724+0000 7f31ca978640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31c40d0ba0 con 0x7f31c40d06f0 2024-04-26T20:49:57.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:49:57.724+0000 7f31ca978640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31c40d2590 con 0x7f31c415f3a0 2024-04-26T20:49:57.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:49:57.724+0000 7f31ca978640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f31c40d0e20 con 0x7f31c40b3860 2024-04-26T20:49:57.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:49:57.724+0000 7f31c37fe640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f31c40d06f0 0x7f31c40d1e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:34078/0 (socket says 172.21.15.73:34078) 2024-04-26T20:49:57.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:49:57.724+0000 7f31c37fe640 1 -- 172.21.15.73:0/1367817279 learned_addr learned my addr 172.21.15.73:0/1367817279 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:49:57.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:49:57.725+0000 7f31c8f15640 1 --2- 172.21.15.73:0/1367817279 >> v2:172.21.15.120:3300/0 conn(0x7f31c40b3860 0x7f31c416a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:49:57.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:49:57.725+0000 7f31c8f15640 1 -- 172.21.15.73:0/1367817279 >> v2:172.21.15.120:3300/0 conn(0x7f31c40b3860 msgr2=0x7f31c416a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-26T20:49:57.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:49:57.725+0000 7f31c8f15640 1 -- 172.21.15.73:0/1367817279 >> v2:172.21.15.120:3300/0 conn(0x7f31c40b3860 msgr2=0x7f31c416a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T20:49:57.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:49:57.725+0000 7f31c8f15640 1 --2- 172.21.15.73:0/1367817279 >> v2:172.21.15.120:3300/0 conn(0x7f31c40b3860 0x7f31c416a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T20:49:57.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:49:57.725+0000 7f31c8f15640 1 --2- 172.21.15.73:0/1367817279 >> v2:172.21.15.120:3300/0 conn(0x7f31c40b3860 0x7f31c416a000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T20:49:57.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:49:57.926+0000 7f31c8f15640 1 --2- 172.21.15.73:0/1367817279 >> v2:172.21.15.120:3300/0 conn(0x7f31c40b3860 0x7f31c416a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:49:57.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:49:57.926+0000 7f31c8f15640 1 -- 172.21.15.73:0/1367817279 >> v2:172.21.15.120:3300/0 conn(0x7f31c40b3860 msgr2=0x7f31c416a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-26T20:49:57.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:49:57.926+0000 7f31c8f15640 1 -- 172.21.15.73:0/1367817279 >> v2:172.21.15.120:3300/0 conn(0x7f31c40b3860 msgr2=0x7f31c416a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T20:49:57.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:49:57.926+0000 7f31c8f15640 1 --2- 172.21.15.73:0/1367817279 >> v2:172.21.15.120:3300/0 conn(0x7f31c40b3860 0x7f31c416a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T20:49:57.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:49:57.926+0000 7f31c8f15640 1 --2- 172.21.15.73:0/1367817279 >> v2:172.21.15.120:3300/0 conn(0x7f31c40b3860 0x7f31c416a000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T20:49:58.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:49:58.327+0000 7f31c8f15640 1 --2- 172.21.15.73:0/1367817279 >> v2:172.21.15.120:3300/0 conn(0x7f31c40b3860 0x7f31c416a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:49:58.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:49:58.327+0000 7f31c8f15640 1 -- 172.21.15.73:0/1367817279 >> v2:172.21.15.120:3300/0 conn(0x7f31c40b3860 msgr2=0x7f31c416a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-26T20:49:58.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:49:58.327+0000 7f31c8f15640 1 -- 172.21.15.73:0/1367817279 >> v2:172.21.15.120:3300/0 conn(0x7f31c40b3860 msgr2=0x7f31c416a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T20:49:58.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:49:58.327+0000 7f31c8f15640 1 --2- 172.21.15.73:0/1367817279 >> v2:172.21.15.120:3300/0 conn(0x7f31c40b3860 0x7f31c416a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T20:49:58.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:49:58.327+0000 7f31c8f15640 1 --2- 172.21.15.73:0/1367817279 >> v2:172.21.15.120:3300/0 conn(0x7f31c40b3860 0x7f31c416a000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-26T20:49:59.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:49:59.128+0000 7f31c8f15640 1 --2- 172.21.15.73:0/1367817279 >> v2:172.21.15.120:3300/0 conn(0x7f31c40b3860 0x7f31c416a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:49:59.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:49:59.128+0000 7f31c8f15640 1 -- 172.21.15.73:0/1367817279 >> v2:172.21.15.120:3300/0 conn(0x7f31c40b3860 msgr2=0x7f31c416a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-26T20:49:59.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:49:59.128+0000 7f31c8f15640 1 -- 172.21.15.73:0/1367817279 >> v2:172.21.15.120:3300/0 conn(0x7f31c40b3860 msgr2=0x7f31c416a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T20:49:59.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:49:59.128+0000 7f31c8f15640 1 --2- 172.21.15.73:0/1367817279 >> v2:172.21.15.120:3300/0 conn(0x7f31c40b3860 0x7f31c416a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T20:49:59.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:49:59.128+0000 7f31c8f15640 1 --2- 172.21.15.73:0/1367817279 >> v2:172.21.15.120:3300/0 conn(0x7f31c40b3860 0x7f31c416a000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-26T20:50:00.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:00.725+0000 7f31c1ffb640 1 -- 172.21.15.73:0/1367817279 >> v2:172.21.15.120:3300/0 conn(0x7f31c40b3860 msgr2=0x7f31c416a000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T20:50:00.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:00.725+0000 7f31c1ffb640 1 --2- 172.21.15.73:0/1367817279 >> v2:172.21.15.120:3300/0 conn(0x7f31c40b3860 0x7f31c416a000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:00.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:00.725+0000 7f31c1ffb640 1 -- 172.21.15.73:0/1367817279 >> v1:172.21.15.120:6789/0 conn(0x7f31c415f3a0 legacy=0x7f31c40d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:00.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:00.725+0000 7f31c1ffb640 1 -- 172.21.15.73:0/1367817279 >> v1:172.21.15.73:6789/0 conn(0x7f31c40d06f0 legacy=0x7f31c40d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:00.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:00.725+0000 7f31c1ffb640 1 --2- 172.21.15.73:0/1367817279 >> v2:172.21.15.73:3300/0 conn(0x7f31b00036e0 0x7f31b0005b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:00.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:00.725+0000 7f31c1ffb640 1 --2- 172.21.15.73:0/1367817279 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31b0006180 0x7f31b0008640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:00.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:00.725+0000 7f31c1ffb640 1 -- 172.21.15.73:0/1367817279 --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f31c40d0e20 con 0x7f31b00036e0 2024-04-26T20:50:00.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:00.725+0000 7f31c1ffb640 1 -- 172.21.15.73:0/1367817279 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f31b0008d70 con 0x7f31b0006180 2024-04-26T20:50:00.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:00.725+0000 7f31c3fff640 1 --2- 172.21.15.73:0/1367817279 >> v2:172.21.15.73:3300/0 conn(0x7f31b00036e0 0x7f31b0005b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:00.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:00.725+0000 7f31c3fff640 1 -- 172.21.15.73:0/1367817279 >> v2:172.21.15.73:3300/0 conn(0x7f31b00036e0 msgr2=0x7f31b0005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-26T20:50:00.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:00.725+0000 7f31c3fff640 1 -- 172.21.15.73:0/1367817279 >> v2:172.21.15.73:3300/0 conn(0x7f31b00036e0 msgr2=0x7f31b0005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T20:50:00.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:00.725+0000 7f31c3fff640 1 --2- 172.21.15.73:0/1367817279 >> v2:172.21.15.73:3300/0 conn(0x7f31b00036e0 0x7f31b0005b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T20:50:00.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:00.725+0000 7f31c3fff640 1 --2- 172.21.15.73:0/1367817279 >> v2:172.21.15.73:3300/0 conn(0x7f31b00036e0 0x7f31b0005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T20:50:00.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:00.725+0000 7f31c37fe640 1 --2- 172.21.15.73:0/1367817279 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31b0006180 0x7f31b0008640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:00.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:00.725+0000 7f31c37fe640 1 -- 172.21.15.73:0/1367817279 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31b0006180 msgr2=0x7f31b0008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-26T20:50:00.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:00.725+0000 7f31c37fe640 1 -- 172.21.15.73:0/1367817279 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31b0006180 msgr2=0x7f31b0008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T20:50:00.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:00.725+0000 7f31c37fe640 1 --2- 172.21.15.73:0/1367817279 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31b0006180 0x7f31b0008640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T20:50:00.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:00.725+0000 7f31c37fe640 1 --2- 172.21.15.73:0/1367817279 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31b0006180 0x7f31b0008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T20:50:00.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:00.926+0000 7f31c3fff640 1 --2- 172.21.15.73:0/1367817279 >> v2:172.21.15.73:3300/0 conn(0x7f31b00036e0 0x7f31b0005b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:00.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:00.926+0000 7f31c3fff640 1 -- 172.21.15.73:0/1367817279 >> v2:172.21.15.73:3300/0 conn(0x7f31b00036e0 msgr2=0x7f31b0005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-26T20:50:00.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:00.926+0000 7f31c3fff640 1 -- 172.21.15.73:0/1367817279 >> v2:172.21.15.73:3300/0 conn(0x7f31b00036e0 msgr2=0x7f31b0005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T20:50:00.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:00.926+0000 7f31c3fff640 1 --2- 172.21.15.73:0/1367817279 >> v2:172.21.15.73:3300/0 conn(0x7f31b00036e0 0x7f31b0005b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T20:50:00.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:00.926+0000 7f31c3fff640 1 --2- 172.21.15.73:0/1367817279 >> v2:172.21.15.73:3300/0 conn(0x7f31b00036e0 0x7f31b0005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T20:50:00.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:00.926+0000 7f31c37fe640 1 --2- 172.21.15.73:0/1367817279 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31b0006180 0x7f31b0008640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:00.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:00.926+0000 7f31c37fe640 1 -- 172.21.15.73:0/1367817279 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31b0006180 msgr2=0x7f31b0008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-26T20:50:00.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:00.926+0000 7f31c37fe640 1 -- 172.21.15.73:0/1367817279 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31b0006180 msgr2=0x7f31b0008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T20:50:00.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:00.926+0000 7f31c37fe640 1 --2- 172.21.15.73:0/1367817279 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31b0006180 0x7f31b0008640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T20:50:00.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:00.926+0000 7f31c37fe640 1 --2- 172.21.15.73:0/1367817279 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31b0006180 0x7f31b0008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T20:50:01.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:01.327+0000 7f31c3fff640 1 --2- 172.21.15.73:0/1367817279 >> v2:172.21.15.73:3300/0 conn(0x7f31b00036e0 0x7f31b0005b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:01.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:01.327+0000 7f31c3fff640 1 -- 172.21.15.73:0/1367817279 >> v2:172.21.15.73:3300/0 conn(0x7f31b00036e0 msgr2=0x7f31b0005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-26T20:50:01.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:01.327+0000 7f31c3fff640 1 -- 172.21.15.73:0/1367817279 >> v2:172.21.15.73:3300/0 conn(0x7f31b00036e0 msgr2=0x7f31b0005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T20:50:01.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:01.327+0000 7f31c3fff640 1 --2- 172.21.15.73:0/1367817279 >> v2:172.21.15.73:3300/0 conn(0x7f31b00036e0 0x7f31b0005b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T20:50:01.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:01.327+0000 7f31c3fff640 1 --2- 172.21.15.73:0/1367817279 >> v2:172.21.15.73:3300/0 conn(0x7f31b00036e0 0x7f31b0005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-26T20:50:01.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:01.327+0000 7f31c37fe640 1 --2- 172.21.15.73:0/1367817279 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31b0006180 0x7f31b0008640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:01.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:01.327+0000 7f31c37fe640 1 -- 172.21.15.73:0/1367817279 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31b0006180 msgr2=0x7f31b0008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-26T20:50:01.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:01.327+0000 7f31c37fe640 1 -- 172.21.15.73:0/1367817279 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31b0006180 msgr2=0x7f31b0008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T20:50:01.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:01.327+0000 7f31c37fe640 1 --2- 172.21.15.73:0/1367817279 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31b0006180 0x7f31b0008640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T20:50:01.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:01.327+0000 7f31c37fe640 1 --2- 172.21.15.73:0/1367817279 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31b0006180 0x7f31b0008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-26T20:50:02.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:02.128+0000 7f31c3fff640 1 --2- 172.21.15.73:0/1367817279 >> v2:172.21.15.73:3300/0 conn(0x7f31b00036e0 0x7f31b0005b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:02.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:02.128+0000 7f31c3fff640 1 -- 172.21.15.73:0/1367817279 >> v2:172.21.15.73:3300/0 conn(0x7f31b00036e0 msgr2=0x7f31b0005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-26T20:50:02.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:02.128+0000 7f31c3fff640 1 -- 172.21.15.73:0/1367817279 >> v2:172.21.15.73:3300/0 conn(0x7f31b00036e0 msgr2=0x7f31b0005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T20:50:02.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:02.128+0000 7f31c3fff640 1 --2- 172.21.15.73:0/1367817279 >> v2:172.21.15.73:3300/0 conn(0x7f31b00036e0 0x7f31b0005b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T20:50:02.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:02.128+0000 7f31c3fff640 1 --2- 172.21.15.73:0/1367817279 >> v2:172.21.15.73:3300/0 conn(0x7f31b00036e0 0x7f31b0005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-26T20:50:02.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:02.128+0000 7f31c37fe640 1 --2- 172.21.15.73:0/1367817279 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31b0006180 0x7f31b0008640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:02.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:02.128+0000 7f31c37fe640 1 -- 172.21.15.73:0/1367817279 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31b0006180 msgr2=0x7f31b0008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-26T20:50:02.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:02.128+0000 7f31c37fe640 1 -- 172.21.15.73:0/1367817279 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31b0006180 msgr2=0x7f31b0008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T20:50:02.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:02.128+0000 7f31c37fe640 1 --2- 172.21.15.73:0/1367817279 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31b0006180 0x7f31b0008640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T20:50:02.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:02.128+0000 7f31c37fe640 1 --2- 172.21.15.73:0/1367817279 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31b0006180 0x7f31b0008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-26T20:50:03.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:03.725+0000 7f31c1ffb640 1 -- 172.21.15.73:0/1367817279 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31b0006180 msgr2=0x7f31b0008640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T20:50:03.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:03.725+0000 7f31c1ffb640 1 --2- 172.21.15.73:0/1367817279 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31b0006180 0x7f31b0008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:03.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:03.725+0000 7f31c1ffb640 1 -- 172.21.15.73:0/1367817279 >> v2:172.21.15.73:3300/0 conn(0x7f31b00036e0 msgr2=0x7f31b0005b60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T20:50:03.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:03.725+0000 7f31c1ffb640 1 --2- 172.21.15.73:0/1367817279 >> v2:172.21.15.73:3300/0 conn(0x7f31b00036e0 0x7f31b0005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:03.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:03.725+0000 7f31c8f15640 1 -- 172.21.15.73:0/1367817279 reap_dead start 2024-04-26T20:50:03.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:03.725+0000 7f31c1ffb640 1 --2- 172.21.15.73:0/1367817279 >> v2:172.21.15.73:3300/0 conn(0x7f31b00091b0 0x7f31b000b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:03.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:03.725+0000 7f31c1ffb640 1 --2- 172.21.15.73:0/1367817279 >> v2:172.21.15.120:3300/0 conn(0x7f31b0005b60 0x7f31b0008640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:03.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:03.725+0000 7f31c1ffb640 1 -- 172.21.15.73:0/1367817279 --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31c40b1d70 con 0x7f31c415f3a0 2024-04-26T20:50:03.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:03.725+0000 7f31c1ffb640 1 -- 172.21.15.73:0/1367817279 --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f31c40d0e20 con 0x7f31b00091b0 2024-04-26T20:50:03.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:03.725+0000 7f31c1ffb640 1 -- 172.21.15.73:0/1367817279 --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f31b0008d70 con 0x7f31b0005b60 2024-04-26T20:50:03.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:03.725+0000 7f31c3fff640 1 --2- 172.21.15.73:0/1367817279 >> v2:172.21.15.73:3300/0 conn(0x7f31b00091b0 0x7f31b000b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:03.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:03.725+0000 7f31c3fff640 1 -- 172.21.15.73:0/1367817279 >> v2:172.21.15.73:3300/0 conn(0x7f31b00091b0 msgr2=0x7f31b000b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-26T20:50:03.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:03.725+0000 7f31c3fff640 1 -- 172.21.15.73:0/1367817279 >> v2:172.21.15.73:3300/0 conn(0x7f31b00091b0 msgr2=0x7f31b000b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T20:50:03.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:03.725+0000 7f31c3fff640 1 --2- 172.21.15.73:0/1367817279 >> v2:172.21.15.73:3300/0 conn(0x7f31b00091b0 0x7f31b000b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T20:50:03.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:03.725+0000 7f31c3fff640 1 --2- 172.21.15.73:0/1367817279 >> v2:172.21.15.73:3300/0 conn(0x7f31b00091b0 0x7f31b000b5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T20:50:03.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:03.725+0000 7f31c8f15640 1 --2- 172.21.15.73:0/1367817279 >> v2:172.21.15.120:3300/0 conn(0x7f31b0005b60 0x7f31b0008640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:03.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:03.725+0000 7f31c8f15640 1 -- 172.21.15.73:0/1367817279 >> v2:172.21.15.120:3300/0 conn(0x7f31b0005b60 msgr2=0x7f31b0008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-26T20:50:03.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:03.725+0000 7f31c8f15640 1 -- 172.21.15.73:0/1367817279 >> v2:172.21.15.120:3300/0 conn(0x7f31b0005b60 msgr2=0x7f31b0008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T20:50:03.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:03.725+0000 7f31c8f15640 1 --2- 172.21.15.73:0/1367817279 >> v2:172.21.15.120:3300/0 conn(0x7f31b0005b60 0x7f31b0008640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T20:50:03.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:03.725+0000 7f31c8f15640 1 --2- 172.21.15.73:0/1367817279 >> v2:172.21.15.120:3300/0 conn(0x7f31b0005b60 0x7f31b0008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T20:50:03.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:03.926+0000 7f31c3fff640 1 --2- 172.21.15.73:0/1367817279 >> v2:172.21.15.73:3300/0 conn(0x7f31b00091b0 0x7f31b000b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:03.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:03.926+0000 7f31c3fff640 1 -- 172.21.15.73:0/1367817279 >> v2:172.21.15.73:3300/0 conn(0x7f31b00091b0 msgr2=0x7f31b000b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-26T20:50:03.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:03.926+0000 7f31c3fff640 1 -- 172.21.15.73:0/1367817279 >> v2:172.21.15.73:3300/0 conn(0x7f31b00091b0 msgr2=0x7f31b000b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T20:50:03.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:03.926+0000 7f31c3fff640 1 --2- 172.21.15.73:0/1367817279 >> v2:172.21.15.73:3300/0 conn(0x7f31b00091b0 0x7f31b000b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T20:50:03.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:03.926+0000 7f31c3fff640 1 --2- 172.21.15.73:0/1367817279 >> v2:172.21.15.73:3300/0 conn(0x7f31b00091b0 0x7f31b000b5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T20:50:03.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:03.926+0000 7f31c8f15640 1 --2- 172.21.15.73:0/1367817279 >> v2:172.21.15.120:3300/0 conn(0x7f31b0005b60 0x7f31b0008640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:03.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:03.926+0000 7f31c8f15640 1 -- 172.21.15.73:0/1367817279 >> v2:172.21.15.120:3300/0 conn(0x7f31b0005b60 msgr2=0x7f31b0008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-26T20:50:03.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:03.926+0000 7f31c8f15640 1 -- 172.21.15.73:0/1367817279 >> v2:172.21.15.120:3300/0 conn(0x7f31b0005b60 msgr2=0x7f31b0008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T20:50:03.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:03.926+0000 7f31c8f15640 1 --2- 172.21.15.73:0/1367817279 >> v2:172.21.15.120:3300/0 conn(0x7f31b0005b60 0x7f31b0008640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T20:50:03.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:03.926+0000 7f31c8f15640 1 --2- 172.21.15.73:0/1367817279 >> v2:172.21.15.120:3300/0 conn(0x7f31b0005b60 0x7f31b0008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T20:50:04.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:04.327+0000 7f31c3fff640 1 --2- 172.21.15.73:0/1367817279 >> v2:172.21.15.73:3300/0 conn(0x7f31b00091b0 0x7f31b000b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:04.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:04.327+0000 7f31c3fff640 1 -- 172.21.15.73:0/1367817279 >> v2:172.21.15.73:3300/0 conn(0x7f31b00091b0 msgr2=0x7f31b000b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-26T20:50:04.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:04.327+0000 7f31c3fff640 1 -- 172.21.15.73:0/1367817279 >> v2:172.21.15.73:3300/0 conn(0x7f31b00091b0 msgr2=0x7f31b000b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T20:50:04.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:04.327+0000 7f31c3fff640 1 --2- 172.21.15.73:0/1367817279 >> v2:172.21.15.73:3300/0 conn(0x7f31b00091b0 0x7f31b000b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T20:50:04.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:04.327+0000 7f31c3fff640 1 --2- 172.21.15.73:0/1367817279 >> v2:172.21.15.73:3300/0 conn(0x7f31b00091b0 0x7f31b000b5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-26T20:50:04.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:04.327+0000 7f31c8f15640 1 --2- 172.21.15.73:0/1367817279 >> v2:172.21.15.120:3300/0 conn(0x7f31b0005b60 0x7f31b0008640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:04.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:04.327+0000 7f31c8f15640 1 -- 172.21.15.73:0/1367817279 >> v2:172.21.15.120:3300/0 conn(0x7f31b0005b60 msgr2=0x7f31b0008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-26T20:50:04.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:04.327+0000 7f31c8f15640 1 -- 172.21.15.73:0/1367817279 >> v2:172.21.15.120:3300/0 conn(0x7f31b0005b60 msgr2=0x7f31b0008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T20:50:04.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:04.327+0000 7f31c8f15640 1 --2- 172.21.15.73:0/1367817279 >> v2:172.21.15.120:3300/0 conn(0x7f31b0005b60 0x7f31b0008640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T20:50:04.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:04.327+0000 7f31c8f15640 1 --2- 172.21.15.73:0/1367817279 >> v2:172.21.15.120:3300/0 conn(0x7f31b0005b60 0x7f31b0008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-26T20:50:05.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:05.128+0000 7f31c3fff640 1 --2- 172.21.15.73:0/1367817279 >> v2:172.21.15.73:3300/0 conn(0x7f31b00091b0 0x7f31b000b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:05.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:05.128+0000 7f31c3fff640 1 -- 172.21.15.73:0/1367817279 >> v2:172.21.15.73:3300/0 conn(0x7f31b00091b0 msgr2=0x7f31b000b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-26T20:50:05.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:05.128+0000 7f31c3fff640 1 -- 172.21.15.73:0/1367817279 >> v2:172.21.15.73:3300/0 conn(0x7f31b00091b0 msgr2=0x7f31b000b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T20:50:05.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:05.128+0000 7f31c3fff640 1 --2- 172.21.15.73:0/1367817279 >> v2:172.21.15.73:3300/0 conn(0x7f31b00091b0 0x7f31b000b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T20:50:05.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:05.128+0000 7f31c3fff640 1 --2- 172.21.15.73:0/1367817279 >> v2:172.21.15.73:3300/0 conn(0x7f31b00091b0 0x7f31b000b5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-26T20:50:05.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:05.128+0000 7f31c8f15640 1 --2- 172.21.15.73:0/1367817279 >> v2:172.21.15.120:3300/0 conn(0x7f31b0005b60 0x7f31b0008640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:05.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:05.128+0000 7f31c8f15640 1 -- 172.21.15.73:0/1367817279 >> v2:172.21.15.120:3300/0 conn(0x7f31b0005b60 msgr2=0x7f31b0008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-26T20:50:05.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:05.128+0000 7f31c8f15640 1 -- 172.21.15.73:0/1367817279 >> v2:172.21.15.120:3300/0 conn(0x7f31b0005b60 msgr2=0x7f31b0008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T20:50:05.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:05.128+0000 7f31c8f15640 1 --2- 172.21.15.73:0/1367817279 >> v2:172.21.15.120:3300/0 conn(0x7f31b0005b60 0x7f31b0008640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T20:50:05.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:05.128+0000 7f31c8f15640 1 --2- 172.21.15.73:0/1367817279 >> v2:172.21.15.120:3300/0 conn(0x7f31b0005b60 0x7f31b0008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-26T20:50:06.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:06.725+0000 7f31c1ffb640 1 -- 172.21.15.73:0/1367817279 >> v2:172.21.15.120:3300/0 conn(0x7f31b0005b60 msgr2=0x7f31b0008640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T20:50:06.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:06.725+0000 7f31c1ffb640 1 --2- 172.21.15.73:0/1367817279 >> v2:172.21.15.120:3300/0 conn(0x7f31b0005b60 0x7f31b0008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:06.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:06.725+0000 7f31c1ffb640 1 -- 172.21.15.73:0/1367817279 >> v2:172.21.15.73:3300/0 conn(0x7f31b00091b0 msgr2=0x7f31b000b5e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T20:50:06.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:06.725+0000 7f31c1ffb640 1 --2- 172.21.15.73:0/1367817279 >> v2:172.21.15.73:3300/0 conn(0x7f31b00091b0 0x7f31b000b5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:06.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:06.725+0000 7f31c1ffb640 1 -- 172.21.15.73:0/1367817279 >> v1:172.21.15.120:6789/0 conn(0x7f31c415f3a0 legacy=0x7f31b000bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:06.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:06.725+0000 7f31c1ffb640 1 --2- 172.21.15.73:0/1367817279 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31b0011690 0x7f31b0013a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:06.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:06.725+0000 7f31c1ffb640 1 -- 172.21.15.73:0/1367817279 --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31b000e400 con 0x7f31b000d540 2024-04-26T20:50:06.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:06.725+0000 7f31c1ffb640 1 -- 172.21.15.73:0/1367817279 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f31c40d0e20 con 0x7f31b0011690 2024-04-26T20:50:06.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:06.726+0000 7f31c37fe640 1 --2- 172.21.15.73:0/1367817279 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31b0011690 0x7f31b0013a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:06.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:06.726+0000 7f31c37fe640 1 -- 172.21.15.73:0/1367817279 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31b0011690 msgr2=0x7f31b0013a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-26T20:50:06.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:06.726+0000 7f31c37fe640 1 -- 172.21.15.73:0/1367817279 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31b0011690 msgr2=0x7f31b0013a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T20:50:06.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:06.726+0000 7f31c37fe640 1 --2- 172.21.15.73:0/1367817279 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31b0011690 0x7f31b0013a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T20:50:06.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:06.726+0000 7f31c37fe640 1 --2- 172.21.15.73:0/1367817279 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31b0011690 0x7f31b0013a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T20:50:06.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:06.927+0000 7f31c37fe640 1 --2- 172.21.15.73:0/1367817279 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31b0011690 0x7f31b0013a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:06.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:06.927+0000 7f31c37fe640 1 -- 172.21.15.73:0/1367817279 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31b0011690 msgr2=0x7f31b0013a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-26T20:50:06.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:06.927+0000 7f31c37fe640 1 -- 172.21.15.73:0/1367817279 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31b0011690 msgr2=0x7f31b0013a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T20:50:06.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:06.927+0000 7f31c37fe640 1 --2- 172.21.15.73:0/1367817279 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31b0011690 0x7f31b0013a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T20:50:06.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:06.927+0000 7f31c37fe640 1 --2- 172.21.15.73:0/1367817279 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31b0011690 0x7f31b0013a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T20:50:07.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.328+0000 7f31c37fe640 1 --2- 172.21.15.73:0/1367817279 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31b0011690 0x7f31b0013a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:07.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.328+0000 7f31c37fe640 1 -- 172.21.15.73:0/1367817279 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31b0011690 msgr2=0x7f31b0013a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-26T20:50:07.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.328+0000 7f31c37fe640 1 -- 172.21.15.73:0/1367817279 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31b0011690 msgr2=0x7f31b0013a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T20:50:07.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.328+0000 7f31c37fe640 1 --2- 172.21.15.73:0/1367817279 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31b0011690 0x7f31b0013a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T20:50:07.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.328+0000 7f31c37fe640 1 --2- 172.21.15.73:0/1367817279 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31b0011690 0x7f31b0013a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-26T20:50:07.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.893+0000 7f31c2ffd640 1 -- 172.21.15.73:0/1367817279 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1014192600 0 0) 0x7f31b000e400 con 0x7f31b000d540 2024-04-26T20:50:07.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.893+0000 7f31c2ffd640 1 -- 172.21.15.73:0/1367817279 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f31ac003520 con 0x7f31b000d540 2024-04-26T20:50:07.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.894+0000 7f31c2ffd640 1 -- 172.21.15.73:0/1367817279 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4027682222 0 0) 0x7f31ac003520 con 0x7f31b000d540 2024-04-26T20:50:07.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.894+0000 7f31c2ffd640 1 -- 172.21.15.73:0/1367817279 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31b0011690 msgr2=0x7f31b0013a80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T20:50:07.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.894+0000 7f31c2ffd640 1 --2- 172.21.15.73:0/1367817279 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31b0011690 0x7f31b0013a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:07.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.894+0000 7f31c2ffd640 1 -- 172.21.15.73:0/1367817279 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31b0015000 con 0x7f31b000d540 2024-04-26T20:50:07.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.894+0000 7f31c2ffd640 1 -- 172.21.15.73:0/1367817279 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7f31b4007da0 con 0x7f31b000d540 2024-04-26T20:50:07.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.894+0000 7f31c2ffd640 1 -- 172.21.15.73:0/1367817279 >> v1:172.21.15.73:6789/0 conn(0x7f31b000d540 legacy=0x7f31b000d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:07.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.894+0000 7f31c8f15640 1 -- 172.21.15.73:0/1367817279 reap_dead start 2024-04-26T20:50:07.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.894+0000 7f31c2ffd640 1 --2- 172.21.15.73:0/1367817279 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31ac0039b0 0x7f31ac003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:07.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.894+0000 7f31c2ffd640 1 --2- 172.21.15.73:0/1367817279 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31b0011690 0x7f31ac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:07.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.894+0000 7f31c2ffd640 1 --2- 172.21.15.73:0/1367817279 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f31ac004d40 0x7f31ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:07.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.894+0000 7f31c2ffd640 1 -- 172.21.15.73:0/1367817279 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f31c40d0e20 con 0x7f31ac0039b0 2024-04-26T20:50:07.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.894+0000 7f31c2ffd640 1 -- 172.21.15.73:0/1367817279 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f31ac00bba0 con 0x7f31ac004d40 2024-04-26T20:50:07.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.894+0000 7f31c2ffd640 1 -- 172.21.15.73:0/1367817279 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f31ac00bda0 con 0x7f31b0011690 2024-04-26T20:50:07.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.894+0000 7f31c3fff640 1 --2- 172.21.15.73:0/1367817279 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31ac0039b0 0x7f31ac003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:07.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.894+0000 7f31c37fe640 1 --2- 172.21.15.73:0/1367817279 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31b0011690 0x7f31ac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:07.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.894+0000 7f31c8f15640 1 --2- 172.21.15.73:0/1367817279 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f31ac004d40 0x7f31ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:07.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.894+0000 7f31c3fff640 1 -- 172.21.15.73:0/1367817279 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31b0011690 msgr2=0x7f31ac004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:07.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.894+0000 7f31c3fff640 1 --2- 172.21.15.73:0/1367817279 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31b0011690 0x7f31ac004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:07.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.894+0000 7f31c3fff640 1 -- 172.21.15.73:0/1367817279 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f31ac004d40 msgr2=0x7f31ac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:07.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.894+0000 7f31c3fff640 1 --2- 172.21.15.73:0/1367817279 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f31ac004d40 0x7f31ac0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:07.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.895+0000 7f31c3fff640 1 -- 172.21.15.73:0/1367817279 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31ac003520 con 0x7f31ac0039b0 2024-04-26T20:50:07.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.895+0000 7f31c8f15640 1 --2- 172.21.15.73:0/1367817279 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f31ac004d40 0x7f31ac0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:50:07.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.895+0000 7f31c3fff640 1 --2- 172.21.15.73:0/1367817279 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31ac0039b0 0x7f31ac003db0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f31b00060d0 tx=0x7f31ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73ed65e27dc41a21 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:07.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.897+0000 7f31c2ffd640 1 -- 172.21.15.73:0/1367817279 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b4039070 con 0x7f31ac0039b0 2024-04-26T20:50:07.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.897+0000 7f31c2ffd640 1 -- 172.21.15.73:0/1367817279 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31b4034410 con 0x7f31ac0039b0 2024-04-26T20:50:07.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.897+0000 7f31c2ffd640 1 -- 172.21.15.73:0/1367817279 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b40035f0 con 0x7f31ac0039b0 2024-04-26T20:50:07.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.898+0000 7f31ca978640 1 -- 172.21.15.73:0/1367817279 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31ac0039b0 msgr2=0x7f31ac003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:07.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.898+0000 7f31ca978640 1 --2- 172.21.15.73:0/1367817279 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31ac0039b0 0x7f31ac003db0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f31b00060d0 tx=0x7f31ac005830 comp rx=0 tx=0).stop 2024-04-26T20:50:07.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.898+0000 7f31ca978640 1 -- 172.21.15.73:0/1367817279 shutdown_connections 2024-04-26T20:50:07.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.898+0000 7f31ca978640 1 --2- 172.21.15.73:0/1367817279 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f31ac004d40 0x7f31ac0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:07.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.898+0000 7f31ca978640 1 --2- 172.21.15.73:0/1367817279 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31b0011690 0x7f31ac004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:07.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.898+0000 7f31ca978640 1 --2- 172.21.15.73:0/1367817279 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31ac0039b0 0x7f31ac003db0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:07.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.898+0000 7f31ca978640 1 -- 172.21.15.73:0/1367817279 >> 172.21.15.73:0/1367817279 conn(0x7f31c40d4f90 msgr2=0x7f31c40d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:07.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.898+0000 7f31ca978640 1 -- 172.21.15.73:0/1367817279 shutdown_connections 2024-04-26T20:50:07.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.898+0000 7f31ca978640 1 -- 172.21.15.73:0/1367817279 wait complete. 2024-04-26T20:50:07.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.900+0000 7f31ca978640 1 Processor -- start 2024-04-26T20:50:07.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.901+0000 7f31ca978640 1 -- start start 2024-04-26T20:50:07.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.901+0000 7f31ca978640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31ac0039b0 0x7f31c4157670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:07.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.901+0000 7f31ca978640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31ac004d40 0x7f31c40a20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:07.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.901+0000 7f31ca978640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f31b0011690 0x7f31c40a04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:07.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.901+0000 7f31ca978640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f31c415f780 con 0x7f31ac0039b0 2024-04-26T20:50:07.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.901+0000 7f31ca978640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f31c40b2330 con 0x7f31b0011690 2024-04-26T20:50:07.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.901+0000 7f31ca978640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f31c415f900 con 0x7f31ac004d40 2024-04-26T20:50:07.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.901+0000 7f31c3fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31ac0039b0 0x7f31c4157670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:07.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.902+0000 7f31c3fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31ac0039b0 0x7f31c4157670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49112/0 (socket says 172.21.15.73:49112) 2024-04-26T20:50:07.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.902+0000 7f31c3fff640 1 -- 172.21.15.73:0/2629065771 learned_addr learned my addr 172.21.15.73:0/2629065771 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:07.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.902+0000 7f31c37fe640 1 --2- 172.21.15.73:0/2629065771 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31ac004d40 0x7f31c40a20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:07.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.902+0000 7f31c8f15640 1 --2- 172.21.15.73:0/2629065771 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f31b0011690 0x7f31c40a04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:07.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.902+0000 7f31c3fff640 1 -- 172.21.15.73:0/2629065771 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31ac004d40 msgr2=0x7f31c40a20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:07.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.902+0000 7f31c3fff640 1 --2- 172.21.15.73:0/2629065771 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31ac004d40 0x7f31c40a20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:07.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.902+0000 7f31c3fff640 1 -- 172.21.15.73:0/2629065771 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f31b0011690 msgr2=0x7f31c40a04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:07.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.902+0000 7f31c3fff640 1 --2- 172.21.15.73:0/2629065771 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f31b0011690 0x7f31c40a04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:07.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.902+0000 7f31c3fff640 1 -- 172.21.15.73:0/2629065771 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31c4157bb0 con 0x7f31ac0039b0 2024-04-26T20:50:07.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.902+0000 7f31c3fff640 1 --2- 172.21.15.73:0/2629065771 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31ac0039b0 0x7f31c4157670 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f31b4037040 tx=0x7f31b4001190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:07.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.902+0000 7f31c0ff9640 1 -- 172.21.15.73:0/2629065771 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b4039040 con 0x7f31ac0039b0 2024-04-26T20:50:07.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.902+0000 7f31c0ff9640 1 -- 172.21.15.73:0/2629065771 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31b4044070 con 0x7f31ac0039b0 2024-04-26T20:50:07.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.902+0000 7f31c0ff9640 1 -- 172.21.15.73:0/2629065771 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b40035f0 con 0x7f31ac0039b0 2024-04-26T20:50:07.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.902+0000 7f31ca978640 1 -- 172.21.15.73:0/2629065771 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31c40a1050 con 0x7f31ac0039b0 2024-04-26T20:50:07.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.902+0000 7f31ca978640 1 -- 172.21.15.73:0/2629065771 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31c40a1610 con 0x7f31ac0039b0 2024-04-26T20:50:07.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.903+0000 7f31c0ff9640 1 -- 172.21.15.73:0/2629065771 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f31b4013040 con 0x7f31ac0039b0 2024-04-26T20:50:07.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.903+0000 7f31c0ff9640 1 -- 172.21.15.73:0/2629065771 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f31b4049460 con 0x7f31ac0039b0 2024-04-26T20:50:07.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.903+0000 7f31ca978640 1 -- 172.21.15.73:0/2629065771 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31c40694f0 con 0x7f31ac0039b0 2024-04-26T20:50:07.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.906+0000 7f31c0ff9640 1 -- 172.21.15.73:0/2629065771 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f31c4157bb0 con 0x7f31ac0039b0 2024-04-26T20:50:07.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.974+0000 7f31ca978640 1 -- 172.21.15.73:0/2629065771 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f31c40b6860 con 0x7f31ac0039b0 2024-04-26T20:50:07.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.976+0000 7f31c0ff9640 1 -- 172.21.15.73:0/2629065771 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f31b403f660 con 0x7f31ac0039b0 2024-04-26T20:50:07.977 INFO:teuthology.orchestra.run.smithi073.stderr:adjusted tunables profile to default 2024-04-26T20:50:07.978 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.977+0000 7f31ca978640 1 -- 172.21.15.73:0/2629065771 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31ac0039b0 msgr2=0x7f31c4157670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:07.978 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.977+0000 7f31ca978640 1 --2- 172.21.15.73:0/2629065771 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31ac0039b0 0x7f31c4157670 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f31b4037040 tx=0x7f31b4001190 comp rx=0 tx=0).stop 2024-04-26T20:50:07.978 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.977+0000 7f31ca978640 1 -- 172.21.15.73:0/2629065771 shutdown_connections 2024-04-26T20:50:07.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.977+0000 7f31ca978640 1 --2- 172.21.15.73:0/2629065771 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f31b0011690 0x7f31c40a04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:07.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.977+0000 7f31ca978640 1 --2- 172.21.15.73:0/2629065771 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f31ac004d40 0x7f31c40a20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:07.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.977+0000 7f31ca978640 1 --2- 172.21.15.73:0/2629065771 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31ac0039b0 0x7f31c4157670 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:07.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.977+0000 7f31ca978640 1 -- 172.21.15.73:0/2629065771 >> 172.21.15.73:0/2629065771 conn(0x7f31c40d4f90 msgr2=0x7f31c406e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:07.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.977+0000 7f31ca978640 1 -- 172.21.15.73:0/2629065771 shutdown_connections 2024-04-26T20:50:07.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:07.977+0000 7f31ca978640 1 -- 172.21.15.73:0/2629065771 wait complete. 2024-04-26T20:50:07.993 INFO:tasks.ceph:check_enable_crimson: False 2024-04-26T20:50:07.993 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-26T20:50:07.994 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-26T20:50:07.995 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-26T20:50:08.008 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-04-26T20:50:08.008+0000 7f5a58a1b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T20:50:08.019 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-26T20:50:08.020 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-26T20:50:08.020 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-26T20:50:08.019+0000 7fba87221040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T20:50:08.045 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-26T20:50:08.045 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-26T20:50:08.108 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-26T20:50:08.109 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-26T20:50:08.151 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-04-26T20:50:08.151+0000 7f5a58a1b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T20:50:08.173 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-26T20:50:08.173 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-26T20:50:08.177 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-26T20:50:08.176+0000 7fba87221040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T20:50:08.199 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-26T20:50:08.199 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-26T20:50:08.206 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-04-26T20:50:08.206+0000 7f5a58a1b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T20:50:08.229 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-26T20:50:08.229 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-26T20:50:08.231 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-26T20:50:08.230+0000 7fba87221040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T20:50:08.292 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-26T20:50:08.292 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-26T20:50:08.356 DEBUG:teuthology.orchestra.run.smithi120:> sudo ceph --cluster ceph osd new 72d6f4f3-214a-4392-a02f-c33cbcab3c7a 0 2024-04-26T20:50:08.470 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-04-26T20:50:08.470+0000 7f5a58a1b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T20:50:08.474 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-26T20:50:08.473+0000 7fba87221040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T20:50:08.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.492+0000 7fd16d87e640 1 Processor -- start 2024-04-26T20:50:08.493 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.493+0000 7fd16d87e640 1 -- start start 2024-04-26T20:50:08.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.493+0000 7fd16d87e640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd1680b88a0 0x7fd1680d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:08.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.493+0000 7fd16d87e640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1680b8c80 con 0x7fd1680cfd50 2024-04-26T20:50:08.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.493+0000 7fd16d87e640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1680cd5b0 con 0x7fd1680b5c10 2024-04-26T20:50:08.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.493+0000 7fd16d87e640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fd1680d2370 con 0x7fd1680b88a0 2024-04-26T20:50:08.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.494+0000 7fd166ffd640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd1680b88a0 0x7fd1680d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:08.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.494+0000 7fd1667fc640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7fd1680b5c10 0x7fd1680cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:35022/0 (socket says 172.21.15.120:35022) 2024-04-26T20:50:08.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.494+0000 7fd1667fc640 1 -- 172.21.15.120:0/2633184134 learned_addr learned my addr 172.21.15.120:0/2633184134 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-04-26T20:50:08.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.494+0000 7fd165ffb640 1 -- 172.21.15.120:0/2633184134 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 944283624 0 0) 0x7fd1680cd5b0 con 0x7fd1680b5c10 2024-04-26T20:50:08.494 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.494+0000 7fd166ffd640 1 -- 172.21.15.120:0/2633184134 >> v1:172.21.15.120:6789/0 conn(0x7fd1680b5c10 legacy=0x7fd1680cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:08.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.494+0000 7fd166ffd640 1 -- 172.21.15.120:0/2633184134 >> v1:172.21.15.73:6789/0 conn(0x7fd1680cfd50 legacy=0x7fd1680ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:08.495 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.494+0000 7fd166ffd640 1 -- 172.21.15.120:0/2633184134 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1680cd890 con 0x7fd1680b88a0 2024-04-26T20:50:08.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.495+0000 7fd166ffd640 1 --2- 172.21.15.120:0/2633184134 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd1680b88a0 0x7fd1680d19f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd1500097d0 tx=0x7fd15002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4126e8be3e59ca59 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:08.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.495+0000 7fd165ffb640 1 -- 172.21.15.120:0/2633184134 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd150037070 con 0x7fd1680b88a0 2024-04-26T20:50:08.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.495+0000 7fd165ffb640 1 -- 172.21.15.120:0/2633184134 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd150032440 con 0x7fd1680b88a0 2024-04-26T20:50:08.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.495+0000 7fd165ffb640 1 -- 172.21.15.120:0/2633184134 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd150032810 con 0x7fd1680b88a0 2024-04-26T20:50:08.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.495+0000 7fd16d87e640 1 -- 172.21.15.120:0/2633184134 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd1680b88a0 msgr2=0x7fd1680d19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:08.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.495+0000 7fd16d87e640 1 --2- 172.21.15.120:0/2633184134 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd1680b88a0 0x7fd1680d19f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd1500097d0 tx=0x7fd15002a210 comp rx=0 tx=0).stop 2024-04-26T20:50:08.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.496+0000 7fd16d87e640 1 -- 172.21.15.120:0/2633184134 shutdown_connections 2024-04-26T20:50:08.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.496+0000 7fd16d87e640 1 --2- 172.21.15.120:0/2633184134 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd1680b88a0 0x7fd1680d19f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:08.496 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.496+0000 7fd16d87e640 1 -- 172.21.15.120:0/2633184134 >> 172.21.15.120:0/2633184134 conn(0x7fd1680d2dc0 msgr2=0x7fd1680d31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:08.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.496+0000 7fd16d87e640 1 -- 172.21.15.120:0/2633184134 shutdown_connections 2024-04-26T20:50:08.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.496+0000 7fd16d87e640 1 -- 172.21.15.120:0/2633184134 wait complete. 2024-04-26T20:50:08.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.497+0000 7fd16d87e640 1 Processor -- start 2024-04-26T20:50:08.497 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.497+0000 7fd16d87e640 1 -- start start 2024-04-26T20:50:08.499 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.498+0000 7fd16d87e640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd1680b5c10 0x7fd16815f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:08.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.498+0000 7fd16d87e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1680b88a0 0x7fd1680cbd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:08.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.498+0000 7fd16d87e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd1680cfd50 0x7fd1680c8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:08.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.498+0000 7fd16d87e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd1680d1f30 con 0x7fd1680cfd50 2024-04-26T20:50:08.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.498+0000 7fd16d87e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd1680cde50 con 0x7fd1680b88a0 2024-04-26T20:50:08.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.498+0000 7fd16d87e640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fd1680d20b0 con 0x7fd1680b5c10 2024-04-26T20:50:08.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.498+0000 7fd166ffd640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd1680b5c10 0x7fd16815f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:08.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.498+0000 7fd166ffd640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd1680b5c10 0x7fd16815f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.120:55348/0 (socket says 172.21.15.120:55348) 2024-04-26T20:50:08.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.498+0000 7fd166ffd640 1 -- 172.21.15.120:0/1509903456 learned_addr learned my addr 172.21.15.120:0/1509903456 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-04-26T20:50:08.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.498+0000 7fd1667fc640 1 --2- 172.21.15.120:0/1509903456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1680b88a0 0x7fd1680cbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:08.500 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.498+0000 7fd1677fe640 1 --2- 172.21.15.120:0/1509903456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd1680cfd50 0x7fd1680c8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:08.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.498+0000 7fd1667fc640 1 -- 172.21.15.120:0/1509903456 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd1680b5c10 msgr2=0x7fd16815f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:08.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.498+0000 7fd1667fc640 1 --2- 172.21.15.120:0/1509903456 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd1680b5c10 0x7fd16815f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:08.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.498+0000 7fd1667fc640 1 -- 172.21.15.120:0/1509903456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd1680cfd50 msgr2=0x7fd1680c8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:08.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.498+0000 7fd1667fc640 1 --2- 172.21.15.120:0/1509903456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd1680cfd50 0x7fd1680c8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:08.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.498+0000 7fd1667fc640 1 -- 172.21.15.120:0/1509903456 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1680cc5a0 con 0x7fd1680b88a0 2024-04-26T20:50:08.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.498+0000 7fd166ffd640 1 --2- 172.21.15.120:0/1509903456 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd1680b5c10 0x7fd16815f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:50:08.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.498+0000 7fd1667fc640 1 --2- 172.21.15.120:0/1509903456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1680b88a0 0x7fd1680cbd00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd1680b5ff0 tx=0x7fd15800c870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:08.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.498+0000 7fd15ffff640 1 -- 172.21.15.120:0/1509903456 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1580074e0 con 0x7fd1680b88a0 2024-04-26T20:50:08.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.498+0000 7fd15ffff640 1 -- 172.21.15.120:0/1509903456 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd158007680 con 0x7fd1680b88a0 2024-04-26T20:50:08.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.498+0000 7fd16d87e640 1 -- 172.21.15.120:0/1509903456 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1680cc880 con 0x7fd1680b88a0 2024-04-26T20:50:08.501 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.498+0000 7fd15ffff640 1 -- 172.21.15.120:0/1509903456 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd158004a00 con 0x7fd1680b88a0 2024-04-26T20:50:08.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.498+0000 7fd16d87e640 1 -- 172.21.15.120:0/1509903456 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1680c85a0 con 0x7fd1680b88a0 2024-04-26T20:50:08.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.499+0000 7fd15ffff640 1 -- 172.21.15.120:0/1509903456 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd158011070 con 0x7fd1680b88a0 2024-04-26T20:50:08.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.499+0000 7fd15ffff640 1 -- 172.21.15.120:0/1509903456 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7fd158013700 con 0x7fd1680b88a0 2024-04-26T20:50:08.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.499+0000 7fd16d87e640 1 -- 172.21.15.120:0/1509903456 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1680694f0 con 0x7fd1680b88a0 2024-04-26T20:50:08.502 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.501+0000 7fd15ffff640 1 -- 172.21.15.120:0/1509903456 <== mon.1 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fd158013950 con 0x7fd1680b88a0 2024-04-26T20:50:08.518 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-04-26T20:50:08.518+0000 7f5a58a1b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T20:50:08.520 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-26T20:50:08.519+0000 7fba87221040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T20:50:08.573 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.573+0000 7fd16d87e640 1 -- 172.21.15.120:0/1509903456 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd new", "uuid": "72d6f4f3-214a-4392-a02f-c33cbcab3c7a", "id": 0} v 0) -- 0x7fd168155810 con 0x7fd1680b88a0 2024-04-26T20:50:08.576 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.576+0000 7fd15ffff640 1 -- 172.21.15.120:0/1509903456 <== mon.1 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "72d6f4f3-214a-4392-a02f-c33cbcab3c7a", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7fd158013b30 con 0x7fd1680b88a0 2024-04-26T20:50:08.576 INFO:teuthology.orchestra.run.smithi120.stdout:0 2024-04-26T20:50:08.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.577+0000 7fd16d87e640 1 -- 172.21.15.120:0/1509903456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1680b88a0 msgr2=0x7fd1680cbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:08.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.577+0000 7fd16d87e640 1 --2- 172.21.15.120:0/1509903456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1680b88a0 0x7fd1680cbd00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd1680b5ff0 tx=0x7fd15800c870 comp rx=0 tx=0).stop 2024-04-26T20:50:08.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.577+0000 7fd16d87e640 1 -- 172.21.15.120:0/1509903456 shutdown_connections 2024-04-26T20:50:08.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.577+0000 7fd16d87e640 1 --2- 172.21.15.120:0/1509903456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd1680cfd50 0x7fd1680c8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:08.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.577+0000 7fd16d87e640 1 --2- 172.21.15.120:0/1509903456 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1680b88a0 0x7fd1680cbd00 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:08.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.577+0000 7fd16d87e640 1 --2- 172.21.15.120:0/1509903456 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd1680b5c10 0x7fd16815f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:08.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.577+0000 7fd16d87e640 1 -- 172.21.15.120:0/1509903456 >> 172.21.15.120:0/1509903456 conn(0x7fd1680d2dc0 msgr2=0x7fd1680d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:08.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.577+0000 7fd16d87e640 1 -- 172.21.15.120:0/1509903456 shutdown_connections 2024-04-26T20:50:08.578 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.577+0000 7fd16d87e640 1 -- 172.21.15.120:0/1509903456 wait complete. 2024-04-26T20:50:08.588 DEBUG:teuthology.orchestra.run.smithi120:> sudo ceph --cluster ceph osd new 786bf8ca-6ddc-4b3c-be6c-9152398bd5fd 1 2024-04-26T20:50:08.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.682+0000 7f4e9c862640 1 Processor -- start 2024-04-26T20:50:08.683 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.682+0000 7f4e9c862640 1 -- start start 2024-04-26T20:50:08.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.683+0000 7f4e9c862640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4e980cfd50 0x7f4e9816a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:08.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.683+0000 7f4e9c862640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e980d2100 con 0x7f4e980b5c10 2024-04-26T20:50:08.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.683+0000 7f4e9c862640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e980b8c80 con 0x7f4e980b88a0 2024-04-26T20:50:08.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.683+0000 7f4e9c862640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f4e980d2370 con 0x7f4e980cfd50 2024-04-26T20:50:08.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.683+0000 7f4e9659c640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f4e980b88a0 0x7f4e980d19f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:35026/0 (socket says 172.21.15.120:35026) 2024-04-26T20:50:08.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.683+0000 7f4e96d9d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4e980cfd50 0x7f4e9816a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:08.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.683+0000 7f4e9659c640 1 -- 172.21.15.120:0/2670712633 learned_addr learned my addr 172.21.15.120:0/2670712633 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-04-26T20:50:08.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.683+0000 7f4e9559a640 1 -- 172.21.15.120:0/2670712633 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1181452638 0 0) 0x7f4e980b8c80 con 0x7f4e980b88a0 2024-04-26T20:50:08.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.683+0000 7f4e9559a640 1 -- 172.21.15.120:0/2670712633 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e80003520 con 0x7f4e980b88a0 2024-04-26T20:50:08.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.683+0000 7f4e96d9d640 1 -- 172.21.15.120:0/2670712633 >> v1:172.21.15.120:6789/0 conn(0x7f4e980b88a0 legacy=0x7f4e980d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:08.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.683+0000 7f4e96d9d640 1 -- 172.21.15.120:0/2670712633 >> v1:172.21.15.73:6789/0 conn(0x7f4e980b5c10 legacy=0x7f4e980cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:08.685 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.683+0000 7f4e9559a640 1 -- 172.21.15.120:0/2670712633 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1508102690 0 0) 0x7f4e980d2100 con 0x7f4e980b5c10 2024-04-26T20:50:08.686 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.683+0000 7f4e96d9d640 1 -- 172.21.15.120:0/2670712633 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e980cd5b0 con 0x7f4e980cfd50 2024-04-26T20:50:08.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.684+0000 7f4e96d9d640 1 --2- 172.21.15.120:0/2670712633 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4e980cfd50 0x7f4e9816a0f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4e90009dc0 tx=0x7f4e9002a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48fe436fb602de33 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:08.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.684+0000 7f4e9559a640 1 -- 172.21.15.120:0/2670712633 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e90007740 con 0x7f4e980cfd50 2024-04-26T20:50:08.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.684+0000 7f4e9559a640 1 -- 172.21.15.120:0/2670712633 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e900078e0 con 0x7f4e980cfd50 2024-04-26T20:50:08.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.684+0000 7f4e9559a640 1 -- 172.21.15.120:0/2670712633 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e90007cb0 con 0x7f4e980cfd50 2024-04-26T20:50:08.687 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.685+0000 7f4e9c862640 1 -- 172.21.15.120:0/2670712633 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4e980cfd50 msgr2=0x7f4e9816a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:08.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.685+0000 7f4e9c862640 1 --2- 172.21.15.120:0/2670712633 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4e980cfd50 0x7f4e9816a0f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4e90009dc0 tx=0x7f4e9002a6d0 comp rx=0 tx=0).stop 2024-04-26T20:50:08.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.685+0000 7f4e9c862640 1 -- 172.21.15.120:0/2670712633 shutdown_connections 2024-04-26T20:50:08.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.685+0000 7f4e9c862640 1 --2- 172.21.15.120:0/2670712633 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4e980cfd50 0x7f4e9816a0f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:08.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.685+0000 7f4e9c862640 1 -- 172.21.15.120:0/2670712633 >> 172.21.15.120:0/2670712633 conn(0x7f4e980d2dc0 msgr2=0x7f4e980d31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:08.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.685+0000 7f4e9c862640 1 -- 172.21.15.120:0/2670712633 shutdown_connections 2024-04-26T20:50:08.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.685+0000 7f4e9c862640 1 -- 172.21.15.120:0/2670712633 wait complete. 2024-04-26T20:50:08.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.686+0000 7f4e9c862640 1 Processor -- start 2024-04-26T20:50:08.688 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.686+0000 7f4e9c862640 1 -- start start 2024-04-26T20:50:08.690 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.687+0000 7f4e9c862640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4e980b5c10 0x7f4e980b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:08.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.687+0000 7f4e9c862640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e980b88a0 0x7f4e981539d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:08.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.687+0000 7f4e9c862640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e980cfd50 0x7f4e98154080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:08.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.687+0000 7f4e9c862640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4e980cde50 con 0x7f4e980b88a0 2024-04-26T20:50:08.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.687+0000 7f4e9c862640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4e980cdb70 con 0x7f4e980cfd50 2024-04-26T20:50:08.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.687+0000 7f4e9c862640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f4e980d04a0 con 0x7f4e980b5c10 2024-04-26T20:50:08.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.687+0000 7f4e9659c640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4e980b5c10 0x7f4e980b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:08.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.687+0000 7f4e96d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e980cfd50 0x7f4e98154080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:08.691 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.687+0000 7f4e96d9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e980cfd50 0x7f4e98154080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38358/0 (socket says 172.21.15.120:38358) 2024-04-26T20:50:08.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.687+0000 7f4e9659c640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4e980b5c10 0x7f4e980b77a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.120:55356/0 (socket says 172.21.15.120:55356) 2024-04-26T20:50:08.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.687+0000 7f4e96d9d640 1 -- 172.21.15.120:0/888375341 learned_addr learned my addr 172.21.15.120:0/888375341 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-04-26T20:50:08.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.687+0000 7f4e95d9b640 1 --2- 172.21.15.120:0/888375341 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e980b88a0 0x7f4e981539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:08.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.688+0000 7f4e9659c640 1 -- 172.21.15.120:0/888375341 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e980cfd50 msgr2=0x7f4e98154080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:08.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.688+0000 7f4e9659c640 1 --2- 172.21.15.120:0/888375341 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e980cfd50 0x7f4e98154080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:08.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.688+0000 7f4e9659c640 1 -- 172.21.15.120:0/888375341 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e980b88a0 msgr2=0x7f4e981539d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:08.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.688+0000 7f4e9659c640 1 --2- 172.21.15.120:0/888375341 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e980b88a0 0x7f4e981539d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:08.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.688+0000 7f4e9659c640 1 -- 172.21.15.120:0/888375341 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e980b6d00 con 0x7f4e980b5c10 2024-04-26T20:50:08.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.688+0000 7f4e96d9d640 1 --2- 172.21.15.120:0/888375341 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e980cfd50 0x7f4e98154080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:50:08.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.688+0000 7f4e9659c640 1 --2- 172.21.15.120:0/888375341 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4e980b5c10 0x7f4e980b77a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c00c400 tx=0x7f4e8c00c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:08.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.688+0000 7f4e86ffd640 1 -- 172.21.15.120:0/888375341 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c017070 con 0x7f4e980b5c10 2024-04-26T20:50:08.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.688+0000 7f4e86ffd640 1 -- 172.21.15.120:0/888375341 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e8c005ce0 con 0x7f4e980b5c10 2024-04-26T20:50:08.692 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.688+0000 7f4e9c862640 1 -- 172.21.15.120:0/888375341 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e9815e220 con 0x7f4e980b5c10 2024-04-26T20:50:08.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.688+0000 7f4e86ffd640 1 -- 172.21.15.120:0/888375341 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c0072f0 con 0x7f4e980b5c10 2024-04-26T20:50:08.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.688+0000 7f4e9c862640 1 -- 172.21.15.120:0/888375341 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e9815e500 con 0x7f4e980b5c10 2024-04-26T20:50:08.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.688+0000 7f4e86ffd640 1 -- 172.21.15.120:0/888375341 <== mon.2 v2:172.21.15.120:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4e8c01a070 con 0x7f4e980b5c10 2024-04-26T20:50:08.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.688+0000 7f4e86ffd640 1 -- 172.21.15.120:0/888375341 <== mon.2 v2:172.21.15.120:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f4e8c0127b0 con 0x7f4e980b5c10 2024-04-26T20:50:08.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.689+0000 7f4e9c862640 1 -- 172.21.15.120:0/888375341 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e980694f0 con 0x7f4e980b5c10 2024-04-26T20:50:08.693 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.691+0000 7f4e86ffd640 1 -- 172.21.15.120:0/888375341 <== mon.2 v2:172.21.15.120:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f4e8c021070 con 0x7f4e980b5c10 2024-04-26T20:50:08.763 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.762+0000 7f4e9c862640 1 -- 172.21.15.120:0/888375341 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_command({"prefix": "osd new", "uuid": "786bf8ca-6ddc-4b3c-be6c-9152398bd5fd", "id": 1} v 0) -- 0x7f4e9815f210 con 0x7f4e980b5c10 2024-04-26T20:50:08.768 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.767+0000 7f4e86ffd640 1 -- 172.21.15.120:0/888375341 <== mon.2 v2:172.21.15.120:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "786bf8ca-6ddc-4b3c-be6c-9152398bd5fd", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f4e8c0075e0 con 0x7f4e980b5c10 2024-04-26T20:50:08.768 INFO:teuthology.orchestra.run.smithi120.stdout:1 2024-04-26T20:50:08.769 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.768+0000 7f4e9c862640 1 -- 172.21.15.120:0/888375341 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4e980b5c10 msgr2=0x7f4e980b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:08.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.768+0000 7f4e9c862640 1 --2- 172.21.15.120:0/888375341 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4e980b5c10 0x7f4e980b77a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c00c400 tx=0x7f4e8c00c8c0 comp rx=0 tx=0).stop 2024-04-26T20:50:08.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.768+0000 7f4e9c862640 1 -- 172.21.15.120:0/888375341 shutdown_connections 2024-04-26T20:50:08.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.768+0000 7f4e9c862640 1 --2- 172.21.15.120:0/888375341 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e980b88a0 0x7f4e981539d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:08.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.768+0000 7f4e9c862640 1 --2- 172.21.15.120:0/888375341 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e980cfd50 0x7f4e98154080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:08.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.768+0000 7f4e9c862640 1 --2- 172.21.15.120:0/888375341 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4e980b5c10 0x7f4e980b77a0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:08.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.768+0000 7f4e9c862640 1 -- 172.21.15.120:0/888375341 >> 172.21.15.120:0/888375341 conn(0x7f4e980d2dc0 msgr2=0x7f4e980706a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:08.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.768+0000 7f4e9c862640 1 -- 172.21.15.120:0/888375341 shutdown_connections 2024-04-26T20:50:08.770 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.768+0000 7f4e9c862640 1 -- 172.21.15.120:0/888375341 wait complete. 2024-04-26T20:50:08.779 DEBUG:teuthology.orchestra.run.smithi120:> sudo ceph --cluster ceph osd new 90963b1a-5437-4ef8-ad8b-6b74c534eeb5 2 2024-04-26T20:50:08.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.875+0000 7ff6aaa85640 1 Processor -- start 2024-04-26T20:50:08.876 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.875+0000 7ff6aaa85640 1 -- start start 2024-04-26T20:50:08.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.875+0000 7ff6aaa85640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7ff6a40b66d0 0x7ff6a40b6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:08.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.875+0000 7ff6aaa85640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff6a406eff0 0x7ff6a406f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:08.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.875+0000 7ff6aaa85640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6a40af900 con 0x7ff6a415f020 2024-04-26T20:50:08.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.875+0000 7ff6aaa85640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7ff6a40b5300 con 0x7ff6a40b66d0 2024-04-26T20:50:08.880 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.875+0000 7ff6aaa85640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7ff6a40b55e0 con 0x7ff6a406eff0 2024-04-26T20:50:08.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.875+0000 7ff6a3fff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7ff6a40b66d0 0x7ff6a40b6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:08.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.876+0000 7ff6a9022640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff6a406eff0 0x7ff6a406f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:08.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.876+0000 7ff6a3fff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7ff6a40b66d0 0x7ff6a40b6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38366/0 (socket says 172.21.15.120:38366) 2024-04-26T20:50:08.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.876+0000 7ff6a3fff640 1 -- 172.21.15.120:0/2109454029 learned_addr learned my addr 172.21.15.120:0/2109454029 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-04-26T20:50:08.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.876+0000 7ff6a9022640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff6a406eff0 0x7ff6a406f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.120:55360/0 (socket says 172.21.15.120:55360) 2024-04-26T20:50:08.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.876+0000 7ff6a3fff640 1 -- 172.21.15.120:0/2109454029 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff6a406eff0 msgr2=0x7ff6a406f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:08.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.876+0000 7ff6a3fff640 1 --2- 172.21.15.120:0/2109454029 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff6a406eff0 0x7ff6a406f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:08.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.876+0000 7ff6a3fff640 1 -- 172.21.15.120:0/2109454029 >> v1:172.21.15.73:6789/0 conn(0x7ff6a415f020 legacy=0x7ff6a40aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:08.881 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.876+0000 7ff6a3fff640 1 -- 172.21.15.120:0/2109454029 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6a40b58c0 con 0x7ff6a40b66d0 2024-04-26T20:50:08.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.876+0000 7ff6a37fe640 1 -- 172.21.15.120:0/2109454029 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1714184276 0 0) 0x7ff6a40af900 con 0x7ff6a415f020 2024-04-26T20:50:08.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.876+0000 7ff6a9022640 1 --2- 172.21.15.120:0/2109454029 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff6a406eff0 0x7ff6a406f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:50:08.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.876+0000 7ff6a3fff640 1 --2- 172.21.15.120:0/2109454029 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff6a40b66d0 0x7ff6a40b6b60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff690009870 tx=0x7ff69002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d126e6c75d728fd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:08.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.877+0000 7ff6a37fe640 1 -- 172.21.15.120:0/2109454029 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff690037070 con 0x7ff6a40b66d0 2024-04-26T20:50:08.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.877+0000 7ff6a37fe640 1 -- 172.21.15.120:0/2109454029 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff690032440 con 0x7ff6a40b66d0 2024-04-26T20:50:08.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.877+0000 7ff6a37fe640 1 -- 172.21.15.120:0/2109454029 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff690032810 con 0x7ff6a40b66d0 2024-04-26T20:50:08.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.877+0000 7ff6aaa85640 1 -- 172.21.15.120:0/2109454029 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff6a40b66d0 msgr2=0x7ff6a40b6b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:08.882 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.877+0000 7ff6aaa85640 1 --2- 172.21.15.120:0/2109454029 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff6a40b66d0 0x7ff6a40b6b60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff690009870 tx=0x7ff69002a270 comp rx=0 tx=0).stop 2024-04-26T20:50:08.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.877+0000 7ff6aaa85640 1 -- 172.21.15.120:0/2109454029 shutdown_connections 2024-04-26T20:50:08.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.877+0000 7ff6aaa85640 1 --2- 172.21.15.120:0/2109454029 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff6a406eff0 0x7ff6a406f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:08.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.877+0000 7ff6aaa85640 1 --2- 172.21.15.120:0/2109454029 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff6a40b66d0 0x7ff6a40b6b60 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:08.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.877+0000 7ff6aaa85640 1 -- 172.21.15.120:0/2109454029 >> 172.21.15.120:0/2109454029 conn(0x7ff6a40b3940 msgr2=0x7ff6a40b3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:08.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.877+0000 7ff6aaa85640 1 -- 172.21.15.120:0/2109454029 shutdown_connections 2024-04-26T20:50:08.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.877+0000 7ff6aaa85640 1 -- 172.21.15.120:0/2109454029 wait complete. 2024-04-26T20:50:08.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.878+0000 7ff6aaa85640 1 Processor -- start 2024-04-26T20:50:08.883 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.879+0000 7ff6aaa85640 1 -- start start 2024-04-26T20:50:08.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.879+0000 7ff6aaa85640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff6a406eff0 0x7ff6a4145600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:08.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.879+0000 7ff6aaa85640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff6a40b66d0 0x7ff6a4145f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:08.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.879+0000 7ff6aaa85640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff6a415f020 0x7ff6a4147fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:08.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.879+0000 7ff6aaa85640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a415f400 con 0x7ff6a415f020 2024-04-26T20:50:08.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.879+0000 7ff6aaa85640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a40b12a0 con 0x7ff6a40b66d0 2024-04-26T20:50:08.886 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.879+0000 7ff6aaa85640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7ff6a415f580 con 0x7ff6a406eff0 2024-04-26T20:50:08.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.879+0000 7ff6a3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff6a40b66d0 0x7ff6a4145f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:08.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.879+0000 7ff6a8821640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff6a406eff0 0x7ff6a4145600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:08.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.879+0000 7ff6a3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff6a40b66d0 0x7ff6a4145f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38380/0 (socket says 172.21.15.120:38380) 2024-04-26T20:50:08.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.879+0000 7ff6a8821640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff6a406eff0 0x7ff6a4145600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.120:55376/0 (socket says 172.21.15.120:55376) 2024-04-26T20:50:08.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.879+0000 7ff6a3fff640 1 -- 172.21.15.120:0/4233420416 learned_addr learned my addr 172.21.15.120:0/4233420416 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-04-26T20:50:08.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.880+0000 7ff6a9022640 1 --2- 172.21.15.120:0/4233420416 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff6a415f020 0x7ff6a4147fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:08.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.880+0000 7ff6a8821640 1 -- 172.21.15.120:0/4233420416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff6a40b66d0 msgr2=0x7ff6a4145f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:08.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.880+0000 7ff6a8821640 1 --2- 172.21.15.120:0/4233420416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff6a40b66d0 0x7ff6a4145f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:08.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.880+0000 7ff6a8821640 1 -- 172.21.15.120:0/4233420416 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff6a415f020 msgr2=0x7ff6a4147fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:08.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.880+0000 7ff6a8821640 1 --2- 172.21.15.120:0/4233420416 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff6a415f020 0x7ff6a4147fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:08.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.880+0000 7ff6a8821640 1 -- 172.21.15.120:0/4233420416 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6a4146990 con 0x7ff6a406eff0 2024-04-26T20:50:08.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.880+0000 7ff6a3fff640 1 --2- 172.21.15.120:0/4233420416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff6a40b66d0 0x7ff6a4145f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:50:08.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.880+0000 7ff6a9022640 1 --2- 172.21.15.120:0/4233420416 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff6a415f020 0x7ff6a4147fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:50:08.887 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.880+0000 7ff6a8821640 1 --2- 172.21.15.120:0/4233420416 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff6a406eff0 0x7ff6a4145600 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff69800c450 tx=0x7ff69800c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:08.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.880+0000 7ff6a17fa640 1 -- 172.21.15.120:0/4233420416 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff698017070 con 0x7ff6a406eff0 2024-04-26T20:50:08.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.880+0000 7ff6a17fa640 1 -- 172.21.15.120:0/4233420416 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6980035f0 con 0x7ff6a406eff0 2024-04-26T20:50:08.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.880+0000 7ff6aaa85640 1 -- 172.21.15.120:0/4233420416 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6a4146c70 con 0x7ff6a406eff0 2024-04-26T20:50:08.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.880+0000 7ff6a17fa640 1 -- 172.21.15.120:0/4233420416 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff698003a20 con 0x7ff6a406eff0 2024-04-26T20:50:08.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.880+0000 7ff6aaa85640 1 -- 172.21.15.120:0/4233420416 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6a4147230 con 0x7ff6a406eff0 2024-04-26T20:50:08.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.880+0000 7ff6a17fa640 1 -- 172.21.15.120:0/4233420416 <== mon.2 v2:172.21.15.120:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff69801a070 con 0x7ff6a406eff0 2024-04-26T20:50:08.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.880+0000 7ff6a17fa640 1 -- 172.21.15.120:0/4233420416 <== mon.2 v2:172.21.15.120:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7ff698012820 con 0x7ff6a406eff0 2024-04-26T20:50:08.888 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.881+0000 7ff6aaa85640 1 -- 172.21.15.120:0/4233420416 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6a40694f0 con 0x7ff6a406eff0 2024-04-26T20:50:08.889 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.885+0000 7ff6a17fa640 1 -- 172.21.15.120:0/4233420416 <== mon.2 v2:172.21.15.120:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7ff698021070 con 0x7ff6a406eff0 2024-04-26T20:50:08.959 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.957+0000 7ff6aaa85640 1 -- 172.21.15.120:0/4233420416 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_command({"prefix": "osd new", "uuid": "90963b1a-5437-4ef8-ad8b-6b74c534eeb5", "id": 2} v 0) -- 0x7ff6a415d3e0 con 0x7ff6a406eff0 2024-04-26T20:50:08.965 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.963+0000 7ff6a17fa640 1 -- 172.21.15.120:0/4233420416 <== mon.2 v2:172.21.15.120:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "90963b1a-5437-4ef8-ad8b-6b74c534eeb5", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7ff6980125f0 con 0x7ff6a406eff0 2024-04-26T20:50:08.965 INFO:teuthology.orchestra.run.smithi120.stdout:2 2024-04-26T20:50:08.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.963+0000 7ff6aaa85640 1 -- 172.21.15.120:0/4233420416 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff6a406eff0 msgr2=0x7ff6a4145600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:08.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.963+0000 7ff6aaa85640 1 --2- 172.21.15.120:0/4233420416 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff6a406eff0 0x7ff6a4145600 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff69800c450 tx=0x7ff69800c850 comp rx=0 tx=0).stop 2024-04-26T20:50:08.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.963+0000 7ff6aaa85640 1 -- 172.21.15.120:0/4233420416 shutdown_connections 2024-04-26T20:50:08.966 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.963+0000 7ff6aaa85640 1 --2- 172.21.15.120:0/4233420416 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff6a415f020 0x7ff6a4147fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:08.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.963+0000 7ff6aaa85640 1 --2- 172.21.15.120:0/4233420416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff6a40b66d0 0x7ff6a4145f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:08.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.963+0000 7ff6aaa85640 1 --2- 172.21.15.120:0/4233420416 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff6a406eff0 0x7ff6a4145600 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:08.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.963+0000 7ff6aaa85640 1 -- 172.21.15.120:0/4233420416 >> 172.21.15.120:0/4233420416 conn(0x7ff6a40b3940 msgr2=0x7ff6a40b3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:08.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.963+0000 7ff6aaa85640 1 -- 172.21.15.120:0/4233420416 shutdown_connections 2024-04-26T20:50:08.967 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:08.963+0000 7ff6aaa85640 1 -- 172.21.15.120:0/4233420416 wait complete. 2024-04-26T20:50:08.975 DEBUG:teuthology.orchestra.run.smithi120:> sudo ceph --cluster ceph osd new bd3d2148-6e38-41b4-a453-fe6e3b8b4ceb 3 2024-04-26T20:50:09.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.070+0000 7fe4ee6fc640 1 Processor -- start 2024-04-26T20:50:09.071 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.070+0000 7fe4ee6fc640 1 -- start start 2024-04-26T20:50:09.072 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.071+0000 7fe4ee6fc640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fe4e80736c0 0x7fe4e815f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:09.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.071+0000 7fe4ee6fc640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4e81605a0 con 0x7fe4e815cf90 2024-04-26T20:50:09.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.071+0000 7fe4ee6fc640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4e815c4b0 con 0x7fe4e80d1350 2024-04-26T20:50:09.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.071+0000 7fe4ee6fc640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fe4e8160880 con 0x7fe4e80736c0 2024-04-26T20:50:09.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.071+0000 7fe4e77fe640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fe4e80736c0 0x7fe4e815f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:09.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.071+0000 7fe4e7fff640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7fe4e80d1350 0x7fe4e8072fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:35032/0 (socket says 172.21.15.120:35032) 2024-04-26T20:50:09.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.071+0000 7fe4e7fff640 1 -- 172.21.15.120:0/1235963537 learned_addr learned my addr 172.21.15.120:0/1235963537 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-04-26T20:50:09.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.071+0000 7fe4e77fe640 1 -- 172.21.15.120:0/1235963537 >> v1:172.21.15.120:6789/0 conn(0x7fe4e80d1350 legacy=0x7fe4e8072fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:09.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.071+0000 7fe4e77fe640 1 -- 172.21.15.120:0/1235963537 >> v1:172.21.15.73:6789/0 conn(0x7fe4e815cf90 legacy=0x7fe4e80c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:09.073 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.071+0000 7fe4e77fe640 1 -- 172.21.15.120:0/1235963537 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4e815c790 con 0x7fe4e80736c0 2024-04-26T20:50:09.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.072+0000 7fe4e77fe640 1 --2- 172.21.15.120:0/1235963537 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4e80736c0 0x7fe4e815f900 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8160a00 tx=0x7fe4d802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8c9c066a7c91d83 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:09.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.072+0000 7fe4e6ffd640 1 -- 172.21.15.120:0/1235963537 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d8037070 con 0x7fe4e80736c0 2024-04-26T20:50:09.074 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.072+0000 7fe4e6ffd640 1 -- 172.21.15.120:0/1235963537 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4d8032440 con 0x7fe4e80736c0 2024-04-26T20:50:09.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.072+0000 7fe4e6ffd640 1 -- 172.21.15.120:0/1235963537 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d8032810 con 0x7fe4e80736c0 2024-04-26T20:50:09.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.073+0000 7fe4ee6fc640 1 -- 172.21.15.120:0/1235963537 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4e80736c0 msgr2=0x7fe4e815f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:09.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.073+0000 7fe4ee6fc640 1 --2- 172.21.15.120:0/1235963537 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4e80736c0 0x7fe4e815f900 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8160a00 tx=0x7fe4d802a180 comp rx=0 tx=0).stop 2024-04-26T20:50:09.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.073+0000 7fe4ee6fc640 1 -- 172.21.15.120:0/1235963537 shutdown_connections 2024-04-26T20:50:09.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.073+0000 7fe4ee6fc640 1 --2- 172.21.15.120:0/1235963537 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4e80736c0 0x7fe4e815f900 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:09.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.073+0000 7fe4ee6fc640 1 -- 172.21.15.120:0/1235963537 >> 172.21.15.120:0/1235963537 conn(0x7fe4e80d1c90 msgr2=0x7fe4e80d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:09.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.073+0000 7fe4ee6fc640 1 -- 172.21.15.120:0/1235963537 shutdown_connections 2024-04-26T20:50:09.075 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.073+0000 7fe4ee6fc640 1 -- 172.21.15.120:0/1235963537 wait complete. 2024-04-26T20:50:09.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.074+0000 7fe4ee6fc640 1 Processor -- start 2024-04-26T20:50:09.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.074+0000 7fe4ee6fc640 1 -- start start 2024-04-26T20:50:09.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.075+0000 7fe4ee6fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe4e80736c0 0x7fe4e80b6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:09.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.075+0000 7fe4ee6fc640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe4e80d1350 0x7fe4e814fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:09.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.075+0000 7fe4ee6fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4e815cf90 0x7fe4e81542a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:09.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.075+0000 7fe4ee6fc640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e80d18b0 con 0x7fe4e80736c0 2024-04-26T20:50:09.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.075+0000 7fe4ee6fc640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e80d1730 con 0x7fe4e815cf90 2024-04-26T20:50:09.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.075+0000 7fe4ee6fc640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fe4e815fe40 con 0x7fe4e80d1350 2024-04-26T20:50:09.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.075+0000 7fe4e77fe640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe4e80d1350 0x7fe4e814fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:09.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.075+0000 7fe4e77fe640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe4e80d1350 0x7fe4e814fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.120:55378/0 (socket says 172.21.15.120:55378) 2024-04-26T20:50:09.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.075+0000 7fe4e77fe640 1 -- 172.21.15.120:0/86209985 learned_addr learned my addr 172.21.15.120:0/86209985 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-04-26T20:50:09.078 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.075+0000 7fe4ecc99640 1 --2- 172.21.15.120:0/86209985 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4e815cf90 0x7fe4e81542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:09.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.075+0000 7fe4e7fff640 1 --2- 172.21.15.120:0/86209985 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe4e80736c0 0x7fe4e80b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:09.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.075+0000 7fe4e77fe640 1 -- 172.21.15.120:0/86209985 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4e815cf90 msgr2=0x7fe4e81542a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:09.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.075+0000 7fe4e77fe640 1 --2- 172.21.15.120:0/86209985 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4e815cf90 0x7fe4e81542a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:09.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.075+0000 7fe4e77fe640 1 -- 172.21.15.120:0/86209985 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe4e80736c0 msgr2=0x7fe4e80b6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:09.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.075+0000 7fe4e77fe640 1 --2- 172.21.15.120:0/86209985 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe4e80736c0 0x7fe4e80b6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:09.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.075+0000 7fe4e77fe640 1 -- 172.21.15.120:0/86209985 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4e80ca4b0 con 0x7fe4e80d1350 2024-04-26T20:50:09.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.075+0000 7fe4ecc99640 1 --2- 172.21.15.120:0/86209985 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4e815cf90 0x7fe4e81542a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:50:09.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.075+0000 7fe4e77fe640 1 --2- 172.21.15.120:0/86209985 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe4e80d1350 0x7fe4e814fc70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8009890 tx=0x7fe4d80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:09.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.076+0000 7fe4e4ff9640 1 -- 172.21.15.120:0/86209985 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d8037040 con 0x7fe4e80d1350 2024-04-26T20:50:09.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.076+0000 7fe4e4ff9640 1 -- 172.21.15.120:0/86209985 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4d8042070 con 0x7fe4e80d1350 2024-04-26T20:50:09.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.076+0000 7fe4ee6fc640 1 -- 172.21.15.120:0/86209985 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4e80b6990 con 0x7fe4e80d1350 2024-04-26T20:50:09.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.076+0000 7fe4e4ff9640 1 -- 172.21.15.120:0/86209985 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d8030a50 con 0x7fe4e80d1350 2024-04-26T20:50:09.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.076+0000 7fe4ee6fc640 1 -- 172.21.15.120:0/86209985 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4e80b6f50 con 0x7fe4e80d1350 2024-04-26T20:50:09.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.076+0000 7fe4e4ff9640 1 -- 172.21.15.120:0/86209985 <== mon.2 v2:172.21.15.120:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe4d8030bf0 con 0x7fe4e80d1350 2024-04-26T20:50:09.079 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.076+0000 7fe4e4ff9640 1 -- 172.21.15.120:0/86209985 <== mon.2 v2:172.21.15.120:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7fe4d8047cd0 con 0x7fe4e80d1350 2024-04-26T20:50:09.080 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.077+0000 7fe4ee6fc640 1 -- 172.21.15.120:0/86209985 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4e80694f0 con 0x7fe4e80d1350 2024-04-26T20:50:09.081 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.080+0000 7fe4e4ff9640 1 -- 172.21.15.120:0/86209985 <== mon.2 v2:172.21.15.120:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fe4d8047460 con 0x7fe4e80d1350 2024-04-26T20:50:09.154 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.153+0000 7fe4ee6fc640 1 -- 172.21.15.120:0/86209985 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_command({"prefix": "osd new", "uuid": "bd3d2148-6e38-41b4-a453-fe6e3b8b4ceb", "id": 3} v 0) -- 0x7fe4e80cfe30 con 0x7fe4e80d1350 2024-04-26T20:50:09.159 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.158+0000 7fe4e4ff9640 1 -- 172.21.15.120:0/86209985 <== mon.2 v2:172.21.15.120:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "bd3d2148-6e38-41b4-a453-fe6e3b8b4ceb", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7fe4d80476e0 con 0x7fe4e80d1350 2024-04-26T20:50:09.160 INFO:teuthology.orchestra.run.smithi120.stdout:3 2024-04-26T20:50:09.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.159+0000 7fe4ee6fc640 1 -- 172.21.15.120:0/86209985 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe4e80d1350 msgr2=0x7fe4e814fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:09.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.159+0000 7fe4ee6fc640 1 --2- 172.21.15.120:0/86209985 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe4e80d1350 0x7fe4e814fc70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8009890 tx=0x7fe4d80046c0 comp rx=0 tx=0).stop 2024-04-26T20:50:09.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.159+0000 7fe4ee6fc640 1 -- 172.21.15.120:0/86209985 shutdown_connections 2024-04-26T20:50:09.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.159+0000 7fe4ee6fc640 1 --2- 172.21.15.120:0/86209985 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe4e815cf90 0x7fe4e81542a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:09.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.159+0000 7fe4ee6fc640 1 --2- 172.21.15.120:0/86209985 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe4e80d1350 0x7fe4e814fc70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:09.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.159+0000 7fe4ee6fc640 1 --2- 172.21.15.120:0/86209985 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe4e80736c0 0x7fe4e80b6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:09.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.159+0000 7fe4ee6fc640 1 -- 172.21.15.120:0/86209985 >> 172.21.15.120:0/86209985 conn(0x7fe4e80d1c90 msgr2=0x7fe4e80d0510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:09.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.160+0000 7fe4ee6fc640 1 -- 172.21.15.120:0/86209985 shutdown_connections 2024-04-26T20:50:09.161 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.160+0000 7fe4ee6fc640 1 -- 172.21.15.120:0/86209985 wait complete. 2024-04-26T20:50:09.171 DEBUG:teuthology.orchestra.run.smithi120:> sudo ceph --cluster ceph osd new e8668dc8-a23f-46d2-8311-cb882da08e74 4 2024-04-26T20:50:09.227 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-26T20:50:09.226+0000 7fba87221040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T20:50:09.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.267+0000 7f0500b9e640 1 Processor -- start 2024-04-26T20:50:09.268 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.267+0000 7f0500b9e640 1 -- start start 2024-04-26T20:50:09.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.267+0000 7f0500b9e640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f04fc072d70 0x7f04fc073150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:09.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.268+0000 7f0500b9e640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f04fc15c230 0x7f04fc0b5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:09.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.268+0000 7f0500b9e640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04fc073690 con 0x7f04fc0b6460 2024-04-26T20:50:09.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.268+0000 7f0500b9e640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f04fc06ed30 con 0x7f04fc15c230 2024-04-26T20:50:09.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.268+0000 7f0500b9e640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f04fc15c7d0 con 0x7f04fc072d70 2024-04-26T20:50:09.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.268+0000 7f04fb7fe640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f04fc072d70 0x7f04fc073150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:09.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.268+0000 7f04f3fff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f04fc15c230 0x7f04fc0b5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:09.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.268+0000 7f04fb7fe640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f04fc072d70 0x7f04fc073150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.120:55386/0 (socket says 172.21.15.120:55386) 2024-04-26T20:50:09.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.268+0000 7f04f3fff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f04fc15c230 0x7f04fc0b5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38406/0 (socket says 172.21.15.120:38406) 2024-04-26T20:50:09.270 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.268+0000 7f04fb7fe640 1 -- 172.21.15.120:0/2377716025 learned_addr learned my addr 172.21.15.120:0/2377716025 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-04-26T20:50:09.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.268+0000 7f04fb7fe640 1 -- 172.21.15.120:0/2377716025 >> v2:172.21.15.120:3300/0 conn(0x7f04fc15c230 msgr2=0x7f04fc0b5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:09.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.268+0000 7f04fb7fe640 1 --2- 172.21.15.120:0/2377716025 >> v2:172.21.15.120:3300/0 conn(0x7f04fc15c230 0x7f04fc0b5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:09.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.268+0000 7f04fb7fe640 1 -- 172.21.15.120:0/2377716025 >> v1:172.21.15.73:6789/0 conn(0x7f04fc0b6460 legacy=0x7f04fc15f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:09.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.268+0000 7f04fb7fe640 1 -- 172.21.15.120:0/2377716025 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04fc15cab0 con 0x7f04fc072d70 2024-04-26T20:50:09.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.268+0000 7f04f3fff640 1 --2- 172.21.15.120:0/2377716025 >> v2:172.21.15.120:3300/0 conn(0x7f04fc15c230 0x7f04fc0b5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:50:09.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.269+0000 7f04fb7fe640 1 --2- 172.21.15.120:0/2377716025 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f04fc072d70 0x7f04fc073150 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f04ec009870 tx=0x7f04ec02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43fde3f5930bd1ce server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:09.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.269+0000 7f04faffd640 1 -- 172.21.15.120:0/2377716025 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ec037070 con 0x7f04fc072d70 2024-04-26T20:50:09.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.269+0000 7f04faffd640 1 -- 172.21.15.120:0/2377716025 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04ec032440 con 0x7f04fc072d70 2024-04-26T20:50:09.271 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.269+0000 7f04faffd640 1 -- 172.21.15.120:0/2377716025 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ec032810 con 0x7f04fc072d70 2024-04-26T20:50:09.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.269+0000 7f0500b9e640 1 -- 172.21.15.120:0/2377716025 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f04fc072d70 msgr2=0x7f04fc073150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:09.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.269+0000 7f0500b9e640 1 --2- 172.21.15.120:0/2377716025 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f04fc072d70 0x7f04fc073150 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f04ec009870 tx=0x7f04ec02a270 comp rx=0 tx=0).stop 2024-04-26T20:50:09.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.270+0000 7f0500b9e640 1 -- 172.21.15.120:0/2377716025 shutdown_connections 2024-04-26T20:50:09.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.270+0000 7f0500b9e640 1 --2- 172.21.15.120:0/2377716025 >> v2:172.21.15.120:3300/0 conn(0x7f04fc15c230 0x7f04fc0b5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:09.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.270+0000 7f0500b9e640 1 --2- 172.21.15.120:0/2377716025 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f04fc072d70 0x7f04fc073150 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:09.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.270+0000 7f0500b9e640 1 -- 172.21.15.120:0/2377716025 >> 172.21.15.120:0/2377716025 conn(0x7f04fc0b38a0 msgr2=0x7f04fc0b3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:09.272 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.270+0000 7f0500b9e640 1 -- 172.21.15.120:0/2377716025 shutdown_connections 2024-04-26T20:50:09.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.270+0000 7f0500b9e640 1 -- 172.21.15.120:0/2377716025 wait complete. 2024-04-26T20:50:09.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.272+0000 7f0500b9e640 1 Processor -- start 2024-04-26T20:50:09.273 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.272+0000 7f0500b9e640 1 -- start start 2024-04-26T20:50:09.275 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.273+0000 7f0500b9e640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f04fc072d70 0x7f04fc14ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:09.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.273+0000 7f0500b9e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04fc15c230 0x7f04fc0cd150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:09.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.273+0000 7f0500b9e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04fc0cd690 0x7f04fc152c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:09.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.273+0000 7f0500b9e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f04fc0b4e40 con 0x7f04fc0cd690 2024-04-26T20:50:09.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.273+0000 7f0500b9e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f04fc15d070 con 0x7f04fc15c230 2024-04-26T20:50:09.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.273+0000 7f0500b9e640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f04fc0701d0 con 0x7f04fc072d70 2024-04-26T20:50:09.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.273+0000 7f04fb7fe640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f04fc072d70 0x7f04fc14ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:09.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.273+0000 7f04f3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04fc15c230 0x7f04fc0cd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:09.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.273+0000 7f04fb7fe640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f04fc072d70 0x7f04fc14ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.120:55402/0 (socket says 172.21.15.120:55402) 2024-04-26T20:50:09.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.273+0000 7f04fb7fe640 1 -- 172.21.15.120:0/3537352325 learned_addr learned my addr 172.21.15.120:0/3537352325 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-04-26T20:50:09.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.273+0000 7f04fbfff640 1 --2- 172.21.15.120:0/3537352325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04fc0cd690 0x7f04fc152c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:09.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.273+0000 7f04f3fff640 1 -- 172.21.15.120:0/3537352325 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f04fc072d70 msgr2=0x7f04fc14ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:09.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.273+0000 7f04f3fff640 1 --2- 172.21.15.120:0/3537352325 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f04fc072d70 0x7f04fc14ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:09.276 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.273+0000 7f04f3fff640 1 -- 172.21.15.120:0/3537352325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04fc0cd690 msgr2=0x7f04fc152c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:09.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.273+0000 7f04f3fff640 1 --2- 172.21.15.120:0/3537352325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04fc0cd690 0x7f04fc152c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:09.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.273+0000 7f04f3fff640 1 -- 172.21.15.120:0/3537352325 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04fc14d210 con 0x7f04fc15c230 2024-04-26T20:50:09.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.273+0000 7f04fb7fe640 1 --2- 172.21.15.120:0/3537352325 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f04fc072d70 0x7f04fc14ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:50:09.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.273+0000 7f04f3fff640 1 --2- 172.21.15.120:0/3537352325 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04fc15c230 0x7f04fc0cd150 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f04e400e870 tx=0x7f04e400ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:09.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.274+0000 7f04f8ff9640 1 -- 172.21.15.120:0/3537352325 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e4017070 con 0x7f04fc15c230 2024-04-26T20:50:09.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.274+0000 7f04f8ff9640 1 -- 172.21.15.120:0/3537352325 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04e400c940 con 0x7f04fc15c230 2024-04-26T20:50:09.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.274+0000 7f04f8ff9640 1 -- 172.21.15.120:0/3537352325 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e400cd10 con 0x7f04fc15c230 2024-04-26T20:50:09.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.274+0000 7f0500b9e640 1 -- 172.21.15.120:0/3537352325 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04fc152840 con 0x7f04fc15c230 2024-04-26T20:50:09.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.274+0000 7f0500b9e640 1 -- 172.21.15.120:0/3537352325 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04fc152560 con 0x7f04fc15c230 2024-04-26T20:50:09.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.274+0000 7f04f8ff9640 1 -- 172.21.15.120:0/3537352325 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f04e401a070 con 0x7f04fc15c230 2024-04-26T20:50:09.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.274+0000 7f04f8ff9640 1 -- 172.21.15.120:0/3537352325 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f04e401c960 con 0x7f04fc15c230 2024-04-26T20:50:09.277 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.275+0000 7f0500b9e640 1 -- 172.21.15.120:0/3537352325 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04fc069a80 con 0x7f04fc15c230 2024-04-26T20:50:09.278 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.277+0000 7f04f8ff9640 1 -- 172.21.15.120:0/3537352325 <== mon.1 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f04fc14d210 con 0x7f04fc15c230 2024-04-26T20:50:09.308 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-04-26T20:50:09.308+0000 7f5a58a1b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T20:50:09.313 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-26T20:50:09.312+0000 7fba87221040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T20:50:09.351 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.349+0000 7f0500b9e640 1 -- 172.21.15.120:0/3537352325 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e8668dc8-a23f-46d2-8311-cb882da08e74", "id": 4} v 0) -- 0x7f04fc0cdf40 con 0x7f04fc15c230 2024-04-26T20:50:09.356 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.355+0000 7f04f8ff9640 1 -- 172.21.15.120:0/3537352325 <== mon.1 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e8668dc8-a23f-46d2-8311-cb882da08e74", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f04e401cbb0 con 0x7f04fc15c230 2024-04-26T20:50:09.356 INFO:teuthology.orchestra.run.smithi120.stdout:4 2024-04-26T20:50:09.357 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-26T20:50:09.355+0000 7fba87221040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T20:50:09.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.356+0000 7f0500b9e640 1 -- 172.21.15.120:0/3537352325 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04fc15c230 msgr2=0x7f04fc0cd150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:09.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.356+0000 7f0500b9e640 1 --2- 172.21.15.120:0/3537352325 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04fc15c230 0x7f04fc0cd150 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f04e400e870 tx=0x7f04e400ed30 comp rx=0 tx=0).stop 2024-04-26T20:50:09.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.356+0000 7f0500b9e640 1 -- 172.21.15.120:0/3537352325 shutdown_connections 2024-04-26T20:50:09.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.356+0000 7f0500b9e640 1 --2- 172.21.15.120:0/3537352325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04fc0cd690 0x7f04fc152c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:09.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.356+0000 7f0500b9e640 1 --2- 172.21.15.120:0/3537352325 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f04fc15c230 0x7f04fc0cd150 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:09.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.356+0000 7f0500b9e640 1 --2- 172.21.15.120:0/3537352325 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f04fc072d70 0x7f04fc14ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:09.358 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.356+0000 7f0500b9e640 1 -- 172.21.15.120:0/3537352325 >> 172.21.15.120:0/3537352325 conn(0x7f04fc0b38a0 msgr2=0x7f04fc160150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:09.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.356+0000 7f0500b9e640 1 -- 172.21.15.120:0/3537352325 shutdown_connections 2024-04-26T20:50:09.359 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.356+0000 7f0500b9e640 1 -- 172.21.15.120:0/3537352325 wait complete. 2024-04-26T20:50:09.368 DEBUG:teuthology.orchestra.run.smithi120:> sudo ceph --cluster ceph osd new cbac0bdb-96bd-4852-bcce-e564495c3fcc 5 2024-04-26T20:50:09.400 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-04-26T20:50:09.399+0000 7f5a58a1b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T20:50:09.444 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-04-26T20:50:09.443+0000 7f5a58a1b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T20:50:09.464 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.463+0000 7f3a81378640 1 Processor -- start 2024-04-26T20:50:09.465 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.463+0000 7f3a81378640 1 -- start start 2024-04-26T20:50:09.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.464+0000 7f3a81378640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3a7c15f3a0 0x7f3a7c0d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:09.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.464+0000 7f3a81378640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3a7c0d1770 0x7f3a7c0d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:09.466 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.464+0000 7f3a81378640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f3a7c0d2130 0x7f3a7c0d2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:09.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.464+0000 7f3a81378640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f3a7c0d25a0 con 0x7f3a7c15f3a0 2024-04-26T20:50:09.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.464+0000 7f3a81378640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f3a7c0d0e20 con 0x7f3a7c0d2130 2024-04-26T20:50:09.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.464+0000 7f3a81378640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f3a7c0b1d70 con 0x7f3a7c0d1770 2024-04-26T20:50:09.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.464+0000 7f3a73fff640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3a7c0d1770 0x7f3a7c0d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:09.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.464+0000 7f3a7b7fe640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f3a7c0d2130 0x7f3a7c0d2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:09.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.464+0000 7f3a73fff640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3a7c0d1770 0x7f3a7c0d1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.120:55412/0 (socket says 172.21.15.120:55412) 2024-04-26T20:50:09.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.464+0000 7f3a73fff640 1 -- 172.21.15.120:0/1645361410 learned_addr learned my addr 172.21.15.120:0/1645361410 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-04-26T20:50:09.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.464+0000 7f3a7b7fe640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f3a7c0d2130 0x7f3a7c0d2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38414/0 (socket says 172.21.15.120:38414) 2024-04-26T20:50:09.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.464+0000 7f3a7affd640 1 --2- 172.21.15.120:0/1645361410 >> v2:172.21.15.73:3300/0 conn(0x7f3a7c15f3a0 0x7f3a7c0d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:09.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.464+0000 7f3a73fff640 1 -- 172.21.15.120:0/1645361410 >> v2:172.21.15.120:3300/0 conn(0x7f3a7c0d2130 msgr2=0x7f3a7c0d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:09.467 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.464+0000 7f3a73fff640 1 --2- 172.21.15.120:0/1645361410 >> v2:172.21.15.120:3300/0 conn(0x7f3a7c0d2130 0x7f3a7c0d2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:09.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.464+0000 7f3a73fff640 1 -- 172.21.15.120:0/1645361410 >> v2:172.21.15.73:3300/0 conn(0x7f3a7c15f3a0 msgr2=0x7f3a7c0d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:09.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.464+0000 7f3a73fff640 1 --2- 172.21.15.120:0/1645361410 >> v2:172.21.15.73:3300/0 conn(0x7f3a7c15f3a0 0x7f3a7c0d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:09.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.464+0000 7f3a73fff640 1 -- 172.21.15.120:0/1645361410 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a7c0b2050 con 0x7f3a7c0d1770 2024-04-26T20:50:09.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.464+0000 7f3a7b7fe640 1 --2- 172.21.15.120:0/1645361410 >> v2:172.21.15.120:3300/0 conn(0x7f3a7c0d2130 0x7f3a7c0d2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:50:09.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.465+0000 7f3a73fff640 1 --2- 172.21.15.120:0/1645361410 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3a7c0d1770 0x7f3a7c0d1bf0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c08e8d0 tx=0x7f3a6402a1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70e1dd6435a2fc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:09.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.465+0000 7f3a7a7fc640 1 -- 172.21.15.120:0/1645361410 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a64037070 con 0x7f3a7c0d1770 2024-04-26T20:50:09.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.465+0000 7f3a7a7fc640 1 -- 172.21.15.120:0/1645361410 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a64032440 con 0x7f3a7c0d1770 2024-04-26T20:50:09.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.465+0000 7f3a7a7fc640 1 -- 172.21.15.120:0/1645361410 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a64032810 con 0x7f3a7c0d1770 2024-04-26T20:50:09.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.466+0000 7f3a81378640 1 -- 172.21.15.120:0/1645361410 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3a7c0d1770 msgr2=0x7f3a7c0d1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:09.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.466+0000 7f3a81378640 1 --2- 172.21.15.120:0/1645361410 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3a7c0d1770 0x7f3a7c0d1bf0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c08e8d0 tx=0x7f3a6402a1e0 comp rx=0 tx=0).stop 2024-04-26T20:50:09.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.466+0000 7f3a81378640 1 -- 172.21.15.120:0/1645361410 shutdown_connections 2024-04-26T20:50:09.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.466+0000 7f3a81378640 1 --2- 172.21.15.120:0/1645361410 >> v2:172.21.15.120:3300/0 conn(0x7f3a7c0d2130 0x7f3a7c0d2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:09.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.466+0000 7f3a81378640 1 --2- 172.21.15.120:0/1645361410 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3a7c0d1770 0x7f3a7c0d1bf0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:09.471 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.466+0000 7f3a81378640 1 --2- 172.21.15.120:0/1645361410 >> v2:172.21.15.73:3300/0 conn(0x7f3a7c15f3a0 0x7f3a7c0d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:09.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.466+0000 7f3a81378640 1 -- 172.21.15.120:0/1645361410 >> 172.21.15.120:0/1645361410 conn(0x7f3a7c0d4f90 msgr2=0x7f3a7c0d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:09.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.466+0000 7f3a81378640 1 -- 172.21.15.120:0/1645361410 shutdown_connections 2024-04-26T20:50:09.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.466+0000 7f3a81378640 1 -- 172.21.15.120:0/1645361410 wait complete. 2024-04-26T20:50:09.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.467+0000 7f3a81378640 1 Processor -- start 2024-04-26T20:50:09.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.467+0000 7f3a81378640 1 -- start start 2024-04-26T20:50:09.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.468+0000 7f3a81378640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3a7c0d1770 0x7f3a7c0c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:09.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.468+0000 7f3a81378640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a7c0d2130 0x7f3a7c15c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:09.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.468+0000 7f3a81378640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3a7c15f3a0 0x7f3a7c0ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:09.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.468+0000 7f3a81378640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7c15f900 con 0x7f3a7c0d2130 2024-04-26T20:50:09.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.468+0000 7f3a81378640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7c15f780 con 0x7f3a7c15f3a0 2024-04-26T20:50:09.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.468+0000 7f3a81378640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f3a7c0d3390 con 0x7f3a7c0d1770 2024-04-26T20:50:09.472 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.468+0000 7f3a7affd640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3a7c0d1770 0x7f3a7c0c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:09.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.468+0000 7f3a7b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3a7c15f3a0 0x7f3a7c0ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:09.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.468+0000 7f3a7b7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3a7c15f3a0 0x7f3a7c0ca3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38430/0 (socket says 172.21.15.120:38430) 2024-04-26T20:50:09.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.468+0000 7f3a7b7fe640 1 -- 172.21.15.120:0/1662734200 learned_addr learned my addr 172.21.15.120:0/1662734200 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-04-26T20:50:09.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.468+0000 7f3a73fff640 1 --2- 172.21.15.120:0/1662734200 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a7c0d2130 0x7f3a7c15c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:09.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.468+0000 7f3a7b7fe640 1 -- 172.21.15.120:0/1662734200 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3a7c0d1770 msgr2=0x7f3a7c0c8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:09.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.468+0000 7f3a7b7fe640 1 --2- 172.21.15.120:0/1662734200 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3a7c0d1770 0x7f3a7c0c8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:09.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.468+0000 7f3a7b7fe640 1 -- 172.21.15.120:0/1662734200 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a7c0d2130 msgr2=0x7f3a7c15c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:09.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.468+0000 7f3a7b7fe640 1 --2- 172.21.15.120:0/1662734200 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a7c0d2130 0x7f3a7c15c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:09.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.468+0000 7f3a7b7fe640 1 -- 172.21.15.120:0/1662734200 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a7c0c9310 con 0x7f3a7c15f3a0 2024-04-26T20:50:09.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.468+0000 7f3a7affd640 1 --2- 172.21.15.120:0/1662734200 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3a7c0d1770 0x7f3a7c0c8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:50:09.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.468+0000 7f3a73fff640 1 --2- 172.21.15.120:0/1662734200 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a7c0d2130 0x7f3a7c15c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:50:09.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.468+0000 7f3a7b7fe640 1 --2- 172.21.15.120:0/1662734200 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3a7c15f3a0 0x7f3a7c0ca3b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3a7400eed0 tx=0x7f3a7400b480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:09.473 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.468+0000 7f3a737fe640 1 -- 172.21.15.120:0/1662734200 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a74018030 con 0x7f3a7c15f3a0 2024-04-26T20:50:09.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.469+0000 7f3a737fe640 1 -- 172.21.15.120:0/1662734200 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a7400bd20 con 0x7f3a7c15f3a0 2024-04-26T20:50:09.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.469+0000 7f3a737fe640 1 -- 172.21.15.120:0/1662734200 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a74003bb0 con 0x7f3a7c15f3a0 2024-04-26T20:50:09.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.469+0000 7f3a81378640 1 -- 172.21.15.120:0/1662734200 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a7c0c95f0 con 0x7f3a7c15f3a0 2024-04-26T20:50:09.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.469+0000 7f3a81378640 1 -- 172.21.15.120:0/1662734200 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a7c0c9e90 con 0x7f3a7c15f3a0 2024-04-26T20:50:09.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.469+0000 7f3a737fe640 1 -- 172.21.15.120:0/1662734200 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3a74005d10 con 0x7f3a7c15f3a0 2024-04-26T20:50:09.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.469+0000 7f3a737fe640 1 -- 172.21.15.120:0/1662734200 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f3a74014980 con 0x7f3a7c15f3a0 2024-04-26T20:50:09.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.469+0000 7f3a81378640 1 -- 172.21.15.120:0/1662734200 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a7c0c97c0 con 0x7f3a7c15f3a0 2024-04-26T20:50:09.474 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.471+0000 7f3a737fe640 1 -- 172.21.15.120:0/1662734200 <== mon.1 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f3a74022070 con 0x7f3a7c15f3a0 2024-04-26T20:50:09.524 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-26T20:50:09.523+0000 7fba87221040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T20:50:09.545 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.544+0000 7f3a81378640 1 -- 172.21.15.120:0/1662734200 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd new", "uuid": "cbac0bdb-96bd-4852-bcce-e564495c3fcc", "id": 5} v 0) -- 0x7f3a7c0694f0 con 0x7f3a7c15f3a0 2024-04-26T20:50:09.550 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.549+0000 7f3a737fe640 1 -- 172.21.15.120:0/1662734200 <== mon.1 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cbac0bdb-96bd-4852-bcce-e564495c3fcc", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f3a74014b80 con 0x7f3a7c15f3a0 2024-04-26T20:50:09.551 INFO:teuthology.orchestra.run.smithi120.stdout:5 2024-04-26T20:50:09.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.550+0000 7f3a81378640 1 -- 172.21.15.120:0/1662734200 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3a7c15f3a0 msgr2=0x7f3a7c0ca3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:09.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.550+0000 7f3a81378640 1 --2- 172.21.15.120:0/1662734200 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3a7c15f3a0 0x7f3a7c0ca3b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3a7400eed0 tx=0x7f3a7400b480 comp rx=0 tx=0).stop 2024-04-26T20:50:09.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.550+0000 7f3a81378640 1 -- 172.21.15.120:0/1662734200 shutdown_connections 2024-04-26T20:50:09.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.550+0000 7f3a81378640 1 --2- 172.21.15.120:0/1662734200 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a7c0d2130 0x7f3a7c15c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:09.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.550+0000 7f3a81378640 1 --2- 172.21.15.120:0/1662734200 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3a7c15f3a0 0x7f3a7c0ca3b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:09.552 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.550+0000 7f3a81378640 1 --2- 172.21.15.120:0/1662734200 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3a7c0d1770 0x7f3a7c0c8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:09.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.550+0000 7f3a81378640 1 -- 172.21.15.120:0/1662734200 >> 172.21.15.120:0/1662734200 conn(0x7f3a7c0d4f90 msgr2=0x7f3a7c06d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:09.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.550+0000 7f3a81378640 1 -- 172.21.15.120:0/1662734200 shutdown_connections 2024-04-26T20:50:09.553 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.550+0000 7f3a81378640 1 -- 172.21.15.120:0/1662734200 wait complete. 2024-04-26T20:50:09.563 DEBUG:teuthology.orchestra.run.smithi120:> sudo ceph --cluster ceph osd new 1884b2a8-569f-4272-96a9-f25a09364dc2 6 2024-04-26T20:50:09.573 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-26T20:50:09.571+0000 7fba87221040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T20:50:09.619 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-04-26T20:50:09.618+0000 7f5a58a1b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T20:50:09.623 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-26T20:50:09.623+0000 7fba87221040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T20:50:09.674 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-04-26T20:50:09.673+0000 7f5a58a1b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T20:50:09.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.698+0000 7fb5d72e1640 1 Processor -- start 2024-04-26T20:50:09.700 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.699+0000 7fb5d72e1640 1 -- start start 2024-04-26T20:50:09.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.699+0000 7fb5d72e1640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fb5d00d1350 0x7fb5d0072fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:09.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.699+0000 7fb5d72e1640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb5d0160010 0x7fb5d00734f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:09.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.699+0000 7fb5d72e1640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5d01605a0 con 0x7fb5d015cf90 2024-04-26T20:50:09.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.699+0000 7fb5d72e1640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fb5d0160880 con 0x7fb5d00d1350 2024-04-26T20:50:09.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.699+0000 7fb5d72e1640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fb5d015c4b0 con 0x7fb5d0160010 2024-04-26T20:50:09.703 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.700+0000 7fb5d587e640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb5d0160010 0x7fb5d00734f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:09.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.700+0000 7fb5d507d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fb5d00d1350 0x7fb5d0072fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:09.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.700+0000 7fb5d587e640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb5d0160010 0x7fb5d00734f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.120:55432/0 (socket says 172.21.15.120:55432) 2024-04-26T20:50:09.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.700+0000 7fb5d587e640 1 -- 172.21.15.120:0/1311917291 learned_addr learned my addr 172.21.15.120:0/1311917291 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-04-26T20:50:09.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.700+0000 7fb5d587e640 1 -- 172.21.15.120:0/1311917291 >> v2:172.21.15.120:3300/0 conn(0x7fb5d00d1350 msgr2=0x7fb5d0072fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:09.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.700+0000 7fb5d587e640 1 --2- 172.21.15.120:0/1311917291 >> v2:172.21.15.120:3300/0 conn(0x7fb5d00d1350 0x7fb5d0072fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:09.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.700+0000 7fb5d587e640 1 -- 172.21.15.120:0/1311917291 >> v1:172.21.15.73:6789/0 conn(0x7fb5d015cf90 legacy=0x7fb5d015f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:09.704 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.700+0000 7fb5d587e640 1 -- 172.21.15.120:0/1311917291 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5d015c790 con 0x7fb5d0160010 2024-04-26T20:50:09.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.701+0000 7fb5d587e640 1 --2- 172.21.15.120:0/1311917291 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb5d0160010 0x7fb5d00734f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb5c8009e50 tx=0x7fb5c802a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33dc9f86e368ea85 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:09.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.701+0000 7fb5c7fff640 1 -- 172.21.15.120:0/1311917291 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c8036070 con 0x7fb5d0160010 2024-04-26T20:50:09.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.701+0000 7fb5c7fff640 1 -- 172.21.15.120:0/1311917291 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5c8031600 con 0x7fb5d0160010 2024-04-26T20:50:09.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.701+0000 7fb5c7fff640 1 -- 172.21.15.120:0/1311917291 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c80319d0 con 0x7fb5d0160010 2024-04-26T20:50:09.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.701+0000 7fb5d72e1640 1 -- 172.21.15.120:0/1311917291 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb5d0160010 msgr2=0x7fb5d00734f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:09.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.701+0000 7fb5d72e1640 1 --2- 172.21.15.120:0/1311917291 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb5d0160010 0x7fb5d00734f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb5c8009e50 tx=0x7fb5c802a680 comp rx=0 tx=0).stop 2024-04-26T20:50:09.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.701+0000 7fb5d72e1640 1 -- 172.21.15.120:0/1311917291 shutdown_connections 2024-04-26T20:50:09.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.701+0000 7fb5d72e1640 1 --2- 172.21.15.120:0/1311917291 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb5d0160010 0x7fb5d00734f0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:09.705 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.701+0000 7fb5d72e1640 1 --2- 172.21.15.120:0/1311917291 >> v2:172.21.15.120:3300/0 conn(0x7fb5d00d1350 0x7fb5d0072fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:09.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.701+0000 7fb5d72e1640 1 -- 172.21.15.120:0/1311917291 >> 172.21.15.120:0/1311917291 conn(0x7fb5d00d1c90 msgr2=0x7fb5d00d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:09.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.701+0000 7fb5d72e1640 1 -- 172.21.15.120:0/1311917291 shutdown_connections 2024-04-26T20:50:09.706 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.701+0000 7fb5d72e1640 1 -- 172.21.15.120:0/1311917291 wait complete. 2024-04-26T20:50:09.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.703+0000 7fb5d72e1640 1 Processor -- start 2024-04-26T20:50:09.708 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.703+0000 7fb5d72e1640 1 -- start start 2024-04-26T20:50:09.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.703+0000 7fb5d72e1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5d00d1350 0x7fb5d0140c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:09.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.703+0000 7fb5d72e1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb5d015cf90 0x7fb5d00b28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:09.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.703+0000 7fb5d72e1640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb5d0160010 0x7fb5d00b2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:09.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.703+0000 7fb5d72e1640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d00d18b0 con 0x7fb5d00d1350 2024-04-26T20:50:09.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.703+0000 7fb5d72e1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d00d1730 con 0x7fb5d015cf90 2024-04-26T20:50:09.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.703+0000 7fb5d72e1640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fb5d00c6b10 con 0x7fb5d0160010 2024-04-26T20:50:09.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.703+0000 7fb5d587e640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb5d0160010 0x7fb5d00b2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:09.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.703+0000 7fb5d487c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb5d015cf90 0x7fb5d00b28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:09.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.703+0000 7fb5d587e640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb5d0160010 0x7fb5d00b2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.120:55448/0 (socket says 172.21.15.120:55448) 2024-04-26T20:50:09.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.703+0000 7fb5d587e640 1 -- 172.21.15.120:0/2346070491 learned_addr learned my addr 172.21.15.120:0/2346070491 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-04-26T20:50:09.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.703+0000 7fb5d507d640 1 --2- 172.21.15.120:0/2346070491 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5d00d1350 0x7fb5d0140c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:09.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.704+0000 7fb5d587e640 1 -- 172.21.15.120:0/2346070491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb5d015cf90 msgr2=0x7fb5d00b28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:09.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.704+0000 7fb5d587e640 1 --2- 172.21.15.120:0/2346070491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb5d015cf90 0x7fb5d00b28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:09.709 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.704+0000 7fb5d587e640 1 -- 172.21.15.120:0/2346070491 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5d00d1350 msgr2=0x7fb5d0140c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:09.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.704+0000 7fb5d587e640 1 --2- 172.21.15.120:0/2346070491 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5d00d1350 0x7fb5d0140c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:09.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.704+0000 7fb5d587e640 1 -- 172.21.15.120:0/2346070491 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5d0144b50 con 0x7fb5d0160010 2024-04-26T20:50:09.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.704+0000 7fb5d487c640 1 --2- 172.21.15.120:0/2346070491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb5d015cf90 0x7fb5d00b28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:50:09.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.704+0000 7fb5d587e640 1 --2- 172.21.15.120:0/2346070491 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb5d0160010 0x7fb5d00b2df0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb5c8034040 tx=0x7fb5c8004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:09.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.704+0000 7fb5c5ffb640 1 -- 172.21.15.120:0/2346070491 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c8041020 con 0x7fb5d0160010 2024-04-26T20:50:09.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.704+0000 7fb5c5ffb640 1 -- 172.21.15.120:0/2346070491 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5c80077a0 con 0x7fb5d0160010 2024-04-26T20:50:09.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.704+0000 7fb5d72e1640 1 -- 172.21.15.120:0/2346070491 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5d0144e30 con 0x7fb5d0160010 2024-04-26T20:50:09.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.704+0000 7fb5c5ffb640 1 -- 172.21.15.120:0/2346070491 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c8036040 con 0x7fb5d0160010 2024-04-26T20:50:09.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.704+0000 7fb5d72e1640 1 -- 172.21.15.120:0/2346070491 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5d01425d0 con 0x7fb5d0160010 2024-04-26T20:50:09.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.704+0000 7fb5c5ffb640 1 -- 172.21.15.120:0/2346070491 <== mon.2 v2:172.21.15.120:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb5c8044020 con 0x7fb5d0160010 2024-04-26T20:50:09.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.704+0000 7fb5c5ffb640 1 -- 172.21.15.120:0/2346070491 <== mon.2 v2:172.21.15.120:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7fb5c803cbf0 con 0x7fb5d0160010 2024-04-26T20:50:09.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.705+0000 7fb5d72e1640 1 -- 172.21.15.120:0/2346070491 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5d00694f0 con 0x7fb5d0160010 2024-04-26T20:50:09.710 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.707+0000 7fb5c5ffb640 1 -- 172.21.15.120:0/2346070491 <== mon.2 v2:172.21.15.120:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fb5c804b030 con 0x7fb5d0160010 2024-04-26T20:50:09.730 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-04-26T20:50:09.729+0000 7f5a58a1b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T20:50:09.734 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-26T20:50:09.734+0000 7fba87221040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T20:50:09.781 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.779+0000 7fb5d72e1640 1 -- 172.21.15.120:0/2346070491 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_command({"prefix": "osd new", "uuid": "1884b2a8-569f-4272-96a9-f25a09364dc2", "id": 6} v 0) -- 0x7fb5d00cc670 con 0x7fb5d0160010 2024-04-26T20:50:09.786 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.785+0000 7fb5c5ffb640 1 -- 172.21.15.120:0/2346070491 <== mon.2 v2:172.21.15.120:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1884b2a8-569f-4272-96a9-f25a09364dc2", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7fb5c8007b90 con 0x7fb5d0160010 2024-04-26T20:50:09.786 INFO:teuthology.orchestra.run.smithi120.stdout:6 2024-04-26T20:50:09.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.786+0000 7fb5d72e1640 1 -- 172.21.15.120:0/2346070491 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb5d0160010 msgr2=0x7fb5d00b2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:09.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.786+0000 7fb5d72e1640 1 --2- 172.21.15.120:0/2346070491 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb5d0160010 0x7fb5d00b2df0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb5c8034040 tx=0x7fb5c8004a40 comp rx=0 tx=0).stop 2024-04-26T20:50:09.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.786+0000 7fb5d72e1640 1 -- 172.21.15.120:0/2346070491 shutdown_connections 2024-04-26T20:50:09.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.786+0000 7fb5d72e1640 1 --2- 172.21.15.120:0/2346070491 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb5d0160010 0x7fb5d00b2df0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:09.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.786+0000 7fb5d72e1640 1 --2- 172.21.15.120:0/2346070491 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb5d015cf90 0x7fb5d00b28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:09.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.786+0000 7fb5d72e1640 1 --2- 172.21.15.120:0/2346070491 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5d00d1350 0x7fb5d0140c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:09.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.786+0000 7fb5d72e1640 1 -- 172.21.15.120:0/2346070491 >> 172.21.15.120:0/2346070491 conn(0x7fb5d00d1c90 msgr2=0x7fb5d0072eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:09.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.786+0000 7fb5d72e1640 1 -- 172.21.15.120:0/2346070491 shutdown_connections 2024-04-26T20:50:09.788 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.786+0000 7fb5d72e1640 1 -- 172.21.15.120:0/2346070491 wait complete. 2024-04-26T20:50:09.797 DEBUG:teuthology.orchestra.run.smithi120:> sudo ceph --cluster ceph osd new c0c3bf6d-a4c5-4f1e-aa2d-330951830666 7 2024-04-26T20:50:09.847 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-04-26T20:50:09.846+0000 7f5a58a1b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T20:50:09.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.933+0000 7f03787c8640 1 Processor -- start 2024-04-26T20:50:09.935 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.933+0000 7f03787c8640 1 -- start start 2024-04-26T20:50:09.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.934+0000 7f03787c8640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f03700ce310 0x7f0370071040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:09.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.934+0000 7f03787c8640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f0370071580 0x7f0370160b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:09.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.934+0000 7f03787c8640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f037015d530 con 0x7f037006c240 2024-04-26T20:50:09.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.934+0000 7f03787c8640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f037015d7f0 con 0x7f03700ce310 2024-04-26T20:50:09.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.934+0000 7f03787c8640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f037015dad0 con 0x7f0370071580 2024-04-26T20:50:09.938 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.934+0000 7f0376564640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f037006c240 0x7f037006c620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.120:35040/0 (socket says 172.21.15.120:35040) 2024-04-26T20:50:09.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.934+0000 7f0376564640 1 -- 172.21.15.120:0/3885114784 learned_addr learned my addr 172.21.15.120:0/3885114784 (peer_addr_for_me v1:172.21.15.120:0/0) 2024-04-26T20:50:09.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.934+0000 7f0376d65640 1 --2- 172.21.15.120:0/3885114784 >> v2:172.21.15.120:3300/0 conn(0x7f0370071580 0x7f0370160b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:09.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.935+0000 7f0375d63640 1 --2- 172.21.15.120:0/3885114784 >> v2:172.21.15.73:3300/0 conn(0x7f03700ce310 0x7f0370071040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:09.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.935+0000 7f0375562640 1 -- 172.21.15.120:0/3885114784 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3553854677 0 0) 0x7f037015d530 con 0x7f037006c240 2024-04-26T20:50:09.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.935+0000 7f0375562640 1 -- 172.21.15.120:0/3885114784 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f035c003520 con 0x7f037006c240 2024-04-26T20:50:09.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.935+0000 7f0376d65640 1 -- 172.21.15.120:0/3885114784 >> v2:172.21.15.73:3300/0 conn(0x7f03700ce310 msgr2=0x7f0370071040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:09.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.935+0000 7f0376d65640 1 --2- 172.21.15.120:0/3885114784 >> v2:172.21.15.73:3300/0 conn(0x7f03700ce310 0x7f0370071040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:09.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.935+0000 7f0376d65640 1 -- 172.21.15.120:0/3885114784 >> v1:172.21.15.120:6789/0 conn(0x7f037006c240 legacy=0x7f037006c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:09.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.935+0000 7f0376d65640 1 -- 172.21.15.120:0/3885114784 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f037015ddb0 con 0x7f0370071580 2024-04-26T20:50:09.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.936+0000 7f0376d65640 1 --2- 172.21.15.120:0/3885114784 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0370071580 0x7f0370160b20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f037006d090 tx=0x7f036802a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99059aed99e8b9a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:09.939 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.936+0000 7f0375562640 1 -- 172.21.15.120:0/3885114784 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0368007740 con 0x7f0370071580 2024-04-26T20:50:09.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.936+0000 7f0375562640 1 -- 172.21.15.120:0/3885114784 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03680078e0 con 0x7f0370071580 2024-04-26T20:50:09.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.936+0000 7f0375562640 1 -- 172.21.15.120:0/3885114784 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0368007cb0 con 0x7f0370071580 2024-04-26T20:50:09.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.936+0000 7f03787c8640 1 -- 172.21.15.120:0/3885114784 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0370071580 msgr2=0x7f0370160b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:09.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.936+0000 7f03787c8640 1 --2- 172.21.15.120:0/3885114784 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0370071580 0x7f0370160b20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f037006d090 tx=0x7f036802a680 comp rx=0 tx=0).stop 2024-04-26T20:50:09.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.936+0000 7f03787c8640 1 -- 172.21.15.120:0/3885114784 shutdown_connections 2024-04-26T20:50:09.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.936+0000 7f03787c8640 1 --2- 172.21.15.120:0/3885114784 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0370071580 0x7f0370160b20 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:09.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.936+0000 7f03787c8640 1 --2- 172.21.15.120:0/3885114784 >> v2:172.21.15.73:3300/0 conn(0x7f03700ce310 0x7f0370071040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:09.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.936+0000 7f03787c8640 1 -- 172.21.15.120:0/3885114784 >> 172.21.15.120:0/3885114784 conn(0x7f03700d1cb0 msgr2=0x7f03700d2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:09.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.936+0000 7f03787c8640 1 -- 172.21.15.120:0/3885114784 shutdown_connections 2024-04-26T20:50:09.940 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.936+0000 7f03787c8640 1 -- 172.21.15.120:0/3885114784 wait complete. 2024-04-26T20:50:09.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.938+0000 7f03787c8640 1 Processor -- start 2024-04-26T20:50:09.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.938+0000 7f03787c8640 1 -- start start 2024-04-26T20:50:09.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.938+0000 7f03787c8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f037006c240 0x7f03701f74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:09.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.938+0000 7f03787c8640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0370071580 0x7f03701f2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:09.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.938+0000 7f03787c8640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03700ce310 0x7f037015b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:09.943 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.938+0000 7f03787c8640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0370161060 con 0x7f03700ce310 2024-04-26T20:50:09.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.938+0000 7f03787c8640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f03700b7850 con 0x7f037006c240 2024-04-26T20:50:09.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.938+0000 7f03787c8640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f03701611e0 con 0x7f0370071580 2024-04-26T20:50:09.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.938+0000 7f0376564640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f037006c240 0x7f03701f74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:09.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.938+0000 7f0376564640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f037006c240 0x7f03701f74c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.120:38466/0 (socket says 172.21.15.120:38466) 2024-04-26T20:50:09.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.938+0000 7f0376564640 1 -- 172.21.15.120:0/228890910 learned_addr learned my addr 172.21.15.120:0/228890910 (peer_addr_for_me v2:172.21.15.120:0/0) 2024-04-26T20:50:09.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.938+0000 7f0375d63640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0370071580 0x7f03701f2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:09.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.939+0000 7f0376d65640 1 --2- 172.21.15.120:0/228890910 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03700ce310 0x7f037015b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:09.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.939+0000 7f0376564640 1 -- 172.21.15.120:0/228890910 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0370071580 msgr2=0x7f03701f2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:09.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.939+0000 7f0376564640 1 --2- 172.21.15.120:0/228890910 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0370071580 0x7f03701f2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:09.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.939+0000 7f0376564640 1 -- 172.21.15.120:0/228890910 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03700ce310 msgr2=0x7f037015b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:09.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.939+0000 7f0376564640 1 --2- 172.21.15.120:0/228890910 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03700ce310 0x7f037015b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:09.944 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.939+0000 7f0376564640 1 -- 172.21.15.120:0/228890910 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03701f7a00 con 0x7f037006c240 2024-04-26T20:50:09.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.939+0000 7f0376564640 1 --2- 172.21.15.120:0/228890910 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f037006c240 0x7f03701f74c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f036000c430 tx=0x7f036000c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:09.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.939+0000 7f0366ffd640 1 -- 172.21.15.120:0/228890910 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0360017070 con 0x7f037006c240 2024-04-26T20:50:09.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.939+0000 7f0366ffd640 1 -- 172.21.15.120:0/228890910 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0360005ce0 con 0x7f037006c240 2024-04-26T20:50:09.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.939+0000 7f03787c8640 1 -- 172.21.15.120:0/228890910 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0370153d40 con 0x7f037006c240 2024-04-26T20:50:09.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.939+0000 7f0366ffd640 1 -- 172.21.15.120:0/228890910 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03600072f0 con 0x7f037006c240 2024-04-26T20:50:09.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.939+0000 7f03787c8640 1 -- 172.21.15.120:0/228890910 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03700c9c60 con 0x7f037006c240 2024-04-26T20:50:09.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.939+0000 7f0366ffd640 1 -- 172.21.15.120:0/228890910 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f036001a070 con 0x7f037006c240 2024-04-26T20:50:09.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.939+0000 7f0366ffd640 1 -- 172.21.15.120:0/228890910 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f0360012a60 con 0x7f037006c240 2024-04-26T20:50:09.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.940+0000 7f03787c8640 1 -- 172.21.15.120:0/228890910 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03700694f0 con 0x7f037006c240 2024-04-26T20:50:09.945 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:09.944+0000 7f0366ffd640 1 -- 172.21.15.120:0/228890910 <== mon.1 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f0360021020 con 0x7f037006c240 2024-04-26T20:50:10.017 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:10.016+0000 7f03787c8640 1 -- 172.21.15.120:0/228890910 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c0c3bf6d-a4c5-4f1e-aa2d-330951830666", "id": 7} v 0) -- 0x7f0370157730 con 0x7f037006c240 2024-04-26T20:50:10.025 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:10.024+0000 7f0366ffd640 1 -- 172.21.15.120:0/228890910 <== mon.1 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c0c3bf6d-a4c5-4f1e-aa2d-330951830666", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f0360007670 con 0x7f037006c240 2024-04-26T20:50:10.025 INFO:teuthology.orchestra.run.smithi120.stdout:7 2024-04-26T20:50:10.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:10.025+0000 7f03787c8640 1 -- 172.21.15.120:0/228890910 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f037006c240 msgr2=0x7f03701f74c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:10.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:10.025+0000 7f03787c8640 1 --2- 172.21.15.120:0/228890910 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f037006c240 0x7f03701f74c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f036000c430 tx=0x7f036000c8f0 comp rx=0 tx=0).stop 2024-04-26T20:50:10.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:10.025+0000 7f03787c8640 1 -- 172.21.15.120:0/228890910 shutdown_connections 2024-04-26T20:50:10.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:10.025+0000 7f03787c8640 1 --2- 172.21.15.120:0/228890910 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03700ce310 0x7f037015b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:10.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:10.025+0000 7f03787c8640 1 --2- 172.21.15.120:0/228890910 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0370071580 0x7f03701f2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:10.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:10.025+0000 7f03787c8640 1 --2- 172.21.15.120:0/228890910 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f037006c240 0x7f03701f74c0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:10.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:10.025+0000 7f03787c8640 1 -- 172.21.15.120:0/228890910 >> 172.21.15.120:0/228890910 conn(0x7f03700d1cb0 msgr2=0x7f03700b1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:10.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:10.025+0000 7f03787c8640 1 -- 172.21.15.120:0/228890910 shutdown_connections 2024-04-26T20:50:10.027 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:50:10.025+0000 7f03787c8640 1 -- 172.21.15.120:0/228890910 wait complete. 2024-04-26T20:50:10.036 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-26T20:50:10.037 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-26T20:50:10.039 INFO:tasks.ceph.osd.0:Started 2024-04-26T20:50:10.039 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-26T20:50:10.039 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-26T20:50:10.041 INFO:tasks.ceph.osd.1:Started 2024-04-26T20:50:10.041 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-26T20:50:10.042 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-26T20:50:10.044 INFO:tasks.ceph.osd.2:Started 2024-04-26T20:50:10.044 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-26T20:50:10.044 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-26T20:50:10.046 INFO:tasks.ceph.osd.3:Started 2024-04-26T20:50:10.046 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-26T20:50:10.047 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-26T20:50:10.079 INFO:tasks.ceph.osd.4:Started 2024-04-26T20:50:10.079 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-26T20:50:10.079 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-26T20:50:10.081 INFO:tasks.ceph.osd.5:Started 2024-04-26T20:50:10.081 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-26T20:50:10.081 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-26T20:50:10.083 INFO:tasks.ceph.osd.6:Started 2024-04-26T20:50:10.083 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-26T20:50:10.084 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-26T20:50:10.087 INFO:tasks.ceph.osd.7:Started 2024-04-26T20:50:10.088 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:50:10.137 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-26T20:50:10.136+0000 7fba87221040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T20:50:10.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.204+0000 7f5850409640 1 Processor -- start 2024-04-26T20:50:10.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.204+0000 7f5850409640 1 -- start start 2024-04-26T20:50:10.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.205+0000 7f5850409640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f5848152fe0 0x7f5848072700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:10.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.205+0000 7f5850409640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5848072c40 0x7f58480b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:10.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.205+0000 7f5850409640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58480b5a50 con 0x7f58480b5220 2024-04-26T20:50:10.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.205+0000 7f5850409640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f5848073080 con 0x7f5848152fe0 2024-04-26T20:50:10.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.205+0000 7f5850409640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f58480b5d30 con 0x7f5848072c40 2024-04-26T20:50:10.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.205+0000 7f584e9a6640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f58480b5220 0x7f584815f9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:46546/0 (socket says 172.21.15.73:46546) 2024-04-26T20:50:10.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.205+0000 7f584e9a6640 1 -- 172.21.15.73:0/211681802 learned_addr learned my addr 172.21.15.73:0/211681802 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:50:10.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.205+0000 7f584d9a4640 1 --2- 172.21.15.73:0/211681802 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5848072c40 0x7f58480b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:10.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.205+0000 7f584e1a5640 1 --2- 172.21.15.73:0/211681802 >> v2:172.21.15.120:3300/0 conn(0x7f5848152fe0 0x7f5848072700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:10.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.205+0000 7f584d1a3640 1 -- 172.21.15.73:0/211681802 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4146396451 0 0) 0x7f58480b5a50 con 0x7f58480b5220 2024-04-26T20:50:10.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.205+0000 7f584d1a3640 1 -- 172.21.15.73:0/211681802 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5834003520 con 0x7f58480b5220 2024-04-26T20:50:10.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.205+0000 7f584d9a4640 1 -- 172.21.15.73:0/211681802 >> v2:172.21.15.120:3300/0 conn(0x7f5848152fe0 msgr2=0x7f5848072700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:10.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.205+0000 7f584d9a4640 1 --2- 172.21.15.73:0/211681802 >> v2:172.21.15.120:3300/0 conn(0x7f5848152fe0 0x7f5848072700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:10.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.205+0000 7f584d9a4640 1 -- 172.21.15.73:0/211681802 >> v1:172.21.15.73:6789/0 conn(0x7f58480b5220 legacy=0x7f584815f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:10.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.205+0000 7f584d9a4640 1 -- 172.21.15.73:0/211681802 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f584806e8e0 con 0x7f5848072c40 2024-04-26T20:50:10.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.206+0000 7f584d9a4640 1 --2- 172.21.15.73:0/211681802 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5848072c40 0x7f58480b4ce0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f58380097a0 tx=0x7f583802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd113e4e99e19107 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:10.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.206+0000 7f584d1a3640 1 -- 172.21.15.73:0/211681802 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5838037070 con 0x7f5848072c40 2024-04-26T20:50:10.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.206+0000 7f584d1a3640 1 -- 172.21.15.73:0/211681802 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5838032440 con 0x7f5848072c40 2024-04-26T20:50:10.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.206+0000 7f584d1a3640 1 -- 172.21.15.73:0/211681802 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5838032810 con 0x7f5848072c40 2024-04-26T20:50:10.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.207+0000 7f5850409640 1 -- 172.21.15.73:0/211681802 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5848072c40 msgr2=0x7f58480b4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:10.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.207+0000 7f5850409640 1 --2- 172.21.15.73:0/211681802 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5848072c40 0x7f58480b4ce0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f58380097a0 tx=0x7f583802a180 comp rx=0 tx=0).stop 2024-04-26T20:50:10.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.207+0000 7f5850409640 1 -- 172.21.15.73:0/211681802 shutdown_connections 2024-04-26T20:50:10.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.207+0000 7f5850409640 1 --2- 172.21.15.73:0/211681802 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5848072c40 0x7f58480b4ce0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:10.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.207+0000 7f5850409640 1 --2- 172.21.15.73:0/211681802 >> v2:172.21.15.120:3300/0 conn(0x7f5848152fe0 0x7f5848072700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:10.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.207+0000 7f5850409640 1 -- 172.21.15.73:0/211681802 >> 172.21.15.73:0/211681802 conn(0x7f58480b3110 msgr2=0x7f58480b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:10.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.207+0000 7f5850409640 1 -- 172.21.15.73:0/211681802 shutdown_connections 2024-04-26T20:50:10.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.207+0000 7f5850409640 1 -- 172.21.15.73:0/211681802 wait complete. 2024-04-26T20:50:10.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.208+0000 7f5850409640 1 Processor -- start 2024-04-26T20:50:10.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.208+0000 7f5850409640 1 -- start start 2024-04-26T20:50:10.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.209+0000 7f5850409640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5848072c40 0x7f5848145de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:10.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.209+0000 7f5850409640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58480b5220 0x7f5848154900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:10.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.209+0000 7f5850409640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5848152fe0 0x7f584806ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:10.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.209+0000 7f5850409640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f58480b5830 con 0x7f5848072c40 2024-04-26T20:50:10.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.209+0000 7f5850409640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f58480b56b0 con 0x7f58480b5220 2024-04-26T20:50:10.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.209+0000 7f5850409640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f58480700f0 con 0x7f5848152fe0 2024-04-26T20:50:10.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.209+0000 7f584e1a5640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5848072c40 0x7f5848145de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:10.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.209+0000 7f584e1a5640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5848072c40 0x7f5848145de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36790/0 (socket says 172.21.15.73:36790) 2024-04-26T20:50:10.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.209+0000 7f584e1a5640 1 -- 172.21.15.73:0/3978026947 learned_addr learned my addr 172.21.15.73:0/3978026947 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:10.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.209+0000 7f584e9a6640 1 --2- 172.21.15.73:0/3978026947 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5848152fe0 0x7f584806ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:10.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.209+0000 7f584d9a4640 1 --2- 172.21.15.73:0/3978026947 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58480b5220 0x7f5848154900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:10.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.209+0000 7f584e1a5640 1 -- 172.21.15.73:0/3978026947 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5848152fe0 msgr2=0x7f584806ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:10.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.209+0000 7f584e1a5640 1 --2- 172.21.15.73:0/3978026947 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5848152fe0 0x7f584806ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:10.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.209+0000 7f584e1a5640 1 -- 172.21.15.73:0/3978026947 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58480b5220 msgr2=0x7f5848154900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:10.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.209+0000 7f584e1a5640 1 --2- 172.21.15.73:0/3978026947 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58480b5220 0x7f5848154900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:10.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.209+0000 7f584e1a5640 1 -- 172.21.15.73:0/3978026947 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5848155230 con 0x7f5848072c40 2024-04-26T20:50:10.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.209+0000 7f584e1a5640 1 --2- 172.21.15.73:0/3978026947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5848072c40 0x7f5848145de0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f584400e870 tx=0x7f584400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:10.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.209+0000 7f583effd640 1 -- 172.21.15.73:0/3978026947 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5844017070 con 0x7f5848072c40 2024-04-26T20:50:10.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.209+0000 7f583effd640 1 -- 172.21.15.73:0/3978026947 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f584400c940 con 0x7f5848072c40 2024-04-26T20:50:10.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.209+0000 7f5850409640 1 -- 172.21.15.73:0/3978026947 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5848155510 con 0x7f5848072c40 2024-04-26T20:50:10.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.209+0000 7f583effd640 1 -- 172.21.15.73:0/3978026947 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584400cd30 con 0x7f5848072c40 2024-04-26T20:50:10.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.209+0000 7f5850409640 1 -- 172.21.15.73:0/3978026947 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5848155ad0 con 0x7f5848072c40 2024-04-26T20:50:10.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.209+0000 7f583effd640 1 -- 172.21.15.73:0/3978026947 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f584401a070 con 0x7f5848072c40 2024-04-26T20:50:10.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.210+0000 7f583effd640 1 -- 172.21.15.73:0/3978026947 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f584401cb20 con 0x7f5848072c40 2024-04-26T20:50:10.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.210+0000 7f5850409640 1 -- 172.21.15.73:0/3978026947 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5848068fc0 con 0x7f5848072c40 2024-04-26T20:50:10.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.212+0000 7f583effd640 1 -- 172.21.15.73:0/3978026947 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f584401c450 con 0x7f5848072c40 2024-04-26T20:50:10.270 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-04-26T20:50:10.269+0000 7f5a58a1b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T20:50:10.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.274+0000 7f5850409640 1 -- 172.21.15.73:0/3978026947 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5848073300 con 0x7f5848072c40 2024-04-26T20:50:10.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.274+0000 7f583effd640 1 -- 172.21.15.73:0/3978026947 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f584401c630 con 0x7f5848072c40 2024-04-26T20:50:10.275 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:50:10.275 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":10,"fsid":"fbd3f548-b547-4713-856b-09f98f5d1764","created":"2024-04-26T20:50:07.876786+0000","modified":"2024-04-26T20:50:10.018400+0000","last_up_change":"0.000000","last_in_change":"2024-04-26T20:50:10.018400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"72d6f4f3-214a-4392-a02f-c33cbcab3c7a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"786bf8ca-6ddc-4b3c-be6c-9152398bd5fd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"90963b1a-5437-4ef8-ad8b-6b74c534eeb5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"bd3d2148-6e38-41b4-a453-fe6e3b8b4ceb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"e8668dc8-a23f-46d2-8311-cb882da08e74","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"cbac0bdb-96bd-4852-bcce-e564495c3fcc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"1884b2a8-569f-4272-96a9-f25a09364dc2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"c0c3bf6d-a4c5-4f1e-aa2d-330951830666","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:50:10.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.275+0000 7f5850409640 1 -- 172.21.15.73:0/3978026947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5848072c40 msgr2=0x7f5848145de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:10.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.275+0000 7f5850409640 1 --2- 172.21.15.73:0/3978026947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5848072c40 0x7f5848145de0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f584400e870 tx=0x7f584400ed30 comp rx=0 tx=0).stop 2024-04-26T20:50:10.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.275+0000 7f5850409640 1 -- 172.21.15.73:0/3978026947 shutdown_connections 2024-04-26T20:50:10.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.275+0000 7f5850409640 1 --2- 172.21.15.73:0/3978026947 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5848152fe0 0x7f584806ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:10.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.275+0000 7f5850409640 1 --2- 172.21.15.73:0/3978026947 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58480b5220 0x7f5848154900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:10.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.275+0000 7f5850409640 1 --2- 172.21.15.73:0/3978026947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5848072c40 0x7f5848145de0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:10.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.275+0000 7f5850409640 1 -- 172.21.15.73:0/3978026947 >> 172.21.15.73:0/3978026947 conn(0x7f58480b3110 msgr2=0x7f584815ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:10.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.275+0000 7f5850409640 1 -- 172.21.15.73:0/3978026947 shutdown_connections 2024-04-26T20:50:10.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:10.275+0000 7f5850409640 1 -- 172.21.15.73:0/3978026947 wait complete. 2024-04-26T20:50:10.287 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-26T20:50:10.287 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-26T20:50:10.406 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-26T20:50:10.405+0000 7fba87221040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T20:50:10.460 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-26T20:50:10.459+0000 7fba87221040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T20:50:10.530 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-04-26T20:50:10.529+0000 7f5a58a1b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T20:50:10.574 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-26T20:50:10.573+0000 7fba87221040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T20:50:10.582 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-04-26T20:50:10.581+0000 7f5a58a1b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T20:50:10.648 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-04-26T20:50:10.648+0000 7f3b27e10640 -1 Falling back to public interface 2024-04-26T20:50:10.649 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-26T20:50:10.648+0000 7f011ec0c640 -1 Falling back to public interface 2024-04-26T20:50:10.661 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-04-26T20:50:10.660+0000 7f3351f2c640 -1 Falling back to public interface 2024-04-26T20:50:10.662 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-04-26T20:50:10.661+0000 7fc65a8ee640 -1 Falling back to public interface 2024-04-26T20:50:10.689 INFO:tasks.ceph.osd.6.smithi120.stderr:2024-04-26T20:50:10.688+0000 7f8eb3968640 -1 Falling back to public interface 2024-04-26T20:50:10.691 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-26T20:50:10.690+0000 7f2b2e0dd640 -1 Falling back to public interface 2024-04-26T20:50:10.698 INFO:tasks.ceph.osd.7.smithi120.stderr:2024-04-26T20:50:10.697+0000 7fd03b3af640 -1 Falling back to public interface 2024-04-26T20:50:10.698 INFO:tasks.ceph.osd.4.smithi120.stderr:2024-04-26T20:50:10.697+0000 7f80cb3d4640 -1 Falling back to public interface 2024-04-26T20:50:10.701 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-04-26T20:50:10.700+0000 7f5a58a1b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T20:50:10.858 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-26T20:50:10.857+0000 7fba87221040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T20:50:10.906 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-26T20:50:10.905+0000 7fba87221040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T20:50:10.986 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-04-26T20:50:10.985+0000 7f5a58a1b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T20:50:11.033 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-04-26T20:50:11.032+0000 7f5a58a1b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T20:50:11.064 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-26T20:50:11.063+0000 7fba87221040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T20:50:11.092 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-26T20:50:11.186 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.181+0000 7f23260ea640 1 Processor -- start 2024-04-26T20:50:11.186 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.181+0000 7f23260ea640 1 -- start start 2024-04-26T20:50:11.186 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.182+0000 7f23260ea640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f232015f4f0 0x7f23200d1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:11.186 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.182+0000 7f23260ea640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f23200d2050 0x7f232006dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:11.186 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.182+0000 7f23260ea640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23200b2610 con 0x7f232006e280 2024-04-26T20:50:11.186 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.182+0000 7f23260ea640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f232015f8d0 con 0x7f23200d2050 2024-04-26T20:50:11.187 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.182+0000 7f23260ea640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f232006e6d0 con 0x7f232015f4f0 2024-04-26T20:50:11.187 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.182+0000 7f231effd640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f23200d2050 0x7f232006dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:11.187 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.182+0000 7f231ffff640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f232006e280 0x7f23200b3690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:46556/0 (socket says 172.21.15.73:46556) 2024-04-26T20:50:11.187 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.182+0000 7f231ffff640 1 -- 172.21.15.73:0/1021302296 learned_addr learned my addr 172.21.15.73:0/1021302296 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:50:11.187 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.183+0000 7f231f7fe640 1 --2- 172.21.15.73:0/1021302296 >> v2:172.21.15.120:3300/0 conn(0x7f232015f4f0 0x7f23200d1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:11.187 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.183+0000 7f231effd640 1 -- 172.21.15.73:0/1021302296 >> v2:172.21.15.120:3300/0 conn(0x7f232015f4f0 msgr2=0x7f23200d1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:11.188 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.183+0000 7f231effd640 1 --2- 172.21.15.73:0/1021302296 >> v2:172.21.15.120:3300/0 conn(0x7f232015f4f0 0x7f23200d1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:11.188 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.183+0000 7f231effd640 1 -- 172.21.15.73:0/1021302296 >> v1:172.21.15.73:6789/0 conn(0x7f232006e280 legacy=0x7f23200b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:11.188 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.183+0000 7f231effd640 1 -- 172.21.15.73:0/1021302296 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23200b28d0 con 0x7f23200d2050 2024-04-26T20:50:11.188 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.183+0000 7f231effd640 1 --2- 172.21.15.73:0/1021302296 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23200d2050 0x7f232006dd40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2320073150 tx=0x7f231002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec8eb4c167c38c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:11.188 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.183+0000 7f231e7fc640 1 -- 172.21.15.73:0/1021302296 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2310037070 con 0x7f23200d2050 2024-04-26T20:50:11.188 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.184+0000 7f231e7fc640 1 -- 172.21.15.73:0/1021302296 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2310032440 con 0x7f23200d2050 2024-04-26T20:50:11.188 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.184+0000 7f231e7fc640 1 -- 172.21.15.73:0/1021302296 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2310032810 con 0x7f23200d2050 2024-04-26T20:50:11.189 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.184+0000 7f23260ea640 1 -- 172.21.15.73:0/1021302296 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23200d2050 msgr2=0x7f232006dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:11.189 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.184+0000 7f23260ea640 1 --2- 172.21.15.73:0/1021302296 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23200d2050 0x7f232006dd40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2320073150 tx=0x7f231002a1a0 comp rx=0 tx=0).stop 2024-04-26T20:50:11.189 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.184+0000 7f23260ea640 1 -- 172.21.15.73:0/1021302296 shutdown_connections 2024-04-26T20:50:11.189 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.184+0000 7f23260ea640 1 --2- 172.21.15.73:0/1021302296 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23200d2050 0x7f232006dd40 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:11.190 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.184+0000 7f23260ea640 1 --2- 172.21.15.73:0/1021302296 >> v2:172.21.15.120:3300/0 conn(0x7f232015f4f0 0x7f23200d1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:11.190 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.184+0000 7f23260ea640 1 -- 172.21.15.73:0/1021302296 >> 172.21.15.73:0/1021302296 conn(0x7f23200d2e00 msgr2=0x7f23200d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:11.190 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.184+0000 7f23260ea640 1 -- 172.21.15.73:0/1021302296 shutdown_connections 2024-04-26T20:50:11.190 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.184+0000 7f23260ea640 1 -- 172.21.15.73:0/1021302296 wait complete. 2024-04-26T20:50:11.190 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-04-26T20:50:11.186+0000 7f5a58a1b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T20:50:11.194 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.185+0000 7f23260ea640 1 Processor -- start 2024-04-26T20:50:11.194 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.186+0000 7f23260ea640 1 -- start start 2024-04-26T20:50:11.194 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.186+0000 7f23260ea640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f232006e280 0x7f2320156240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:11.194 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.186+0000 7f23260ea640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23200d2050 0x7f23200b7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:11.194 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.186+0000 7f23260ea640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f232015f4f0 0x7f2320161eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:11.195 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.186+0000 7f23260ea640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f23200b2e90 con 0x7f23200d2050 2024-04-26T20:50:11.195 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.186+0000 7f23260ea640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f23200b3200 con 0x7f232015f4f0 2024-04-26T20:50:11.195 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.186+0000 7f23260ea640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f23200afad0 con 0x7f232006e280 2024-04-26T20:50:11.195 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.186+0000 7f231effd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23200d2050 0x7f23200b7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:11.195 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.186+0000 7f231effd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23200d2050 0x7f23200b7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36804/0 (socket says 172.21.15.73:36804) 2024-04-26T20:50:11.195 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.186+0000 7f231effd640 1 -- 172.21.15.73:0/2396481482 learned_addr learned my addr 172.21.15.73:0/2396481482 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:11.196 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.186+0000 7f231f7fe640 1 --2- 172.21.15.73:0/2396481482 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f232006e280 0x7f2320156240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:11.196 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.186+0000 7f231ffff640 1 --2- 172.21.15.73:0/2396481482 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f232015f4f0 0x7f2320161eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:11.196 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.187+0000 7f231effd640 1 -- 172.21.15.73:0/2396481482 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f232006e280 msgr2=0x7f2320156240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:11.196 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.187+0000 7f231effd640 1 --2- 172.21.15.73:0/2396481482 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f232006e280 0x7f2320156240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:11.196 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.187+0000 7f231effd640 1 -- 172.21.15.73:0/2396481482 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f232015f4f0 msgr2=0x7f2320161eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:11.196 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.187+0000 7f231effd640 1 --2- 172.21.15.73:0/2396481482 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f232015f4f0 0x7f2320161eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:11.196 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.187+0000 7f231effd640 1 -- 172.21.15.73:0/2396481482 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23200b5c50 con 0x7f23200d2050 2024-04-26T20:50:11.197 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.187+0000 7f231effd640 1 --2- 172.21.15.73:0/2396481482 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23200d2050 0x7f23200b7ff0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2310035040 tx=0x7f23100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:11.197 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.187+0000 7f2307fff640 1 -- 172.21.15.73:0/2396481482 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2310037040 con 0x7f23200d2050 2024-04-26T20:50:11.197 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.187+0000 7f2307fff640 1 -- 172.21.15.73:0/2396481482 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2310042070 con 0x7f23200d2050 2024-04-26T20:50:11.197 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.187+0000 7f23260ea640 1 -- 172.21.15.73:0/2396481482 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23200cd9a0 con 0x7f23200d2050 2024-04-26T20:50:11.197 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.187+0000 7f2307fff640 1 -- 172.21.15.73:0/2396481482 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2310030960 con 0x7f23200d2050 2024-04-26T20:50:11.197 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.187+0000 7f23260ea640 1 -- 172.21.15.73:0/2396481482 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23200cdf60 con 0x7f23200d2050 2024-04-26T20:50:11.197 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.187+0000 7f2307fff640 1 -- 172.21.15.73:0/2396481482 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2310030b00 con 0x7f23200d2050 2024-04-26T20:50:11.198 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.187+0000 7f2307fff640 1 -- 172.21.15.73:0/2396481482 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f2310047b40 con 0x7f23200d2050 2024-04-26T20:50:11.198 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.188+0000 7f23260ea640 1 -- 172.21.15.73:0/2396481482 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2320076610 con 0x7f23200d2050 2024-04-26T20:50:11.198 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.190+0000 7f2307fff640 1 -- 172.21.15.73:0/2396481482 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f2310047460 con 0x7f23200d2050 2024-04-26T20:50:11.206 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-26T20:50:11.206+0000 7fba87221040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T20:50:11.253 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.252+0000 7f23260ea640 1 -- 172.21.15.73:0/2396481482 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f23200b3df0 con 0x7f23200d2050 2024-04-26T20:50:11.253 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.252+0000 7f2307fff640 1 -- 172.21.15.73:0/2396481482 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f2310047640 con 0x7f23200d2050 2024-04-26T20:50:11.253 INFO:teuthology.misc.health.smithi073.stdout: 2024-04-26T20:50:11.254 INFO:teuthology.misc.health.smithi073.stdout:{"epoch":10,"fsid":"fbd3f548-b547-4713-856b-09f98f5d1764","created":"2024-04-26T20:50:07.876786+0000","modified":"2024-04-26T20:50:10.018400+0000","last_up_change":"0.000000","last_in_change":"2024-04-26T20:50:10.018400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"72d6f4f3-214a-4392-a02f-c33cbcab3c7a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"786bf8ca-6ddc-4b3c-be6c-9152398bd5fd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"90963b1a-5437-4ef8-ad8b-6b74c534eeb5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"bd3d2148-6e38-41b4-a453-fe6e3b8b4ceb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"e8668dc8-a23f-46d2-8311-cb882da08e74","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"cbac0bdb-96bd-4852-bcce-e564495c3fcc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"1884b2a8-569f-4272-96a9-f25a09364dc2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"c0c3bf6d-a4c5-4f1e-aa2d-330951830666","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:50:11.255 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.253+0000 7f23260ea640 1 -- 172.21.15.73:0/2396481482 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23200d2050 msgr2=0x7f23200b7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:11.255 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.253+0000 7f23260ea640 1 --2- 172.21.15.73:0/2396481482 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23200d2050 0x7f23200b7ff0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2310035040 tx=0x7f23100046c0 comp rx=0 tx=0).stop 2024-04-26T20:50:11.255 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.253+0000 7f23260ea640 1 -- 172.21.15.73:0/2396481482 shutdown_connections 2024-04-26T20:50:11.255 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.253+0000 7f23260ea640 1 --2- 172.21.15.73:0/2396481482 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23200d2050 0x7f23200b7ff0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:11.255 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.253+0000 7f23260ea640 1 --2- 172.21.15.73:0/2396481482 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f232015f4f0 0x7f2320161eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:11.255 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.253+0000 7f23260ea640 1 --2- 172.21.15.73:0/2396481482 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f232006e280 0x7f2320156240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:11.255 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.253+0000 7f23260ea640 1 -- 172.21.15.73:0/2396481482 >> 172.21.15.73:0/2396481482 conn(0x7f23200d2e00 msgr2=0x7f23200b4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:11.255 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.253+0000 7f23260ea640 1 -- 172.21.15.73:0/2396481482 shutdown_connections 2024-04-26T20:50:11.256 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:11.253+0000 7f23260ea640 1 -- 172.21.15.73:0/2396481482 wait complete. 2024-04-26T20:50:11.258 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-26T20:50:11.257+0000 7fba87221040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T20:50:11.263 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-26T20:50:11.324 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-04-26T20:50:11.323+0000 7f5a58a1b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T20:50:11.329 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-26T20:50:11.328+0000 7fba87221040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T20:50:11.373 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-04-26T20:50:11.371+0000 7f5a58a1b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T20:50:11.374 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-26T20:50:11.373+0000 7fba87221040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T20:50:11.441 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-04-26T20:50:11.440+0000 7f5a58a1b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T20:50:11.487 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-04-26T20:50:11.486+0000 7f5a58a1b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T20:50:11.853 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-26T20:50:11.852+0000 7fba87221040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T20:50:11.967 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-04-26T20:50:11.966+0000 7f5a58a1b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T20:50:11.969 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-26T20:50:11.968+0000 7fba87221040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T20:50:12.090 INFO:tasks.ceph.mgr.x.smithi120.stderr:2024-04-26T20:50:12.089+0000 7f5a58a1b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T20:50:12.982 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-26T20:50:12.981+0000 7fba27136640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-26T20:50:12.982 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-26T20:50:12.981+0000 7fba27136640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-26T20:50:14.130 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-04-26T20:50:14.129+0000 7f3b27e10640 -1 osd.2 0 log_to_monitors true 2024-04-26T20:50:14.175 INFO:tasks.ceph.osd.4.smithi120.stderr:2024-04-26T20:50:14.174+0000 7f80cb3d4640 -1 osd.4 0 log_to_monitors true 2024-04-26T20:50:14.183 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-26T20:50:14.183+0000 7f011ec0c640 -1 osd.0 0 log_to_monitors true 2024-04-26T20:50:14.187 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-04-26T20:50:14.187+0000 7fc65a8ee640 -1 osd.3 0 log_to_monitors true 2024-04-26T20:50:14.193 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-04-26T20:50:14.193+0000 7f3351f2c640 -1 osd.1 0 log_to_monitors true 2024-04-26T20:50:14.206 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-26T20:50:14.205+0000 7f2b2e0dd640 -1 osd.5 0 log_to_monitors true 2024-04-26T20:50:14.210 INFO:tasks.ceph.osd.6.smithi120.stderr:2024-04-26T20:50:14.209+0000 7f8eb3968640 -1 osd.6 0 log_to_monitors true 2024-04-26T20:50:14.218 INFO:tasks.ceph.osd.7.smithi120.stderr:2024-04-26T20:50:14.217+0000 7fd03b3af640 -1 osd.7 0 log_to_monitors true 2024-04-26T20:50:18.062 INFO:tasks.ceph.osd.7.smithi120.stderr:2024-04-26T20:50:18.059+0000 7fd037345640 -1 osd.7 0 waiting for initial osdmap 2024-04-26T20:50:18.071 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-26T20:50:18.079 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-26T20:50:18.078+0000 7f2b2a073640 -1 osd.5 0 waiting for initial osdmap 2024-04-26T20:50:18.083 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-04-26T20:50:18.082+0000 7f3b23851640 -1 osd.2 0 waiting for initial osdmap 2024-04-26T20:50:18.084 INFO:tasks.ceph.osd.7.smithi120.stderr:2024-04-26T20:50:18.083+0000 7fd02f165640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T20:50:18.090 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-26T20:50:18.086+0000 7f011a48c640 -1 osd.0 0 waiting for initial osdmap 2024-04-26T20:50:18.093 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-04-26T20:50:18.092+0000 7f334dec2640 -1 osd.1 0 waiting for initial osdmap 2024-04-26T20:50:18.095 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-26T20:50:18.094+0000 7f2b21e93640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T20:50:18.101 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-04-26T20:50:18.100+0000 7f3b1b671640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T20:50:18.105 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-26T20:50:18.104+0000 7f01122ac640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T20:50:18.107 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-04-26T20:50:18.107+0000 7f3345ce2640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T20:50:18.112 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-04-26T20:50:18.111+0000 7fc656884640 -1 osd.3 0 waiting for initial osdmap 2024-04-26T20:50:18.126 INFO:tasks.ceph.osd.3.smithi073.stderr:2024-04-26T20:50:18.125+0000 7fc64e6a4640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T20:50:18.131 INFO:tasks.ceph.osd.6.smithi120.stderr:2024-04-26T20:50:18.130+0000 7f8eaf8fe640 -1 osd.6 0 waiting for initial osdmap 2024-04-26T20:50:18.151 INFO:tasks.ceph.osd.4.smithi120.stderr:2024-04-26T20:50:18.150+0000 7f80c736a640 -1 osd.4 0 waiting for initial osdmap 2024-04-26T20:50:18.154 INFO:tasks.ceph.osd.6.smithi120.stderr:2024-04-26T20:50:18.153+0000 7f8ea771e640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T20:50:18.166 INFO:tasks.ceph.osd.4.smithi120.stderr:2024-04-26T20:50:18.165+0000 7f80bf18a640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T20:50:18.174 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.173+0000 7f7b1f862640 1 Processor -- start 2024-04-26T20:50:18.177 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.174+0000 7f7b1f862640 1 -- start start 2024-04-26T20:50:18.182 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.174+0000 7f7b1f862640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7b18072860 0x7f7b180720f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:18.183 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.174+0000 7f7b1f862640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b1815f740 con 0x7f7b1815f360 2024-04-26T20:50:18.183 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.174+0000 7f7b1f862640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b1806e320 con 0x7f7b18072c90 2024-04-26T20:50:18.183 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.174+0000 7f7b1f862640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f7b180cf2b0 con 0x7f7b18072860 2024-04-26T20:50:18.183 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.174+0000 7f7b1cdfd640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7b18072860 0x7f7b180720f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:18.183 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.175+0000 7f7b1d5fe640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f7b1815f360 0x7f7b180b4600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:46560/0 (socket says 172.21.15.73:46560) 2024-04-26T20:50:18.183 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.175+0000 7f7b1d5fe640 1 -- 172.21.15.73:0/668806081 learned_addr learned my addr 172.21.15.73:0/668806081 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:50:18.183 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.175+0000 7f7b1cdfd640 1 -- 172.21.15.73:0/668806081 >> v1:172.21.15.120:6789/0 conn(0x7f7b18072c90 legacy=0x7f7b180cfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:18.183 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.175+0000 7f7b1cdfd640 1 -- 172.21.15.73:0/668806081 >> v1:172.21.15.73:6789/0 conn(0x7f7b1815f360 legacy=0x7f7b180b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:18.183 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.175+0000 7f7b1cdfd640 1 -- 172.21.15.73:0/668806081 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b180cf590 con 0x7f7b18072860 2024-04-26T20:50:18.184 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.175+0000 7f7b0bfff640 1 -- 172.21.15.73:0/668806081 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2447084580 0 0) 0x7f7b1815f740 con 0x7f7b1815f360 2024-04-26T20:50:18.184 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.175+0000 7f7b1cdfd640 1 --2- 172.21.15.73:0/668806081 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b18072860 0x7f7b180720f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7b140097a0 tx=0x7f7b1402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75374b45ad8254b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:18.184 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.176+0000 7f7b0bfff640 1 -- 172.21.15.73:0/668806081 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b14037070 con 0x7f7b18072860 2024-04-26T20:50:18.184 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.176+0000 7f7b0bfff640 1 -- 172.21.15.73:0/668806081 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b14032440 con 0x7f7b18072860 2024-04-26T20:50:18.184 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.176+0000 7f7b0bfff640 1 -- 172.21.15.73:0/668806081 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b14032810 con 0x7f7b18072860 2024-04-26T20:50:18.184 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.176+0000 7f7b1f862640 1 -- 172.21.15.73:0/668806081 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b18072860 msgr2=0x7f7b180720f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:18.184 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.176+0000 7f7b1f862640 1 --2- 172.21.15.73:0/668806081 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b18072860 0x7f7b180720f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7b140097a0 tx=0x7f7b1402a230 comp rx=0 tx=0).stop 2024-04-26T20:50:18.184 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.176+0000 7f7b1f862640 1 -- 172.21.15.73:0/668806081 shutdown_connections 2024-04-26T20:50:18.184 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.176+0000 7f7b1f862640 1 --2- 172.21.15.73:0/668806081 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b18072860 0x7f7b180720f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:18.184 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.176+0000 7f7b1f862640 1 -- 172.21.15.73:0/668806081 >> 172.21.15.73:0/668806081 conn(0x7f7b180d0b30 msgr2=0x7f7b180d0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:18.185 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.176+0000 7f7b1f862640 1 -- 172.21.15.73:0/668806081 shutdown_connections 2024-04-26T20:50:18.185 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.176+0000 7f7b1f862640 1 -- 172.21.15.73:0/668806081 wait complete. 2024-04-26T20:50:18.185 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.177+0000 7f7b1f862640 1 Processor -- start 2024-04-26T20:50:18.185 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.177+0000 7f7b1f862640 1 -- start start 2024-04-26T20:50:18.185 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.178+0000 7f7b1f862640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b18072860 0x7f7b181438a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:18.185 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.178+0000 7f7b1f862640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7b18072c90 0x7f7b180aa4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:18.186 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.178+0000 7f7b1f862640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b1815f360 0x7f7b180aaa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:18.186 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.178+0000 7f7b1f862640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1806e0c0 con 0x7f7b18072860 2024-04-26T20:50:18.186 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.178+0000 7f7b1f862640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1806df40 con 0x7f7b1815f360 2024-04-26T20:50:18.186 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.178+0000 7f7b1f862640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f7b180d04a0 con 0x7f7b18072c90 2024-04-26T20:50:18.186 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.178+0000 7f7b1d5fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b18072860 0x7f7b181438a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:18.186 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.178+0000 7f7b1d5fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b18072860 0x7f7b181438a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36904/0 (socket says 172.21.15.73:36904) 2024-04-26T20:50:18.186 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.178+0000 7f7b1d5fe640 1 -- 172.21.15.73:0/3359517344 learned_addr learned my addr 172.21.15.73:0/3359517344 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:18.186 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.178+0000 7f7b1d5fe640 1 -- 172.21.15.73:0/3359517344 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7b18072c90 msgr2=0x7f7b180aa4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:18.186 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.178+0000 7f7b1d5fe640 1 --2- 172.21.15.73:0/3359517344 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7b18072c90 0x7f7b180aa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:18.186 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.178+0000 7f7b1d5fe640 1 -- 172.21.15.73:0/3359517344 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b1815f360 msgr2=0x7f7b180aaa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:18.187 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.178+0000 7f7b1ddff640 1 --2- 172.21.15.73:0/3359517344 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b1815f360 0x7f7b180aaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:18.187 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.178+0000 7f7b1d5fe640 1 --2- 172.21.15.73:0/3359517344 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b1815f360 0x7f7b180aaa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:18.187 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.178+0000 7f7b1d5fe640 1 -- 172.21.15.73:0/3359517344 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b18143de0 con 0x7f7b18072860 2024-04-26T20:50:18.187 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.179+0000 7f7b1d5fe640 1 --2- 172.21.15.73:0/3359517344 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b18072860 0x7f7b181438a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c00c420 tx=0x7f7b0c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:18.187 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.179+0000 7f7b09ffb640 1 -- 172.21.15.73:0/3359517344 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c0074e0 con 0x7f7b18072860 2024-04-26T20:50:18.187 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.179+0000 7f7b09ffb640 1 -- 172.21.15.73:0/3359517344 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b0c0047e0 con 0x7f7b18072860 2024-04-26T20:50:18.187 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.179+0000 7f7b1f862640 1 -- 172.21.15.73:0/3359517344 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b181f6e80 con 0x7f7b18072860 2024-04-26T20:50:18.187 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.179+0000 7f7b09ffb640 1 -- 172.21.15.73:0/3359517344 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c004c10 con 0x7f7b18072860 2024-04-26T20:50:18.187 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.179+0000 7f7b1f862640 1 -- 172.21.15.73:0/3359517344 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b181464e0 con 0x7f7b18072860 2024-04-26T20:50:18.187 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.180+0000 7f7b1f862640 1 -- 172.21.15.73:0/3359517344 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b18076610 con 0x7f7b18072860 2024-04-26T20:50:18.187 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.180+0000 7f7b09ffb640 1 -- 172.21.15.73:0/3359517344 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7f7b0c011070 con 0x7f7b18072860 2024-04-26T20:50:18.187 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.180+0000 7f7b09ffb640 1 --2- 172.21.15.73:0/3359517344 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f7afc06c860 0x7f7afc06ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:18.187 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.180+0000 7f7b1cdfd640 1 --2- 172.21.15.73:0/3359517344 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f7afc06c860 0x7f7afc06ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:18.187 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.180+0000 7f7b09ffb640 1 -- 172.21.15.73:0/3359517344 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7f7b0c08a950 con 0x7f7b18072860 2024-04-26T20:50:18.188 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.181+0000 7f7b1cdfd640 1 --2- 172.21.15.73:0/3359517344 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f7afc06c860 0x7f7afc06ed20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7b14034000 tx=0x7f7b1402c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:18.188 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.185+0000 7f7b09ffb640 1 -- 172.21.15.73:0/3359517344 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7b0c00e280 con 0x7f7b18072860 2024-04-26T20:50:18.363 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.362+0000 7f7b1f862640 1 -- 172.21.15.73:0/3359517344 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7b180d19d0 con 0x7f7b18072860 2024-04-26T20:50:18.363 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.362+0000 7f7b09ffb640 1 -- 172.21.15.73:0/3359517344 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7f7b0c059bf0 con 0x7f7b18072860 2024-04-26T20:50:18.363 INFO:teuthology.misc.health.smithi073.stdout: 2024-04-26T20:50:18.363 INFO:teuthology.misc.health.smithi073.stdout:{"epoch":12,"fsid":"fbd3f548-b547-4713-856b-09f98f5d1764","created":"2024-04-26T20:50:07.876786+0000","modified":"2024-04-26T20:50:15.986915+0000","last_up_change":"0.000000","last_in_change":"2024-04-26T20:50:10.018400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"72d6f4f3-214a-4392-a02f-c33cbcab3c7a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"786bf8ca-6ddc-4b3c-be6c-9152398bd5fd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"90963b1a-5437-4ef8-ad8b-6b74c534eeb5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"bd3d2148-6e38-41b4-a453-fe6e3b8b4ceb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"e8668dc8-a23f-46d2-8311-cb882da08e74","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"cbac0bdb-96bd-4852-bcce-e564495c3fcc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"1884b2a8-569f-4272-96a9-f25a09364dc2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"c0c3bf6d-a4c5-4f1e-aa2d-330951830666","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:50:18.366 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.364+0000 7f7b1f862640 1 -- 172.21.15.73:0/3359517344 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f7afc06c860 msgr2=0x7f7afc06ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:18.366 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.364+0000 7f7b1f862640 1 --2- 172.21.15.73:0/3359517344 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f7afc06c860 0x7f7afc06ed20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7b14034000 tx=0x7f7b1402c000 comp rx=0 tx=0).stop 2024-04-26T20:50:18.366 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.364+0000 7f7b1f862640 1 -- 172.21.15.73:0/3359517344 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b18072860 msgr2=0x7f7b181438a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:18.366 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.364+0000 7f7b1f862640 1 --2- 172.21.15.73:0/3359517344 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b18072860 0x7f7b181438a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c00c420 tx=0x7f7b0c00c8e0 comp rx=0 tx=0).stop 2024-04-26T20:50:18.366 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.364+0000 7f7b1f862640 1 -- 172.21.15.73:0/3359517344 shutdown_connections 2024-04-26T20:50:18.366 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.364+0000 7f7b1f862640 1 --2- 172.21.15.73:0/3359517344 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f7afc06c860 0x7f7afc06ed20 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:18.366 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.364+0000 7f7b1f862640 1 --2- 172.21.15.73:0/3359517344 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7b1815f360 0x7f7b180aaa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:18.366 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.364+0000 7f7b1f862640 1 --2- 172.21.15.73:0/3359517344 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7b18072c90 0x7f7b180aa4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:18.366 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.364+0000 7f7b1f862640 1 --2- 172.21.15.73:0/3359517344 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b18072860 0x7f7b181438a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:18.367 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.364+0000 7f7b1f862640 1 -- 172.21.15.73:0/3359517344 >> 172.21.15.73:0/3359517344 conn(0x7f7b180d0b30 msgr2=0x7f7b180b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:18.367 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.365+0000 7f7b1f862640 1 -- 172.21.15.73:0/3359517344 shutdown_connections 2024-04-26T20:50:18.367 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:18.365+0000 7f7b1f862640 1 -- 172.21.15.73:0/3359517344 wait complete. 2024-04-26T20:50:18.375 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-26T20:50:21.966 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:50:21.969 DEBUG:teuthology.orchestra.run.smithi120:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:50:25.182 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-26T20:50:25.282 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.281+0000 7fcf887f0640 1 Processor -- start 2024-04-26T20:50:25.282 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.281+0000 7fcf887f0640 1 -- start start 2024-04-26T20:50:25.285 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.282+0000 7fcf887f0640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fcf800b64e0 0x7fcf800d1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:25.285 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.282+0000 7fcf887f0640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fcf800b40b0 0x7fcf8015fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:25.285 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.282+0000 7fcf887f0640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf800d25f0 con 0x7fcf8015f830 2024-04-26T20:50:25.285 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.282+0000 7fcf887f0640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fcf800d1080 con 0x7fcf800b40b0 2024-04-26T20:50:25.285 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.282+0000 7fcf887f0640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fcf800d3aa0 con 0x7fcf800b64e0 2024-04-26T20:50:25.285 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.282+0000 7fcf8658c640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fcf8015f830 0x7fcf800d1790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:46498/0 (socket says 172.21.15.73:46498) 2024-04-26T20:50:25.285 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.282+0000 7fcf8658c640 1 -- 172.21.15.73:0/4218448853 learned_addr learned my addr 172.21.15.73:0/4218448853 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:50:25.285 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.282+0000 7fcf86d8d640 1 --2- 172.21.15.73:0/4218448853 >> v2:172.21.15.120:3300/0 conn(0x7fcf800b40b0 0x7fcf8015fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:25.285 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.282+0000 7fcf8558a640 1 -- 172.21.15.73:0/4218448853 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 782842798 0 0) 0x7fcf800d25f0 con 0x7fcf8015f830 2024-04-26T20:50:25.285 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.282+0000 7fcf85d8b640 1 --2- 172.21.15.73:0/4218448853 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fcf800b64e0 0x7fcf800d1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:25.285 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.282+0000 7fcf8558a640 1 -- 172.21.15.73:0/4218448853 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf6c003520 con 0x7fcf8015f830 2024-04-26T20:50:25.286 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.283+0000 7fcf8558a640 1 -- 172.21.15.73:0/4218448853 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3525136077 0 0) 0x7fcf6c003520 con 0x7fcf8015f830 2024-04-26T20:50:25.286 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.283+0000 7fcf8558a640 1 -- 172.21.15.73:0/4218448853 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fcf800b64e0 msgr2=0x7fcf800d1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:25.286 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.283+0000 7fcf8558a640 1 --2- 172.21.15.73:0/4218448853 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fcf800b64e0 0x7fcf800d1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:25.286 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.283+0000 7fcf8558a640 1 -- 172.21.15.73:0/4218448853 >> v2:172.21.15.120:3300/0 conn(0x7fcf800b40b0 msgr2=0x7fcf8015fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:25.286 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.283+0000 7fcf8558a640 1 --2- 172.21.15.73:0/4218448853 >> v2:172.21.15.120:3300/0 conn(0x7fcf800b40b0 0x7fcf8015fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:25.286 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.283+0000 7fcf8558a640 1 -- 172.21.15.73:0/4218448853 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf800d3d80 con 0x7fcf8015f830 2024-04-26T20:50:25.286 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.283+0000 7fcf8558a640 1 -- 172.21.15.73:0/4218448853 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7fcf7c001670 con 0x7fcf8015f830 2024-04-26T20:50:25.286 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.283+0000 7fcf8558a640 1 -- 172.21.15.73:0/4218448853 >> v1:172.21.15.73:6789/0 conn(0x7fcf8015f830 legacy=0x7fcf800d1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:25.286 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.283+0000 7fcf8558a640 1 --2- 172.21.15.73:0/4218448853 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fcf800b64e0 0x7fcf6c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:25.286 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.283+0000 7fcf8558a640 1 --2- 172.21.15.73:0/4218448853 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf6c003f30 0x7fcf6c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:25.286 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.283+0000 7fcf8558a640 1 --2- 172.21.15.73:0/4218448853 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf6c004d40 0x7fcf6c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:25.286 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.283+0000 7fcf8558a640 1 -- 172.21.15.73:0/4218448853 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fcf800d3aa0 con 0x7fcf6c004d40 2024-04-26T20:50:25.286 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.283+0000 7fcf8558a640 1 -- 172.21.15.73:0/4218448853 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6c00b9c0 con 0x7fcf6c003f30 2024-04-26T20:50:25.286 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.283+0000 7fcf8558a640 1 -- 172.21.15.73:0/4218448853 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fcf6c00bb40 con 0x7fcf800b64e0 2024-04-26T20:50:25.287 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.283+0000 7fcf86d8d640 1 --2- 172.21.15.73:0/4218448853 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf6c004d40 0x7fcf6c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:25.287 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.283+0000 7fcf85d8b640 1 --2- 172.21.15.73:0/4218448853 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf6c003f30 0x7fcf6c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:25.287 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.283+0000 7fcf8658c640 1 --2- 172.21.15.73:0/4218448853 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fcf800b64e0 0x7fcf6c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:25.287 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.283+0000 7fcf86d8d640 1 -- 172.21.15.73:0/4218448853 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fcf800b64e0 msgr2=0x7fcf6c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:25.287 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.283+0000 7fcf86d8d640 1 --2- 172.21.15.73:0/4218448853 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fcf800b64e0 0x7fcf6c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:25.287 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.283+0000 7fcf86d8d640 1 -- 172.21.15.73:0/4218448853 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf6c003f30 msgr2=0x7fcf6c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:25.287 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.283+0000 7fcf86d8d640 1 --2- 172.21.15.73:0/4218448853 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf6c003f30 0x7fcf6c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:25.287 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.283+0000 7fcf86d8d640 1 -- 172.21.15.73:0/4218448853 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf6c003520 con 0x7fcf6c004d40 2024-04-26T20:50:25.287 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.284+0000 7fcf86d8d640 1 --2- 172.21.15.73:0/4218448853 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf6c004d40 0x7fcf6c0051c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcf78009fc0 tx=0x7fcf6c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2908bc5354e7a3a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:25.287 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.284+0000 7fcf8558a640 1 -- 172.21.15.73:0/4218448853 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf78009c10 con 0x7fcf6c004d40 2024-04-26T20:50:25.287 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.284+0000 7fcf8558a640 1 -- 172.21.15.73:0/4218448853 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf78036070 con 0x7fcf6c004d40 2024-04-26T20:50:25.287 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.284+0000 7fcf8558a640 1 -- 172.21.15.73:0/4218448853 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7802d070 con 0x7fcf6c004d40 2024-04-26T20:50:25.291 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.284+0000 7fcf887f0640 1 -- 172.21.15.73:0/4218448853 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf6c004d40 msgr2=0x7fcf6c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:25.292 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.284+0000 7fcf887f0640 1 --2- 172.21.15.73:0/4218448853 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf6c004d40 0x7fcf6c0051c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcf78009fc0 tx=0x7fcf6c005830 comp rx=0 tx=0).stop 2024-04-26T20:50:25.292 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.285+0000 7fcf86d8d640 1 -- 172.21.15.73:0/4218448853 reap_dead start 2024-04-26T20:50:25.292 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.285+0000 7fcf887f0640 1 -- 172.21.15.73:0/4218448853 shutdown_connections 2024-04-26T20:50:25.292 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.285+0000 7fcf887f0640 1 -- 172.21.15.73:0/4218448853 >> 172.21.15.73:0/4218448853 conn(0x7fcf800d4fd0 msgr2=0x7fcf800d53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:25.292 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.285+0000 7fcf887f0640 1 -- 172.21.15.73:0/4218448853 shutdown_connections 2024-04-26T20:50:25.292 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.285+0000 7fcf887f0640 1 -- 172.21.15.73:0/4218448853 wait complete. 2024-04-26T20:50:25.292 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.286+0000 7fcf887f0640 1 Processor -- start 2024-04-26T20:50:25.292 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.286+0000 7fcf887f0640 1 -- start start 2024-04-26T20:50:25.292 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.287+0000 7fcf887f0640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf6c004d40 0x7fcf80152440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:25.292 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.287+0000 7fcf887f0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf800cdbd0 0x7fcf800ce030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:25.292 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.287+0000 7fcf887f0640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fcf800d0960 0x7fcf800d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:25.292 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.287+0000 7fcf887f0640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fcf800b47a0 con 0x7fcf6c004d40 2024-04-26T20:50:25.292 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.287+0000 7fcf887f0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fcf800b4620 con 0x7fcf800cdbd0 2024-04-26T20:50:25.292 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.287+0000 7fcf887f0640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fcf800d1ea0 con 0x7fcf800d0960 2024-04-26T20:50:25.293 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.287+0000 7fcf8658c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf6c004d40 0x7fcf80152440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:25.293 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.287+0000 7fcf8658c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf6c004d40 0x7fcf80152440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47338/0 (socket says 172.21.15.73:47338) 2024-04-26T20:50:25.293 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.287+0000 7fcf8658c640 1 -- 172.21.15.73:0/3580004485 learned_addr learned my addr 172.21.15.73:0/3580004485 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:25.293 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.287+0000 7fcf85d8b640 1 --2- 172.21.15.73:0/3580004485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf800cdbd0 0x7fcf800ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:25.293 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.287+0000 7fcf86d8d640 1 --2- 172.21.15.73:0/3580004485 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fcf800d0960 0x7fcf800d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:25.293 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.287+0000 7fcf8658c640 1 -- 172.21.15.73:0/3580004485 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fcf800d0960 msgr2=0x7fcf800d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:25.293 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.287+0000 7fcf8658c640 1 --2- 172.21.15.73:0/3580004485 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fcf800d0960 0x7fcf800d0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:25.293 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.287+0000 7fcf8658c640 1 -- 172.21.15.73:0/3580004485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf800cdbd0 msgr2=0x7fcf800ce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:25.293 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.287+0000 7fcf8658c640 1 --2- 172.21.15.73:0/3580004485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf800cdbd0 0x7fcf800ce030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:25.293 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.287+0000 7fcf8658c640 1 -- 172.21.15.73:0/3580004485 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf800c8660 con 0x7fcf6c004d40 2024-04-26T20:50:25.293 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.287+0000 7fcf86d8d640 1 --2- 172.21.15.73:0/3580004485 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fcf800d0960 0x7fcf800d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:50:25.293 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.287+0000 7fcf8658c640 1 --2- 172.21.15.73:0/3580004485 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf6c004d40 0x7fcf80152440 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c007340 tx=0x7fcf7c007740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:25.293 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.287+0000 7fcf76ffd640 1 -- 172.21.15.73:0/3580004485 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c00f940 con 0x7fcf6c004d40 2024-04-26T20:50:25.293 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.287+0000 7fcf76ffd640 1 -- 172.21.15.73:0/3580004485 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf7c018070 con 0x7fcf6c004d40 2024-04-26T20:50:25.293 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.287+0000 7fcf887f0640 1 -- 172.21.15.73:0/3580004485 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf800c8920 con 0x7fcf6c004d40 2024-04-26T20:50:25.294 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.287+0000 7fcf76ffd640 1 -- 172.21.15.73:0/3580004485 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c00fd40 con 0x7fcf6c004d40 2024-04-26T20:50:25.294 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.287+0000 7fcf887f0640 1 -- 172.21.15.73:0/3580004485 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf800c8ee0 con 0x7fcf6c004d40 2024-04-26T20:50:25.294 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.288+0000 7fcf887f0640 1 -- 172.21.15.73:0/3580004485 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf80076610 con 0x7fcf6c004d40 2024-04-26T20:50:25.294 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.289+0000 7fcf76ffd640 1 -- 172.21.15.73:0/3580004485 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcf7c01d460 con 0x7fcf6c004d40 2024-04-26T20:50:25.294 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.289+0000 7fcf76ffd640 1 --2- 172.21.15.73:0/3580004485 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fcf6406c920 0x7fcf6406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:25.294 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.289+0000 7fcf85d8b640 1 --2- 172.21.15.73:0/3580004485 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fcf6406c920 0x7fcf6406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:25.294 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.289+0000 7fcf76ffd640 1 -- 172.21.15.73:0/3580004485 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fcf7c08b7e0 con 0x7fcf6c004d40 2024-04-26T20:50:25.294 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.290+0000 7fcf85d8b640 1 --2- 172.21.15.73:0/3580004485 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fcf6406c920 0x7fcf6406ede0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcf800b3e90 tx=0x7fcf70009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:25.294 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.294+0000 7fcf76ffd640 1 -- 172.21.15.73:0/3580004485 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcf7c059ff0 con 0x7fcf6c004d40 2024-04-26T20:50:25.470 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.470+0000 7fcf887f0640 1 -- 172.21.15.73:0/3580004485 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcf800d5da0 con 0x7fcf6c004d40 2024-04-26T20:50:25.471 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.470+0000 7fcf76ffd640 1 -- 172.21.15.73:0/3580004485 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12884 (secure 0 0 0) 0x7fcf7c0598f0 con 0x7fcf6c004d40 2024-04-26T20:50:25.471 INFO:teuthology.misc.health.smithi073.stdout: 2024-04-26T20:50:25.472 INFO:teuthology.misc.health.smithi073.stdout:{"epoch":16,"fsid":"fbd3f548-b547-4713-856b-09f98f5d1764","created":"2024-04-26T20:50:07.876786+0000","modified":"2024-04-26T20:50:22.089366+0000","last_up_change":"2024-04-26T20:50:19.082741+0000","last_in_change":"2024-04-26T20:50:10.018400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:50:19.993731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d6f4f3-214a-4392-a02f-c33cbcab3c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6809","nonce":3461705124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6811","nonce":3461705124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6815","nonce":3461705124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6813","nonce":3461705124}]},"public_addr":"172.21.15.73:6809/3461705124","cluster_addr":"172.21.15.73:6811/3461705124","heartbeat_back_addr":"172.21.15.73:6815/3461705124","heartbeat_front_addr":"172.21.15.73:6813/3461705124","state":["exists","up"]},{"osd":1,"uuid":"786bf8ca-6ddc-4b3c-be6c-9152398bd5fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6817","nonce":2827619403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6819","nonce":2827619403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6825","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6827","nonce":2827619403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6821","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6823","nonce":2827619403}]},"public_addr":"172.21.15.73:6817/2827619403","cluster_addr":"172.21.15.73:6819/2827619403","heartbeat_back_addr":"172.21.15.73:6827/2827619403","heartbeat_front_addr":"172.21.15.73:6823/2827619403","state":["exists","up"]},{"osd":2,"uuid":"90963b1a-5437-4ef8-ad8b-6b74c534eeb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6801","nonce":2542681064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6803","nonce":2542681064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6808","nonce":2542681064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6805","nonce":2542681064}]},"public_addr":"172.21.15.73:6801/2542681064","cluster_addr":"172.21.15.73:6803/2542681064","heartbeat_back_addr":"172.21.15.73:6808/2542681064","heartbeat_front_addr":"172.21.15.73:6805/2542681064","state":["exists","up"]},{"osd":3,"uuid":"bd3d2148-6e38-41b4-a453-fe6e3b8b4ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6822","nonce":1604042718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6826","nonce":1604042718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6831","nonce":1604042718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6829","nonce":1604042718}]},"public_addr":"172.21.15.73:6822/1604042718","cluster_addr":"172.21.15.73:6826/1604042718","heartbeat_back_addr":"172.21.15.73:6831/1604042718","heartbeat_front_addr":"172.21.15.73:6829/1604042718","state":["exists","up"]},{"osd":4,"uuid":"e8668dc8-a23f-46d2-8311-cb882da08e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6817","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6819","nonce":3591055561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6824","nonce":3591055561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6831","nonce":3591055561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6828","nonce":3591055561}]},"public_addr":"172.21.15.120:6819/3591055561","cluster_addr":"172.21.15.120:6824/3591055561","heartbeat_back_addr":"172.21.15.120:6831/3591055561","heartbeat_front_addr":"172.21.15.120:6828/3591055561","state":["exists","up"]},{"osd":5,"uuid":"cbac0bdb-96bd-4852-bcce-e564495c3fcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6809","nonce":819450433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6811","nonce":819450433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6815","nonce":819450433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6813","nonce":819450433}]},"public_addr":"172.21.15.120:6809/819450433","cluster_addr":"172.21.15.120:6811/819450433","heartbeat_back_addr":"172.21.15.120:6815/819450433","heartbeat_front_addr":"172.21.15.120:6813/819450433","state":["exists","up"]},{"osd":6,"uuid":"1884b2a8-569f-4272-96a9-f25a09364dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6801","nonce":3946210727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6803","nonce":3946210727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6807","nonce":3946210727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6805","nonce":3946210727}]},"public_addr":"172.21.15.120:6801/3946210727","cluster_addr":"172.21.15.120:6803/3946210727","heartbeat_back_addr":"172.21.15.120:6807/3946210727","heartbeat_front_addr":"172.21.15.120:6805/3946210727","state":["exists","up"]},{"osd":7,"uuid":"c0c3bf6d-a4c5-4f1e-aa2d-330951830666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6818","nonce":2915574674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6821","nonce":2915574674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6827","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6829","nonce":2915574674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6823","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6825","nonce":2915574674}]},"public_addr":"172.21.15.120:6818/2915574674","cluster_addr":"172.21.15.120:6821/2915574674","heartbeat_back_addr":"172.21.15.120:6829/2915574674","heartbeat_front_addr":"172.21.15.120:6825/2915574674","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.218216+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.208710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.112705+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.182789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.164324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.174212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.225396+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.261559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:50:25.474 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.472+0000 7fcf887f0640 1 -- 172.21.15.73:0/3580004485 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fcf6406c920 msgr2=0x7fcf6406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:25.474 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.472+0000 7fcf887f0640 1 --2- 172.21.15.73:0/3580004485 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fcf6406c920 0x7fcf6406ede0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fcf800b3e90 tx=0x7fcf70009190 comp rx=0 tx=0).stop 2024-04-26T20:50:25.474 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.472+0000 7fcf887f0640 1 -- 172.21.15.73:0/3580004485 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf6c004d40 msgr2=0x7fcf80152440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:25.474 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.472+0000 7fcf887f0640 1 --2- 172.21.15.73:0/3580004485 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf6c004d40 0x7fcf80152440 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c007340 tx=0x7fcf7c007740 comp rx=0 tx=0).stop 2024-04-26T20:50:25.474 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.472+0000 7fcf887f0640 1 -- 172.21.15.73:0/3580004485 shutdown_connections 2024-04-26T20:50:25.474 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.472+0000 7fcf887f0640 1 --2- 172.21.15.73:0/3580004485 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fcf6406c920 0x7fcf6406ede0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:25.474 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.472+0000 7fcf887f0640 1 --2- 172.21.15.73:0/3580004485 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fcf800d0960 0x7fcf800d0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:25.474 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.472+0000 7fcf887f0640 1 --2- 172.21.15.73:0/3580004485 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fcf800cdbd0 0x7fcf800ce030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:25.474 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.472+0000 7fcf887f0640 1 --2- 172.21.15.73:0/3580004485 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf6c004d40 0x7fcf80152440 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:25.474 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.472+0000 7fcf887f0640 1 -- 172.21.15.73:0/3580004485 >> 172.21.15.73:0/3580004485 conn(0x7fcf800d4fd0 msgr2=0x7fcf800cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:25.475 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.473+0000 7fcf887f0640 1 -- 172.21.15.73:0/3580004485 shutdown_connections 2024-04-26T20:50:25.475 INFO:teuthology.misc.health.smithi073.stderr:2024-04-26T20:50:25.473+0000 7fcf887f0640 1 -- 172.21.15.73:0/3580004485 wait complete. 2024-04-26T20:50:25.483 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-26T20:50:25.483 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-26T20:50:25.484 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T20:50:25.484 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T20:50:25.486 INFO:tasks.ceph.mds.a:Started 2024-04-26T20:50:25.486 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T20:50:25.486 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T20:50:25.488 INFO:tasks.ceph.mds.c:Started 2024-04-26T20:50:25.489 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T20:50:25.489 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T20:50:25.491 INFO:tasks.ceph.mds.b:Started 2024-04-26T20:50:25.491 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T20:50:25.491 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T20:50:25.494 INFO:tasks.ceph.mds.d:Started 2024-04-26T20:50:25.494 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-26T20:50:25.494 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:50:25.559 INFO:tasks.ceph.mds.a.smithi073.stdout:starting mds.a at 2024-04-26T20:50:25.559 INFO:tasks.ceph.mds.c.smithi073.stdout:starting mds.c at 2024-04-26T20:50:25.574 INFO:tasks.ceph.mds.b.smithi120.stdout:starting mds.b at 2024-04-26T20:50:25.575 INFO:tasks.ceph.mds.d.smithi120.stdout:starting mds.d at 2024-04-26T20:50:25.575 INFO:tasks.ceph.mds.b.smithi120.stderr:2024-04-26T20:50:25.574+0000 7fcd63384640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-04-26T20:50:25.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.594+0000 7f3745397640 1 Processor -- start 2024-04-26T20:50:25.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.594+0000 7f3745397640 1 -- start start 2024-04-26T20:50:25.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.595+0000 7f3745397640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f37400b88a0 0x7f37400d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:25.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.595+0000 7f3745397640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f37400b5c10 0x7f37400cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:25.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.595+0000 7f3745397640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37400b8c80 con 0x7f37400d04a0 2024-04-26T20:50:25.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.595+0000 7f3745397640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f37400d2370 con 0x7f37400b88a0 2024-04-26T20:50:25.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.595+0000 7f3745397640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f37400cd5b0 con 0x7f37400b5c10 2024-04-26T20:50:25.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.595+0000 7f373effd640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f37400b88a0 0x7f37400d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:25.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.595+0000 7f373effd640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f37400b88a0 0x7f37400d19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47382/0 (socket says 172.21.15.73:47382) 2024-04-26T20:50:25.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.595+0000 7f373effd640 1 -- 172.21.15.73:0/2359257180 learned_addr learned my addr 172.21.15.73:0/2359257180 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:25.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.595+0000 7f3737fff640 1 --2- 172.21.15.73:0/2359257180 >> v2:172.21.15.120:3300/0 conn(0x7f37400b5c10 0x7f37400cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:25.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.596+0000 7f373effd640 1 -- 172.21.15.73:0/2359257180 >> v2:172.21.15.120:3300/0 conn(0x7f37400b5c10 msgr2=0x7f37400cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:25.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.596+0000 7f373effd640 1 --2- 172.21.15.73:0/2359257180 >> v2:172.21.15.120:3300/0 conn(0x7f37400b5c10 0x7f37400cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:25.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.596+0000 7f373effd640 1 -- 172.21.15.73:0/2359257180 >> v1:172.21.15.120:6789/0 conn(0x7f37400d04a0 legacy=0x7f374006d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:25.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.596+0000 7f373effd640 1 -- 172.21.15.73:0/2359257180 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37400cd890 con 0x7f37400b88a0 2024-04-26T20:50:25.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.596+0000 7f373effd640 1 --2- 172.21.15.73:0/2359257180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37400b88a0 0x7f37400d19f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3740073720 tx=0x7f373002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa35eb67cb0ebe33 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:25.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.597+0000 7f373e7fc640 1 -- 172.21.15.73:0/2359257180 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3730037070 con 0x7f37400b88a0 2024-04-26T20:50:25.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.597+0000 7f373e7fc640 1 -- 172.21.15.73:0/2359257180 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3730032440 con 0x7f37400b88a0 2024-04-26T20:50:25.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.597+0000 7f373e7fc640 1 -- 172.21.15.73:0/2359257180 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3730032810 con 0x7f37400b88a0 2024-04-26T20:50:25.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.598+0000 7f3745397640 1 -- 172.21.15.73:0/2359257180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37400b88a0 msgr2=0x7f37400d19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:25.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.598+0000 7f3745397640 1 --2- 172.21.15.73:0/2359257180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37400b88a0 0x7f37400d19f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3740073720 tx=0x7f373002a270 comp rx=0 tx=0).stop 2024-04-26T20:50:25.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.598+0000 7f3745397640 1 -- 172.21.15.73:0/2359257180 shutdown_connections 2024-04-26T20:50:25.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.598+0000 7f3745397640 1 --2- 172.21.15.73:0/2359257180 >> v2:172.21.15.120:3300/0 conn(0x7f37400b5c10 0x7f37400cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:25.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.598+0000 7f3745397640 1 --2- 172.21.15.73:0/2359257180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37400b88a0 0x7f37400d19f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:25.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.598+0000 7f3745397640 1 -- 172.21.15.73:0/2359257180 >> 172.21.15.73:0/2359257180 conn(0x7f37400d2dc0 msgr2=0x7f37400d31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:25.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.598+0000 7f3745397640 1 -- 172.21.15.73:0/2359257180 shutdown_connections 2024-04-26T20:50:25.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.598+0000 7f3745397640 1 -- 172.21.15.73:0/2359257180 wait complete. 2024-04-26T20:50:25.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.599+0000 7f3745397640 1 Processor -- start 2024-04-26T20:50:25.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.599+0000 7f3745397640 1 -- start start 2024-04-26T20:50:25.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.600+0000 7f3745397640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f37400b5c10 0x7f37400c96a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:25.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.600+0000 7f3745397640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37400b88a0 0x7f374015bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:25.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.600+0000 7f3745397640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37400d04a0 0x7f3740161570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:25.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.600+0000 7f3745397640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f37400d1f30 con 0x7f37400b88a0 2024-04-26T20:50:25.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.600+0000 7f3745397640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f37400cde50 con 0x7f37400d04a0 2024-04-26T20:50:25.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.600+0000 7f3745397640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f37400d20b0 con 0x7f37400b5c10 2024-04-26T20:50:25.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.600+0000 7f3737fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37400b88a0 0x7f374015bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:25.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.600+0000 7f3737fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37400b88a0 0x7f374015bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47388/0 (socket says 172.21.15.73:47388) 2024-04-26T20:50:25.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.600+0000 7f3737fff640 1 -- 172.21.15.73:0/2793373017 learned_addr learned my addr 172.21.15.73:0/2793373017 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:25.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.600+0000 7f373f7fe640 1 --2- 172.21.15.73:0/2793373017 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37400d04a0 0x7f3740161570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:25.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.600+0000 7f373effd640 1 --2- 172.21.15.73:0/2793373017 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f37400b5c10 0x7f37400c96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:25.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.600+0000 7f3737fff640 1 -- 172.21.15.73:0/2793373017 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f37400b5c10 msgr2=0x7f37400c96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:25.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.600+0000 7f3737fff640 1 --2- 172.21.15.73:0/2793373017 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f37400b5c10 0x7f37400c96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:25.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.600+0000 7f3737fff640 1 -- 172.21.15.73:0/2793373017 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37400d04a0 msgr2=0x7f3740161570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:25.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.600+0000 7f3737fff640 1 --2- 172.21.15.73:0/2793373017 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37400d04a0 0x7f3740161570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:25.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.600+0000 7f3737fff640 1 -- 172.21.15.73:0/2793373017 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f374014cfd0 con 0x7f37400b88a0 2024-04-26T20:50:25.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.600+0000 7f3737fff640 1 --2- 172.21.15.73:0/2793373017 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37400b88a0 0x7f374015bfb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f372800e9b0 tx=0x7f372800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:25.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.601+0000 7f37377fe640 1 -- 172.21.15.73:0/2793373017 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3728017070 con 0x7f37400b88a0 2024-04-26T20:50:25.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.601+0000 7f37377fe640 1 -- 172.21.15.73:0/2793373017 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f372800c9a0 con 0x7f37400b88a0 2024-04-26T20:50:25.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.601+0000 7f37377fe640 1 -- 172.21.15.73:0/2793373017 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372800cd90 con 0x7f37400b88a0 2024-04-26T20:50:25.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.601+0000 7f3745397640 1 -- 172.21.15.73:0/2793373017 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37400c9be0 con 0x7f37400b88a0 2024-04-26T20:50:25.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.601+0000 7f3745397640 1 -- 172.21.15.73:0/2793373017 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37401607b0 con 0x7f37400b88a0 2024-04-26T20:50:25.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.602+0000 7f3745397640 1 -- 172.21.15.73:0/2793373017 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37400694f0 con 0x7f37400b88a0 2024-04-26T20:50:25.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.602+0000 7f37377fe640 1 -- 172.21.15.73:0/2793373017 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f372801a070 con 0x7f37400b88a0 2024-04-26T20:50:25.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.602+0000 7f37377fe640 1 --2- 172.21.15.73:0/2793373017 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f372006c8f0 0x7f372006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:25.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.602+0000 7f373effd640 1 --2- 172.21.15.73:0/2793373017 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f372006c8f0 0x7f372006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:25.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.602+0000 7f37377fe640 1 -- 172.21.15.73:0/2793373017 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f372808b6d0 con 0x7f37400b88a0 2024-04-26T20:50:25.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.603+0000 7f373effd640 1 --2- 172.21.15.73:0/2793373017 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f372006c8f0 0x7f372006edb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3730034000 tx=0x7f373002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:25.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.607+0000 7f37377fe640 1 -- 172.21.15.73:0/2793373017 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f374014cfd0 con 0x7f37400b88a0 2024-04-26T20:50:25.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.782+0000 7f3745397640 1 -- 172.21.15.73:0/2793373017 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f37400d3690 con 0x7f37400b88a0 2024-04-26T20:50:25.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.783+0000 7f37377fe640 1 -- 172.21.15.73:0/2793373017 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12884 (secure 0 0 0) 0x7f3728059890 con 0x7f37400b88a0 2024-04-26T20:50:25.784 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:50:25.784 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":16,"fsid":"fbd3f548-b547-4713-856b-09f98f5d1764","created":"2024-04-26T20:50:07.876786+0000","modified":"2024-04-26T20:50:22.089366+0000","last_up_change":"2024-04-26T20:50:19.082741+0000","last_in_change":"2024-04-26T20:50:10.018400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:50:19.993731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d6f4f3-214a-4392-a02f-c33cbcab3c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6809","nonce":3461705124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6811","nonce":3461705124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6815","nonce":3461705124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6813","nonce":3461705124}]},"public_addr":"172.21.15.73:6809/3461705124","cluster_addr":"172.21.15.73:6811/3461705124","heartbeat_back_addr":"172.21.15.73:6815/3461705124","heartbeat_front_addr":"172.21.15.73:6813/3461705124","state":["exists","up"]},{"osd":1,"uuid":"786bf8ca-6ddc-4b3c-be6c-9152398bd5fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6817","nonce":2827619403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6819","nonce":2827619403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6825","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6827","nonce":2827619403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6821","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6823","nonce":2827619403}]},"public_addr":"172.21.15.73:6817/2827619403","cluster_addr":"172.21.15.73:6819/2827619403","heartbeat_back_addr":"172.21.15.73:6827/2827619403","heartbeat_front_addr":"172.21.15.73:6823/2827619403","state":["exists","up"]},{"osd":2,"uuid":"90963b1a-5437-4ef8-ad8b-6b74c534eeb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6801","nonce":2542681064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6803","nonce":2542681064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6808","nonce":2542681064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6805","nonce":2542681064}]},"public_addr":"172.21.15.73:6801/2542681064","cluster_addr":"172.21.15.73:6803/2542681064","heartbeat_back_addr":"172.21.15.73:6808/2542681064","heartbeat_front_addr":"172.21.15.73:6805/2542681064","state":["exists","up"]},{"osd":3,"uuid":"bd3d2148-6e38-41b4-a453-fe6e3b8b4ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6822","nonce":1604042718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6826","nonce":1604042718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6831","nonce":1604042718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6829","nonce":1604042718}]},"public_addr":"172.21.15.73:6822/1604042718","cluster_addr":"172.21.15.73:6826/1604042718","heartbeat_back_addr":"172.21.15.73:6831/1604042718","heartbeat_front_addr":"172.21.15.73:6829/1604042718","state":["exists","up"]},{"osd":4,"uuid":"e8668dc8-a23f-46d2-8311-cb882da08e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6817","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6819","nonce":3591055561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6824","nonce":3591055561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6831","nonce":3591055561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6828","nonce":3591055561}]},"public_addr":"172.21.15.120:6819/3591055561","cluster_addr":"172.21.15.120:6824/3591055561","heartbeat_back_addr":"172.21.15.120:6831/3591055561","heartbeat_front_addr":"172.21.15.120:6828/3591055561","state":["exists","up"]},{"osd":5,"uuid":"cbac0bdb-96bd-4852-bcce-e564495c3fcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6809","nonce":819450433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6811","nonce":819450433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6815","nonce":819450433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6813","nonce":819450433}]},"public_addr":"172.21.15.120:6809/819450433","cluster_addr":"172.21.15.120:6811/819450433","heartbeat_back_addr":"172.21.15.120:6815/819450433","heartbeat_front_addr":"172.21.15.120:6813/819450433","state":["exists","up"]},{"osd":6,"uuid":"1884b2a8-569f-4272-96a9-f25a09364dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6801","nonce":3946210727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6803","nonce":3946210727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6807","nonce":3946210727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6805","nonce":3946210727}]},"public_addr":"172.21.15.120:6801/3946210727","cluster_addr":"172.21.15.120:6803/3946210727","heartbeat_back_addr":"172.21.15.120:6807/3946210727","heartbeat_front_addr":"172.21.15.120:6805/3946210727","state":["exists","up"]},{"osd":7,"uuid":"c0c3bf6d-a4c5-4f1e-aa2d-330951830666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6818","nonce":2915574674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6821","nonce":2915574674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6827","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6829","nonce":2915574674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6823","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6825","nonce":2915574674}]},"public_addr":"172.21.15.120:6818/2915574674","cluster_addr":"172.21.15.120:6821/2915574674","heartbeat_back_addr":"172.21.15.120:6829/2915574674","heartbeat_front_addr":"172.21.15.120:6825/2915574674","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.218216+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.208710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.112705+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.182789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.164324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.174212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.225396+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.261559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:50:25.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.785+0000 7f3745397640 1 -- 172.21.15.73:0/2793373017 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f372006c8f0 msgr2=0x7f372006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:25.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.785+0000 7f3745397640 1 --2- 172.21.15.73:0/2793373017 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f372006c8f0 0x7f372006edb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3730034000 tx=0x7f373002e000 comp rx=0 tx=0).stop 2024-04-26T20:50:25.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.785+0000 7f3745397640 1 -- 172.21.15.73:0/2793373017 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37400b88a0 msgr2=0x7f374015bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:25.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.785+0000 7f3745397640 1 --2- 172.21.15.73:0/2793373017 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37400b88a0 0x7f374015bfb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f372800e9b0 tx=0x7f372800ee70 comp rx=0 tx=0).stop 2024-04-26T20:50:25.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.785+0000 7f3745397640 1 -- 172.21.15.73:0/2793373017 shutdown_connections 2024-04-26T20:50:25.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.785+0000 7f3745397640 1 --2- 172.21.15.73:0/2793373017 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37400b88a0 0x7f374015bfb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:25.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.785+0000 7f3745397640 1 --2- 172.21.15.73:0/2793373017 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f372006c8f0 0x7f372006edb0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:25.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.785+0000 7f3745397640 1 --2- 172.21.15.73:0/2793373017 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f37400d04a0 0x7f3740161570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:25.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.785+0000 7f3745397640 1 --2- 172.21.15.73:0/2793373017 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f37400b5c10 0x7f37400c96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:25.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.785+0000 7f3745397640 1 -- 172.21.15.73:0/2793373017 >> 172.21.15.73:0/2793373017 conn(0x7f37400d2dc0 msgr2=0x7f37400b5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:25.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.785+0000 7f3745397640 1 -- 172.21.15.73:0/2793373017 shutdown_connections 2024-04-26T20:50:25.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.785+0000 7f3745397640 1 -- 172.21.15.73:0/2793373017 wait complete. 2024-04-26T20:50:25.797 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:50:19.993731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:50:25.797 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:50:25.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.897+0000 7fdc5be85640 1 Processor -- start 2024-04-26T20:50:25.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.897+0000 7fdc5be85640 1 -- start start 2024-04-26T20:50:25.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.898+0000 7fdc5be85640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdc5415f550 0x7fdc540d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:25.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.898+0000 7fdc5be85640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc540d1040 con 0x7fdc5415f170 2024-04-26T20:50:25.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.898+0000 7fdc5be85640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc540c9680 con 0x7fdc540d3bc0 2024-04-26T20:50:25.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.898+0000 7fdc5be85640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7fdc540c9430 con 0x7fdc5415f550 2024-04-26T20:50:25.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.898+0000 7fdc59420640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdc5415f550 0x7fdc540d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:25.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.898+0000 7fdc59c21640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fdc5415f170 0x7fdc540d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:46500/0 (socket says 172.21.15.73:46500) 2024-04-26T20:50:25.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.898+0000 7fdc59c21640 1 -- 172.21.15.73:0/2549644140 learned_addr learned my addr 172.21.15.73:0/2549644140 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:50:25.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.898+0000 7fdc59420640 1 -- 172.21.15.73:0/2549644140 >> v1:172.21.15.120:6789/0 conn(0x7fdc540d3bc0 legacy=0x7fdc540ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:25.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.898+0000 7fdc59420640 1 -- 172.21.15.73:0/2549644140 >> v1:172.21.15.73:6789/0 conn(0x7fdc5415f170 legacy=0x7fdc540d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:25.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.898+0000 7fdc58c1f640 1 -- 172.21.15.73:0/2549644140 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3617924413 0 0) 0x7fdc50001110 con 0x7fdc5415f170 2024-04-26T20:50:25.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.898+0000 7fdc59420640 1 -- 172.21.15.73:0/2549644140 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc540c9960 con 0x7fdc5415f550 2024-04-26T20:50:25.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.899+0000 7fdc59420640 1 --2- 172.21.15.73:0/2549644140 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc5415f550 0x7fdc540d5a80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdc540c95b0 tx=0x7fdc4402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91b309c7bbc48ccf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:25.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.899+0000 7fdc58c1f640 1 -- 172.21.15.73:0/2549644140 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc44037070 con 0x7fdc5415f550 2024-04-26T20:50:25.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.899+0000 7fdc58c1f640 1 -- 172.21.15.73:0/2549644140 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc44032440 con 0x7fdc5415f550 2024-04-26T20:50:25.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.899+0000 7fdc58c1f640 1 -- 172.21.15.73:0/2549644140 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc44032810 con 0x7fdc5415f550 2024-04-26T20:50:25.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.899+0000 7fdc5be85640 1 -- 172.21.15.73:0/2549644140 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc5415f550 msgr2=0x7fdc540d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:25.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.899+0000 7fdc5be85640 1 --2- 172.21.15.73:0/2549644140 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc5415f550 0x7fdc540d5a80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdc540c95b0 tx=0x7fdc4402a200 comp rx=0 tx=0).stop 2024-04-26T20:50:25.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.899+0000 7fdc5be85640 1 -- 172.21.15.73:0/2549644140 shutdown_connections 2024-04-26T20:50:25.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.899+0000 7fdc5be85640 1 --2- 172.21.15.73:0/2549644140 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc5415f550 0x7fdc540d5a80 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:25.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.899+0000 7fdc5be85640 1 -- 172.21.15.73:0/2549644140 >> 172.21.15.73:0/2549644140 conn(0x7fdc540d4f60 msgr2=0x7fdc540d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:25.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.900+0000 7fdc5be85640 1 -- 172.21.15.73:0/2549644140 shutdown_connections 2024-04-26T20:50:25.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.900+0000 7fdc5be85640 1 -- 172.21.15.73:0/2549644140 wait complete. 2024-04-26T20:50:25.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.901+0000 7fdc5be85640 1 Processor -- start 2024-04-26T20:50:25.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.901+0000 7fdc5be85640 1 -- start start 2024-04-26T20:50:25.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.901+0000 7fdc5be85640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc540d3bc0 0x7fdc540ba5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:25.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.901+0000 7fdc5be85640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdc5415f170 0x7fdc540cdf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:25.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.901+0000 7fdc5be85640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc540ca6b0 0x7fdc540cc030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:25.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.901+0000 7fdc5be85640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdc540d2e30 con 0x7fdc540ca6b0 2024-04-26T20:50:25.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.901+0000 7fdc5be85640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdc540d2cb0 con 0x7fdc540d3bc0 2024-04-26T20:50:25.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.901+0000 7fdc5be85640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fdc540d2fb0 con 0x7fdc5415f170 2024-04-26T20:50:25.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.901+0000 7fdc5a422640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc540ca6b0 0x7fdc540cc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:25.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.902+0000 7fdc5a422640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc540ca6b0 0x7fdc540cc030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47404/0 (socket says 172.21.15.73:47404) 2024-04-26T20:50:25.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.902+0000 7fdc5a422640 1 -- 172.21.15.73:0/1188199645 learned_addr learned my addr 172.21.15.73:0/1188199645 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:25.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.902+0000 7fdc59c21640 1 --2- 172.21.15.73:0/1188199645 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc540d3bc0 0x7fdc540ba5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:25.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.902+0000 7fdc59420640 1 --2- 172.21.15.73:0/1188199645 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdc5415f170 0x7fdc540cdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:25.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.902+0000 7fdc5a422640 1 -- 172.21.15.73:0/1188199645 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdc5415f170 msgr2=0x7fdc540cdf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:25.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.902+0000 7fdc5a422640 1 --2- 172.21.15.73:0/1188199645 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdc5415f170 0x7fdc540cdf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:25.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.902+0000 7fdc5a422640 1 -- 172.21.15.73:0/1188199645 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc540d3bc0 msgr2=0x7fdc540ba5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:25.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.902+0000 7fdc5a422640 1 --2- 172.21.15.73:0/1188199645 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc540d3bc0 0x7fdc540ba5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:25.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.902+0000 7fdc5a422640 1 -- 172.21.15.73:0/1188199645 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc540ce750 con 0x7fdc540ca6b0 2024-04-26T20:50:25.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.902+0000 7fdc5a422640 1 --2- 172.21.15.73:0/1188199645 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc540ca6b0 0x7fdc540cc030 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c00c7b0 tx=0x7fdc4c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:25.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.902+0000 7fdc4a7fc640 1 -- 172.21.15.73:0/1188199645 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4c017070 con 0x7fdc540ca6b0 2024-04-26T20:50:25.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.902+0000 7fdc4a7fc640 1 -- 172.21.15.73:0/1188199645 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc4c003da0 con 0x7fdc540ca6b0 2024-04-26T20:50:25.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.902+0000 7fdc5be85640 1 -- 172.21.15.73:0/1188199645 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc540cea30 con 0x7fdc540ca6b0 2024-04-26T20:50:25.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.902+0000 7fdc4a7fc640 1 -- 172.21.15.73:0/1188199645 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4c00ce60 con 0x7fdc540ca6b0 2024-04-26T20:50:25.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.902+0000 7fdc5be85640 1 -- 172.21.15.73:0/1188199645 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc540ceff0 con 0x7fdc540ca6b0 2024-04-26T20:50:25.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.903+0000 7fdc5be85640 1 -- 172.21.15.73:0/1188199645 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc540d4640 con 0x7fdc540ca6b0 2024-04-26T20:50:25.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.903+0000 7fdc4a7fc640 1 -- 172.21.15.73:0/1188199645 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdc4c01a070 con 0x7fdc540ca6b0 2024-04-26T20:50:25.909 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.904+0000 7fdc4a7fc640 1 --2- 172.21.15.73:0/1188199645 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fdc3806c920 0x7fdc3806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:25.909 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.904+0000 7fdc59c21640 1 --2- 172.21.15.73:0/1188199645 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fdc3806c920 0x7fdc3806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:25.909 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.904+0000 7fdc4a7fc640 1 -- 172.21.15.73:0/1188199645 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fdc4c08ada0 con 0x7fdc540ca6b0 2024-04-26T20:50:25.909 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.904+0000 7fdc59c21640 1 --2- 172.21.15.73:0/1188199645 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fdc3806c920 0x7fdc3806ede0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdc50000f60 tx=0x7fdc50007b50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:25.909 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:25.908+0000 7fdc4a7fc640 1 -- 172.21.15.73:0/1188199645 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdc540d4640 con 0x7fdc540ca6b0 2024-04-26T20:50:26.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.083+0000 7fdc5be85640 1 -- 172.21.15.73:0/1188199645 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdc540694f0 con 0x7fdc540ca6b0 2024-04-26T20:50:26.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.084+0000 7fdc4a7fc640 1 -- 172.21.15.73:0/1188199645 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7fdc4c058e40 con 0x7fdc540ca6b0 2024-04-26T20:50:26.085 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-26T20:50:26.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.086+0000 7fdc5be85640 1 -- 172.21.15.73:0/1188199645 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fdc3806c920 msgr2=0x7fdc3806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:26.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.086+0000 7fdc5be85640 1 --2- 172.21.15.73:0/1188199645 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fdc3806c920 0x7fdc3806ede0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdc50000f60 tx=0x7fdc50007b50 comp rx=0 tx=0).stop 2024-04-26T20:50:26.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.086+0000 7fdc5be85640 1 -- 172.21.15.73:0/1188199645 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc540ca6b0 msgr2=0x7fdc540cc030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:26.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.086+0000 7fdc5be85640 1 --2- 172.21.15.73:0/1188199645 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc540ca6b0 0x7fdc540cc030 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c00c7b0 tx=0x7fdc4c00cc70 comp rx=0 tx=0).stop 2024-04-26T20:50:26.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.086+0000 7fdc5be85640 1 -- 172.21.15.73:0/1188199645 shutdown_connections 2024-04-26T20:50:26.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.086+0000 7fdc5be85640 1 --2- 172.21.15.73:0/1188199645 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc540ca6b0 0x7fdc540cc030 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:26.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.086+0000 7fdc5be85640 1 --2- 172.21.15.73:0/1188199645 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fdc3806c920 0x7fdc3806ede0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:26.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.086+0000 7fdc5be85640 1 --2- 172.21.15.73:0/1188199645 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdc5415f170 0x7fdc540cdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:26.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.086+0000 7fdc5be85640 1 --2- 172.21.15.73:0/1188199645 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc540d3bc0 0x7fdc540ba5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:26.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.086+0000 7fdc5be85640 1 -- 172.21.15.73:0/1188199645 >> 172.21.15.73:0/1188199645 conn(0x7fdc540d4f60 msgr2=0x7fdc540d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:26.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.086+0000 7fdc5be85640 1 -- 172.21.15.73:0/1188199645 shutdown_connections 2024-04-26T20:50:26.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.086+0000 7fdc5be85640 1 -- 172.21.15.73:0/1188199645 wait complete. 2024-04-26T20:50:26.097 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:50:26.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.193+0000 7f5b49a45640 1 Processor -- start 2024-04-26T20:50:26.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.193+0000 7f5b49a45640 1 -- start start 2024-04-26T20:50:26.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.194+0000 7f5b49a45640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5b440b15a0 0x7f5b440d0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:26.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.194+0000 7f5b49a45640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5b44070460 0x7f5b4406f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:26.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.194+0000 7f5b49a45640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b44073a70 con 0x7f5b4415bfe0 2024-04-26T20:50:26.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.194+0000 7f5b49a45640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f5b44073d50 con 0x7f5b44070460 2024-04-26T20:50:26.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.194+0000 7f5b49a45640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f5b44160560 con 0x7f5b440b15a0 2024-04-26T20:50:26.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.194+0000 7f5b437fe640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5b44070460 0x7f5b4406f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:26.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.194+0000 7f5b42ffd640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f5b4415bfe0 0x7f5b440d1890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:46502/0 (socket says 172.21.15.73:46502) 2024-04-26T20:50:26.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.194+0000 7f5b42ffd640 1 -- 172.21.15.73:0/3596650365 learned_addr learned my addr 172.21.15.73:0/3596650365 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:50:26.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.194+0000 7f5b427fc640 1 --2- 172.21.15.73:0/3596650365 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5b440b15a0 0x7f5b440d0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:26.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.194+0000 7f5b437fe640 1 -- 172.21.15.73:0/3596650365 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5b440b15a0 msgr2=0x7f5b440d0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:26.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.194+0000 7f5b437fe640 1 --2- 172.21.15.73:0/3596650365 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5b440b15a0 0x7f5b440d0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:26.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.194+0000 7f5b437fe640 1 -- 172.21.15.73:0/3596650365 >> v1:172.21.15.73:6789/0 conn(0x7f5b4415bfe0 legacy=0x7f5b440d1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:26.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.194+0000 7f5b41ffb640 1 -- 172.21.15.73:0/3596650365 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 780230649 0 0) 0x7f5b44073a70 con 0x7f5b4415bfe0 2024-04-26T20:50:26.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.194+0000 7f5b437fe640 1 -- 172.21.15.73:0/3596650365 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b44160840 con 0x7f5b44070460 2024-04-26T20:50:26.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.195+0000 7f5b437fe640 1 --2- 172.21.15.73:0/3596650365 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b44070460 0x7f5b4406f030 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c0097a0 tx=0x7f5b3c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caaab9aaf25f8cce server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:26.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.195+0000 7f5b41ffb640 1 -- 172.21.15.73:0/3596650365 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c037070 con 0x7f5b44070460 2024-04-26T20:50:26.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.195+0000 7f5b41ffb640 1 -- 172.21.15.73:0/3596650365 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b3c032440 con 0x7f5b44070460 2024-04-26T20:50:26.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.195+0000 7f5b41ffb640 1 -- 172.21.15.73:0/3596650365 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c032810 con 0x7f5b44070460 2024-04-26T20:50:26.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.196+0000 7f5b49a45640 1 -- 172.21.15.73:0/3596650365 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b44070460 msgr2=0x7f5b4406f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:26.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.196+0000 7f5b49a45640 1 --2- 172.21.15.73:0/3596650365 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b44070460 0x7f5b4406f030 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c0097a0 tx=0x7f5b3c02a180 comp rx=0 tx=0).stop 2024-04-26T20:50:26.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.196+0000 7f5b49a45640 1 -- 172.21.15.73:0/3596650365 shutdown_connections 2024-04-26T20:50:26.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.196+0000 7f5b49a45640 1 --2- 172.21.15.73:0/3596650365 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5b440b15a0 0x7f5b440d0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:26.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.196+0000 7f5b49a45640 1 --2- 172.21.15.73:0/3596650365 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b44070460 0x7f5b4406f030 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:26.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.196+0000 7f5b49a45640 1 -- 172.21.15.73:0/3596650365 >> 172.21.15.73:0/3596650365 conn(0x7f5b44068090 msgr2=0x7f5b440684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:26.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.196+0000 7f5b49a45640 1 -- 172.21.15.73:0/3596650365 shutdown_connections 2024-04-26T20:50:26.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.196+0000 7f5b49a45640 1 -- 172.21.15.73:0/3596650365 wait complete. 2024-04-26T20:50:26.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.197+0000 7f5b49a45640 1 Processor -- start 2024-04-26T20:50:26.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.197+0000 7f5b49a45640 1 -- start start 2024-04-26T20:50:26.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.198+0000 7f5b49a45640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b44070460 0x7f5b4415b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:26.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.198+0000 7f5b49a45640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5b440b15a0 0x7f5b440b6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:26.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.198+0000 7f5b49a45640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5b4415bfe0 0x7f5b44152890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:26.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.198+0000 7f5b49a45640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5b440d1fa0 con 0x7f5b44070460 2024-04-26T20:50:26.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.198+0000 7f5b49a45640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5b44070e10 con 0x7f5b440b15a0 2024-04-26T20:50:26.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.198+0000 7f5b49a45640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f5b440d2120 con 0x7f5b4415bfe0 2024-04-26T20:50:26.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.198+0000 7f5b42ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b44070460 0x7f5b4415b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:26.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.198+0000 7f5b42ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b44070460 0x7f5b4415b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47418/0 (socket says 172.21.15.73:47418) 2024-04-26T20:50:26.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.198+0000 7f5b42ffd640 1 -- 172.21.15.73:0/3964872370 learned_addr learned my addr 172.21.15.73:0/3964872370 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:26.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.198+0000 7f5b427fc640 1 --2- 172.21.15.73:0/3964872370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5b440b15a0 0x7f5b440b6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:26.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.198+0000 7f5b437fe640 1 --2- 172.21.15.73:0/3964872370 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5b4415bfe0 0x7f5b44152890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:26.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.198+0000 7f5b42ffd640 1 -- 172.21.15.73:0/3964872370 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5b4415bfe0 msgr2=0x7f5b44152890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:26.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.198+0000 7f5b42ffd640 1 --2- 172.21.15.73:0/3964872370 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5b4415bfe0 0x7f5b44152890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:26.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.198+0000 7f5b42ffd640 1 -- 172.21.15.73:0/3964872370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5b440b15a0 msgr2=0x7f5b440b6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:26.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.198+0000 7f5b42ffd640 1 --2- 172.21.15.73:0/3964872370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5b440b15a0 0x7f5b440b6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:26.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.198+0000 7f5b42ffd640 1 -- 172.21.15.73:0/3964872370 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b4415b7a0 con 0x7f5b44070460 2024-04-26T20:50:26.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.198+0000 7f5b437fe640 1 --2- 172.21.15.73:0/3964872370 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5b4415bfe0 0x7f5b44152890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:50:26.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.198+0000 7f5b42ffd640 1 --2- 172.21.15.73:0/3964872370 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b44070460 0x7f5b4415b260 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5b3400c7c0 tx=0x7f5b3400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:26.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.198+0000 7f5b2bfff640 1 -- 172.21.15.73:0/3964872370 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b34018070 con 0x7f5b44070460 2024-04-26T20:50:26.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.199+0000 7f5b2bfff640 1 -- 172.21.15.73:0/3964872370 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b3400a640 con 0x7f5b44070460 2024-04-26T20:50:26.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.199+0000 7f5b49a45640 1 -- 172.21.15.73:0/3964872370 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b4415a1c0 con 0x7f5b44070460 2024-04-26T20:50:26.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.199+0000 7f5b2bfff640 1 -- 172.21.15.73:0/3964872370 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3400aa30 con 0x7f5b44070460 2024-04-26T20:50:26.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.199+0000 7f5b49a45640 1 -- 172.21.15.73:0/3964872370 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b4415a780 con 0x7f5b44070460 2024-04-26T20:50:26.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.200+0000 7f5b49a45640 1 -- 172.21.15.73:0/3964872370 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b440694f0 con 0x7f5b44070460 2024-04-26T20:50:26.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.200+0000 7f5b2bfff640 1 -- 172.21.15.73:0/3964872370 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5b3401b070 con 0x7f5b44070460 2024-04-26T20:50:26.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.200+0000 7f5b2bfff640 1 --2- 172.21.15.73:0/3964872370 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f5b3006c8f0 0x7f5b3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:26.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.201+0000 7f5b427fc640 1 --2- 172.21.15.73:0/3964872370 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f5b3006c8f0 0x7f5b3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:26.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.201+0000 7f5b2bfff640 1 -- 172.21.15.73:0/3964872370 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f5b3408ad60 con 0x7f5b44070460 2024-04-26T20:50:26.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.201+0000 7f5b427fc640 1 --2- 172.21.15.73:0/3964872370 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f5b3006c8f0 0x7f5b3006edb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5b440d09a0 tx=0x7f5b38008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:26.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.205+0000 7f5b2bfff640 1 -- 172.21.15.73:0/3964872370 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5b34058e90 con 0x7f5b44070460 2024-04-26T20:50:26.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.416+0000 7f5b49a45640 1 -- 172.21.15.73:0/3964872370 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b440b0930 con 0x7f5b44070460 2024-04-26T20:50:26.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.416+0000 7f5b2bfff640 1 -- 172.21.15.73:0/3964872370 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1392 (secure 0 0 0) 0x7f5b3405d190 con 0x7f5b44070460 2024-04-26T20:50:26.417 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:50:26.417 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2146325725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2146325725},{"type":"v1","addr":"172.21.15.73:6835","nonce":2146325725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-26T20:50:26.418 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 2 2024-04-26T20:50:26.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.418+0000 7f5b49a45640 1 -- 172.21.15.73:0/3964872370 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f5b3006c8f0 msgr2=0x7f5b3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:26.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.418+0000 7f5b49a45640 1 --2- 172.21.15.73:0/3964872370 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f5b3006c8f0 0x7f5b3006edb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5b440d09a0 tx=0x7f5b38008040 comp rx=0 tx=0).stop 2024-04-26T20:50:26.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.418+0000 7f5b49a45640 1 -- 172.21.15.73:0/3964872370 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b44070460 msgr2=0x7f5b4415b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:26.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.418+0000 7f5b49a45640 1 --2- 172.21.15.73:0/3964872370 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b44070460 0x7f5b4415b260 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5b3400c7c0 tx=0x7f5b3400cc80 comp rx=0 tx=0).stop 2024-04-26T20:50:26.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.418+0000 7f5b49a45640 1 -- 172.21.15.73:0/3964872370 shutdown_connections 2024-04-26T20:50:26.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.418+0000 7f5b49a45640 1 --2- 172.21.15.73:0/3964872370 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f5b3006c8f0 0x7f5b3006edb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:26.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.418+0000 7f5b49a45640 1 --2- 172.21.15.73:0/3964872370 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5b4415bfe0 0x7f5b44152890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:26.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.418+0000 7f5b49a45640 1 --2- 172.21.15.73:0/3964872370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5b440b15a0 0x7f5b440b6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:26.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.418+0000 7f5b49a45640 1 --2- 172.21.15.73:0/3964872370 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b44070460 0x7f5b4415b260 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:26.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.418+0000 7f5b49a45640 1 -- 172.21.15.73:0/3964872370 >> 172.21.15.73:0/3964872370 conn(0x7f5b44068090 msgr2=0x7f5b4415f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:26.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.418+0000 7f5b49a45640 1 -- 172.21.15.73:0/3964872370 shutdown_connections 2024-04-26T20:50:26.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:26.418+0000 7f5b49a45640 1 -- 172.21.15.73:0/3964872370 wait complete. 2024-04-26T20:50:28.430 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:50:28.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.539+0000 7f6af1f5f640 1 Processor -- start 2024-04-26T20:50:28.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.539+0000 7f6af1f5f640 1 -- start start 2024-04-26T20:50:28.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.540+0000 7f6af1f5f640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6aec0b0b70 0x7f6aec0b0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:28.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.540+0000 7f6af1f5f640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6aec0aed90 0x7f6aec0af240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:28.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.540+0000 7f6af1f5f640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6aec0b1490 con 0x7f6aec0b4d40 2024-04-26T20:50:28.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.540+0000 7f6af1f5f640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f6aec073bc0 con 0x7f6aec0aed90 2024-04-26T20:50:28.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.540+0000 7f6af1f5f640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f6aec072fb0 con 0x7f6aec0b0b70 2024-04-26T20:50:28.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.540+0000 7f6aebfff640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6aec0aed90 0x7f6aec0af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:28.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.540+0000 7f6aeaffd640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f6aec0b4d40 0x7f6aec0b6710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:46510/0 (socket says 172.21.15.73:46510) 2024-04-26T20:50:28.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.540+0000 7f6aeaffd640 1 -- 172.21.15.73:0/3673608574 learned_addr learned my addr 172.21.15.73:0/3673608574 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:50:28.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.540+0000 7f6aebfff640 1 -- 172.21.15.73:0/3673608574 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6aec0b0b70 msgr2=0x7f6aec0b0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:28.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.540+0000 7f6aeb7fe640 1 --2- 172.21.15.73:0/3673608574 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6aec0b0b70 0x7f6aec0b0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:28.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.540+0000 7f6aebfff640 1 --2- 172.21.15.73:0/3673608574 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6aec0b0b70 0x7f6aec0b0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:28.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.540+0000 7f6aebfff640 1 -- 172.21.15.73:0/3673608574 >> v1:172.21.15.73:6789/0 conn(0x7f6aec0b4d40 legacy=0x7f6aec0b6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:28.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.540+0000 7f6aebfff640 1 -- 172.21.15.73:0/3673608574 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aec073290 con 0x7f6aec0aed90 2024-04-26T20:50:28.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.540+0000 7f6aea7fc640 1 -- 172.21.15.73:0/3673608574 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3746176835 0 0) 0x7f6aec0b1490 con 0x7f6aec0b4d40 2024-04-26T20:50:28.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.541+0000 7f6aebfff640 1 --2- 172.21.15.73:0/3673608574 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6aec0aed90 0x7f6aec0af240 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4009d90 tx=0x7f6ae402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aa68c68cdff62e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:28.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.541+0000 7f6aea7fc640 1 -- 172.21.15.73:0/3673608574 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4007740 con 0x7f6aec0aed90 2024-04-26T20:50:28.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.541+0000 7f6aea7fc640 1 -- 172.21.15.73:0/3673608574 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ae40078e0 con 0x7f6aec0aed90 2024-04-26T20:50:28.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.541+0000 7f6aea7fc640 1 -- 172.21.15.73:0/3673608574 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4007cb0 con 0x7f6aec0aed90 2024-04-26T20:50:28.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.542+0000 7f6af1f5f640 1 -- 172.21.15.73:0/3673608574 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6aec0aed90 msgr2=0x7f6aec0af240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:28.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.542+0000 7f6af1f5f640 1 --2- 172.21.15.73:0/3673608574 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6aec0aed90 0x7f6aec0af240 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4009d90 tx=0x7f6ae402a660 comp rx=0 tx=0).stop 2024-04-26T20:50:28.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.542+0000 7f6af1f5f640 1 -- 172.21.15.73:0/3673608574 shutdown_connections 2024-04-26T20:50:28.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.542+0000 7f6af1f5f640 1 --2- 172.21.15.73:0/3673608574 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6aec0aed90 0x7f6aec0af240 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:28.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.542+0000 7f6af1f5f640 1 --2- 172.21.15.73:0/3673608574 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6aec0b0b70 0x7f6aec0b0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:28.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.542+0000 7f6af1f5f640 1 -- 172.21.15.73:0/3673608574 >> 172.21.15.73:0/3673608574 conn(0x7f6aec0b39b0 msgr2=0x7f6aec0b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:28.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.542+0000 7f6af1f5f640 1 -- 172.21.15.73:0/3673608574 shutdown_connections 2024-04-26T20:50:28.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.542+0000 7f6af1f5f640 1 -- 172.21.15.73:0/3673608574 wait complete. 2024-04-26T20:50:28.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.543+0000 7f6af1f5f640 1 Processor -- start 2024-04-26T20:50:28.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.543+0000 7f6af1f5f640 1 -- start start 2024-04-26T20:50:28.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.544+0000 7f6af1f5f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6aec0aed90 0x7f6aec157c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:28.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.544+0000 7f6af1f5f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6aec0b0b70 0x7f6aec1557e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:28.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.544+0000 7f6af1f5f640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6aec0b4d40 0x7f6aec156260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:28.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.544+0000 7f6af1f5f640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6aec0b6e20 con 0x7f6aec0b0b70 2024-04-26T20:50:28.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.544+0000 7f6af1f5f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6aec073850 con 0x7f6aec0aed90 2024-04-26T20:50:28.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.544+0000 7f6af1f5f640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f6aec160360 con 0x7f6aec0b4d40 2024-04-26T20:50:28.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.544+0000 7f6aeaffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6aec0b0b70 0x7f6aec1557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:28.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.544+0000 7f6aeaffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6aec0b0b70 0x7f6aec1557e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47436/0 (socket says 172.21.15.73:47436) 2024-04-26T20:50:28.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.544+0000 7f6aeaffd640 1 -- 172.21.15.73:0/3954863304 learned_addr learned my addr 172.21.15.73:0/3954863304 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:28.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.544+0000 7f6aeb7fe640 1 --2- 172.21.15.73:0/3954863304 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6aec0aed90 0x7f6aec157c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:28.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.544+0000 7f6aebfff640 1 --2- 172.21.15.73:0/3954863304 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6aec0b4d40 0x7f6aec156260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:28.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.544+0000 7f6aeaffd640 1 -- 172.21.15.73:0/3954863304 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6aec0b4d40 msgr2=0x7f6aec156260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:28.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.544+0000 7f6aeaffd640 1 --2- 172.21.15.73:0/3954863304 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6aec0b4d40 0x7f6aec156260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:28.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.544+0000 7f6aeaffd640 1 -- 172.21.15.73:0/3954863304 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6aec0aed90 msgr2=0x7f6aec157c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:28.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.544+0000 7f6aeaffd640 1 --2- 172.21.15.73:0/3954863304 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6aec0aed90 0x7f6aec157c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:28.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.544+0000 7f6aeaffd640 1 -- 172.21.15.73:0/3954863304 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aec156b90 con 0x7f6aec0b0b70 2024-04-26T20:50:28.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.544+0000 7f6aeaffd640 1 --2- 172.21.15.73:0/3954863304 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6aec0b0b70 0x7f6aec1557e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6ae000c4e0 tx=0x7f6ae000c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:28.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.545+0000 7f6ad3fff640 1 -- 172.21.15.73:0/3954863304 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0018070 con 0x7f6aec0b0b70 2024-04-26T20:50:28.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.545+0000 7f6ad3fff640 1 -- 172.21.15.73:0/3954863304 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ae0007500 con 0x7f6aec0b0b70 2024-04-26T20:50:28.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.545+0000 7f6ad3fff640 1 -- 172.21.15.73:0/3954863304 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0004a20 con 0x7f6aec0b0b70 2024-04-26T20:50:28.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.545+0000 7f6af1f5f640 1 -- 172.21.15.73:0/3954863304 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6aec156e70 con 0x7f6aec0b0b70 2024-04-26T20:50:28.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.545+0000 7f6af1f5f640 1 -- 172.21.15.73:0/3954863304 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6aec157430 con 0x7f6aec0b0b70 2024-04-26T20:50:28.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.546+0000 7f6af1f5f640 1 -- 172.21.15.73:0/3954863304 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6aec0694f0 con 0x7f6aec0b0b70 2024-04-26T20:50:28.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.546+0000 7f6ad3fff640 1 -- 172.21.15.73:0/3954863304 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6ae001b070 con 0x7f6aec0b0b70 2024-04-26T20:50:28.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.546+0000 7f6ad3fff640 1 --2- 172.21.15.73:0/3954863304 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f6ad806c8f0 0x7f6ad806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:28.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.546+0000 7f6ad3fff640 1 -- 172.21.15.73:0/3954863304 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f6ae008aab0 con 0x7f6aec0b0b70 2024-04-26T20:50:28.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.546+0000 7f6aeb7fe640 1 --2- 172.21.15.73:0/3954863304 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f6ad806c8f0 0x7f6ad806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:28.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.547+0000 7f6aeb7fe640 1 --2- 172.21.15.73:0/3954863304 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f6ad806c8f0 0x7f6ad806edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6adc00aac0 tx=0x7f6adc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:28.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.551+0000 7f6ad3fff640 1 -- 172.21.15.73:0/3954863304 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6ae0058ca0 con 0x7f6aec0b0b70 2024-04-26T20:50:28.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.763+0000 7f6af1f5f640 1 -- 172.21.15.73:0/3954863304 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6aec06dfb0 con 0x7f6aec0b0b70 2024-04-26T20:50:28.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.764+0000 7f6ad3fff640 1 -- 172.21.15.73:0/3954863304 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3867 (secure 0 0 0) 0x7f6ae005cfa0 con 0x7f6aec0b0b70 2024-04-26T20:50:28.765 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:50:28.765 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/4263167680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6833","nonce":4263167680},{"type":"v1","addr":"172.21.15.120:6835","nonce":4263167680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6834/1614342510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1614342510},{"type":"v1","addr":"172.21.15.120:6834","nonce":1614342510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2146325725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2146325725},{"type":"v1","addr":"172.21.15.73:6835","nonce":2146325725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/181202544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":181202544},{"type":"v1","addr":"172.21.15.73:6837","nonce":181202544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-26T20:50:28.765 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 3 2024-04-26T20:50:28.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.766+0000 7f6af1f5f640 1 -- 172.21.15.73:0/3954863304 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f6ad806c8f0 msgr2=0x7f6ad806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:28.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.766+0000 7f6af1f5f640 1 --2- 172.21.15.73:0/3954863304 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f6ad806c8f0 0x7f6ad806edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6adc00aac0 tx=0x7f6adc008040 comp rx=0 tx=0).stop 2024-04-26T20:50:28.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.766+0000 7f6af1f5f640 1 -- 172.21.15.73:0/3954863304 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6aec0b0b70 msgr2=0x7f6aec1557e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:28.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.766+0000 7f6af1f5f640 1 --2- 172.21.15.73:0/3954863304 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6aec0b0b70 0x7f6aec1557e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6ae000c4e0 tx=0x7f6ae000c9a0 comp rx=0 tx=0).stop 2024-04-26T20:50:28.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.766+0000 7f6af1f5f640 1 -- 172.21.15.73:0/3954863304 shutdown_connections 2024-04-26T20:50:28.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.766+0000 7f6af1f5f640 1 --2- 172.21.15.73:0/3954863304 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6aec0b0b70 0x7f6aec1557e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:28.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.766+0000 7f6af1f5f640 1 --2- 172.21.15.73:0/3954863304 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f6ad806c8f0 0x7f6ad806edb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:28.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.766+0000 7f6af1f5f640 1 --2- 172.21.15.73:0/3954863304 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6aec0b4d40 0x7f6aec156260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:28.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.766+0000 7f6af1f5f640 1 --2- 172.21.15.73:0/3954863304 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6aec0aed90 0x7f6aec157c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:28.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.766+0000 7f6af1f5f640 1 -- 172.21.15.73:0/3954863304 >> 172.21.15.73:0/3954863304 conn(0x7f6aec0b39b0 msgr2=0x7f6aec0b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:28.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.766+0000 7f6af1f5f640 1 -- 172.21.15.73:0/3954863304 shutdown_connections 2024-04-26T20:50:28.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.766+0000 7f6af1f5f640 1 -- 172.21.15.73:0/3954863304 wait complete. 2024-04-26T20:50:28.778 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:50:28.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.873+0000 7f0809324640 1 Processor -- start 2024-04-26T20:50:28.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.873+0000 7f0809324640 1 -- start start 2024-04-26T20:50:28.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.874+0000 7f0809324640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f08040d06f0 0x7f08040d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:28.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.874+0000 7f0809324640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f08040b2850 0x7f08040b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:28.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.874+0000 7f0809324640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08040d0ba0 con 0x7f080415f3a0 2024-04-26T20:50:28.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.874+0000 7f0809324640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f08040d0e20 con 0x7f08040d06f0 2024-04-26T20:50:28.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.874+0000 7f0809324640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f08040b1d70 con 0x7f08040b2850 2024-04-26T20:50:28.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.874+0000 7f0802ffd640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f080415f3a0 0x7f08040d1770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:46512/0 (socket says 172.21.15.73:46512) 2024-04-26T20:50:28.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.874+0000 7f0802ffd640 1 -- 172.21.15.73:0/780960637 learned_addr learned my addr 172.21.15.73:0/780960637 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:50:28.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.874+0000 7f08027fc640 1 -- 172.21.15.73:0/780960637 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1818524635 0 0) 0x7f07f4001110 con 0x7f080415f3a0 2024-04-26T20:50:28.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.874+0000 7f08027fc640 1 -- 172.21.15.73:0/780960637 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f07f0003520 con 0x7f080415f3a0 2024-04-26T20:50:28.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.874+0000 7f08037fe640 1 --2- 172.21.15.73:0/780960637 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f08040b2850 0x7f08040b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:28.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.874+0000 7f07fbfff640 1 --2- 172.21.15.73:0/780960637 >> v2:172.21.15.120:3300/0 conn(0x7f08040d06f0 0x7f08040d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:28.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.874+0000 7f08027fc640 1 -- 172.21.15.73:0/780960637 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3339088940 0 0) 0x7f07f0003520 con 0x7f080415f3a0 2024-04-26T20:50:28.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.874+0000 7f08027fc640 1 -- 172.21.15.73:0/780960637 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f08040b2850 msgr2=0x7f08040b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:28.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.874+0000 7f08027fc640 1 --2- 172.21.15.73:0/780960637 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f08040b2850 0x7f08040b2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:28.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.874+0000 7f08027fc640 1 -- 172.21.15.73:0/780960637 >> v2:172.21.15.120:3300/0 conn(0x7f08040d06f0 msgr2=0x7f08040d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:28.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.874+0000 7f08027fc640 1 --2- 172.21.15.73:0/780960637 >> v2:172.21.15.120:3300/0 conn(0x7f08040d06f0 0x7f08040d1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:28.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.874+0000 7f08027fc640 1 -- 172.21.15.73:0/780960637 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08040b2050 con 0x7f080415f3a0 2024-04-26T20:50:28.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.874+0000 7f08027fc640 1 -- 172.21.15.73:0/780960637 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7f07f4001840 con 0x7f080415f3a0 2024-04-26T20:50:28.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.874+0000 7f08027fc640 1 -- 172.21.15.73:0/780960637 >> v1:172.21.15.73:6789/0 conn(0x7f080415f3a0 legacy=0x7f08040d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:28.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.874+0000 7f08027fc640 1 --2- 172.21.15.73:0/780960637 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f08040b2850 0x7f07f00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:28.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.874+0000 7f08027fc640 1 --2- 172.21.15.73:0/780960637 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f07f0003fe0 0x7f07f0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:28.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.874+0000 7f08027fc640 1 --2- 172.21.15.73:0/780960637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07f0004d40 0x7f07f00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:28.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.874+0000 7f08027fc640 1 -- 172.21.15.73:0/780960637 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f08040d0e20 con 0x7f07f0004d40 2024-04-26T20:50:28.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.874+0000 7f08027fc640 1 -- 172.21.15.73:0/780960637 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f08040b1d70 con 0x7f07f0003fe0 2024-04-26T20:50:28.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.874+0000 7f08027fc640 1 -- 172.21.15.73:0/780960637 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f07f000bac0 con 0x7f08040b2850 2024-04-26T20:50:28.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.875+0000 7f0802ffd640 1 --2- 172.21.15.73:0/780960637 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f08040b2850 0x7f07f00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:28.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.875+0000 7f08037fe640 1 --2- 172.21.15.73:0/780960637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07f0004d40 0x7f07f00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:28.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.875+0000 7f07fbfff640 1 --2- 172.21.15.73:0/780960637 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f07f0003fe0 0x7f07f0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:28.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.875+0000 7f08037fe640 1 -- 172.21.15.73:0/780960637 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f08040b2850 msgr2=0x7f07f00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:28.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.875+0000 7f08037fe640 1 --2- 172.21.15.73:0/780960637 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f08040b2850 0x7f07f00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:28.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.875+0000 7f08037fe640 1 -- 172.21.15.73:0/780960637 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f07f0003fe0 msgr2=0x7f07f0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:28.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.875+0000 7f08037fe640 1 --2- 172.21.15.73:0/780960637 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f07f0003fe0 0x7f07f0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:28.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.875+0000 7f08037fe640 1 -- 172.21.15.73:0/780960637 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07f0003520 con 0x7f07f0004d40 2024-04-26T20:50:28.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.875+0000 7f08037fe640 1 --2- 172.21.15.73:0/780960637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07f0004d40 0x7f07f00051c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f07fc007360 tx=0x7f07f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8792943c323be3f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:28.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.876+0000 7f08027fc640 1 -- 172.21.15.73:0/780960637 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07fc009be0 con 0x7f07f0004d40 2024-04-26T20:50:28.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.876+0000 7f08027fc640 1 -- 172.21.15.73:0/780960637 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07fc036600 con 0x7f07f0004d40 2024-04-26T20:50:28.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.876+0000 7f08027fc640 1 -- 172.21.15.73:0/780960637 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07fc0369d0 con 0x7f07f0004d40 2024-04-26T20:50:28.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.876+0000 7f0809324640 1 -- 172.21.15.73:0/780960637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07f0004d40 msgr2=0x7f07f00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:28.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.876+0000 7f0809324640 1 --2- 172.21.15.73:0/780960637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07f0004d40 0x7f07f00051c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f07fc007360 tx=0x7f07f0005830 comp rx=0 tx=0).stop 2024-04-26T20:50:28.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.876+0000 7f08037fe640 1 -- 172.21.15.73:0/780960637 reap_dead start 2024-04-26T20:50:28.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.876+0000 7f0809324640 1 -- 172.21.15.73:0/780960637 shutdown_connections 2024-04-26T20:50:28.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.876+0000 7f0809324640 1 -- 172.21.15.73:0/780960637 >> 172.21.15.73:0/780960637 conn(0x7f08040d4f90 msgr2=0x7f08040d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:28.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.876+0000 7f0809324640 1 -- 172.21.15.73:0/780960637 shutdown_connections 2024-04-26T20:50:28.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.876+0000 7f0809324640 1 -- 172.21.15.73:0/780960637 wait complete. 2024-04-26T20:50:28.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.877+0000 7f0809324640 1 Processor -- start 2024-04-26T20:50:28.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.877+0000 7f0809324640 1 -- start start 2024-04-26T20:50:28.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.878+0000 7f0809324640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f080415de20 0x7f08040caa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:28.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.878+0000 7f0809324640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f080415e200 0x7f08040caf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:28.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.878+0000 7f0809324640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08040b7b80 0x7f08040b8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:28.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.878+0000 7f0809324640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f080415f900 con 0x7f08040b7b80 2024-04-26T20:50:28.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.878+0000 7f0809324640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f080415f780 con 0x7f080415de20 2024-04-26T20:50:28.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.878+0000 7f0809324640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f08040d23c0 con 0x7f080415e200 2024-04-26T20:50:28.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.878+0000 7f08037fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08040b7b80 0x7f08040b8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:28.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.878+0000 7f08037fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08040b7b80 0x7f08040b8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47456/0 (socket says 172.21.15.73:47456) 2024-04-26T20:50:28.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.878+0000 7f08037fe640 1 -- 172.21.15.73:0/1222714201 learned_addr learned my addr 172.21.15.73:0/1222714201 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:28.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.878+0000 7f0802ffd640 1 --2- 172.21.15.73:0/1222714201 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f080415de20 0x7f08040caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:28.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.878+0000 7f07fbfff640 1 --2- 172.21.15.73:0/1222714201 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f080415e200 0x7f08040caf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:28.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.878+0000 7f08037fe640 1 -- 172.21.15.73:0/1222714201 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f080415e200 msgr2=0x7f08040caf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:28.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.878+0000 7f08037fe640 1 --2- 172.21.15.73:0/1222714201 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f080415e200 0x7f08040caf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:28.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.878+0000 7f08037fe640 1 -- 172.21.15.73:0/1222714201 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f080415de20 msgr2=0x7f08040caa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:28.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.878+0000 7f08037fe640 1 --2- 172.21.15.73:0/1222714201 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f080415de20 0x7f08040caa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:28.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.878+0000 7f08037fe640 1 -- 172.21.15.73:0/1222714201 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08040b8760 con 0x7f08040b7b80 2024-04-26T20:50:28.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.878+0000 7f0802ffd640 1 --2- 172.21.15.73:0/1222714201 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f080415de20 0x7f08040caa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:50:28.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.879+0000 7f08037fe640 1 --2- 172.21.15.73:0/1222714201 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08040b7b80 0x7f08040b8030 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f07fc007480 tx=0x7f07fc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:28.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.879+0000 7f07fb7fe640 1 -- 172.21.15.73:0/1222714201 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07fc009be0 con 0x7f08040b7b80 2024-04-26T20:50:28.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.879+0000 7f07fb7fe640 1 -- 172.21.15.73:0/1222714201 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07fc036400 con 0x7f08040b7b80 2024-04-26T20:50:28.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.879+0000 7f0809324640 1 -- 172.21.15.73:0/1222714201 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08040b8a40 con 0x7f08040b7b80 2024-04-26T20:50:28.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.879+0000 7f07fb7fe640 1 -- 172.21.15.73:0/1222714201 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07fc038520 con 0x7f08040b7b80 2024-04-26T20:50:28.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.879+0000 7f0809324640 1 -- 172.21.15.73:0/1222714201 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08040b9000 con 0x7f08040b7b80 2024-04-26T20:50:28.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.880+0000 7f0809324640 1 -- 172.21.15.73:0/1222714201 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08040694f0 con 0x7f08040b7b80 2024-04-26T20:50:28.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.880+0000 7f07fb7fe640 1 -- 172.21.15.73:0/1222714201 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f07fc02e620 con 0x7f08040b7b80 2024-04-26T20:50:28.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.880+0000 7f07fb7fe640 1 --2- 172.21.15.73:0/1222714201 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f07e406c8f0 0x7f07e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:28.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.880+0000 7f0802ffd640 1 --2- 172.21.15.73:0/1222714201 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f07e406c8f0 0x7f07e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:28.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.881+0000 7f07fb7fe640 1 -- 172.21.15.73:0/1222714201 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f07fc0ac070 con 0x7f08040b7b80 2024-04-26T20:50:28.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.881+0000 7f0802ffd640 1 --2- 172.21.15.73:0/1222714201 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f07e406c8f0 0x7f07e406edb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f07f40012b0 tx=0x7f07f4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:28.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:28.885+0000 7f07fb7fe640 1 -- 172.21.15.73:0/1222714201 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f07fc048060 con 0x7f08040b7b80 2024-04-26T20:50:29.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.060+0000 7f0809324640 1 -- 172.21.15.73:0/1222714201 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f08040b6860 con 0x7f08040b7b80 2024-04-26T20:50:29.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.061+0000 7f07fb7fe640 1 -- 172.21.15.73:0/1222714201 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12884 (secure 0 0 0) 0x7f07fc032030 con 0x7f08040b7b80 2024-04-26T20:50:29.062 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:50:29.062 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":16,"fsid":"fbd3f548-b547-4713-856b-09f98f5d1764","created":"2024-04-26T20:50:07.876786+0000","modified":"2024-04-26T20:50:22.089366+0000","last_up_change":"2024-04-26T20:50:19.082741+0000","last_in_change":"2024-04-26T20:50:10.018400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:50:19.993731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d6f4f3-214a-4392-a02f-c33cbcab3c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6809","nonce":3461705124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6811","nonce":3461705124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6815","nonce":3461705124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6813","nonce":3461705124}]},"public_addr":"172.21.15.73:6809/3461705124","cluster_addr":"172.21.15.73:6811/3461705124","heartbeat_back_addr":"172.21.15.73:6815/3461705124","heartbeat_front_addr":"172.21.15.73:6813/3461705124","state":["exists","up"]},{"osd":1,"uuid":"786bf8ca-6ddc-4b3c-be6c-9152398bd5fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6817","nonce":2827619403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6819","nonce":2827619403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6825","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6827","nonce":2827619403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6821","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6823","nonce":2827619403}]},"public_addr":"172.21.15.73:6817/2827619403","cluster_addr":"172.21.15.73:6819/2827619403","heartbeat_back_addr":"172.21.15.73:6827/2827619403","heartbeat_front_addr":"172.21.15.73:6823/2827619403","state":["exists","up"]},{"osd":2,"uuid":"90963b1a-5437-4ef8-ad8b-6b74c534eeb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6801","nonce":2542681064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6803","nonce":2542681064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6808","nonce":2542681064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6805","nonce":2542681064}]},"public_addr":"172.21.15.73:6801/2542681064","cluster_addr":"172.21.15.73:6803/2542681064","heartbeat_back_addr":"172.21.15.73:6808/2542681064","heartbeat_front_addr":"172.21.15.73:6805/2542681064","state":["exists","up"]},{"osd":3,"uuid":"bd3d2148-6e38-41b4-a453-fe6e3b8b4ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6822","nonce":1604042718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6826","nonce":1604042718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6831","nonce":1604042718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6829","nonce":1604042718}]},"public_addr":"172.21.15.73:6822/1604042718","cluster_addr":"172.21.15.73:6826/1604042718","heartbeat_back_addr":"172.21.15.73:6831/1604042718","heartbeat_front_addr":"172.21.15.73:6829/1604042718","state":["exists","up"]},{"osd":4,"uuid":"e8668dc8-a23f-46d2-8311-cb882da08e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6817","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6819","nonce":3591055561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6824","nonce":3591055561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6831","nonce":3591055561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6828","nonce":3591055561}]},"public_addr":"172.21.15.120:6819/3591055561","cluster_addr":"172.21.15.120:6824/3591055561","heartbeat_back_addr":"172.21.15.120:6831/3591055561","heartbeat_front_addr":"172.21.15.120:6828/3591055561","state":["exists","up"]},{"osd":5,"uuid":"cbac0bdb-96bd-4852-bcce-e564495c3fcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6809","nonce":819450433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6811","nonce":819450433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6815","nonce":819450433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6813","nonce":819450433}]},"public_addr":"172.21.15.120:6809/819450433","cluster_addr":"172.21.15.120:6811/819450433","heartbeat_back_addr":"172.21.15.120:6815/819450433","heartbeat_front_addr":"172.21.15.120:6813/819450433","state":["exists","up"]},{"osd":6,"uuid":"1884b2a8-569f-4272-96a9-f25a09364dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6801","nonce":3946210727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6803","nonce":3946210727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6807","nonce":3946210727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6805","nonce":3946210727}]},"public_addr":"172.21.15.120:6801/3946210727","cluster_addr":"172.21.15.120:6803/3946210727","heartbeat_back_addr":"172.21.15.120:6807/3946210727","heartbeat_front_addr":"172.21.15.120:6805/3946210727","state":["exists","up"]},{"osd":7,"uuid":"c0c3bf6d-a4c5-4f1e-aa2d-330951830666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6818","nonce":2915574674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6821","nonce":2915574674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6827","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6829","nonce":2915574674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6823","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6825","nonce":2915574674}]},"public_addr":"172.21.15.120:6818/2915574674","cluster_addr":"172.21.15.120:6821/2915574674","heartbeat_back_addr":"172.21.15.120:6829/2915574674","heartbeat_front_addr":"172.21.15.120:6825/2915574674","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.218216+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.208710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.112705+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.182789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.164324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.174212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.225396+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.261559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:50:29.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.063+0000 7f0809324640 1 -- 172.21.15.73:0/1222714201 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f07e406c8f0 msgr2=0x7f07e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:29.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.063+0000 7f0809324640 1 --2- 172.21.15.73:0/1222714201 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f07e406c8f0 0x7f07e406edb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f07f40012b0 tx=0x7f07f4009040 comp rx=0 tx=0).stop 2024-04-26T20:50:29.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.063+0000 7f0809324640 1 -- 172.21.15.73:0/1222714201 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08040b7b80 msgr2=0x7f08040b8030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:29.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.063+0000 7f0809324640 1 --2- 172.21.15.73:0/1222714201 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08040b7b80 0x7f08040b8030 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f07fc007480 tx=0x7f07fc004ab0 comp rx=0 tx=0).stop 2024-04-26T20:50:29.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.063+0000 7f0809324640 1 -- 172.21.15.73:0/1222714201 shutdown_connections 2024-04-26T20:50:29.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.063+0000 7f0809324640 1 --2- 172.21.15.73:0/1222714201 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08040b7b80 0x7f08040b8030 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:29.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.063+0000 7f0809324640 1 --2- 172.21.15.73:0/1222714201 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f07e406c8f0 0x7f07e406edb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:29.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.063+0000 7f0809324640 1 --2- 172.21.15.73:0/1222714201 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f080415e200 0x7f08040caf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:29.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.063+0000 7f0809324640 1 --2- 172.21.15.73:0/1222714201 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f080415de20 0x7f08040caa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:29.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.063+0000 7f0809324640 1 -- 172.21.15.73:0/1222714201 >> 172.21.15.73:0/1222714201 conn(0x7f08040d4f90 msgr2=0x7f08040d2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:29.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.063+0000 7f0809324640 1 -- 172.21.15.73:0/1222714201 shutdown_connections 2024-04-26T20:50:29.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.063+0000 7f0809324640 1 -- 172.21.15.73:0/1222714201 wait complete. 2024-04-26T20:50:29.075 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:50:19.993731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:50:29.075 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:50:29.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.174+0000 7f2fdbfff640 1 Processor -- start 2024-04-26T20:50:29.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.174+0000 7f2fdbfff640 1 -- start start 2024-04-26T20:50:29.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.175+0000 7f2fdbfff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f2fdc0b5120 0x7f2fdc0b5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:29.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.175+0000 7f2fdbfff640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2fdc0b5500 0x7f2fdc0b6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:29.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.175+0000 7f2fdbfff640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2fdc0b59b0 0x7f2fdc0b6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:29.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.175+0000 7f2fdbfff640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f2fdc0d03b0 con 0x7f2fdc0b59b0 2024-04-26T20:50:29.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.175+0000 7f2fdbfff640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f2fdc0d3dc0 con 0x7f2fdc0b5120 2024-04-26T20:50:29.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.175+0000 7f2fdbfff640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f2fdc0b75b0 con 0x7f2fdc0b5500 2024-04-26T20:50:29.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.175+0000 7f2fdb7fe640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2fdc0b59b0 0x7f2fdc0b6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:29.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.175+0000 7f2fdb7fe640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2fdc0b59b0 0x7f2fdc0b6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47464/0 (socket says 172.21.15.73:47464) 2024-04-26T20:50:29.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.175+0000 7f2fdb7fe640 1 -- 172.21.15.73:0/3782515926 learned_addr learned my addr 172.21.15.73:0/3782515926 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:29.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.175+0000 7f2fdb7fe640 1 -- 172.21.15.73:0/3782515926 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2fdc0b5500 msgr2=0x7f2fdc0b6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:29.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.175+0000 7f2fdaffd640 1 --2- 172.21.15.73:0/3782515926 >> v2:172.21.15.120:3300/0 conn(0x7f2fdc0b5120 0x7f2fdc0b5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:29.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.175+0000 7f2fdb7fe640 1 --2- 172.21.15.73:0/3782515926 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2fdc0b5500 0x7f2fdc0b6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:29.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.175+0000 7f2fdb7fe640 1 -- 172.21.15.73:0/3782515926 >> v2:172.21.15.120:3300/0 conn(0x7f2fdc0b5120 msgr2=0x7f2fdc0b5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:29.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.175+0000 7f2fdb7fe640 1 --2- 172.21.15.73:0/3782515926 >> v2:172.21.15.120:3300/0 conn(0x7f2fdc0b5120 0x7f2fdc0b5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:29.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.175+0000 7f2fdb7fe640 1 -- 172.21.15.73:0/3782515926 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fdc0d0600 con 0x7f2fdc0b59b0 2024-04-26T20:50:29.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.176+0000 7f2fdb7fe640 1 --2- 172.21.15.73:0/3782515926 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2fdc0b59b0 0x7f2fdc0b6d70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc06c7a0 tx=0x7f2fd402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d72af6992ea0071 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:29.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.176+0000 7f2fda7fc640 1 -- 172.21.15.73:0/3782515926 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd4036070 con 0x7f2fdc0b59b0 2024-04-26T20:50:29.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.176+0000 7f2fda7fc640 1 -- 172.21.15.73:0/3782515926 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fd4031600 con 0x7f2fdc0b59b0 2024-04-26T20:50:29.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.176+0000 7f2fda7fc640 1 -- 172.21.15.73:0/3782515926 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd40319d0 con 0x7f2fdc0b59b0 2024-04-26T20:50:29.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.176+0000 7f2fdbfff640 1 -- 172.21.15.73:0/3782515926 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2fdc0b59b0 msgr2=0x7f2fdc0b6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:29.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.176+0000 7f2fdbfff640 1 --2- 172.21.15.73:0/3782515926 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2fdc0b59b0 0x7f2fdc0b6d70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc06c7a0 tx=0x7f2fd402a610 comp rx=0 tx=0).stop 2024-04-26T20:50:29.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.176+0000 7f2fdbfff640 1 -- 172.21.15.73:0/3782515926 shutdown_connections 2024-04-26T20:50:29.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.176+0000 7f2fdbfff640 1 --2- 172.21.15.73:0/3782515926 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2fdc0b59b0 0x7f2fdc0b6d70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:29.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.176+0000 7f2fdbfff640 1 --2- 172.21.15.73:0/3782515926 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2fdc0b5500 0x7f2fdc0b6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:29.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.176+0000 7f2fdbfff640 1 --2- 172.21.15.73:0/3782515926 >> v2:172.21.15.120:3300/0 conn(0x7f2fdc0b5120 0x7f2fdc0b5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:29.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.176+0000 7f2fdbfff640 1 -- 172.21.15.73:0/3782515926 >> 172.21.15.73:0/3782515926 conn(0x7f2fdc0d4db0 msgr2=0x7f2fdc0d51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:29.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.176+0000 7f2fdbfff640 1 -- 172.21.15.73:0/3782515926 shutdown_connections 2024-04-26T20:50:29.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.176+0000 7f2fdbfff640 1 -- 172.21.15.73:0/3782515926 wait complete. 2024-04-26T20:50:29.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.178+0000 7f2fdbfff640 1 Processor -- start 2024-04-26T20:50:29.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.178+0000 7f2fdbfff640 1 -- start start 2024-04-26T20:50:29.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.178+0000 7f2fdbfff640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2fdc0b5120 0x7f2fdc0b9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:29.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.178+0000 7f2fdbfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2fdc0b5500 0x7f2fdc14f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:29.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.178+0000 7f2fdbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2fdc0b59b0 0x7f2fdc14fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:29.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.178+0000 7f2fdbfff640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2fdc0b72b0 con 0x7f2fdc0b5500 2024-04-26T20:50:29.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.178+0000 7f2fdbfff640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2fdc0d0bc0 con 0x7f2fdc0b59b0 2024-04-26T20:50:29.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.178+0000 7f2fdbfff640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f2fdc0d2b00 con 0x7f2fdc0b5120 2024-04-26T20:50:29.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.178+0000 7f2fd3fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2fdc0b5500 0x7f2fdc14f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:29.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.179+0000 7f2fd3fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2fdc0b5500 0x7f2fdc14f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47472/0 (socket says 172.21.15.73:47472) 2024-04-26T20:50:29.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.179+0000 7f2fd3fff640 1 -- 172.21.15.73:0/4140846447 learned_addr learned my addr 172.21.15.73:0/4140846447 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:29.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.179+0000 7f2fdaffd640 1 --2- 172.21.15.73:0/4140846447 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2fdc0b5120 0x7f2fdc0b9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:29.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.179+0000 7f2fdb7fe640 1 --2- 172.21.15.73:0/4140846447 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2fdc0b59b0 0x7f2fdc14fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:29.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.179+0000 7f2fd3fff640 1 -- 172.21.15.73:0/4140846447 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2fdc0b5120 msgr2=0x7f2fdc0b9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:29.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.179+0000 7f2fd3fff640 1 --2- 172.21.15.73:0/4140846447 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2fdc0b5120 0x7f2fdc0b9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:29.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.179+0000 7f2fd3fff640 1 -- 172.21.15.73:0/4140846447 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2fdc0b59b0 msgr2=0x7f2fdc14fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:29.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.179+0000 7f2fd3fff640 1 --2- 172.21.15.73:0/4140846447 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2fdc0b59b0 0x7f2fdc14fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:29.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.179+0000 7f2fd3fff640 1 -- 172.21.15.73:0/4140846447 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fdc0b9b50 con 0x7f2fdc0b5500 2024-04-26T20:50:29.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.179+0000 7f2fd3fff640 1 --2- 172.21.15.73:0/4140846447 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2fdc0b5500 0x7f2fdc14f380 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2fc400b450 tx=0x7f2fc400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:29.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.179+0000 7f2fd8ff9640 1 -- 172.21.15.73:0/4140846447 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc4015020 con 0x7f2fdc0b5500 2024-04-26T20:50:29.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.179+0000 7f2fd8ff9640 1 -- 172.21.15.73:0/4140846447 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fc4011890 con 0x7f2fdc0b5500 2024-04-26T20:50:29.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.179+0000 7f2fd8ff9640 1 -- 172.21.15.73:0/4140846447 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc4011c80 con 0x7f2fdc0b5500 2024-04-26T20:50:29.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.179+0000 7f2fdbfff640 1 -- 172.21.15.73:0/4140846447 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fdc0a1080 con 0x7f2fdc0b5500 2024-04-26T20:50:29.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.179+0000 7f2fdbfff640 1 -- 172.21.15.73:0/4140846447 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fdc0a1920 con 0x7f2fdc0b5500 2024-04-26T20:50:29.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.180+0000 7f2fdbfff640 1 -- 172.21.15.73:0/4140846447 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fdc069370 con 0x7f2fdc0b5500 2024-04-26T20:50:29.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.181+0000 7f2fd8ff9640 1 -- 172.21.15.73:0/4140846447 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2fc4018020 con 0x7f2fdc0b5500 2024-04-26T20:50:29.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.181+0000 7f2fd8ff9640 1 --2- 172.21.15.73:0/4140846447 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f2fbc06c8f0 0x7f2fbc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:29.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.181+0000 7f2fdaffd640 1 --2- 172.21.15.73:0/4140846447 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f2fbc06c8f0 0x7f2fbc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:29.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.181+0000 7f2fd8ff9640 1 -- 172.21.15.73:0/4140846447 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f2fc408b910 con 0x7f2fdc0b5500 2024-04-26T20:50:29.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.181+0000 7f2fdaffd640 1 --2- 172.21.15.73:0/4140846447 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f2fbc06c8f0 0x7f2fbc06edb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc00a970 tx=0x7f2fcc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:29.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.185+0000 7f2fd8ff9640 1 -- 172.21.15.73:0/4140846447 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2fc401f020 con 0x7f2fdc0b5500 2024-04-26T20:50:29.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.360+0000 7f2fdbfff640 1 -- 172.21.15.73:0/4140846447 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2fdc14f8c0 con 0x7f2fdc0b5500 2024-04-26T20:50:29.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.361+0000 7f2fd8ff9640 1 -- 172.21.15.73:0/4140846447 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f2fc400d040 con 0x7f2fdc0b5500 2024-04-26T20:50:29.362 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-26T20:50:29.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.363+0000 7f2fdbfff640 1 -- 172.21.15.73:0/4140846447 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f2fbc06c8f0 msgr2=0x7f2fbc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:29.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.363+0000 7f2fdbfff640 1 --2- 172.21.15.73:0/4140846447 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f2fbc06c8f0 0x7f2fbc06edb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc00a970 tx=0x7f2fcc008040 comp rx=0 tx=0).stop 2024-04-26T20:50:29.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.363+0000 7f2fdbfff640 1 -- 172.21.15.73:0/4140846447 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2fdc0b5500 msgr2=0x7f2fdc14f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:29.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.363+0000 7f2fdbfff640 1 --2- 172.21.15.73:0/4140846447 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2fdc0b5500 0x7f2fdc14f380 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2fc400b450 tx=0x7f2fc400b910 comp rx=0 tx=0).stop 2024-04-26T20:50:29.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.363+0000 7f2fdbfff640 1 -- 172.21.15.73:0/4140846447 shutdown_connections 2024-04-26T20:50:29.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.363+0000 7f2fdbfff640 1 --2- 172.21.15.73:0/4140846447 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2fdc0b5500 0x7f2fdc14f380 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:29.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.363+0000 7f2fdbfff640 1 --2- 172.21.15.73:0/4140846447 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f2fbc06c8f0 0x7f2fbc06edb0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:29.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.363+0000 7f2fdbfff640 1 --2- 172.21.15.73:0/4140846447 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2fdc0b59b0 0x7f2fdc14fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:29.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.363+0000 7f2fdbfff640 1 --2- 172.21.15.73:0/4140846447 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2fdc0b5120 0x7f2fdc0b9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:29.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.363+0000 7f2fdbfff640 1 -- 172.21.15.73:0/4140846447 >> 172.21.15.73:0/4140846447 conn(0x7f2fdc0d4db0 msgr2=0x7f2fdc0b7f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:29.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.363+0000 7f2fdbfff640 1 -- 172.21.15.73:0/4140846447 shutdown_connections 2024-04-26T20:50:29.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.363+0000 7f2fdbfff640 1 -- 172.21.15.73:0/4140846447 wait complete. 2024-04-26T20:50:29.375 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T20:50:29.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.471+0000 7f0f3cc52640 1 Processor -- start 2024-04-26T20:50:29.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.471+0000 7f0f3cc52640 1 -- start start 2024-04-26T20:50:29.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.472+0000 7f0f3cc52640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0f3806fc40 0x7f0f38073360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:29.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.472+0000 7f0f3cc52640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f38156840 con 0x7f0f38156460 2024-04-26T20:50:29.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.472+0000 7f0f3cc52640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f38070090 con 0x7f0f3806f790 2024-04-26T20:50:29.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.472+0000 7f0f3cc52640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f0f380cbf20 con 0x7f0f3806fc40 2024-04-26T20:50:29.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.472+0000 7f0f3659c640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f0f38156460 0x7f0f380cb4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51976/0 (socket says 172.21.15.73:51976) 2024-04-26T20:50:29.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.472+0000 7f0f3659c640 1 -- 172.21.15.73:0/265913404 learned_addr learned my addr 172.21.15.73:0/265913404 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:50:29.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.472+0000 7f0f36d9d640 1 --2- 172.21.15.73:0/265913404 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0f3806fc40 0x7f0f38073360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:29.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.472+0000 7f0f35d9b640 1 -- 172.21.15.73:0/265913404 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3670427224 0 0) 0x7f0f38156840 con 0x7f0f38156460 2024-04-26T20:50:29.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.472+0000 7f0f35d9b640 1 -- 172.21.15.73:0/265913404 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0f24003520 con 0x7f0f38156460 2024-04-26T20:50:29.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.472+0000 7f0f35d9b640 1 -- 172.21.15.73:0/265913404 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3804613507 0 0) 0x7f0f24003520 con 0x7f0f38156460 2024-04-26T20:50:29.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.472+0000 7f0f35d9b640 1 -- 172.21.15.73:0/265913404 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0f3806fc40 msgr2=0x7f0f38073360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:29.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.472+0000 7f0f35d9b640 1 --2- 172.21.15.73:0/265913404 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0f3806fc40 0x7f0f38073360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:29.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.472+0000 7f0f35d9b640 1 -- 172.21.15.73:0/265913404 >> v1:172.21.15.120:6789/0 conn(0x7f0f3806f790 legacy=0x7f0f38072c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:29.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.472+0000 7f0f35d9b640 1 -- 172.21.15.73:0/265913404 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f380cc200 con 0x7f0f38156460 2024-04-26T20:50:29.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.472+0000 7f0f35d9b640 1 -- 172.21.15.73:0/265913404 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7f0f28001670 con 0x7f0f38156460 2024-04-26T20:50:29.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.472+0000 7f0f35d9b640 1 -- 172.21.15.73:0/265913404 >> v1:172.21.15.73:6789/0 conn(0x7f0f38156460 legacy=0x7f0f380cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:29.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.472+0000 7f0f35d9b640 1 --2- 172.21.15.73:0/265913404 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0f240039e0 0x7f0f24003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:29.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.472+0000 7f0f35d9b640 1 --2- 172.21.15.73:0/265913404 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f24004340 0x7f0f24004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:29.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.472+0000 7f0f35d9b640 1 --2- 172.21.15.73:0/265913404 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f24005020 0x7f0f2400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:29.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.472+0000 7f0f35d9b640 1 -- 172.21.15.73:0/265913404 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0f380cbf20 con 0x7f0f24004340 2024-04-26T20:50:29.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.472+0000 7f0f35d9b640 1 -- 172.21.15.73:0/265913404 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2400bf30 con 0x7f0f24005020 2024-04-26T20:50:29.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.472+0000 7f0f35d9b640 1 -- 172.21.15.73:0/265913404 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f0f2400c130 con 0x7f0f240039e0 2024-04-26T20:50:29.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.472+0000 7f0f36d9d640 1 --2- 172.21.15.73:0/265913404 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0f3806fc40 0x7f0f38073360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:50:29.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.472+0000 7f0f2ffff640 1 --2- 172.21.15.73:0/265913404 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f24004340 0x7f0f24004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:29.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.473+0000 7f0f3659c640 1 --2- 172.21.15.73:0/265913404 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0f240039e0 0x7f0f24003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:29.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.473+0000 7f0f36d9d640 1 --2- 172.21.15.73:0/265913404 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f24005020 0x7f0f2400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:29.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.473+0000 7f0f2ffff640 1 -- 172.21.15.73:0/265913404 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0f240039e0 msgr2=0x7f0f24003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:29.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.473+0000 7f0f2ffff640 1 --2- 172.21.15.73:0/265913404 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0f240039e0 0x7f0f24003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:29.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.473+0000 7f0f2ffff640 1 -- 172.21.15.73:0/265913404 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f24005020 msgr2=0x7f0f2400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:29.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.473+0000 7f0f2ffff640 1 --2- 172.21.15.73:0/265913404 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f24005020 0x7f0f2400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:29.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.473+0000 7f0f2ffff640 1 -- 172.21.15.73:0/265913404 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f24003520 con 0x7f0f24004340 2024-04-26T20:50:29.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.473+0000 7f0f2ffff640 1 --2- 172.21.15.73:0/265913404 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f24004340 0x7f0f24004820 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0f20001860 tx=0x7f0f24005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84211b432288427 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:29.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.473+0000 7f0f35d9b640 1 -- 172.21.15.73:0/265913404 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f20037070 con 0x7f0f24004340 2024-04-26T20:50:29.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.474+0000 7f0f35d9b640 1 -- 172.21.15.73:0/265913404 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f2002abf0 con 0x7f0f24004340 2024-04-26T20:50:29.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.474+0000 7f0f35d9b640 1 -- 172.21.15.73:0/265913404 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f20001430 con 0x7f0f24004340 2024-04-26T20:50:29.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.474+0000 7f0f3cc52640 1 -- 172.21.15.73:0/265913404 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f24004340 msgr2=0x7f0f24004820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:29.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.474+0000 7f0f3cc52640 1 --2- 172.21.15.73:0/265913404 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f24004340 0x7f0f24004820 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0f20001860 tx=0x7f0f24005830 comp rx=0 tx=0).stop 2024-04-26T20:50:29.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.474+0000 7f0f36d9d640 1 -- 172.21.15.73:0/265913404 reap_dead start 2024-04-26T20:50:29.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.474+0000 7f0f3cc52640 1 -- 172.21.15.73:0/265913404 shutdown_connections 2024-04-26T20:50:29.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.474+0000 7f0f3cc52640 1 -- 172.21.15.73:0/265913404 >> 172.21.15.73:0/265913404 conn(0x7f0f380d0be0 msgr2=0x7f0f380d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:29.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.474+0000 7f0f3cc52640 1 -- 172.21.15.73:0/265913404 shutdown_connections 2024-04-26T20:50:29.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.474+0000 7f0f3cc52640 1 -- 172.21.15.73:0/265913404 wait complete. 2024-04-26T20:50:29.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.475+0000 7f0f3cc52640 1 Processor -- start 2024-04-26T20:50:29.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.475+0000 7f0f3cc52640 1 -- start start 2024-04-26T20:50:29.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.476+0000 7f0f3cc52640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0f24004340 0x7f0f381f28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:29.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.476+0000 7f0f3cc52640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f38153460 0x7f0f381efd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:29.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.476+0000 7f0f3cc52640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f380b0e70 0x7f0f380b12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:29.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.476+0000 7f0f3cc52640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0f38072700 con 0x7f0f38153460 2024-04-26T20:50:29.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.476+0000 7f0f3cc52640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0f380cc7c0 con 0x7f0f380b0e70 2024-04-26T20:50:29.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.476+0000 7f0f3cc52640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f0f380cbbe0 con 0x7f0f24004340 2024-04-26T20:50:29.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.476+0000 7f0f2ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f38153460 0x7f0f381efd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:29.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.476+0000 7f0f2ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f38153460 0x7f0f381efd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34684/0 (socket says 172.21.15.73:34684) 2024-04-26T20:50:29.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.476+0000 7f0f2ffff640 1 -- 172.21.15.73:0/1814711585 learned_addr learned my addr 172.21.15.73:0/1814711585 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:29.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.476+0000 7f0f3659c640 1 --2- 172.21.15.73:0/1814711585 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0f24004340 0x7f0f381f28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:29.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.476+0000 7f0f36d9d640 1 --2- 172.21.15.73:0/1814711585 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f380b0e70 0x7f0f380b12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:29.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.476+0000 7f0f2ffff640 1 -- 172.21.15.73:0/1814711585 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0f24004340 msgr2=0x7f0f381f28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:29.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.476+0000 7f0f2ffff640 1 --2- 172.21.15.73:0/1814711585 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0f24004340 0x7f0f381f28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:29.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.476+0000 7f0f2ffff640 1 -- 172.21.15.73:0/1814711585 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f380b0e70 msgr2=0x7f0f380b12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:29.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.476+0000 7f0f2ffff640 1 --2- 172.21.15.73:0/1814711585 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f380b0e70 0x7f0f380b12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:29.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.476+0000 7f0f2ffff640 1 -- 172.21.15.73:0/1814711585 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f380c5c80 con 0x7f0f38153460 2024-04-26T20:50:29.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.477+0000 7f0f2ffff640 1 --2- 172.21.15.73:0/1814711585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f38153460 0x7f0f381efd00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0f20001980 tx=0x7f0f20001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:29.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.477+0000 7f0f2effd640 1 -- 172.21.15.73:0/1814711585 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f2003f020 con 0x7f0f38153460 2024-04-26T20:50:29.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.477+0000 7f0f2effd640 1 -- 172.21.15.73:0/1814711585 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f20005270 con 0x7f0f38153460 2024-04-26T20:50:29.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.477+0000 7f0f3cc52640 1 -- 172.21.15.73:0/1814711585 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f381538e0 con 0x7f0f38153460 2024-04-26T20:50:29.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.477+0000 7f0f2effd640 1 -- 172.21.15.73:0/1814711585 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f20037040 con 0x7f0f38153460 2024-04-26T20:50:29.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.477+0000 7f0f3cc52640 1 -- 172.21.15.73:0/1814711585 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f381f3250 con 0x7f0f38153460 2024-04-26T20:50:29.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.478+0000 7f0f3cc52640 1 -- 172.21.15.73:0/1814711585 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f380694f0 con 0x7f0f38153460 2024-04-26T20:50:29.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.478+0000 7f0f2effd640 1 -- 172.21.15.73:0/1814711585 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0f20042020 con 0x7f0f38153460 2024-04-26T20:50:29.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.479+0000 7f0f2effd640 1 --2- 172.21.15.73:0/1814711585 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f0f1806c8f0 0x7f0f1806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:29.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.479+0000 7f0f2effd640 1 -- 172.21.15.73:0/1814711585 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f0f200ab570 con 0x7f0f38153460 2024-04-26T20:50:29.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.479+0000 7f0f3659c640 1 --2- 172.21.15.73:0/1814711585 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f0f1806c8f0 0x7f0f1806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:29.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.479+0000 7f0f3659c640 1 --2- 172.21.15.73:0/1814711585 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f0f1806c8f0 0x7f0f1806edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0f28007fd0 tx=0x7f0f28009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:29.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.483+0000 7f0f2effd640 1 -- 172.21.15.73:0/1814711585 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0f20049030 con 0x7f0f38153460 2024-04-26T20:50:29.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.661+0000 7f0f3cc52640 1 -- 172.21.15.73:0/1814711585 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f0f3806de80 con 0x7f0f38153460 2024-04-26T20:50:29.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.661+0000 7f0f2effd640 1 -- 172.21.15.73:0/1814711585 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7f0f20079cf0 con 0x7f0f38153460 2024-04-26T20:50:29.662 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:50:29.662 INFO:teuthology.orchestra.run.smithi073.stdout:[ 2024-04-26T20:50:29.662 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-26T20:50:29.663 INFO:teuthology.orchestra.run.smithi073.stdout: "poolnum": 1, 2024-04-26T20:50:29.663 INFO:teuthology.orchestra.run.smithi073.stdout: "poolname": ".mgr" 2024-04-26T20:50:29.663 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-26T20:50:29.663 INFO:teuthology.orchestra.run.smithi073.stdout:] 2024-04-26T20:50:29.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.663+0000 7f0f3cc52640 1 -- 172.21.15.73:0/1814711585 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f0f1806c8f0 msgr2=0x7f0f1806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:29.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.663+0000 7f0f3cc52640 1 --2- 172.21.15.73:0/1814711585 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f0f1806c8f0 0x7f0f1806edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0f28007fd0 tx=0x7f0f28009040 comp rx=0 tx=0).stop 2024-04-26T20:50:29.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.663+0000 7f0f3cc52640 1 -- 172.21.15.73:0/1814711585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f38153460 msgr2=0x7f0f381efd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:29.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.663+0000 7f0f3cc52640 1 --2- 172.21.15.73:0/1814711585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f38153460 0x7f0f381efd00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0f20001980 tx=0x7f0f20001230 comp rx=0 tx=0).stop 2024-04-26T20:50:29.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.663+0000 7f0f3cc52640 1 -- 172.21.15.73:0/1814711585 shutdown_connections 2024-04-26T20:50:29.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.663+0000 7f0f3cc52640 1 --2- 172.21.15.73:0/1814711585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f38153460 0x7f0f381efd00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:29.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.663+0000 7f0f3cc52640 1 --2- 172.21.15.73:0/1814711585 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f0f1806c8f0 0x7f0f1806edb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:29.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.663+0000 7f0f3cc52640 1 --2- 172.21.15.73:0/1814711585 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f380b0e70 0x7f0f380b12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:29.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.663+0000 7f0f3cc52640 1 --2- 172.21.15.73:0/1814711585 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0f24004340 0x7f0f381f28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:29.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.663+0000 7f0f3cc52640 1 -- 172.21.15.73:0/1814711585 >> 172.21.15.73:0/1814711585 conn(0x7f0f380d0be0 msgr2=0x7f0f380d0050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:29.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.663+0000 7f0f3cc52640 1 -- 172.21.15.73:0/1814711585 shutdown_connections 2024-04-26T20:50:29.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.663+0000 7f0f3cc52640 1 -- 172.21.15.73:0/1814711585 wait complete. 2024-04-26T20:50:29.674 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T20:50:29.674 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T20:50:29.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.775+0000 7faf7c7c6640 1 Processor -- start 2024-04-26T20:50:29.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.775+0000 7faf7c7c6640 1 -- start start 2024-04-26T20:50:29.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.776+0000 7faf7c7c6640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7faf741629a0 0x7faf740b4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:29.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.776+0000 7faf7c7c6640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf74162d80 con 0x7faf740b5fd0 2024-04-26T20:50:29.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.776+0000 7faf7c7c6640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf740d39d0 con 0x7faf740d1950 2024-04-26T20:50:29.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.776+0000 7faf7c7c6640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7faf740d5f30 con 0x7faf741629a0 2024-04-26T20:50:29.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.776+0000 7faf7a562640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7faf741629a0 0x7faf740b4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:29.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.776+0000 7faf79d61640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7faf740b5fd0 0x7faf740d5820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51978/0 (socket says 172.21.15.73:51978) 2024-04-26T20:50:29.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.776+0000 7faf79d61640 1 -- 172.21.15.73:0/2644230807 learned_addr learned my addr 172.21.15.73:0/2644230807 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:50:29.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.776+0000 7faf7a562640 1 -- 172.21.15.73:0/2644230807 >> v1:172.21.15.120:6789/0 conn(0x7faf740d1950 legacy=0x7faf740d02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:29.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.776+0000 7faf7a562640 1 -- 172.21.15.73:0/2644230807 >> v1:172.21.15.73:6789/0 conn(0x7faf740b5fd0 legacy=0x7faf740d5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:29.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.776+0000 7faf7a562640 1 -- 172.21.15.73:0/2644230807 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf740d3cb0 con 0x7faf741629a0 2024-04-26T20:50:29.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.776+0000 7faf79560640 1 -- 172.21.15.73:0/2644230807 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2078439620 0 0) 0x7faf74162d80 con 0x7faf740b5fd0 2024-04-26T20:50:29.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.777+0000 7faf7a562640 1 --2- 172.21.15.73:0/2644230807 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf741629a0 0x7faf740b4890 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7faf700097a0 tx=0x7faf7002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6c6493b4ccd665f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:29.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.777+0000 7faf79560640 1 -- 172.21.15.73:0/2644230807 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf70037070 con 0x7faf741629a0 2024-04-26T20:50:29.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.777+0000 7faf79560640 1 -- 172.21.15.73:0/2644230807 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf70032440 con 0x7faf741629a0 2024-04-26T20:50:29.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.777+0000 7faf79560640 1 -- 172.21.15.73:0/2644230807 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf70032810 con 0x7faf741629a0 2024-04-26T20:50:29.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.777+0000 7faf7c7c6640 1 -- 172.21.15.73:0/2644230807 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf741629a0 msgr2=0x7faf740b4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:29.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.777+0000 7faf7c7c6640 1 --2- 172.21.15.73:0/2644230807 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf741629a0 0x7faf740b4890 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7faf700097a0 tx=0x7faf7002a1e0 comp rx=0 tx=0).stop 2024-04-26T20:50:29.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.777+0000 7faf7c7c6640 1 -- 172.21.15.73:0/2644230807 shutdown_connections 2024-04-26T20:50:29.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.777+0000 7faf7c7c6640 1 --2- 172.21.15.73:0/2644230807 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf741629a0 0x7faf740b4890 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:29.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.777+0000 7faf7c7c6640 1 -- 172.21.15.73:0/2644230807 >> 172.21.15.73:0/2644230807 conn(0x7faf7409e3e0 msgr2=0x7faf740d5110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:29.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.777+0000 7faf7c7c6640 1 -- 172.21.15.73:0/2644230807 shutdown_connections 2024-04-26T20:50:29.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.777+0000 7faf7c7c6640 1 -- 172.21.15.73:0/2644230807 wait complete. 2024-04-26T20:50:29.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.779+0000 7faf7c7c6640 1 Processor -- start 2024-04-26T20:50:29.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.779+0000 7faf7c7c6640 1 -- start start 2024-04-26T20:50:29.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.779+0000 7faf7c7c6640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7faf740b5fd0 0x7faf740b7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:29.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.779+0000 7faf7c7c6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf740d1950 0x7faf74161310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:29.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.779+0000 7faf7c7c6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faf741629a0 0x7faf740b9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:29.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.779+0000 7faf7c7c6640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7faf740d3f90 con 0x7faf740d1950 2024-04-26T20:50:29.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.779+0000 7faf7c7c6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7faf740d36a0 con 0x7faf741629a0 2024-04-26T20:50:29.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.779+0000 7faf7c7c6640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7faf740d1de0 con 0x7faf740b5fd0 2024-04-26T20:50:29.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.779+0000 7faf79d61640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf740d1950 0x7faf74161310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:29.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.779+0000 7faf79d61640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf740d1950 0x7faf74161310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34714/0 (socket says 172.21.15.73:34714) 2024-04-26T20:50:29.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.779+0000 7faf79d61640 1 -- 172.21.15.73:0/2942175183 learned_addr learned my addr 172.21.15.73:0/2942175183 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:29.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.780+0000 7faf79d61640 1 -- 172.21.15.73:0/2942175183 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7faf740b5fd0 msgr2=0x7faf740b7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:29.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.780+0000 7faf79d61640 1 --2- 172.21.15.73:0/2942175183 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7faf740b5fd0 0x7faf740b7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:29.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.780+0000 7faf79d61640 1 -- 172.21.15.73:0/2942175183 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faf741629a0 msgr2=0x7faf740b9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:29.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.780+0000 7faf7ad63640 1 --2- 172.21.15.73:0/2942175183 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faf741629a0 0x7faf740b9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:29.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.780+0000 7faf79d61640 1 --2- 172.21.15.73:0/2942175183 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faf741629a0 0x7faf740b9980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:29.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.780+0000 7faf79d61640 1 -- 172.21.15.73:0/2942175183 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf740ca490 con 0x7faf740d1950 2024-04-26T20:50:29.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.780+0000 7faf79d61640 1 --2- 172.21.15.73:0/2942175183 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf740d1950 0x7faf74161310 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7faf6400c4d0 tx=0x7faf6400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:29.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.780+0000 7faf6affd640 1 -- 172.21.15.73:0/2942175183 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf64018070 con 0x7faf740d1950 2024-04-26T20:50:29.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.780+0000 7faf6affd640 1 -- 172.21.15.73:0/2942175183 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf64007500 con 0x7faf740d1950 2024-04-26T20:50:29.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.780+0000 7faf7c7c6640 1 -- 172.21.15.73:0/2942175183 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf740ca770 con 0x7faf740d1950 2024-04-26T20:50:29.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.780+0000 7faf6affd640 1 -- 172.21.15.73:0/2942175183 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf64004a20 con 0x7faf740d1950 2024-04-26T20:50:29.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.780+0000 7faf7c7c6640 1 -- 172.21.15.73:0/2942175183 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf740cad30 con 0x7faf740d1950 2024-04-26T20:50:29.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.781+0000 7faf7c7c6640 1 -- 172.21.15.73:0/2942175183 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf740caf30 con 0x7faf740d1950 2024-04-26T20:50:29.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.781+0000 7faf6affd640 1 -- 172.21.15.73:0/2942175183 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faf6401b070 con 0x7faf740d1950 2024-04-26T20:50:29.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.782+0000 7faf6affd640 1 --2- 172.21.15.73:0/2942175183 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7faf5806c8f0 0x7faf5806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:29.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.782+0000 7faf7a562640 1 --2- 172.21.15.73:0/2942175183 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7faf5806c8f0 0x7faf5806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:29.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.782+0000 7faf6affd640 1 -- 172.21.15.73:0/2942175183 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7faf6408ab50 con 0x7faf740d1950 2024-04-26T20:50:29.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.782+0000 7faf7a562640 1 --2- 172.21.15.73:0/2942175183 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7faf5806c8f0 0x7faf5806edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7faf70034000 tx=0x7faf7002c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:29.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.786+0000 7faf6affd640 1 -- 172.21.15.73:0/2942175183 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faf740caf30 con 0x7faf740d1950 2024-04-26T20:50:29.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:29.960+0000 7faf7c7c6640 1 -- 172.21.15.73:0/2942175183 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7faf740d26c0 con 0x7faf740d1950 2024-04-26T20:50:30.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.608+0000 7faf6affd640 1 -- 172.21.15.73:0/2942175183 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7faf640592d0 con 0x7faf740d1950 2024-04-26T20:50:30.609 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_metadata' created 2024-04-26T20:50:30.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.610+0000 7faf7c7c6640 1 -- 172.21.15.73:0/2942175183 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7faf5806c8f0 msgr2=0x7faf5806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:30.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.610+0000 7faf7c7c6640 1 --2- 172.21.15.73:0/2942175183 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7faf5806c8f0 0x7faf5806edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7faf70034000 tx=0x7faf7002c000 comp rx=0 tx=0).stop 2024-04-26T20:50:30.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.610+0000 7faf7c7c6640 1 -- 172.21.15.73:0/2942175183 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf740d1950 msgr2=0x7faf74161310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:30.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.611+0000 7faf7c7c6640 1 --2- 172.21.15.73:0/2942175183 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf740d1950 0x7faf74161310 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7faf6400c4d0 tx=0x7faf6400c990 comp rx=0 tx=0).stop 2024-04-26T20:50:30.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.611+0000 7faf7c7c6640 1 -- 172.21.15.73:0/2942175183 shutdown_connections 2024-04-26T20:50:30.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.611+0000 7faf7c7c6640 1 --2- 172.21.15.73:0/2942175183 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf740d1950 0x7faf74161310 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:30.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.611+0000 7faf7c7c6640 1 --2- 172.21.15.73:0/2942175183 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7faf5806c8f0 0x7faf5806edb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:30.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.611+0000 7faf7c7c6640 1 --2- 172.21.15.73:0/2942175183 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faf741629a0 0x7faf740b9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:30.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.611+0000 7faf7c7c6640 1 --2- 172.21.15.73:0/2942175183 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7faf740b5fd0 0x7faf740b7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:30.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.611+0000 7faf7c7c6640 1 -- 172.21.15.73:0/2942175183 >> 172.21.15.73:0/2942175183 conn(0x7faf7409e3e0 msgr2=0x7faf7406cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:30.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.611+0000 7faf7c7c6640 1 -- 172.21.15.73:0/2942175183 shutdown_connections 2024-04-26T20:50:30.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.611+0000 7faf7c7c6640 1 -- 172.21.15.73:0/2942175183 wait complete. 2024-04-26T20:50:30.626 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-26T20:50:30.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.730+0000 7fa1fe1e9640 1 Processor -- start 2024-04-26T20:50:30.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.731+0000 7fa1fe1e9640 1 -- start start 2024-04-26T20:50:30.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.731+0000 7fa1fe1e9640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa1f80b0b70 0x7fa1f80b0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:30.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.731+0000 7fa1fe1e9640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fa1f80b4d40 0x7fa1f80b6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:30.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.731+0000 7fa1fe1e9640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1f80b1490 con 0x7fa1f80b6c50 2024-04-26T20:50:30.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.731+0000 7fa1fe1e9640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7fa1f8073bc0 con 0x7fa1f80b0b70 2024-04-26T20:50:30.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.731+0000 7fa1fe1e9640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fa1f8072fb0 con 0x7fa1f80b4d40 2024-04-26T20:50:30.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.731+0000 7fa1f77fe640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa1f80b0b70 0x7fa1f80b0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:30.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.731+0000 7fa1f77fe640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa1f80b0b70 0x7fa1f80b0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34730/0 (socket says 172.21.15.73:34730) 2024-04-26T20:50:30.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.731+0000 7fa1f77fe640 1 -- 172.21.15.73:0/3327508373 learned_addr learned my addr 172.21.15.73:0/3327508373 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:30.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.732+0000 7fa1f77fe640 1 -- 172.21.15.73:0/3327508373 >> v2:172.21.15.120:3300/0 conn(0x7fa1f80b4d40 msgr2=0x7fa1f80b6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:30.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.732+0000 7fa1f77fe640 1 --2- 172.21.15.73:0/3327508373 >> v2:172.21.15.120:3300/0 conn(0x7fa1f80b4d40 0x7fa1f80b6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:30.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.732+0000 7fa1f77fe640 1 -- 172.21.15.73:0/3327508373 >> v1:172.21.15.120:6789/0 conn(0x7fa1f80b6c50 legacy=0x7fa1f815f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:30.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.732+0000 7fa1f77fe640 1 -- 172.21.15.73:0/3327508373 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f8073290 con 0x7fa1f80b0b70 2024-04-26T20:50:30.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.732+0000 7fa1f77fe640 1 --2- 172.21.15.73:0/3327508373 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1f80b0b70 0x7fa1f80b0f50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8074840 tx=0x7fa1e802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1f5ec40a8fdff77 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:30.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.732+0000 7fa1f67fc640 1 -- 172.21.15.73:0/3327508373 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8037070 con 0x7fa1f80b0b70 2024-04-26T20:50:30.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.733+0000 7fa1f67fc640 1 -- 172.21.15.73:0/3327508373 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1e8032440 con 0x7fa1f80b0b70 2024-04-26T20:50:30.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.733+0000 7fa1f67fc640 1 -- 172.21.15.73:0/3327508373 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8032810 con 0x7fa1f80b0b70 2024-04-26T20:50:30.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.733+0000 7fa1fe1e9640 1 -- 172.21.15.73:0/3327508373 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1f80b0b70 msgr2=0x7fa1f80b0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:30.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.733+0000 7fa1fe1e9640 1 --2- 172.21.15.73:0/3327508373 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1f80b0b70 0x7fa1f80b0f50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8074840 tx=0x7fa1e802a230 comp rx=0 tx=0).stop 2024-04-26T20:50:30.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.733+0000 7fa1fe1e9640 1 -- 172.21.15.73:0/3327508373 shutdown_connections 2024-04-26T20:50:30.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.733+0000 7fa1fe1e9640 1 --2- 172.21.15.73:0/3327508373 >> v2:172.21.15.120:3300/0 conn(0x7fa1f80b4d40 0x7fa1f80b6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:30.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.733+0000 7fa1fe1e9640 1 --2- 172.21.15.73:0/3327508373 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1f80b0b70 0x7fa1f80b0f50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:30.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.733+0000 7fa1fe1e9640 1 -- 172.21.15.73:0/3327508373 >> 172.21.15.73:0/3327508373 conn(0x7fa1f80b39b0 msgr2=0x7fa1f80b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:30.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.733+0000 7fa1fe1e9640 1 -- 172.21.15.73:0/3327508373 shutdown_connections 2024-04-26T20:50:30.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.733+0000 7fa1fe1e9640 1 -- 172.21.15.73:0/3327508373 wait complete. 2024-04-26T20:50:30.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.734+0000 7fa1fe1e9640 1 Processor -- start 2024-04-26T20:50:30.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.734+0000 7fa1fe1e9640 1 -- start start 2024-04-26T20:50:30.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.735+0000 7fa1fe1e9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1f80b0b70 0x7fa1f81f4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:30.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.735+0000 7fa1fe1e9640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa1f80b4d40 0x7fa1f81555d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:30.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.735+0000 7fa1fe1e9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1f80b6c50 0x7fa1f8149760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:30.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.735+0000 7fa1fe1e9640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f80afda0 con 0x7fa1f80b0b70 2024-04-26T20:50:30.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.735+0000 7fa1fe1e9640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f8073850 con 0x7fa1f80b6c50 2024-04-26T20:50:30.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.735+0000 7fa1fe1e9640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fa1f80aff20 con 0x7fa1f80b4d40 2024-04-26T20:50:30.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.735+0000 7fa1f77fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1f80b0b70 0x7fa1f81f4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:30.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.735+0000 7fa1f77fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1f80b0b70 0x7fa1f81f4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34736/0 (socket says 172.21.15.73:34736) 2024-04-26T20:50:30.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.735+0000 7fa1f77fe640 1 -- 172.21.15.73:0/568899592 learned_addr learned my addr 172.21.15.73:0/568899592 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:30.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.735+0000 7fa1f6ffd640 1 --2- 172.21.15.73:0/568899592 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa1f80b4d40 0x7fa1f81555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:30.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.735+0000 7fa1f7fff640 1 --2- 172.21.15.73:0/568899592 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1f80b6c50 0x7fa1f8149760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:30.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.735+0000 7fa1f77fe640 1 -- 172.21.15.73:0/568899592 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa1f80b4d40 msgr2=0x7fa1f81555d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:30.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.735+0000 7fa1f77fe640 1 --2- 172.21.15.73:0/568899592 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa1f80b4d40 0x7fa1f81555d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:30.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.735+0000 7fa1f77fe640 1 -- 172.21.15.73:0/568899592 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1f80b6c50 msgr2=0x7fa1f8149760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:30.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.735+0000 7fa1f77fe640 1 --2- 172.21.15.73:0/568899592 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1f80b6c50 0x7fa1f8149760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:30.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.735+0000 7fa1f77fe640 1 -- 172.21.15.73:0/568899592 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f81f5350 con 0x7fa1f80b0b70 2024-04-26T20:50:30.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.736+0000 7fa1f77fe640 1 --2- 172.21.15.73:0/568899592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1f80b0b70 0x7fa1f81f4c90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa1e802a270 tx=0x7fa1e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:30.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.736+0000 7fa1dffff640 1 -- 172.21.15.73:0/568899592 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8041070 con 0x7fa1f80b0b70 2024-04-26T20:50:30.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.736+0000 7fa1dffff640 1 -- 172.21.15.73:0/568899592 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1e803c870 con 0x7fa1f80b0b70 2024-04-26T20:50:30.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.736+0000 7fa1fe1e9640 1 -- 172.21.15.73:0/568899592 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1f81f5630 con 0x7fa1f80b0b70 2024-04-26T20:50:30.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.736+0000 7fa1dffff640 1 -- 172.21.15.73:0/568899592 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8037040 con 0x7fa1f80b0b70 2024-04-26T20:50:30.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.736+0000 7fa1fe1e9640 1 -- 172.21.15.73:0/568899592 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1f8149ca0 con 0x7fa1f80b0b70 2024-04-26T20:50:30.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.737+0000 7fa1fe1e9640 1 -- 172.21.15.73:0/568899592 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1f80694f0 con 0x7fa1f80b0b70 2024-04-26T20:50:30.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.737+0000 7fa1dffff640 1 -- 172.21.15.73:0/568899592 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa1e8044070 con 0x7fa1f80b0b70 2024-04-26T20:50:30.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.738+0000 7fa1dffff640 1 --2- 172.21.15.73:0/568899592 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa1e406c8f0 0x7fa1e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:30.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.738+0000 7fa1f6ffd640 1 --2- 172.21.15.73:0/568899592 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa1e406c8f0 0x7fa1e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:30.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.738+0000 7fa1dffff640 1 -- 172.21.15.73:0/568899592 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7fa1e80ac4b0 con 0x7fa1f80b0b70 2024-04-26T20:50:30.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.738+0000 7fa1f6ffd640 1 --2- 172.21.15.73:0/568899592 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa1e406c8f0 0x7fa1e406edb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8158cf0 tx=0x7fa1ec008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:30.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.742+0000 7fa1dffff640 1 -- 172.21.15.73:0/568899592 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa1e807ac70 con 0x7fa1f80b0b70 2024-04-26T20:50:30.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:30.916+0000 7fa1fe1e9640 1 -- 172.21.15.73:0/568899592 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7fa1f806cf10 con 0x7fa1f80b0b70 2024-04-26T20:50:31.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.610+0000 7fa1dffff640 1 -- 172.21.15.73:0/568899592 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v18) ==== 171+0+0 (secure 0 0 0) 0x7fa1e807a570 con 0x7fa1f80b0b70 2024-04-26T20:50:31.611 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data' created 2024-04-26T20:50:31.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.612+0000 7fa1fe1e9640 1 -- 172.21.15.73:0/568899592 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa1e406c8f0 msgr2=0x7fa1e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:31.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.612+0000 7fa1fe1e9640 1 --2- 172.21.15.73:0/568899592 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa1e406c8f0 0x7fa1e406edb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8158cf0 tx=0x7fa1ec008040 comp rx=0 tx=0).stop 2024-04-26T20:50:31.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.612+0000 7fa1fe1e9640 1 -- 172.21.15.73:0/568899592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1f80b0b70 msgr2=0x7fa1f81f4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:31.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.612+0000 7fa1fe1e9640 1 --2- 172.21.15.73:0/568899592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1f80b0b70 0x7fa1f81f4c90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa1e802a270 tx=0x7fa1e80046c0 comp rx=0 tx=0).stop 2024-04-26T20:50:31.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.612+0000 7fa1fe1e9640 1 -- 172.21.15.73:0/568899592 shutdown_connections 2024-04-26T20:50:31.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.612+0000 7fa1fe1e9640 1 --2- 172.21.15.73:0/568899592 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa1e406c8f0 0x7fa1e406edb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:31.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.612+0000 7fa1fe1e9640 1 --2- 172.21.15.73:0/568899592 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa1f80b6c50 0x7fa1f8149760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:31.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.612+0000 7fa1fe1e9640 1 --2- 172.21.15.73:0/568899592 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa1f80b4d40 0x7fa1f81555d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:31.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.612+0000 7fa1fe1e9640 1 --2- 172.21.15.73:0/568899592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1f80b0b70 0x7fa1f81f4c90 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:31.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.612+0000 7fa1fe1e9640 1 -- 172.21.15.73:0/568899592 >> 172.21.15.73:0/568899592 conn(0x7fa1f80b39b0 msgr2=0x7fa1f806c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:31.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.613+0000 7fa1fe1e9640 1 -- 172.21.15.73:0/568899592 shutdown_connections 2024-04-26T20:50:31.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.613+0000 7fa1fe1e9640 1 -- 172.21.15.73:0/568899592 wait complete. 2024-04-26T20:50:31.638 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T20:50:31.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.739+0000 7faed3378640 1 Processor -- start 2024-04-26T20:50:31.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.740+0000 7faed3378640 1 -- start start 2024-04-26T20:50:31.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.740+0000 7faed3378640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7faecc072fb0 0x7faecc0b56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:31.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.740+0000 7faed3378640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7faecc0b06f0 0x7faecc06fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:31.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.740+0000 7faed3378640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faecc0b5e30 con 0x7faecc073390 2024-04-26T20:50:31.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.740+0000 7faed3378640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7faecc0b0b80 con 0x7faecc0b06f0 2024-04-26T20:50:31.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.740+0000 7faed3378640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7faecc0b5c00 con 0x7faecc072fb0 2024-04-26T20:50:31.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.741+0000 7faed1915640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7faecc0b06f0 0x7faecc06fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:31.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.741+0000 7faed1915640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7faecc0b06f0 0x7faecc06fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34738/0 (socket says 172.21.15.73:34738) 2024-04-26T20:50:31.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.741+0000 7faed1915640 1 -- 172.21.15.73:0/3376365170 learned_addr learned my addr 172.21.15.73:0/3376365170 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:31.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.741+0000 7faed1915640 1 -- 172.21.15.73:0/3376365170 >> v2:172.21.15.120:3300/0 conn(0x7faecc072fb0 msgr2=0x7faecc0b56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:31.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.741+0000 7faed1915640 1 --2- 172.21.15.73:0/3376365170 >> v2:172.21.15.120:3300/0 conn(0x7faecc072fb0 0x7faecc0b56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:31.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.741+0000 7faed1915640 1 -- 172.21.15.73:0/3376365170 >> v1:172.21.15.120:6789/0 conn(0x7faecc073390 legacy=0x7faecc06f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:31.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.741+0000 7faed1915640 1 -- 172.21.15.73:0/3376365170 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faecc0b6110 con 0x7faecc0b06f0 2024-04-26T20:50:31.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.741+0000 7faed1915640 1 --2- 172.21.15.73:0/3376365170 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faecc0b06f0 0x7faecc06fe50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7faecc050730 tx=0x7faec402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ce9a5536b24ea8a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:31.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.742+0000 7faec3fff640 1 -- 172.21.15.73:0/3376365170 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec4037070 con 0x7faecc0b06f0 2024-04-26T20:50:31.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.742+0000 7faec3fff640 1 -- 172.21.15.73:0/3376365170 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faec4032440 con 0x7faecc0b06f0 2024-04-26T20:50:31.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.742+0000 7faec3fff640 1 -- 172.21.15.73:0/3376365170 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec4032810 con 0x7faecc0b06f0 2024-04-26T20:50:31.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.742+0000 7faed3378640 1 -- 172.21.15.73:0/3376365170 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faecc0b06f0 msgr2=0x7faecc06fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:31.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.742+0000 7faed3378640 1 --2- 172.21.15.73:0/3376365170 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faecc0b06f0 0x7faecc06fe50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7faecc050730 tx=0x7faec402a230 comp rx=0 tx=0).stop 2024-04-26T20:50:31.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.742+0000 7faed3378640 1 -- 172.21.15.73:0/3376365170 shutdown_connections 2024-04-26T20:50:31.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.742+0000 7faed3378640 1 --2- 172.21.15.73:0/3376365170 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faecc0b06f0 0x7faecc06fe50 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:31.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.742+0000 7faed3378640 1 --2- 172.21.15.73:0/3376365170 >> v2:172.21.15.120:3300/0 conn(0x7faecc072fb0 0x7faecc0b56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:31.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.742+0000 7faed3378640 1 -- 172.21.15.73:0/3376365170 >> 172.21.15.73:0/3376365170 conn(0x7faecc0b39b0 msgr2=0x7faecc0b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:31.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.742+0000 7faed3378640 1 -- 172.21.15.73:0/3376365170 shutdown_connections 2024-04-26T20:50:31.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.742+0000 7faed3378640 1 -- 172.21.15.73:0/3376365170 wait complete. 2024-04-26T20:50:31.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.743+0000 7faed3378640 1 Processor -- start 2024-04-26T20:50:31.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.743+0000 7faed3378640 1 -- start start 2024-04-26T20:50:31.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.744+0000 7faed3378640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7faecc072fb0 0x7faecc147720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:31.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.744+0000 7faed3378640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faecc073390 0x7faecc15e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:31.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.744+0000 7faed3378640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faecc0b06f0 0x7faecc15ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:31.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.744+0000 7faed3378640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7faecc0737f0 con 0x7faecc073390 2024-04-26T20:50:31.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.744+0000 7faed3378640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7faecc06f2b0 con 0x7faecc0b06f0 2024-04-26T20:50:31.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.744+0000 7faed3378640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7faecc15c160 con 0x7faecc072fb0 2024-04-26T20:50:31.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.744+0000 7faed0913640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faecc073390 0x7faecc15e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:31.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.744+0000 7faed0913640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faecc073390 0x7faecc15e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34744/0 (socket says 172.21.15.73:34744) 2024-04-26T20:50:31.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.744+0000 7faed0913640 1 -- 172.21.15.73:0/1320117236 learned_addr learned my addr 172.21.15.73:0/1320117236 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:31.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.744+0000 7faed1915640 1 --2- 172.21.15.73:0/1320117236 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faecc0b06f0 0x7faecc15ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:31.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.744+0000 7faed0913640 1 -- 172.21.15.73:0/1320117236 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7faecc072fb0 msgr2=0x7faecc147720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:31.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.744+0000 7faed0913640 1 --2- 172.21.15.73:0/1320117236 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7faecc072fb0 0x7faecc147720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:31.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.744+0000 7faed0913640 1 -- 172.21.15.73:0/1320117236 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faecc0b06f0 msgr2=0x7faecc15ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:31.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.744+0000 7faed0913640 1 --2- 172.21.15.73:0/1320117236 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faecc0b06f0 0x7faecc15ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:31.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.744+0000 7faed0913640 1 -- 172.21.15.73:0/1320117236 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faecc06d020 con 0x7faecc073390 2024-04-26T20:50:31.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.745+0000 7faed0913640 1 --2- 172.21.15.73:0/1320117236 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faecc073390 0x7faecc15e500 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7faebc00c410 tx=0x7faebc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:31.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.745+0000 7faec1ffb640 1 -- 172.21.15.73:0/1320117236 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faebc017070 con 0x7faecc073390 2024-04-26T20:50:31.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.745+0000 7faec1ffb640 1 -- 172.21.15.73:0/1320117236 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faebc0035f0 con 0x7faecc073390 2024-04-26T20:50:31.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.745+0000 7faec1ffb640 1 -- 172.21.15.73:0/1320117236 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faebc003a20 con 0x7faecc073390 2024-04-26T20:50:31.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.745+0000 7faed3378640 1 -- 172.21.15.73:0/1320117236 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faecc1f5260 con 0x7faecc073390 2024-04-26T20:50:31.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.745+0000 7faed3378640 1 -- 172.21.15.73:0/1320117236 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faecc1f5820 con 0x7faecc073390 2024-04-26T20:50:31.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.746+0000 7faed3378640 1 -- 172.21.15.73:0/1320117236 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faecc0694f0 con 0x7faecc073390 2024-04-26T20:50:31.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.746+0000 7faec1ffb640 1 -- 172.21.15.73:0/1320117236 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faebc01a070 con 0x7faecc073390 2024-04-26T20:50:31.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.747+0000 7faec1ffb640 1 --2- 172.21.15.73:0/1320117236 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7faeb006c8f0 0x7faeb006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:31.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.747+0000 7faed1114640 1 --2- 172.21.15.73:0/1320117236 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7faeb006c8f0 0x7faeb006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:31.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.747+0000 7faec1ffb640 1 -- 172.21.15.73:0/1320117236 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7faebc08b000 con 0x7faecc073390 2024-04-26T20:50:31.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.747+0000 7faed1114640 1 --2- 172.21.15.73:0/1320117236 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7faeb006c8f0 0x7faeb006edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7faec800ab90 tx=0x7faec8009250 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:31.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.751+0000 7faec1ffb640 1 -- 172.21.15.73:0/1320117236 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faecc06d020 con 0x7faecc073390 2024-04-26T20:50:31.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:31.963+0000 7faed3378640 1 -- 172.21.15.73:0/1320117236 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7faecc06d250 con 0x7faecc073390 2024-04-26T20:50:32.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.630+0000 7faec1ffb640 1 -- 172.21.15.73:0/1320117236 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7faebc059490 con 0x7faecc073390 2024-04-26T20:50:32.632 INFO:teuthology.orchestra.run.smithi073.stderr:new fs with metadata pool 2 and data pool 3 2024-04-26T20:50:32.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.638+0000 7faed3378640 1 -- 172.21.15.73:0/1320117236 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7faeb006c8f0 msgr2=0x7faeb006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:32.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.638+0000 7faed3378640 1 --2- 172.21.15.73:0/1320117236 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7faeb006c8f0 0x7faeb006edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7faec800ab90 tx=0x7faec8009250 comp rx=0 tx=0).stop 2024-04-26T20:50:32.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.638+0000 7faed3378640 1 -- 172.21.15.73:0/1320117236 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faecc073390 msgr2=0x7faecc15e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:32.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.638+0000 7faed3378640 1 --2- 172.21.15.73:0/1320117236 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faecc073390 0x7faecc15e500 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7faebc00c410 tx=0x7faebc00c8d0 comp rx=0 tx=0).stop 2024-04-26T20:50:32.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.640+0000 7faed3378640 1 -- 172.21.15.73:0/1320117236 shutdown_connections 2024-04-26T20:50:32.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.640+0000 7faed3378640 1 --2- 172.21.15.73:0/1320117236 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faecc073390 0x7faecc15e500 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:32.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.640+0000 7faed3378640 1 --2- 172.21.15.73:0/1320117236 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7faeb006c8f0 0x7faeb006edb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:32.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.640+0000 7faed3378640 1 --2- 172.21.15.73:0/1320117236 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faecc0b06f0 0x7faecc15ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:32.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.640+0000 7faed3378640 1 --2- 172.21.15.73:0/1320117236 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7faecc072fb0 0x7faecc147720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:32.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.640+0000 7faed3378640 1 -- 172.21.15.73:0/1320117236 >> 172.21.15.73:0/1320117236 conn(0x7faecc0b39b0 msgr2=0x7faecc15a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:32.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.645+0000 7faed3378640 1 -- 172.21.15.73:0/1320117236 shutdown_connections 2024-04-26T20:50:32.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.645+0000 7faed3378640 1 -- 172.21.15.73:0/1320117236 wait complete. 2024-04-26T20:50:32.664 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-26T20:50:32.665 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-26T20:50:32.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.767+0000 7f8db2852640 1 Processor -- start 2024-04-26T20:50:32.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.767+0000 7f8db2852640 1 -- start start 2024-04-26T20:50:32.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.767+0000 7f8db2852640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f8dac0b5490 0x7f8dac0cff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:32.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.767+0000 7f8db2852640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8dac070240 con 0x7f8dac0ce1e0 2024-04-26T20:50:32.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.767+0000 7f8db2852640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8dac0d0c10 con 0x7f8dac160a20 2024-04-26T20:50:32.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.767+0000 7f8db2852640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f8dac0d0ef0 con 0x7f8dac0b5490 2024-04-26T20:50:32.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.768+0000 7f8db0def640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f8dac0ce1e0 0x7f8dac0ce650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51990/0 (socket says 172.21.15.73:51990) 2024-04-26T20:50:32.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.768+0000 7f8db0def640 1 -- 172.21.15.73:0/375455898 learned_addr learned my addr 172.21.15.73:0/375455898 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:50:32.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.768+0000 7f8dab7fe640 1 --2- 172.21.15.73:0/375455898 >> v2:172.21.15.120:3300/0 conn(0x7f8dac0b5490 0x7f8dac0cff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:32.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.768+0000 7f8daaffd640 1 -- 172.21.15.73:0/375455898 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2556706768 0 0) 0x7f8dac070240 con 0x7f8dac0ce1e0 2024-04-26T20:50:32.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.768+0000 7f8daaffd640 1 -- 172.21.15.73:0/375455898 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8d94003520 con 0x7f8dac0ce1e0 2024-04-26T20:50:32.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.768+0000 7f8daaffd640 1 -- 172.21.15.73:0/375455898 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3315239292 0 0) 0x7f8d94003520 con 0x7f8dac0ce1e0 2024-04-26T20:50:32.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.768+0000 7f8daaffd640 1 -- 172.21.15.73:0/375455898 >> v2:172.21.15.120:3300/0 conn(0x7f8dac0b5490 msgr2=0x7f8dac0cff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:32.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.768+0000 7f8daaffd640 1 --2- 172.21.15.73:0/375455898 >> v2:172.21.15.120:3300/0 conn(0x7f8dac0b5490 0x7f8dac0cff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:32.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.768+0000 7f8daaffd640 1 -- 172.21.15.73:0/375455898 >> v1:172.21.15.120:6789/0 conn(0x7f8dac160a20 legacy=0x7f8dac0cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:32.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.768+0000 7f8daaffd640 1 -- 172.21.15.73:0/375455898 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dac0d1c30 con 0x7f8dac0ce1e0 2024-04-26T20:50:32.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.768+0000 7f8daaffd640 1 -- 172.21.15.73:0/375455898 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7f8da40019f0 con 0x7f8dac0ce1e0 2024-04-26T20:50:32.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.768+0000 7f8daaffd640 1 -- 172.21.15.73:0/375455898 >> v1:172.21.15.73:6789/0 conn(0x7f8dac0ce1e0 legacy=0x7f8dac0ce650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:32.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.768+0000 7f8daaffd640 1 --2- 172.21.15.73:0/375455898 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8d940039b0 0x7f8d94003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:32.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.768+0000 7f8daaffd640 1 --2- 172.21.15.73:0/375455898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d94004800 0x7f8d94004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:32.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.768+0000 7f8daaffd640 1 --2- 172.21.15.73:0/375455898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d940051c0 0x7f8d9400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:32.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.768+0000 7f8daaffd640 1 -- 172.21.15.73:0/375455898 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8dac0d0ef0 con 0x7f8d940051c0 2024-04-26T20:50:32.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.768+0000 7f8daaffd640 1 -- 172.21.15.73:0/375455898 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9400c160 con 0x7f8d94004800 2024-04-26T20:50:32.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.768+0000 7f8daaffd640 1 -- 172.21.15.73:0/375455898 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f8d9400c360 con 0x7f8d940039b0 2024-04-26T20:50:32.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.768+0000 7f8db0def640 1 --2- 172.21.15.73:0/375455898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d940051c0 0x7f8d9400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:32.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.768+0000 7f8dab7fe640 1 --2- 172.21.15.73:0/375455898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d94004800 0x7f8d94004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:32.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.768+0000 7f8dabfff640 1 --2- 172.21.15.73:0/375455898 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8d940039b0 0x7f8d94003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:32.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.768+0000 7f8db0def640 1 -- 172.21.15.73:0/375455898 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8d940039b0 msgr2=0x7f8d94003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:32.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.768+0000 7f8db0def640 1 --2- 172.21.15.73:0/375455898 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8d940039b0 0x7f8d94003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:32.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.768+0000 7f8db0def640 1 -- 172.21.15.73:0/375455898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d94004800 msgr2=0x7f8d94004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:32.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.768+0000 7f8db0def640 1 --2- 172.21.15.73:0/375455898 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8d94004800 0x7f8d94004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:32.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.768+0000 7f8db0def640 1 -- 172.21.15.73:0/375455898 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d94003520 con 0x7f8d940051c0 2024-04-26T20:50:32.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.769+0000 7f8db0def640 1 -- 172.21.15.73:0/375455898 reap_dead start 2024-04-26T20:50:32.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.769+0000 7f8db0def640 1 --2- 172.21.15.73:0/375455898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d940051c0 0x7f8d9400b9c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8da400a8d0 tx=0x7f8d94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57ff85a431412fea server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:32.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.769+0000 7f8daaffd640 1 -- 172.21.15.73:0/375455898 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da400a4a0 con 0x7f8d940051c0 2024-04-26T20:50:32.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.769+0000 7f8daaffd640 1 -- 172.21.15.73:0/375455898 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8da402f070 con 0x7f8d940051c0 2024-04-26T20:50:32.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.769+0000 7f8daaffd640 1 -- 172.21.15.73:0/375455898 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da402bcb0 con 0x7f8d940051c0 2024-04-26T20:50:32.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.770+0000 7f8db2852640 1 -- 172.21.15.73:0/375455898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d940051c0 msgr2=0x7f8d9400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:32.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.770+0000 7f8db2852640 1 --2- 172.21.15.73:0/375455898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d940051c0 0x7f8d9400b9c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8da400a8d0 tx=0x7f8d94005830 comp rx=0 tx=0).stop 2024-04-26T20:50:32.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.770+0000 7f8db2852640 1 -- 172.21.15.73:0/375455898 shutdown_connections 2024-04-26T20:50:32.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.770+0000 7f8db2852640 1 --2- 172.21.15.73:0/375455898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d940051c0 0x7f8d9400b9c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:32.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.770+0000 7f8db2852640 1 -- 172.21.15.73:0/375455898 >> 172.21.15.73:0/375455898 conn(0x7f8dac0b4970 msgr2=0x7f8dac0b4d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:32.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.770+0000 7f8db2852640 1 -- 172.21.15.73:0/375455898 shutdown_connections 2024-04-26T20:50:32.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.770+0000 7f8db2852640 1 -- 172.21.15.73:0/375455898 wait complete. 2024-04-26T20:50:32.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.771+0000 7f8db2852640 1 Processor -- start 2024-04-26T20:50:32.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.771+0000 7f8db2852640 1 -- start start 2024-04-26T20:50:32.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.772+0000 7f8db2852640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8d940051c0 0x7f8dac14fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:32.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.772+0000 7f8db2852640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8dac0cc6c0 0x7f8dac15f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:32.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.772+0000 7f8db2852640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8dac14d020 0x7f8dac0b81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:32.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.772+0000 7f8db2852640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8dac0cdc50 con 0x7f8dac14d020 2024-04-26T20:50:32.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.772+0000 7f8db2852640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8dac0b58d0 con 0x7f8dac0cc6c0 2024-04-26T20:50:32.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.772+0000 7f8db2852640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f8dac0ce650 con 0x7f8d940051c0 2024-04-26T20:50:32.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.772+0000 7f8db0def640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8dac14d020 0x7f8dac0b81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:32.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.772+0000 7f8db0def640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8dac14d020 0x7f8dac0b81d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34758/0 (socket says 172.21.15.73:34758) 2024-04-26T20:50:32.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.772+0000 7f8db0def640 1 -- 172.21.15.73:0/2672937676 learned_addr learned my addr 172.21.15.73:0/2672937676 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:32.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.772+0000 7f8dab7fe640 1 --2- 172.21.15.73:0/2672937676 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8dac0cc6c0 0x7f8dac15f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:32.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.772+0000 7f8dabfff640 1 --2- 172.21.15.73:0/2672937676 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8d940051c0 0x7f8dac14fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:32.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.772+0000 7f8db0def640 1 -- 172.21.15.73:0/2672937676 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8d940051c0 msgr2=0x7f8dac14fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:32.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.772+0000 7f8db0def640 1 --2- 172.21.15.73:0/2672937676 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8d940051c0 0x7f8dac14fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:32.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.772+0000 7f8db0def640 1 -- 172.21.15.73:0/2672937676 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8dac0cc6c0 msgr2=0x7f8dac15f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:32.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.772+0000 7f8db0def640 1 --2- 172.21.15.73:0/2672937676 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8dac0cc6c0 0x7f8dac15f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:32.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.772+0000 7f8db0def640 1 -- 172.21.15.73:0/2672937676 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dac0b7150 con 0x7f8dac14d020 2024-04-26T20:50:32.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.772+0000 7f8db0def640 1 --2- 172.21.15.73:0/2672937676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8dac14d020 0x7f8dac0b81d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8da4000f80 tx=0x7f8da4001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:32.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.773+0000 7f8da8ff9640 1 -- 172.21.15.73:0/2672937676 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da402b850 con 0x7f8dac14d020 2024-04-26T20:50:32.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.773+0000 7f8da8ff9640 1 -- 172.21.15.73:0/2672937676 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8da402b9f0 con 0x7f8dac14d020 2024-04-26T20:50:32.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.773+0000 7f8db2852640 1 -- 172.21.15.73:0/2672937676 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dac0b7410 con 0x7f8dac14d020 2024-04-26T20:50:32.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.773+0000 7f8da8ff9640 1 -- 172.21.15.73:0/2672937676 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da402bda0 con 0x7f8dac14d020 2024-04-26T20:50:32.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.773+0000 7f8db2852640 1 -- 172.21.15.73:0/2672937676 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dac0b79d0 con 0x7f8dac14d020 2024-04-26T20:50:32.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.774+0000 7f8db2852640 1 -- 172.21.15.73:0/2672937676 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dac069600 con 0x7f8dac14d020 2024-04-26T20:50:32.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.774+0000 7f8da8ff9640 1 -- 172.21.15.73:0/2672937676 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8da4034470 con 0x7f8dac14d020 2024-04-26T20:50:32.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.774+0000 7f8da8ff9640 1 --2- 172.21.15.73:0/2672937676 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f8d9806c920 0x7f8d9806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:32.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.774+0000 7f8da8ff9640 1 -- 172.21.15.73:0/2672937676 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f8da40ab6a0 con 0x7f8dac14d020 2024-04-26T20:50:32.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.775+0000 7f8dabfff640 1 --2- 172.21.15.73:0/2672937676 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f8d9806c920 0x7f8d9806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:32.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.775+0000 7f8dabfff640 1 --2- 172.21.15.73:0/2672937676 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f8d9806c920 0x7f8d9806ede0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c000c00 tx=0x7f8d9c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:32.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.779+0000 7f8da8ff9640 1 -- 172.21.15.73:0/2672937676 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8da4079480 con 0x7f8dac14d020 2024-04-26T20:50:32.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:32.964+0000 7f8db2852640 1 -- 172.21.15.73:0/2672937676 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f8dac070240 con 0x7f8dac14d020 2024-04-26T20:50:33.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.632+0000 7f8da8ff9640 1 -- 172.21.15.73:0/2672937676 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) ==== 157+0+0 (secure 0 0 0) 0x7f8da407d780 con 0x7f8dac14d020 2024-04-26T20:50:33.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.636+0000 7f8db2852640 1 -- 172.21.15.73:0/2672937676 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f8d9806c920 msgr2=0x7f8d9806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:33.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.636+0000 7f8db2852640 1 --2- 172.21.15.73:0/2672937676 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f8d9806c920 0x7f8d9806ede0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c000c00 tx=0x7f8d9c009040 comp rx=0 tx=0).stop 2024-04-26T20:50:33.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.636+0000 7f8db2852640 1 -- 172.21.15.73:0/2672937676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8dac14d020 msgr2=0x7f8dac0b81d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:33.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.636+0000 7f8db2852640 1 --2- 172.21.15.73:0/2672937676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8dac14d020 0x7f8dac0b81d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8da4000f80 tx=0x7f8da4001dc0 comp rx=0 tx=0).stop 2024-04-26T20:50:33.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.636+0000 7f8db2852640 1 -- 172.21.15.73:0/2672937676 shutdown_connections 2024-04-26T20:50:33.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.636+0000 7f8db2852640 1 --2- 172.21.15.73:0/2672937676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8dac14d020 0x7f8dac0b81d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:33.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.636+0000 7f8db2852640 1 --2- 172.21.15.73:0/2672937676 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f8d9806c920 0x7f8d9806ede0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:33.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.636+0000 7f8db2852640 1 --2- 172.21.15.73:0/2672937676 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8dac0cc6c0 0x7f8dac15f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:33.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.636+0000 7f8db2852640 1 --2- 172.21.15.73:0/2672937676 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8d940051c0 0x7f8dac14fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:33.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.636+0000 7f8db2852640 1 -- 172.21.15.73:0/2672937676 >> 172.21.15.73:0/2672937676 conn(0x7f8dac0b4970 msgr2=0x7f8dac0b0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:33.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.636+0000 7f8db2852640 1 -- 172.21.15.73:0/2672937676 shutdown_connections 2024-04-26T20:50:33.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.636+0000 7f8db2852640 1 -- 172.21.15.73:0/2672937676 wait complete. 2024-04-26T20:50:33.651 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T20:50:33.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.757+0000 7f9d19807640 1 Processor -- start 2024-04-26T20:50:33.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.757+0000 7f9d19807640 1 -- start start 2024-04-26T20:50:33.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.758+0000 7f9d19807640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9d140d5710 0x7f9d140d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:33.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.758+0000 7f9d19807640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9d140d0d60 0x7f9d140b7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:33.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.758+0000 7f9d19807640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d140d5c10 con 0x7f9d140d1880 2024-04-26T20:50:33.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.758+0000 7f9d19807640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f9d140cfb10 con 0x7f9d140d5710 2024-04-26T20:50:33.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.758+0000 7f9d19807640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f9d140d5ef0 con 0x7f9d140d0d60 2024-04-26T20:50:33.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.758+0000 7f9d0bfff640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9d140d5710 0x7f9d140d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:33.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.758+0000 7f9d0bfff640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9d140d5710 0x7f9d140d0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34764/0 (socket says 172.21.15.73:34764) 2024-04-26T20:50:33.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.758+0000 7f9d0bfff640 1 -- 172.21.15.73:0/2311637761 learned_addr learned my addr 172.21.15.73:0/2311637761 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:33.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.758+0000 7f9d0bfff640 1 -- 172.21.15.73:0/2311637761 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9d140d0d60 msgr2=0x7f9d140b7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:33.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.758+0000 7f9d0bfff640 1 --2- 172.21.15.73:0/2311637761 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9d140d0d60 0x7f9d140b7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:33.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.758+0000 7f9d0bfff640 1 -- 172.21.15.73:0/2311637761 >> v1:172.21.15.120:6789/0 conn(0x7f9d140d1880 legacy=0x7f9d140b60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:33.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.758+0000 7f9d0bfff640 1 -- 172.21.15.73:0/2311637761 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d140cfd40 con 0x7f9d140d5710 2024-04-26T20:50:33.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.759+0000 7f9d0bfff640 1 --2- 172.21.15.73:0/2311637761 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d140d5710 0x7f9d140d0820 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc0098a0 tx=0x7f9cfc02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27f4b0ec43449480 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:33.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.759+0000 7f9d127fc640 1 -- 172.21.15.73:0/2311637761 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc037070 con 0x7f9d140d5710 2024-04-26T20:50:33.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.759+0000 7f9d127fc640 1 -- 172.21.15.73:0/2311637761 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cfc032640 con 0x7f9d140d5710 2024-04-26T20:50:33.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.759+0000 7f9d127fc640 1 -- 172.21.15.73:0/2311637761 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc032a10 con 0x7f9d140d5710 2024-04-26T20:50:33.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.759+0000 7f9d19807640 1 -- 172.21.15.73:0/2311637761 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d140d5710 msgr2=0x7f9d140d0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:33.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.759+0000 7f9d19807640 1 --2- 172.21.15.73:0/2311637761 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d140d5710 0x7f9d140d0820 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc0098a0 tx=0x7f9cfc02a300 comp rx=0 tx=0).stop 2024-04-26T20:50:33.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.760+0000 7f9d19807640 1 -- 172.21.15.73:0/2311637761 shutdown_connections 2024-04-26T20:50:33.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.760+0000 7f9d19807640 1 --2- 172.21.15.73:0/2311637761 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9d140d0d60 0x7f9d140b7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:33.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.760+0000 7f9d19807640 1 --2- 172.21.15.73:0/2311637761 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d140d5710 0x7f9d140d0820 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:33.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.760+0000 7f9d19807640 1 -- 172.21.15.73:0/2311637761 >> 172.21.15.73:0/2311637761 conn(0x7f9d1409e360 msgr2=0x7f9d140d5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:33.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.760+0000 7f9d19807640 1 -- 172.21.15.73:0/2311637761 shutdown_connections 2024-04-26T20:50:33.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.760+0000 7f9d19807640 1 -- 172.21.15.73:0/2311637761 wait complete. 2024-04-26T20:50:33.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.761+0000 7f9d19807640 1 Processor -- start 2024-04-26T20:50:33.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.761+0000 7f9d19807640 1 -- start start 2024-04-26T20:50:33.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.761+0000 7f9d19807640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d140d1880 0x7f9d1415cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:33.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.761+0000 7f9d19807640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d140d5710 0x7f9d140c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:33.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.761+0000 7f9d19807640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9d140b53b0 0x7f9d1414fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:33.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.761+0000 7f9d19807640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9d140d1c60 con 0x7f9d140d5710 2024-04-26T20:50:33.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.761+0000 7f9d19807640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9d140d0300 con 0x7f9d140d1880 2024-04-26T20:50:33.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.761+0000 7f9d19807640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f9d140d1de0 con 0x7f9d140b53b0 2024-04-26T20:50:33.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.761+0000 7f9d0bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d140d5710 0x7f9d140c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:33.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.762+0000 7f9d0bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d140d5710 0x7f9d140c9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34766/0 (socket says 172.21.15.73:34766) 2024-04-26T20:50:33.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.762+0000 7f9d0bfff640 1 -- 172.21.15.73:0/1331731290 learned_addr learned my addr 172.21.15.73:0/1331731290 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:33.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.762+0000 7f9d12ffd640 1 --2- 172.21.15.73:0/1331731290 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d140d1880 0x7f9d1415cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:33.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.762+0000 7f9d137fe640 1 --2- 172.21.15.73:0/1331731290 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9d140b53b0 0x7f9d1414fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:33.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.762+0000 7f9d0bfff640 1 -- 172.21.15.73:0/1331731290 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9d140b53b0 msgr2=0x7f9d1414fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:33.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.762+0000 7f9d0bfff640 1 --2- 172.21.15.73:0/1331731290 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9d140b53b0 0x7f9d1414fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:33.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.762+0000 7f9d0bfff640 1 -- 172.21.15.73:0/1331731290 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d140d1880 msgr2=0x7f9d1415cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:33.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.762+0000 7f9d0bfff640 1 --2- 172.21.15.73:0/1331731290 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d140d1880 0x7f9d1415cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:33.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.762+0000 7f9d0bfff640 1 -- 172.21.15.73:0/1331731290 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d140ca0c0 con 0x7f9d140d5710 2024-04-26T20:50:33.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.762+0000 7f9d0bfff640 1 --2- 172.21.15.73:0/1331731290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d140d5710 0x7f9d140c9b80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc0098a0 tx=0x7f9cfc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:33.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.762+0000 7f9d10ff9640 1 -- 172.21.15.73:0/1331731290 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc037040 con 0x7f9d140d5710 2024-04-26T20:50:33.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.762+0000 7f9d10ff9640 1 -- 172.21.15.73:0/1331731290 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cfc004230 con 0x7f9d140d5710 2024-04-26T20:50:33.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.762+0000 7f9d10ff9640 1 -- 172.21.15.73:0/1331731290 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc03c730 con 0x7f9d140d5710 2024-04-26T20:50:33.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.762+0000 7f9d19807640 1 -- 172.21.15.73:0/1331731290 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d140ca360 con 0x7f9d140d5710 2024-04-26T20:50:33.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.762+0000 7f9d19807640 1 -- 172.21.15.73:0/1331731290 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d140ca920 con 0x7f9d140d5710 2024-04-26T20:50:33.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.763+0000 7f9d19807640 1 -- 172.21.15.73:0/1331731290 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d14069590 con 0x7f9d140d5710 2024-04-26T20:50:33.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.764+0000 7f9d10ff9640 1 -- 172.21.15.73:0/1331731290 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9cfc03a070 con 0x7f9d140d5710 2024-04-26T20:50:33.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.764+0000 7f9d10ff9640 1 --2- 172.21.15.73:0/1331731290 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f9cf406c920 0x7f9cf406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:33.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.764+0000 7f9d10ff9640 1 -- 172.21.15.73:0/1331731290 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 6007+0+0 (secure 0 0 0) 0x7f9cfc0ac4c0 con 0x7f9d140d5710 2024-04-26T20:50:33.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.764+0000 7f9d12ffd640 1 --2- 172.21.15.73:0/1331731290 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f9cf406c920 0x7f9cf406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:33.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.764+0000 7f9d12ffd640 1 --2- 172.21.15.73:0/1331731290 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f9cf406c920 0x7f9cf406ede0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9d04007fd0 tx=0x7f9d04009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:33.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.768+0000 7f9d10ff9640 1 -- 172.21.15.73:0/1331731290 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9cfc041020 con 0x7f9d140d5710 2024-04-26T20:50:33.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:33.947+0000 7f9d19807640 1 -- 172.21.15.73:0/1331731290 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f9d140d25b0 con 0x7f9d140d5710 2024-04-26T20:50:35.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.651+0000 7f9d10ff9640 1 -- 172.21.15.73:0/1331731290 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v22) ==== 228+0+0 (secure 0 0 0) 0x7f9cfc07a7c0 con 0x7f9d140d5710 2024-04-26T20:50:35.652 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data_ec' created 2024-04-26T20:50:35.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.655+0000 7f9d19807640 1 -- 172.21.15.73:0/1331731290 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f9cf406c920 msgr2=0x7f9cf406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:35.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.655+0000 7f9d19807640 1 --2- 172.21.15.73:0/1331731290 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f9cf406c920 0x7f9cf406ede0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9d04007fd0 tx=0x7f9d04009040 comp rx=0 tx=0).stop 2024-04-26T20:50:35.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.655+0000 7f9d19807640 1 -- 172.21.15.73:0/1331731290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d140d5710 msgr2=0x7f9d140c9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:35.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.655+0000 7f9d19807640 1 --2- 172.21.15.73:0/1331731290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d140d5710 0x7f9d140c9b80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc0098a0 tx=0x7f9cfc0046c0 comp rx=0 tx=0).stop 2024-04-26T20:50:35.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.655+0000 7f9d19807640 1 -- 172.21.15.73:0/1331731290 shutdown_connections 2024-04-26T20:50:35.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.655+0000 7f9d19807640 1 --2- 172.21.15.73:0/1331731290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d140d5710 0x7f9d140c9b80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:35.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.655+0000 7f9d19807640 1 --2- 172.21.15.73:0/1331731290 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f9cf406c920 0x7f9cf406ede0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:35.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.655+0000 7f9d19807640 1 --2- 172.21.15.73:0/1331731290 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9d140b53b0 0x7f9d1414fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:35.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.655+0000 7f9d19807640 1 --2- 172.21.15.73:0/1331731290 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9d140d1880 0x7f9d1415cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:35.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.655+0000 7f9d19807640 1 -- 172.21.15.73:0/1331731290 >> 172.21.15.73:0/1331731290 conn(0x7f9d1409e360 msgr2=0x7f9d1406cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:35.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.655+0000 7f9d19807640 1 -- 172.21.15.73:0/1331731290 shutdown_connections 2024-04-26T20:50:35.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.655+0000 7f9d19807640 1 -- 172.21.15.73:0/1331731290 wait complete. 2024-04-26T20:50:35.676 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-26T20:50:35.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.782+0000 7efe43c37640 1 Processor -- start 2024-04-26T20:50:35.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.782+0000 7efe43c37640 1 -- start start 2024-04-26T20:50:35.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.783+0000 7efe43c37640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efe3c15f460 0x7efe3c0d3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:35.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.783+0000 7efe43c37640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7efe3c0d0760 0x7efe3c0d0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:35.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.783+0000 7efe43c37640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe3c15f840 con 0x7efe3c0d1a40 2024-04-26T20:50:35.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.783+0000 7efe43c37640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7efe3c0d41c0 con 0x7efe3c0d0760 2024-04-26T20:50:35.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.783+0000 7efe43c37640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7efe3c0d37f0 con 0x7efe3c15f460 2024-04-26T20:50:35.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.783+0000 7efe411d2640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7efe3c0d0760 0x7efe3c0d0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:35.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.783+0000 7efe421d4640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7efe3c0d1a40 0x7efe3c06ce20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51998/0 (socket says 172.21.15.73:51998) 2024-04-26T20:50:35.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.783+0000 7efe421d4640 1 -- 172.21.15.73:0/1750378145 learned_addr learned my addr 172.21.15.73:0/1750378145 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:50:35.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.783+0000 7efe419d3640 1 --2- 172.21.15.73:0/1750378145 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efe3c15f460 0x7efe3c0d3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:35.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.783+0000 7efe411d2640 1 -- 172.21.15.73:0/1750378145 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efe3c15f460 msgr2=0x7efe3c0d3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:35.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.783+0000 7efe411d2640 1 --2- 172.21.15.73:0/1750378145 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efe3c15f460 0x7efe3c0d3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:35.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.783+0000 7efe411d2640 1 -- 172.21.15.73:0/1750378145 >> v1:172.21.15.73:6789/0 conn(0x7efe3c0d1a40 legacy=0x7efe3c06ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:35.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.783+0000 7efe411d2640 1 -- 172.21.15.73:0/1750378145 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe3c0d2ec0 con 0x7efe3c0d0760 2024-04-26T20:50:35.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.783+0000 7efe409d1640 1 -- 172.21.15.73:0/1750378145 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3568319205 0 0) 0x7efe3c15f840 con 0x7efe3c0d1a40 2024-04-26T20:50:35.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.784+0000 7efe419d3640 1 --2- 172.21.15.73:0/1750378145 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efe3c15f460 0x7efe3c0d3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:50:35.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.784+0000 7efe411d2640 1 --2- 172.21.15.73:0/1750378145 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe3c0d0760 0x7efe3c0d0bc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7efe2c0097a0 tx=0x7efe2c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23de2c168ec8ea18 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:35.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.784+0000 7efe409d1640 1 -- 172.21.15.73:0/1750378145 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2c037070 con 0x7efe3c0d0760 2024-04-26T20:50:35.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.784+0000 7efe409d1640 1 -- 172.21.15.73:0/1750378145 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe2c02ae50 con 0x7efe3c0d0760 2024-04-26T20:50:35.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.784+0000 7efe409d1640 1 -- 172.21.15.73:0/1750378145 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2c032660 con 0x7efe3c0d0760 2024-04-26T20:50:35.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.785+0000 7efe43c37640 1 -- 172.21.15.73:0/1750378145 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe3c0d0760 msgr2=0x7efe3c0d0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:35.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.785+0000 7efe43c37640 1 --2- 172.21.15.73:0/1750378145 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe3c0d0760 0x7efe3c0d0bc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7efe2c0097a0 tx=0x7efe2c02a100 comp rx=0 tx=0).stop 2024-04-26T20:50:35.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.785+0000 7efe43c37640 1 -- 172.21.15.73:0/1750378145 shutdown_connections 2024-04-26T20:50:35.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.785+0000 7efe43c37640 1 --2- 172.21.15.73:0/1750378145 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe3c0d0760 0x7efe3c0d0bc0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:35.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.785+0000 7efe43c37640 1 --2- 172.21.15.73:0/1750378145 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efe3c15f460 0x7efe3c0d3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:35.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.785+0000 7efe43c37640 1 -- 172.21.15.73:0/1750378145 >> 172.21.15.73:0/1750378145 conn(0x7efe3c09e3f0 msgr2=0x7efe3c0d50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:35.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.785+0000 7efe43c37640 1 -- 172.21.15.73:0/1750378145 shutdown_connections 2024-04-26T20:50:35.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.785+0000 7efe43c37640 1 -- 172.21.15.73:0/1750378145 wait complete. 2024-04-26T20:50:35.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.786+0000 7efe43c37640 1 Processor -- start 2024-04-26T20:50:35.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.786+0000 7efe43c37640 1 -- start start 2024-04-26T20:50:35.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.787+0000 7efe43c37640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe3c0d0760 0x7efe3c1579f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:35.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.787+0000 7efe43c37640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efe3c0d1a40 0x7efe3c150530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:35.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.787+0000 7efe43c37640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe3c15f460 0x7efe3c0cadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:35.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.787+0000 7efe43c37640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efe3c0d5f30 con 0x7efe3c0d0760 2024-04-26T20:50:35.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.787+0000 7efe43c37640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efe3c0d3480 con 0x7efe3c15f460 2024-04-26T20:50:35.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.787+0000 7efe43c37640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7efe3c0d1ed0 con 0x7efe3c0d1a40 2024-04-26T20:50:35.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.787+0000 7efe419d3640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe3c0d0760 0x7efe3c1579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:35.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.787+0000 7efe419d3640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe3c0d0760 0x7efe3c1579f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34790/0 (socket says 172.21.15.73:34790) 2024-04-26T20:50:35.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.787+0000 7efe419d3640 1 -- 172.21.15.73:0/1163257410 learned_addr learned my addr 172.21.15.73:0/1163257410 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:35.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.787+0000 7efe421d4640 1 --2- 172.21.15.73:0/1163257410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe3c15f460 0x7efe3c0cadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:35.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.787+0000 7efe419d3640 1 -- 172.21.15.73:0/1163257410 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efe3c0d1a40 msgr2=0x7efe3c150530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:35.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.787+0000 7efe411d2640 1 --2- 172.21.15.73:0/1163257410 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efe3c0d1a40 0x7efe3c150530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:35.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.787+0000 7efe419d3640 1 --2- 172.21.15.73:0/1163257410 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efe3c0d1a40 0x7efe3c150530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:35.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.787+0000 7efe419d3640 1 -- 172.21.15.73:0/1163257410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe3c15f460 msgr2=0x7efe3c0cadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:35.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.787+0000 7efe419d3640 1 --2- 172.21.15.73:0/1163257410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe3c15f460 0x7efe3c0cadc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:35.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.787+0000 7efe419d3640 1 -- 172.21.15.73:0/1163257410 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe3c150a70 con 0x7efe3c0d0760 2024-04-26T20:50:35.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.787+0000 7efe419d3640 1 --2- 172.21.15.73:0/1163257410 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe3c0d0760 0x7efe3c1579f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7efe3800c820 tx=0x7efe3800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:35.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.787+0000 7efe327fc640 1 -- 172.21.15.73:0/1163257410 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe38015070 con 0x7efe3c0d0760 2024-04-26T20:50:35.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.787+0000 7efe327fc640 1 -- 172.21.15.73:0/1163257410 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe38007a30 con 0x7efe3c0d0760 2024-04-26T20:50:35.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.787+0000 7efe43c37640 1 -- 172.21.15.73:0/1163257410 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe3c15fb60 con 0x7efe3c0d0760 2024-04-26T20:50:35.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.787+0000 7efe327fc640 1 -- 172.21.15.73:0/1163257410 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe38007de0 con 0x7efe3c0d0760 2024-04-26T20:50:35.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.787+0000 7efe43c37640 1 -- 172.21.15.73:0/1163257410 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe3c160120 con 0x7efe3c0d0760 2024-04-26T20:50:35.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.788+0000 7efe43c37640 1 -- 172.21.15.73:0/1163257410 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe3c069600 con 0x7efe3c0d0760 2024-04-26T20:50:35.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.789+0000 7efe327fc640 1 -- 172.21.15.73:0/1163257410 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efe38018070 con 0x7efe3c0d0760 2024-04-26T20:50:35.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.789+0000 7efe327fc640 1 --2- 172.21.15.73:0/1163257410 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7efe2006c8f0 0x7efe2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:35.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.790+0000 7efe327fc640 1 -- 172.21.15.73:0/1163257410 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 6500+0+0 (secure 0 0 0) 0x7efe3808c0a0 con 0x7efe3c0d0760 2024-04-26T20:50:35.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.790+0000 7efe411d2640 1 --2- 172.21.15.73:0/1163257410 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7efe2006c8f0 0x7efe2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:35.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.790+0000 7efe411d2640 1 --2- 172.21.15.73:0/1163257410 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7efe2006c8f0 0x7efe2006edb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7efe2c034000 tx=0x7efe2c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:35.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.794+0000 7efe327fc640 1 -- 172.21.15.73:0/1163257410 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efe3c150a70 con 0x7efe3c0d0760 2024-04-26T20:50:35.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:35.969+0000 7efe43c37640 1 -- 172.21.15.73:0/1163257410 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7efe3c0d29a0 con 0x7efe3c0d0760 2024-04-26T20:50:36.572 INFO:tasks.ceph.mds.c.smithi073.stderr:2024-04-26T20:50:36.571+0000 7fcfebb5b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T20:50:36.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.655+0000 7efe327fc640 1 -- 172.21.15.73:0/1163257410 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v23) ==== 169+0+0 (secure 0 0 0) 0x7efe3805a320 con 0x7efe3c0d0760 2024-04-26T20:50:36.656 INFO:teuthology.orchestra.run.smithi073.stderr:set pool 4 allow_ec_overwrites to true 2024-04-26T20:50:36.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.657+0000 7efe43c37640 1 -- 172.21.15.73:0/1163257410 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7efe2006c8f0 msgr2=0x7efe2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:36.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.657+0000 7efe43c37640 1 --2- 172.21.15.73:0/1163257410 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7efe2006c8f0 0x7efe2006edb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7efe2c034000 tx=0x7efe2c02e000 comp rx=0 tx=0).stop 2024-04-26T20:50:36.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.657+0000 7efe43c37640 1 -- 172.21.15.73:0/1163257410 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe3c0d0760 msgr2=0x7efe3c1579f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:36.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.657+0000 7efe43c37640 1 --2- 172.21.15.73:0/1163257410 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe3c0d0760 0x7efe3c1579f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7efe3800c820 tx=0x7efe3800cce0 comp rx=0 tx=0).stop 2024-04-26T20:50:36.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.657+0000 7efe43c37640 1 -- 172.21.15.73:0/1163257410 shutdown_connections 2024-04-26T20:50:36.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.657+0000 7efe43c37640 1 --2- 172.21.15.73:0/1163257410 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7efe2006c8f0 0x7efe2006edb0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:36.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.657+0000 7efe43c37640 1 --2- 172.21.15.73:0/1163257410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe3c15f460 0x7efe3c0cadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:36.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.657+0000 7efe43c37640 1 --2- 172.21.15.73:0/1163257410 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efe3c0d1a40 0x7efe3c150530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:36.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.657+0000 7efe43c37640 1 --2- 172.21.15.73:0/1163257410 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe3c0d0760 0x7efe3c1579f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:36.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.657+0000 7efe43c37640 1 -- 172.21.15.73:0/1163257410 >> 172.21.15.73:0/1163257410 conn(0x7efe3c09e3f0 msgr2=0x7efe3c0b3cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:36.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.658+0000 7efe43c37640 1 -- 172.21.15.73:0/1163257410 shutdown_connections 2024-04-26T20:50:36.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.658+0000 7efe43c37640 1 -- 172.21.15.73:0/1163257410 wait complete. 2024-04-26T20:50:36.681 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-26T20:50:36.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.781+0000 7fe24672b640 1 Processor -- start 2024-04-26T20:50:36.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.781+0000 7fe24672b640 1 -- start start 2024-04-26T20:50:36.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.782+0000 7fe24672b640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe2400d3bc0 0x7fe2400d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:36.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.782+0000 7fe24672b640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2400b7e50 con 0x7fe240162880 2024-04-26T20:50:36.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.782+0000 7fe24672b640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2400d1800 con 0x7fe2400b5f20 2024-04-26T20:50:36.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.782+0000 7fe24672b640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fe2400d0580 con 0x7fe2400d3bc0 2024-04-26T20:50:36.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.782+0000 7fe23ffff640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fe240162880 0x7fe2400d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:52010/0 (socket says 172.21.15.73:52010) 2024-04-26T20:50:36.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.782+0000 7fe23ffff640 1 -- 172.21.15.73:0/3311454477 learned_addr learned my addr 172.21.15.73:0/3311454477 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:50:36.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.782+0000 7fe23effd640 1 -- 172.21.15.73:0/3311454477 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 933744410 0 0) 0x7fe2400b7e50 con 0x7fe240162880 2024-04-26T20:50:36.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.782+0000 7fe23effd640 1 -- 172.21.15.73:0/3311454477 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe228003520 con 0x7fe240162880 2024-04-26T20:50:36.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.782+0000 7fe23f7fe640 1 --2- 172.21.15.73:0/3311454477 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe2400d3bc0 0x7fe2400d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:36.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.782+0000 7fe23effd640 1 -- 172.21.15.73:0/3311454477 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1205717266 0 0) 0x7fe228003520 con 0x7fe240162880 2024-04-26T20:50:36.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.782+0000 7fe23effd640 1 -- 172.21.15.73:0/3311454477 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe2400d3bc0 msgr2=0x7fe2400d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:36.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.782+0000 7fe23effd640 1 --2- 172.21.15.73:0/3311454477 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe2400d3bc0 0x7fe2400d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:36.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.782+0000 7fe23effd640 1 -- 172.21.15.73:0/3311454477 >> v1:172.21.15.120:6789/0 conn(0x7fe2400b5f20 legacy=0x7fe2400b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:36.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.782+0000 7fe23effd640 1 -- 172.21.15.73:0/3311454477 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2400d07d0 con 0x7fe240162880 2024-04-26T20:50:36.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.782+0000 7fe23effd640 1 -- 172.21.15.73:0/3311454477 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7fe230001670 con 0x7fe240162880 2024-04-26T20:50:36.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.782+0000 7fe23effd640 1 -- 172.21.15.73:0/3311454477 >> v1:172.21.15.73:6789/0 conn(0x7fe240162880 legacy=0x7fe2400d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:36.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.782+0000 7fe23effd640 1 --2- 172.21.15.73:0/3311454477 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe2280039e0 0x7fe228003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:36.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.782+0000 7fe23effd640 1 --2- 172.21.15.73:0/3311454477 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe2400d3bc0 0x7fe228004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:36.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.782+0000 7fe23effd640 1 --2- 172.21.15.73:0/3311454477 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe228004d40 0x7fe2280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:36.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.782+0000 7fe23effd640 1 -- 172.21.15.73:0/3311454477 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe2400d0580 con 0x7fe2280039e0 2024-04-26T20:50:36.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.782+0000 7fe23effd640 1 -- 172.21.15.73:0/3311454477 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe22800bb50 con 0x7fe228004d40 2024-04-26T20:50:36.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.782+0000 7fe23effd640 1 -- 172.21.15.73:0/3311454477 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fe22800bd20 con 0x7fe2400d3bc0 2024-04-26T20:50:36.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.782+0000 7fe23ffff640 1 --2- 172.21.15.73:0/3311454477 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe2280039e0 0x7fe228003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:36.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.782+0000 7fe23f7fe640 1 --2- 172.21.15.73:0/3311454477 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe2400d3bc0 0x7fe228004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:36.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.782+0000 7fe244cc8640 1 --2- 172.21.15.73:0/3311454477 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe228004d40 0x7fe2280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:36.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.783+0000 7fe23ffff640 1 -- 172.21.15.73:0/3311454477 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe2400d3bc0 msgr2=0x7fe228004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:36.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.783+0000 7fe23ffff640 1 --2- 172.21.15.73:0/3311454477 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe2400d3bc0 0x7fe228004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:36.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.783+0000 7fe23ffff640 1 -- 172.21.15.73:0/3311454477 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe228004d40 msgr2=0x7fe2280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:36.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.783+0000 7fe23ffff640 1 --2- 172.21.15.73:0/3311454477 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe228004d40 0x7fe2280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:36.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.783+0000 7fe23ffff640 1 -- 172.21.15.73:0/3311454477 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe228003520 con 0x7fe2280039e0 2024-04-26T20:50:36.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.783+0000 7fe23ffff640 1 --2- 172.21.15.73:0/3311454477 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe2280039e0 0x7fe228003e00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe23000c270 tx=0x7fe228005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d98241e487626926 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:36.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.783+0000 7fe23effd640 1 -- 172.21.15.73:0/3311454477 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe230007d10 con 0x7fe2280039e0 2024-04-26T20:50:36.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.783+0000 7fe23effd640 1 -- 172.21.15.73:0/3311454477 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe230005ad0 con 0x7fe2280039e0 2024-04-26T20:50:36.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.783+0000 7fe23effd640 1 -- 172.21.15.73:0/3311454477 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe230007070 con 0x7fe2280039e0 2024-04-26T20:50:36.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.784+0000 7fe24672b640 1 -- 172.21.15.73:0/3311454477 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe2280039e0 msgr2=0x7fe228003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:36.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.784+0000 7fe24672b640 1 --2- 172.21.15.73:0/3311454477 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe2280039e0 0x7fe228003e00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe23000c270 tx=0x7fe228005830 comp rx=0 tx=0).stop 2024-04-26T20:50:36.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.784+0000 7fe244cc8640 1 -- 172.21.15.73:0/3311454477 reap_dead start 2024-04-26T20:50:36.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.784+0000 7fe24672b640 1 -- 172.21.15.73:0/3311454477 shutdown_connections 2024-04-26T20:50:36.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.784+0000 7fe24672b640 1 -- 172.21.15.73:0/3311454477 >> 172.21.15.73:0/3311454477 conn(0x7fe2400d4f60 msgr2=0x7fe2400d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:36.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.784+0000 7fe24672b640 1 -- 172.21.15.73:0/3311454477 shutdown_connections 2024-04-26T20:50:36.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.784+0000 7fe24672b640 1 -- 172.21.15.73:0/3311454477 wait complete. 2024-04-26T20:50:36.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.785+0000 7fe24672b640 1 Processor -- start 2024-04-26T20:50:36.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.785+0000 7fe24672b640 1 -- start start 2024-04-26T20:50:36.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.786+0000 7fe24672b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe2280039e0 0x7fe2400ce8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:36.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.786+0000 7fe24672b640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe240159390 0x7fe24006e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:36.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.786+0000 7fe24672b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe24006e670 0x7fe2400cc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:36.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.786+0000 7fe24672b640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe240162c60 con 0x7fe24006e670 2024-04-26T20:50:36.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.786+0000 7fe24672b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe2400d0d90 con 0x7fe2280039e0 2024-04-26T20:50:36.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.786+0000 7fe24672b640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fe240162de0 con 0x7fe240159390 2024-04-26T20:50:36.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.786+0000 7fe244cc8640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe24006e670 0x7fe2400cc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:36.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.786+0000 7fe244cc8640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe24006e670 0x7fe2400cc0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34802/0 (socket says 172.21.15.73:34802) 2024-04-26T20:50:36.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.786+0000 7fe244cc8640 1 -- 172.21.15.73:0/3781620524 learned_addr learned my addr 172.21.15.73:0/3781620524 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:36.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.786+0000 7fe23ffff640 1 --2- 172.21.15.73:0/3781620524 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe2280039e0 0x7fe2400ce8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:36.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.786+0000 7fe23f7fe640 1 --2- 172.21.15.73:0/3781620524 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe240159390 0x7fe24006e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:36.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.786+0000 7fe244cc8640 1 -- 172.21.15.73:0/3781620524 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe240159390 msgr2=0x7fe24006e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:36.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.786+0000 7fe244cc8640 1 --2- 172.21.15.73:0/3781620524 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe240159390 0x7fe24006e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:36.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.786+0000 7fe244cc8640 1 -- 172.21.15.73:0/3781620524 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe2280039e0 msgr2=0x7fe2400ce8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:36.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.786+0000 7fe244cc8640 1 --2- 172.21.15.73:0/3781620524 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe2280039e0 0x7fe2400ce8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:36.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.786+0000 7fe244cc8640 1 -- 172.21.15.73:0/3781620524 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2400cce60 con 0x7fe24006e670 2024-04-26T20:50:36.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.786+0000 7fe244cc8640 1 --2- 172.21.15.73:0/3781620524 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe24006e670 0x7fe2400cc0a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe24008d460 tx=0x7fe23800fc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:36.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.787+0000 7fe23cff9640 1 -- 172.21.15.73:0/3781620524 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23801a070 con 0x7fe24006e670 2024-04-26T20:50:36.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.787+0000 7fe23cff9640 1 -- 172.21.15.73:0/3781620524 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe238006d60 con 0x7fe24006e670 2024-04-26T20:50:36.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.787+0000 7fe23cff9640 1 -- 172.21.15.73:0/3781620524 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23800fea0 con 0x7fe24006e670 2024-04-26T20:50:36.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.787+0000 7fe24672b640 1 -- 172.21.15.73:0/3781620524 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2400cde00 con 0x7fe24006e670 2024-04-26T20:50:36.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.787+0000 7fe24672b640 1 -- 172.21.15.73:0/3781620524 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe240159810 con 0x7fe24006e670 2024-04-26T20:50:36.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.788+0000 7fe24672b640 1 -- 172.21.15.73:0/3781620524 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2400694f0 con 0x7fe24006e670 2024-04-26T20:50:36.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.788+0000 7fe23cff9640 1 -- 172.21.15.73:0/3781620524 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe23801d070 con 0x7fe24006e670 2024-04-26T20:50:36.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.788+0000 7fe23cff9640 1 --2- 172.21.15.73:0/3781620524 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe22c06c920 0x7fe22c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:36.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.788+0000 7fe23cff9640 1 -- 172.21.15.73:0/3781620524 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6500+0+0 (secure 0 0 0) 0x7fe23808b330 con 0x7fe24006e670 2024-04-26T20:50:36.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.788+0000 7fe23ffff640 1 --2- 172.21.15.73:0/3781620524 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe22c06c920 0x7fe22c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:36.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.789+0000 7fe23ffff640 1 --2- 172.21.15.73:0/3781620524 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe22c06c920 0x7fe22c06ede0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe230033000 tx=0x7fe230009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:36.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:36.793+0000 7fe23cff9640 1 -- 172.21.15.73:0/3781620524 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe2400d1800 con 0x7fe24006e670 2024-04-26T20:50:37.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.006+0000 7fe24672b640 1 -- 172.21.15.73:0/3781620524 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7fe2400cbbb0 con 0x7fe24006e670 2024-04-26T20:50:37.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.656+0000 7fe23cff9640 1 -- 172.21.15.73:0/3781620524 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v6) ==== 137+0+0 (secure 0 0 0) 0x7fe238058e80 con 0x7fe24006e670 2024-04-26T20:50:37.658 INFO:teuthology.orchestra.run.smithi073.stderr:added data pool 4 to fsmap 2024-04-26T20:50:37.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.660+0000 7fe24672b640 1 -- 172.21.15.73:0/3781620524 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe22c06c920 msgr2=0x7fe22c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:37.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.660+0000 7fe24672b640 1 --2- 172.21.15.73:0/3781620524 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe22c06c920 0x7fe22c06ede0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe230033000 tx=0x7fe230009030 comp rx=0 tx=0).stop 2024-04-26T20:50:37.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.660+0000 7fe24672b640 1 -- 172.21.15.73:0/3781620524 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe24006e670 msgr2=0x7fe2400cc0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:37.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.660+0000 7fe24672b640 1 --2- 172.21.15.73:0/3781620524 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe24006e670 0x7fe2400cc0a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe24008d460 tx=0x7fe23800fc50 comp rx=0 tx=0).stop 2024-04-26T20:50:37.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.660+0000 7fe24672b640 1 -- 172.21.15.73:0/3781620524 shutdown_connections 2024-04-26T20:50:37.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.660+0000 7fe24672b640 1 --2- 172.21.15.73:0/3781620524 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe24006e670 0x7fe2400cc0a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:37.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.660+0000 7fe24672b640 1 --2- 172.21.15.73:0/3781620524 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe22c06c920 0x7fe22c06ede0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:37.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.660+0000 7fe24672b640 1 --2- 172.21.15.73:0/3781620524 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe240159390 0x7fe24006e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:37.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.660+0000 7fe24672b640 1 --2- 172.21.15.73:0/3781620524 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe2280039e0 0x7fe2400ce8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:37.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.660+0000 7fe24672b640 1 -- 172.21.15.73:0/3781620524 >> 172.21.15.73:0/3781620524 conn(0x7fe2400d4f60 msgr2=0x7fe2400d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:37.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.660+0000 7fe24672b640 1 -- 172.21.15.73:0/3781620524 shutdown_connections 2024-04-26T20:50:37.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.660+0000 7fe24672b640 1 -- 172.21.15.73:0/3781620524 wait complete. 2024-04-26T20:50:37.675 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:50:37.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.775+0000 7f91df787640 1 Processor -- start 2024-04-26T20:50:37.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.776+0000 7f91df787640 1 -- start start 2024-04-26T20:50:37.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.776+0000 7f91df787640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f91d815b600 0x7f91d8071f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:37.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.777+0000 7f91df787640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f91d80cf0e0 0x7f91d806f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:37.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.777+0000 7f91df787640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91d8072450 con 0x7f91d80cf5e0 2024-04-26T20:50:37.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.777+0000 7f91df787640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f91d80cb9b0 con 0x7f91d815b600 2024-04-26T20:50:37.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.777+0000 7f91df787640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f91d815b9e0 con 0x7f91d80cf0e0 2024-04-26T20:50:37.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.777+0000 7f91dd523640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f91d815b600 0x7f91d8071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:37.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.777+0000 7f91dd523640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f91d815b600 0x7f91d8071f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34810/0 (socket says 172.21.15.73:34810) 2024-04-26T20:50:37.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.777+0000 7f91dd523640 1 -- 172.21.15.73:0/144986000 learned_addr learned my addr 172.21.15.73:0/144986000 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:37.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.777+0000 7f91ddd24640 1 --2- 172.21.15.73:0/144986000 >> v2:172.21.15.120:3300/0 conn(0x7f91d80cf0e0 0x7f91d806f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:37.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.777+0000 7f91dd523640 1 -- 172.21.15.73:0/144986000 >> v2:172.21.15.120:3300/0 conn(0x7f91d80cf0e0 msgr2=0x7f91d806f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:37.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.777+0000 7f91dd523640 1 --2- 172.21.15.73:0/144986000 >> v2:172.21.15.120:3300/0 conn(0x7f91d80cf0e0 0x7f91d806f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:37.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.777+0000 7f91dd523640 1 -- 172.21.15.73:0/144986000 >> v1:172.21.15.120:6789/0 conn(0x7f91d80cf5e0 legacy=0x7f91d80ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:37.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.777+0000 7f91dd523640 1 -- 172.21.15.73:0/144986000 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91d80d0740 con 0x7f91d815b600 2024-04-26T20:50:37.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.778+0000 7f91dd523640 1 --2- 172.21.15.73:0/144986000 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91d815b600 0x7f91d8071f10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f91d8071a80 tx=0x7f91d002c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c185057f0246eab4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:37.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.778+0000 7f91cffff640 1 -- 172.21.15.73:0/144986000 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d0009600 con 0x7f91d815b600 2024-04-26T20:50:37.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.778+0000 7f91cffff640 1 -- 172.21.15.73:0/144986000 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91d00097a0 con 0x7f91d815b600 2024-04-26T20:50:37.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.778+0000 7f91cffff640 1 -- 172.21.15.73:0/144986000 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d0009b70 con 0x7f91d815b600 2024-04-26T20:50:37.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.778+0000 7f91df787640 1 -- 172.21.15.73:0/144986000 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91d815b600 msgr2=0x7f91d8071f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:37.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.778+0000 7f91df787640 1 --2- 172.21.15.73:0/144986000 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91d815b600 0x7f91d8071f10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f91d8071a80 tx=0x7f91d002c900 comp rx=0 tx=0).stop 2024-04-26T20:50:37.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.778+0000 7f91df787640 1 -- 172.21.15.73:0/144986000 shutdown_connections 2024-04-26T20:50:37.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.778+0000 7f91df787640 1 --2- 172.21.15.73:0/144986000 >> v2:172.21.15.120:3300/0 conn(0x7f91d80cf0e0 0x7f91d806f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:37.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.778+0000 7f91df787640 1 --2- 172.21.15.73:0/144986000 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91d815b600 0x7f91d8071f10 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:37.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.778+0000 7f91df787640 1 -- 172.21.15.73:0/144986000 >> 172.21.15.73:0/144986000 conn(0x7f91d80d0be0 msgr2=0x7f91d80d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:37.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.778+0000 7f91df787640 1 -- 172.21.15.73:0/144986000 shutdown_connections 2024-04-26T20:50:37.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.778+0000 7f91df787640 1 -- 172.21.15.73:0/144986000 wait complete. 2024-04-26T20:50:37.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.779+0000 7f91df787640 1 Processor -- start 2024-04-26T20:50:37.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.780+0000 7f91df787640 1 -- start start 2024-04-26T20:50:37.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.780+0000 7f91df787640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91d80cf0e0 0x7f91d80c9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:37.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.780+0000 7f91df787640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f91d80cf5e0 0x7f91d81522b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:37.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.780+0000 7f91df787640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91d815b600 0x7f91d81f1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:37.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.780+0000 7f91df787640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f91d80d02f0 con 0x7f91d80cf0e0 2024-04-26T20:50:37.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.780+0000 7f91df787640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f91d80cb640 con 0x7f91d815b600 2024-04-26T20:50:37.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.780+0000 7f91df787640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f91d80d0470 con 0x7f91d80cf5e0 2024-04-26T20:50:37.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.780+0000 7f91dd523640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91d80cf0e0 0x7f91d80c9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:37.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.780+0000 7f91dd523640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91d80cf0e0 0x7f91d80c9820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34818/0 (socket says 172.21.15.73:34818) 2024-04-26T20:50:37.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.780+0000 7f91dd523640 1 -- 172.21.15.73:0/1525007940 learned_addr learned my addr 172.21.15.73:0/1525007940 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:37.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.781+0000 7f91dd523640 1 -- 172.21.15.73:0/1525007940 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f91d80cf5e0 msgr2=0x7f91d81522b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:37.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.781+0000 7f91dd523640 1 --2- 172.21.15.73:0/1525007940 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f91d80cf5e0 0x7f91d81522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:37.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.781+0000 7f91dd523640 1 -- 172.21.15.73:0/1525007940 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91d815b600 msgr2=0x7f91d81f1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:37.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.781+0000 7f91dd523640 1 --2- 172.21.15.73:0/1525007940 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91d815b600 0x7f91d81f1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:37.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.781+0000 7f91dd523640 1 -- 172.21.15.73:0/1525007940 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91d80b6350 con 0x7f91d80cf0e0 2024-04-26T20:50:37.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.781+0000 7f91dd523640 1 --2- 172.21.15.73:0/1525007940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91d80cf0e0 0x7f91d80c9820 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f91d000d240 tx=0x7f91d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:37.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.781+0000 7f91cdffb640 1 -- 172.21.15.73:0/1525007940 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d0043070 con 0x7f91d80cf0e0 2024-04-26T20:50:37.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.781+0000 7f91cdffb640 1 -- 172.21.15.73:0/1525007940 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91d0034c80 con 0x7f91d80cf0e0 2024-04-26T20:50:37.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.781+0000 7f91cdffb640 1 -- 172.21.15.73:0/1525007940 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d0009600 con 0x7f91d80cf0e0 2024-04-26T20:50:37.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.781+0000 7f91df787640 1 -- 172.21.15.73:0/1525007940 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91d81f0d50 con 0x7f91d80cf0e0 2024-04-26T20:50:37.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.781+0000 7f91df787640 1 -- 172.21.15.73:0/1525007940 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91d81f1d70 con 0x7f91d80cf0e0 2024-04-26T20:50:37.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.782+0000 7f91df787640 1 -- 172.21.15.73:0/1525007940 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91d80694f0 con 0x7f91d80cf0e0 2024-04-26T20:50:37.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.783+0000 7f91cdffb640 1 -- 172.21.15.73:0/1525007940 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f91d003b040 con 0x7f91d80cf0e0 2024-04-26T20:50:37.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.783+0000 7f91cdffb640 1 --2- 172.21.15.73:0/1525007940 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f91bc06c8f0 0x7f91bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:37.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.783+0000 7f91cdffb640 1 -- 172.21.15.73:0/1525007940 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f91d00ae770 con 0x7f91d80cf0e0 2024-04-26T20:50:37.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.783+0000 7f91dcd22640 1 --2- 172.21.15.73:0/1525007940 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f91bc06c8f0 0x7f91bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:37.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.783+0000 7f91dcd22640 1 --2- 172.21.15.73:0/1525007940 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f91bc06c8f0 0x7f91bc06edb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f91d815e2c0 tx=0x7f91c80079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:37.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:37.787+0000 7f91cdffb640 1 -- 172.21.15.73:0/1525007940 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f91d8072450 con 0x7f91d80cf0e0 2024-04-26T20:50:38.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.000+0000 7f91df787640 1 -- 172.21.15.73:0/1525007940 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f91d80ce7b0 con 0x7f91d80cf0e0 2024-04-26T20:50:38.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.001+0000 7f91cdffb640 1 -- 172.21.15.73:0/1525007940 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5180 (secure 0 0 0) 0x7f91d007c860 con 0x7f91d80cf0e0 2024-04-26T20:50:38.002 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:50:38.002 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/4263167680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6833","nonce":4263167680},{"type":"v1","addr":"172.21.15.120:6835","nonce":4263167680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6834/1614342510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1614342510},{"type":"v1","addr":"172.21.15.120:6834","nonce":1614342510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2146325725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2146325725},{"type":"v1","addr":"172.21.15.73:6835","nonce":2146325725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:50:31.964317+0000","modified":"2024-04-26T20:50:37.007492+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/181202544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":181202544},{"type":"v1","addr":"172.21.15.73:6837","nonce":181202544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4281,"qdb_cluster":[4281]},"id":1}]} 2024-04-26T20:50:38.002 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 6 2024-04-26T20:50:38.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.003+0000 7f91df787640 1 -- 172.21.15.73:0/1525007940 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f91bc06c8f0 msgr2=0x7f91bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:38.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.003+0000 7f91df787640 1 --2- 172.21.15.73:0/1525007940 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f91bc06c8f0 0x7f91bc06edb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f91d815e2c0 tx=0x7f91c80079b0 comp rx=0 tx=0).stop 2024-04-26T20:50:38.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.003+0000 7f91df787640 1 -- 172.21.15.73:0/1525007940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91d80cf0e0 msgr2=0x7f91d80c9820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:38.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.003+0000 7f91df787640 1 --2- 172.21.15.73:0/1525007940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91d80cf0e0 0x7f91d80c9820 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f91d000d240 tx=0x7f91d0004a40 comp rx=0 tx=0).stop 2024-04-26T20:50:38.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.003+0000 7f91df787640 1 -- 172.21.15.73:0/1525007940 shutdown_connections 2024-04-26T20:50:38.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.003+0000 7f91df787640 1 --2- 172.21.15.73:0/1525007940 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f91bc06c8f0 0x7f91bc06edb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:38.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.003+0000 7f91df787640 1 --2- 172.21.15.73:0/1525007940 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91d815b600 0x7f91d81f1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:38.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.003+0000 7f91df787640 1 --2- 172.21.15.73:0/1525007940 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f91d80cf5e0 0x7f91d81522b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:38.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.003+0000 7f91df787640 1 --2- 172.21.15.73:0/1525007940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91d80cf0e0 0x7f91d80c9820 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:38.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.003+0000 7f91df787640 1 -- 172.21.15.73:0/1525007940 >> 172.21.15.73:0/1525007940 conn(0x7f91d80d0be0 msgr2=0x7f91d80b0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:38.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.003+0000 7f91df787640 1 -- 172.21.15.73:0/1525007940 shutdown_connections 2024-04-26T20:50:38.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.003+0000 7f91df787640 1 -- 172.21.15.73:0/1525007940 wait complete. 2024-04-26T20:50:38.015 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:50:38.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.111+0000 7fec8a1c6640 1 Processor -- start 2024-04-26T20:50:38.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.111+0000 7fec8a1c6640 1 -- start start 2024-04-26T20:50:38.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.112+0000 7fec8a1c6640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fec84162810 0x7fec84070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:38.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.112+0000 7fec8a1c6640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fec840b2340 0x7fec840cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:38.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.112+0000 7fec8a1c6640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec840d10d0 con 0x7fec840cd930 2024-04-26T20:50:38.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.112+0000 7fec8a1c6640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fec840d0750 con 0x7fec840b2340 2024-04-26T20:50:38.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.112+0000 7fec8a1c6640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fec840d16a0 con 0x7fec84162810 2024-04-26T20:50:38.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.112+0000 7fec82ffd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fec840b2340 0x7fec840cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:38.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.112+0000 7fec837fe640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fec84162810 0x7fec84070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:38.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.112+0000 7fec82ffd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fec840b2340 0x7fec840cd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.73:33982/0 (socket says 172.21.15.73:33982) 2024-04-26T20:50:38.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.112+0000 7fec837fe640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fec84162810 0x7fec84070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.73:54880/0 (socket says 172.21.15.73:54880) 2024-04-26T20:50:38.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.112+0000 7fec82ffd640 1 -- 172.21.15.73:0/3198627461 learned_addr learned my addr 172.21.15.73:0/3198627461 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:38.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.112+0000 7fec83fff640 1 --1- 172.21.15.73:0/3198627461 >> v1:172.21.15.120:6789/0 conn(0x7fec840cd930 0x7fec840d1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.73:46174/0 (socket says 172.21.15.73:46174) 2024-04-26T20:50:38.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.113+0000 7fec837fe640 1 -- 172.21.15.73:0/3198627461 >> v2:172.21.15.120:3300/0 conn(0x7fec840b2340 msgr2=0x7fec840cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:38.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.113+0000 7fec837fe640 1 --2- 172.21.15.73:0/3198627461 >> v2:172.21.15.120:3300/0 conn(0x7fec840b2340 0x7fec840cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:38.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.113+0000 7fec837fe640 1 -- 172.21.15.73:0/3198627461 >> v1:172.21.15.120:6789/0 conn(0x7fec840cd930 legacy=0x7fec840d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:38.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.113+0000 7fec837fe640 1 -- 172.21.15.73:0/3198627461 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec840d1370 con 0x7fec84162810 2024-04-26T20:50:38.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.113+0000 7fec837fe640 1 --2- 172.21.15.73:0/3198627461 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fec84162810 0x7fec84070350 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fec740097a0 tx=0x7fec7402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c554837749bf751 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:38.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.113+0000 7fec827fc640 1 -- 172.21.15.73:0/3198627461 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec74037070 con 0x7fec84162810 2024-04-26T20:50:38.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.114+0000 7fec827fc640 1 -- 172.21.15.73:0/3198627461 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec74032440 con 0x7fec84162810 2024-04-26T20:50:38.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.114+0000 7fec827fc640 1 -- 172.21.15.73:0/3198627461 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec74032810 con 0x7fec84162810 2024-04-26T20:50:38.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.114+0000 7fec8a1c6640 1 -- 172.21.15.73:0/3198627461 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fec84162810 msgr2=0x7fec84070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:38.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.114+0000 7fec8a1c6640 1 --2- 172.21.15.73:0/3198627461 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fec84162810 0x7fec84070350 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fec740097a0 tx=0x7fec7402a180 comp rx=0 tx=0).stop 2024-04-26T20:50:38.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.114+0000 7fec8a1c6640 1 -- 172.21.15.73:0/3198627461 shutdown_connections 2024-04-26T20:50:38.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.114+0000 7fec8a1c6640 1 --2- 172.21.15.73:0/3198627461 >> v2:172.21.15.120:3300/0 conn(0x7fec840b2340 0x7fec840cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:38.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.114+0000 7fec8a1c6640 1 --2- 172.21.15.73:0/3198627461 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fec84162810 0x7fec84070350 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:38.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.114+0000 7fec8a1c6640 1 -- 172.21.15.73:0/3198627461 >> 172.21.15.73:0/3198627461 conn(0x7fec840d2da0 msgr2=0x7fec840d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:38.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.114+0000 7fec8a1c6640 1 -- 172.21.15.73:0/3198627461 shutdown_connections 2024-04-26T20:50:38.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.114+0000 7fec8a1c6640 1 -- 172.21.15.73:0/3198627461 wait complete. 2024-04-26T20:50:38.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.115+0000 7fec8a1c6640 1 Processor -- start 2024-04-26T20:50:38.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.115+0000 7fec8a1c6640 1 -- start start 2024-04-26T20:50:38.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.116+0000 7fec8a1c6640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fec840b2340 0x7fec8409f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:38.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.116+0000 7fec8a1c6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec840cd930 0x7fec8414e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:38.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.116+0000 7fec8a1c6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec84162810 0x7fec8409cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:38.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.116+0000 7fec8a1c6640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fec84070890 con 0x7fec84162810 2024-04-26T20:50:38.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.116+0000 7fec8a1c6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fec840d0470 con 0x7fec840cd930 2024-04-26T20:50:38.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.116+0000 7fec8a1c6640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fec840d2240 con 0x7fec840b2340 2024-04-26T20:50:38.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.116+0000 7fec83fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec84162810 0x7fec8409cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:38.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.116+0000 7fec83fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec84162810 0x7fec8409cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34822/0 (socket says 172.21.15.73:34822) 2024-04-26T20:50:38.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.116+0000 7fec83fff640 1 -- 172.21.15.73:0/2113059738 learned_addr learned my addr 172.21.15.73:0/2113059738 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:38.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.116+0000 7fec837fe640 1 --2- 172.21.15.73:0/2113059738 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fec840b2340 0x7fec8409f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:38.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.116+0000 7fec82ffd640 1 --2- 172.21.15.73:0/2113059738 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec840cd930 0x7fec8414e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:38.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.116+0000 7fec83fff640 1 -- 172.21.15.73:0/2113059738 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fec840b2340 msgr2=0x7fec8409f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:38.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.116+0000 7fec83fff640 1 --2- 172.21.15.73:0/2113059738 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fec840b2340 0x7fec8409f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:38.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.116+0000 7fec83fff640 1 -- 172.21.15.73:0/2113059738 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec840cd930 msgr2=0x7fec8414e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:38.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.116+0000 7fec83fff640 1 --2- 172.21.15.73:0/2113059738 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec840cd930 0x7fec8414e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:38.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.116+0000 7fec83fff640 1 -- 172.21.15.73:0/2113059738 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec8409f840 con 0x7fec84162810 2024-04-26T20:50:38.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.117+0000 7fec83fff640 1 --2- 172.21.15.73:0/2113059738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec84162810 0x7fec8409cca0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fec7c00c730 tx=0x7fec7c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:38.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.117+0000 7fec6bfff640 1 -- 172.21.15.73:0/2113059738 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec7c018070 con 0x7fec84162810 2024-04-26T20:50:38.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.117+0000 7fec6bfff640 1 -- 172.21.15.73:0/2113059738 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec7c004d60 con 0x7fec84162810 2024-04-26T20:50:38.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.117+0000 7fec6bfff640 1 -- 172.21.15.73:0/2113059738 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec7c00a880 con 0x7fec84162810 2024-04-26T20:50:38.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.117+0000 7fec8a1c6640 1 -- 172.21.15.73:0/2113059738 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec8409fb20 con 0x7fec84162810 2024-04-26T20:50:38.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.117+0000 7fec8a1c6640 1 -- 172.21.15.73:0/2113059738 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec840a00e0 con 0x7fec84162810 2024-04-26T20:50:38.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.118+0000 7fec8a1c6640 1 -- 172.21.15.73:0/2113059738 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec840694f0 con 0x7fec84162810 2024-04-26T20:50:38.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.118+0000 7fec6bfff640 1 -- 172.21.15.73:0/2113059738 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fec7c01b070 con 0x7fec84162810 2024-04-26T20:50:38.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.119+0000 7fec6bfff640 1 --2- 172.21.15.73:0/2113059738 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fec7006c8f0 0x7fec7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:38.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.119+0000 7fec6bfff640 1 -- 172.21.15.73:0/2113059738 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fec7c08b430 con 0x7fec84162810 2024-04-26T20:50:38.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.119+0000 7fec837fe640 1 --2- 172.21.15.73:0/2113059738 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fec7006c8f0 0x7fec7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:38.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.119+0000 7fec837fe640 1 --2- 172.21.15.73:0/2113059738 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fec7006c8f0 0x7fec7006edb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fec74034000 tx=0x7fec7402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:38.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.123+0000 7fec6bfff640 1 -- 172.21.15.73:0/2113059738 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fec7c058f40 con 0x7fec84162810 2024-04-26T20:50:38.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.303+0000 7fec8a1c6640 1 -- 172.21.15.73:0/2113059738 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fec840b18b0 con 0x7fec84162810 2024-04-26T20:50:38.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.304+0000 7fec6bfff640 1 -- 172.21.15.73:0/2113059738 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18527 (secure 0 0 0) 0x7fec7c05d240 con 0x7fec84162810 2024-04-26T20:50:38.305 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:50:38.305 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":24,"fsid":"fbd3f548-b547-4713-856b-09f98f5d1764","created":"2024-04-26T20:50:07.876786+0000","modified":"2024-04-26T20:50:37.007296+0000","last_up_change":"2024-04-26T20:50:19.082741+0000","last_in_change":"2024-04-26T20:50:10.018400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:50:19.993731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:50:29.964420+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:50:30.920082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:50:34.650207+0000","flags":32773,"flags_names":"hashpspool,ec_overwrites,creating","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"72d6f4f3-214a-4392-a02f-c33cbcab3c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6809","nonce":3461705124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6811","nonce":3461705124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6815","nonce":3461705124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6813","nonce":3461705124}]},"public_addr":"172.21.15.73:6809/3461705124","cluster_addr":"172.21.15.73:6811/3461705124","heartbeat_back_addr":"172.21.15.73:6815/3461705124","heartbeat_front_addr":"172.21.15.73:6813/3461705124","state":["exists","up"]},{"osd":1,"uuid":"786bf8ca-6ddc-4b3c-be6c-9152398bd5fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6817","nonce":2827619403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6819","nonce":2827619403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6825","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6827","nonce":2827619403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6821","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6823","nonce":2827619403}]},"public_addr":"172.21.15.73:6817/2827619403","cluster_addr":"172.21.15.73:6819/2827619403","heartbeat_back_addr":"172.21.15.73:6827/2827619403","heartbeat_front_addr":"172.21.15.73:6823/2827619403","state":["exists","up"]},{"osd":2,"uuid":"90963b1a-5437-4ef8-ad8b-6b74c534eeb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6801","nonce":2542681064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6803","nonce":2542681064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6808","nonce":2542681064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6805","nonce":2542681064}]},"public_addr":"172.21.15.73:6801/2542681064","cluster_addr":"172.21.15.73:6803/2542681064","heartbeat_back_addr":"172.21.15.73:6808/2542681064","heartbeat_front_addr":"172.21.15.73:6805/2542681064","state":["exists","up"]},{"osd":3,"uuid":"bd3d2148-6e38-41b4-a453-fe6e3b8b4ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6822","nonce":1604042718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6826","nonce":1604042718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6831","nonce":1604042718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6829","nonce":1604042718}]},"public_addr":"172.21.15.73:6822/1604042718","cluster_addr":"172.21.15.73:6826/1604042718","heartbeat_back_addr":"172.21.15.73:6831/1604042718","heartbeat_front_addr":"172.21.15.73:6829/1604042718","state":["exists","up"]},{"osd":4,"uuid":"e8668dc8-a23f-46d2-8311-cb882da08e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6817","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6819","nonce":3591055561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6824","nonce":3591055561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6831","nonce":3591055561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6828","nonce":3591055561}]},"public_addr":"172.21.15.120:6819/3591055561","cluster_addr":"172.21.15.120:6824/3591055561","heartbeat_back_addr":"172.21.15.120:6831/3591055561","heartbeat_front_addr":"172.21.15.120:6828/3591055561","state":["exists","up"]},{"osd":5,"uuid":"cbac0bdb-96bd-4852-bcce-e564495c3fcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6809","nonce":819450433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6811","nonce":819450433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6815","nonce":819450433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6813","nonce":819450433}]},"public_addr":"172.21.15.120:6809/819450433","cluster_addr":"172.21.15.120:6811/819450433","heartbeat_back_addr":"172.21.15.120:6815/819450433","heartbeat_front_addr":"172.21.15.120:6813/819450433","state":["exists","up"]},{"osd":6,"uuid":"1884b2a8-569f-4272-96a9-f25a09364dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6801","nonce":3946210727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6803","nonce":3946210727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6807","nonce":3946210727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6805","nonce":3946210727}]},"public_addr":"172.21.15.120:6801/3946210727","cluster_addr":"172.21.15.120:6803/3946210727","heartbeat_back_addr":"172.21.15.120:6807/3946210727","heartbeat_front_addr":"172.21.15.120:6805/3946210727","state":["exists","up"]},{"osd":7,"uuid":"c0c3bf6d-a4c5-4f1e-aa2d-330951830666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6818","nonce":2915574674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6821","nonce":2915574674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6827","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6829","nonce":2915574674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6823","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6825","nonce":2915574674}]},"public_addr":"172.21.15.120:6818/2915574674","cluster_addr":"172.21.15.120:6821/2915574674","heartbeat_back_addr":"172.21.15.120:6829/2915574674","heartbeat_front_addr":"172.21.15.120:6825/2915574674","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.218216+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.208710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.112705+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.182789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.164324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.174212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.225396+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.261559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:50:38.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.306+0000 7fec8a1c6640 1 -- 172.21.15.73:0/2113059738 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fec7006c8f0 msgr2=0x7fec7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:38.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.306+0000 7fec8a1c6640 1 --2- 172.21.15.73:0/2113059738 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fec7006c8f0 0x7fec7006edb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fec74034000 tx=0x7fec7402e000 comp rx=0 tx=0).stop 2024-04-26T20:50:38.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.306+0000 7fec8a1c6640 1 -- 172.21.15.73:0/2113059738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec84162810 msgr2=0x7fec8409cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:38.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.306+0000 7fec8a1c6640 1 --2- 172.21.15.73:0/2113059738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec84162810 0x7fec8409cca0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fec7c00c730 tx=0x7fec7c00cbf0 comp rx=0 tx=0).stop 2024-04-26T20:50:38.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.306+0000 7fec8a1c6640 1 -- 172.21.15.73:0/2113059738 shutdown_connections 2024-04-26T20:50:38.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.306+0000 7fec8a1c6640 1 --2- 172.21.15.73:0/2113059738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec84162810 0x7fec8409cca0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:38.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.306+0000 7fec8a1c6640 1 --2- 172.21.15.73:0/2113059738 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fec7006c8f0 0x7fec7006edb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:38.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.306+0000 7fec8a1c6640 1 --2- 172.21.15.73:0/2113059738 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec840cd930 0x7fec8414e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:38.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.306+0000 7fec8a1c6640 1 --2- 172.21.15.73:0/2113059738 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fec840b2340 0x7fec8409f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:38.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.306+0000 7fec8a1c6640 1 -- 172.21.15.73:0/2113059738 >> 172.21.15.73:0/2113059738 conn(0x7fec840d2da0 msgr2=0x7fec840cee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:38.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.306+0000 7fec8a1c6640 1 -- 172.21.15.73:0/2113059738 shutdown_connections 2024-04-26T20:50:38.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.306+0000 7fec8a1c6640 1 -- 172.21.15.73:0/2113059738 wait complete. 2024-04-26T20:50:38.318 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:50:38.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.412+0000 7fde17319640 1 Processor -- start 2024-04-26T20:50:38.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.413+0000 7fde17319640 1 -- start start 2024-04-26T20:50:38.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.413+0000 7fde17319640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fde100b0380 0x7fde100ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:38.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.413+0000 7fde17319640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde100cfd80 con 0x7fde100b10b0 2024-04-26T20:50:38.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.413+0000 7fde17319640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde100d0020 con 0x7fde1006c950 2024-04-26T20:50:38.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.413+0000 7fde17319640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fde100af8e0 con 0x7fde100b0380 2024-04-26T20:50:38.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.414+0000 7fde148b4640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fde100b10b0 0x7fde1006c240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:52024/0 (socket says 172.21.15.73:52024) 2024-04-26T20:50:38.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.414+0000 7fde148b4640 1 -- 172.21.15.73:0/2019493925 learned_addr learned my addr 172.21.15.73:0/2019493925 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:50:38.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.414+0000 7fde150b5640 1 --2- 172.21.15.73:0/2019493925 >> v2:172.21.15.120:3300/0 conn(0x7fde100b0380 0x7fde100ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:38.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.414+0000 7fde03fff640 1 -- 172.21.15.73:0/2019493925 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3600566660 0 0) 0x7fde100cfd80 con 0x7fde100b10b0 2024-04-26T20:50:38.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.414+0000 7fde03fff640 1 -- 172.21.15.73:0/2019493925 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fddfc003520 con 0x7fde100b10b0 2024-04-26T20:50:38.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.414+0000 7fde03fff640 1 -- 172.21.15.73:0/2019493925 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3137934441 0 0) 0x7fddfc003520 con 0x7fde100b10b0 2024-04-26T20:50:38.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.414+0000 7fde03fff640 1 -- 172.21.15.73:0/2019493925 >> v2:172.21.15.120:3300/0 conn(0x7fde100b0380 msgr2=0x7fde100ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:38.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.414+0000 7fde03fff640 1 --2- 172.21.15.73:0/2019493925 >> v2:172.21.15.120:3300/0 conn(0x7fde100b0380 0x7fde100ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:38.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.414+0000 7fde03fff640 1 -- 172.21.15.73:0/2019493925 >> v1:172.21.15.120:6789/0 conn(0x7fde1006c950 legacy=0x7fde1015e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:38.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.414+0000 7fde03fff640 1 -- 172.21.15.73:0/2019493925 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde100d0300 con 0x7fde100b10b0 2024-04-26T20:50:38.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.414+0000 7fde03fff640 1 -- 172.21.15.73:0/2019493925 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7fde0c001690 con 0x7fde100b10b0 2024-04-26T20:50:38.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.414+0000 7fde03fff640 1 -- 172.21.15.73:0/2019493925 >> v1:172.21.15.73:6789/0 conn(0x7fde100b10b0 legacy=0x7fde1006c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:38.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.414+0000 7fde03fff640 1 --2- 172.21.15.73:0/2019493925 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fddfc0039b0 0x7fddfc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:38.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.414+0000 7fde03fff640 1 --2- 172.21.15.73:0/2019493925 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fddfc004800 0x7fddfc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:38.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.414+0000 7fde03fff640 1 --2- 172.21.15.73:0/2019493925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddfc0051c0 0x7fddfc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:38.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.414+0000 7fde03fff640 1 -- 172.21.15.73:0/2019493925 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fde100af8e0 con 0x7fddfc0051c0 2024-04-26T20:50:38.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.414+0000 7fde03fff640 1 -- 172.21.15.73:0/2019493925 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fddfc00c160 con 0x7fddfc0039b0 2024-04-26T20:50:38.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.414+0000 7fde03fff640 1 -- 172.21.15.73:0/2019493925 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fddfc00c360 con 0x7fddfc004800 2024-04-26T20:50:38.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.414+0000 7fde158b6640 1 --2- 172.21.15.73:0/2019493925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddfc0051c0 0x7fddfc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:38.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.414+0000 7fde150b5640 1 --2- 172.21.15.73:0/2019493925 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fddfc0039b0 0x7fddfc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:38.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.414+0000 7fde148b4640 1 --2- 172.21.15.73:0/2019493925 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fddfc004800 0x7fddfc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:38.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.414+0000 7fde158b6640 1 -- 172.21.15.73:0/2019493925 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fddfc004800 msgr2=0x7fddfc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:38.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.414+0000 7fde158b6640 1 --2- 172.21.15.73:0/2019493925 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fddfc004800 0x7fddfc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:38.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.414+0000 7fde158b6640 1 -- 172.21.15.73:0/2019493925 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fddfc0039b0 msgr2=0x7fddfc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:38.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.414+0000 7fde158b6640 1 --2- 172.21.15.73:0/2019493925 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fddfc0039b0 0x7fddfc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:38.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.414+0000 7fde158b6640 1 -- 172.21.15.73:0/2019493925 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddfc003520 con 0x7fddfc0051c0 2024-04-26T20:50:38.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.415+0000 7fde158b6640 1 -- 172.21.15.73:0/2019493925 reap_dead start 2024-04-26T20:50:38.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.415+0000 7fde158b6640 1 --2- 172.21.15.73:0/2019493925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddfc0051c0 0x7fddfc00b9c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fde08001e20 tx=0x7fddfc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7e15633d643055f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:38.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.415+0000 7fde03fff640 1 -- 172.21.15.73:0/2019493925 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde08001a70 con 0x7fddfc0051c0 2024-04-26T20:50:38.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.415+0000 7fde03fff640 1 -- 172.21.15.73:0/2019493925 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde0802ae40 con 0x7fddfc0051c0 2024-04-26T20:50:38.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.415+0000 7fde03fff640 1 -- 172.21.15.73:0/2019493925 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde08003bb0 con 0x7fddfc0051c0 2024-04-26T20:50:38.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.416+0000 7fde17319640 1 -- 172.21.15.73:0/2019493925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddfc0051c0 msgr2=0x7fddfc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:38.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.416+0000 7fde17319640 1 --2- 172.21.15.73:0/2019493925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddfc0051c0 0x7fddfc00b9c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fde08001e20 tx=0x7fddfc005830 comp rx=0 tx=0).stop 2024-04-26T20:50:38.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.416+0000 7fde17319640 1 -- 172.21.15.73:0/2019493925 shutdown_connections 2024-04-26T20:50:38.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.416+0000 7fde17319640 1 --2- 172.21.15.73:0/2019493925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddfc0051c0 0x7fddfc00b9c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:38.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.416+0000 7fde17319640 1 -- 172.21.15.73:0/2019493925 >> 172.21.15.73:0/2019493925 conn(0x7fde100d1cf0 msgr2=0x7fde100d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:38.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.416+0000 7fde17319640 1 -- 172.21.15.73:0/2019493925 shutdown_connections 2024-04-26T20:50:38.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.416+0000 7fde17319640 1 -- 172.21.15.73:0/2019493925 wait complete. 2024-04-26T20:50:38.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.417+0000 7fde17319640 1 Processor -- start 2024-04-26T20:50:38.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.417+0000 7fde17319640 1 -- start start 2024-04-26T20:50:38.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.418+0000 7fde17319640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddfc0051c0 0x7fde10160400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:38.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.418+0000 7fde17319640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fde100b6f50 0x7fde100b73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:38.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.418+0000 7fde17319640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde100b29a0 0x7fde100b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:38.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.418+0000 7fde17319640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fde100b0760 con 0x7fddfc0051c0 2024-04-26T20:50:38.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.418+0000 7fde17319640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fde100d1980 con 0x7fde100b29a0 2024-04-26T20:50:38.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.418+0000 7fde17319640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fde100d15c0 con 0x7fde100b6f50 2024-04-26T20:50:38.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.418+0000 7fde150b5640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddfc0051c0 0x7fde10160400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:38.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.418+0000 7fde150b5640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddfc0051c0 0x7fde10160400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34848/0 (socket says 172.21.15.73:34848) 2024-04-26T20:50:38.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.418+0000 7fde150b5640 1 -- 172.21.15.73:0/3395815927 learned_addr learned my addr 172.21.15.73:0/3395815927 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:38.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.418+0000 7fde148b4640 1 --2- 172.21.15.73:0/3395815927 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fde100b6f50 0x7fde100b73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:38.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.418+0000 7fde158b6640 1 --2- 172.21.15.73:0/3395815927 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde100b29a0 0x7fde100b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:38.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.418+0000 7fde150b5640 1 -- 172.21.15.73:0/3395815927 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fde100b6f50 msgr2=0x7fde100b73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:38.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.418+0000 7fde150b5640 1 --2- 172.21.15.73:0/3395815927 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fde100b6f50 0x7fde100b73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:38.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.418+0000 7fde150b5640 1 -- 172.21.15.73:0/3395815927 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde100b29a0 msgr2=0x7fde100b2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:38.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.418+0000 7fde150b5640 1 --2- 172.21.15.73:0/3395815927 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde100b29a0 0x7fde100b2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:38.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.418+0000 7fde150b5640 1 -- 172.21.15.73:0/3395815927 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde10160940 con 0x7fddfc0051c0 2024-04-26T20:50:38.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.418+0000 7fde150b5640 1 --2- 172.21.15.73:0/3395815927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddfc0051c0 0x7fde10160400 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fde04011910 tx=0x7fde04011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:38.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.418+0000 7fde01ffb640 1 -- 172.21.15.73:0/3395815927 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde0400fa30 con 0x7fddfc0051c0 2024-04-26T20:50:38.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.418+0000 7fde01ffb640 1 -- 172.21.15.73:0/3395815927 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde0400fbd0 con 0x7fddfc0051c0 2024-04-26T20:50:38.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.418+0000 7fde17319640 1 -- 172.21.15.73:0/3395815927 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde101f50e0 con 0x7fddfc0051c0 2024-04-26T20:50:38.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.418+0000 7fde01ffb640 1 -- 172.21.15.73:0/3395815927 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde04014440 con 0x7fddfc0051c0 2024-04-26T20:50:38.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.419+0000 7fde17319640 1 -- 172.21.15.73:0/3395815927 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde101f56a0 con 0x7fddfc0051c0 2024-04-26T20:50:38.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.420+0000 7fde17319640 1 -- 172.21.15.73:0/3395815927 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde100694f0 con 0x7fddfc0051c0 2024-04-26T20:50:38.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.420+0000 7fde01ffb640 1 -- 172.21.15.73:0/3395815927 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fde04012040 con 0x7fddfc0051c0 2024-04-26T20:50:38.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.420+0000 7fde01ffb640 1 --2- 172.21.15.73:0/3395815927 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fddf406c8f0 0x7fddf406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:38.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.420+0000 7fde01ffb640 1 -- 172.21.15.73:0/3395815927 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fde0408c370 con 0x7fddfc0051c0 2024-04-26T20:50:38.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.420+0000 7fde148b4640 1 --2- 172.21.15.73:0/3395815927 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fddf406c8f0 0x7fddf406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:38.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.421+0000 7fde148b4640 1 --2- 172.21.15.73:0/3395815927 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fddf406c8f0 0x7fddf406edb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fde100b7a00 tx=0x7fde0c001b30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:38.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.425+0000 7fde01ffb640 1 -- 172.21.15.73:0/3395815927 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fde10160940 con 0x7fddfc0051c0 2024-04-26T20:50:38.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.600+0000 7fde17319640 1 -- 172.21.15.73:0/3395815927 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fde100cc7f0 con 0x7fddfc0051c0 2024-04-26T20:50:38.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.601+0000 7fde01ffb640 1 -- 172.21.15.73:0/3395815927 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18527 (secure 0 0 0) 0x7fde04019030 con 0x7fddfc0051c0 2024-04-26T20:50:38.602 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:50:38.602 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":24,"fsid":"fbd3f548-b547-4713-856b-09f98f5d1764","created":"2024-04-26T20:50:07.876786+0000","modified":"2024-04-26T20:50:37.007296+0000","last_up_change":"2024-04-26T20:50:19.082741+0000","last_in_change":"2024-04-26T20:50:10.018400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:50:19.993731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:50:29.964420+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:50:30.920082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:50:34.650207+0000","flags":32773,"flags_names":"hashpspool,ec_overwrites,creating","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"72d6f4f3-214a-4392-a02f-c33cbcab3c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6809","nonce":3461705124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6811","nonce":3461705124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6815","nonce":3461705124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6813","nonce":3461705124}]},"public_addr":"172.21.15.73:6809/3461705124","cluster_addr":"172.21.15.73:6811/3461705124","heartbeat_back_addr":"172.21.15.73:6815/3461705124","heartbeat_front_addr":"172.21.15.73:6813/3461705124","state":["exists","up"]},{"osd":1,"uuid":"786bf8ca-6ddc-4b3c-be6c-9152398bd5fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6817","nonce":2827619403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6819","nonce":2827619403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6825","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6827","nonce":2827619403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6821","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6823","nonce":2827619403}]},"public_addr":"172.21.15.73:6817/2827619403","cluster_addr":"172.21.15.73:6819/2827619403","heartbeat_back_addr":"172.21.15.73:6827/2827619403","heartbeat_front_addr":"172.21.15.73:6823/2827619403","state":["exists","up"]},{"osd":2,"uuid":"90963b1a-5437-4ef8-ad8b-6b74c534eeb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6801","nonce":2542681064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6803","nonce":2542681064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6808","nonce":2542681064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6805","nonce":2542681064}]},"public_addr":"172.21.15.73:6801/2542681064","cluster_addr":"172.21.15.73:6803/2542681064","heartbeat_back_addr":"172.21.15.73:6808/2542681064","heartbeat_front_addr":"172.21.15.73:6805/2542681064","state":["exists","up"]},{"osd":3,"uuid":"bd3d2148-6e38-41b4-a453-fe6e3b8b4ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6822","nonce":1604042718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6826","nonce":1604042718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6831","nonce":1604042718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6829","nonce":1604042718}]},"public_addr":"172.21.15.73:6822/1604042718","cluster_addr":"172.21.15.73:6826/1604042718","heartbeat_back_addr":"172.21.15.73:6831/1604042718","heartbeat_front_addr":"172.21.15.73:6829/1604042718","state":["exists","up"]},{"osd":4,"uuid":"e8668dc8-a23f-46d2-8311-cb882da08e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6817","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6819","nonce":3591055561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6824","nonce":3591055561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6831","nonce":3591055561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6828","nonce":3591055561}]},"public_addr":"172.21.15.120:6819/3591055561","cluster_addr":"172.21.15.120:6824/3591055561","heartbeat_back_addr":"172.21.15.120:6831/3591055561","heartbeat_front_addr":"172.21.15.120:6828/3591055561","state":["exists","up"]},{"osd":5,"uuid":"cbac0bdb-96bd-4852-bcce-e564495c3fcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6809","nonce":819450433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6811","nonce":819450433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6815","nonce":819450433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6813","nonce":819450433}]},"public_addr":"172.21.15.120:6809/819450433","cluster_addr":"172.21.15.120:6811/819450433","heartbeat_back_addr":"172.21.15.120:6815/819450433","heartbeat_front_addr":"172.21.15.120:6813/819450433","state":["exists","up"]},{"osd":6,"uuid":"1884b2a8-569f-4272-96a9-f25a09364dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6801","nonce":3946210727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6803","nonce":3946210727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6807","nonce":3946210727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6805","nonce":3946210727}]},"public_addr":"172.21.15.120:6801/3946210727","cluster_addr":"172.21.15.120:6803/3946210727","heartbeat_back_addr":"172.21.15.120:6807/3946210727","heartbeat_front_addr":"172.21.15.120:6805/3946210727","state":["exists","up"]},{"osd":7,"uuid":"c0c3bf6d-a4c5-4f1e-aa2d-330951830666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6818","nonce":2915574674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6821","nonce":2915574674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6827","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6829","nonce":2915574674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6823","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6825","nonce":2915574674}]},"public_addr":"172.21.15.120:6818/2915574674","cluster_addr":"172.21.15.120:6821/2915574674","heartbeat_back_addr":"172.21.15.120:6829/2915574674","heartbeat_front_addr":"172.21.15.120:6825/2915574674","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.218216+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.208710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.112705+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.182789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.164324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.174212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.225396+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.261559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:50:38.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.603+0000 7fde17319640 1 -- 172.21.15.73:0/3395815927 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fddf406c8f0 msgr2=0x7fddf406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:38.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.603+0000 7fde17319640 1 --2- 172.21.15.73:0/3395815927 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fddf406c8f0 0x7fddf406edb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fde100b7a00 tx=0x7fde0c001b30 comp rx=0 tx=0).stop 2024-04-26T20:50:38.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.603+0000 7fde17319640 1 -- 172.21.15.73:0/3395815927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddfc0051c0 msgr2=0x7fde10160400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:38.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.603+0000 7fde17319640 1 --2- 172.21.15.73:0/3395815927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddfc0051c0 0x7fde10160400 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fde04011910 tx=0x7fde04011d10 comp rx=0 tx=0).stop 2024-04-26T20:50:38.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.603+0000 7fde17319640 1 -- 172.21.15.73:0/3395815927 shutdown_connections 2024-04-26T20:50:38.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.603+0000 7fde17319640 1 --2- 172.21.15.73:0/3395815927 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fddf406c8f0 0x7fddf406edb0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:38.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.603+0000 7fde17319640 1 --2- 172.21.15.73:0/3395815927 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fde100b29a0 0x7fde100b2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:38.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.603+0000 7fde17319640 1 --2- 172.21.15.73:0/3395815927 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fde100b6f50 0x7fde100b73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:38.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.603+0000 7fde17319640 1 --2- 172.21.15.73:0/3395815927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddfc0051c0 0x7fde10160400 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:38.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.603+0000 7fde17319640 1 -- 172.21.15.73:0/3395815927 >> 172.21.15.73:0/3395815927 conn(0x7fde100d1cf0 msgr2=0x7fde100ccd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:38.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.603+0000 7fde17319640 1 -- 172.21.15.73:0/3395815927 shutdown_connections 2024-04-26T20:50:38.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.603+0000 7fde17319640 1 -- 172.21.15.73:0/3395815927 wait complete. 2024-04-26T20:50:38.629 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:50:38.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.728+0000 7f59ce9e7640 1 Processor -- start 2024-04-26T20:50:38.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.728+0000 7f59ce9e7640 1 -- start start 2024-04-26T20:50:38.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.729+0000 7f59ce9e7640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f59c80cf0e0 0x7f59c806f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:38.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.729+0000 7f59ce9e7640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59c815b9e0 con 0x7f59c815b600 2024-04-26T20:50:38.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.729+0000 7f59ce9e7640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59c80d0740 con 0x7f59c80cf5e0 2024-04-26T20:50:38.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.729+0000 7f59ce9e7640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f59c80cb9b0 con 0x7f59c80cf0e0 2024-04-26T20:50:38.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.729+0000 7f59c7fff640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f59c815b600 0x7f59c8071f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:52034/0 (socket says 172.21.15.73:52034) 2024-04-26T20:50:38.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.729+0000 7f59c7fff640 1 -- 172.21.15.73:0/959783239 learned_addr learned my addr 172.21.15.73:0/959783239 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:50:38.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.729+0000 7f59ccf84640 1 --2- 172.21.15.73:0/959783239 >> v2:172.21.15.120:3300/0 conn(0x7f59c80cf0e0 0x7f59c806f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:38.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.729+0000 7f59c6ffd640 1 -- 172.21.15.73:0/959783239 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3741543472 0 0) 0x7f59c815b9e0 con 0x7f59c815b600 2024-04-26T20:50:38.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.729+0000 7f59c6ffd640 1 -- 172.21.15.73:0/959783239 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f59b0003520 con 0x7f59c815b600 2024-04-26T20:50:38.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.729+0000 7f59c6ffd640 1 -- 172.21.15.73:0/959783239 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3649273778 0 0) 0x7f59b0003520 con 0x7f59c815b600 2024-04-26T20:50:38.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.729+0000 7f59c6ffd640 1 -- 172.21.15.73:0/959783239 >> v2:172.21.15.120:3300/0 conn(0x7f59c80cf0e0 msgr2=0x7f59c806f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:38.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.729+0000 7f59c6ffd640 1 --2- 172.21.15.73:0/959783239 >> v2:172.21.15.120:3300/0 conn(0x7f59c80cf0e0 0x7f59c806f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:38.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.729+0000 7f59c6ffd640 1 -- 172.21.15.73:0/959783239 >> v1:172.21.15.120:6789/0 conn(0x7f59c80cf5e0 legacy=0x7f59c80ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:38.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.729+0000 7f59c6ffd640 1 -- 172.21.15.73:0/959783239 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59c80cb360 con 0x7f59c815b600 2024-04-26T20:50:38.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.729+0000 7f59c6ffd640 1 -- 172.21.15.73:0/959783239 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7f59b8001690 con 0x7f59c815b600 2024-04-26T20:50:38.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.729+0000 7f59c6ffd640 1 -- 172.21.15.73:0/959783239 >> v1:172.21.15.73:6789/0 conn(0x7f59c815b600 legacy=0x7f59c8071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:38.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.729+0000 7f59c6ffd640 1 --2- 172.21.15.73:0/959783239 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59b00039b0 0x7f59b0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:38.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.729+0000 7f59c6ffd640 1 --2- 172.21.15.73:0/959783239 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59b0004800 0x7f59b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:38.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.729+0000 7f59c6ffd640 1 --2- 172.21.15.73:0/959783239 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f59b00051c0 0x7f59b000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:38.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.729+0000 7f59c6ffd640 1 -- 172.21.15.73:0/959783239 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f59c80cb9b0 con 0x7f59b0004800 2024-04-26T20:50:38.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.729+0000 7f59c6ffd640 1 -- 172.21.15.73:0/959783239 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f59b000c060 con 0x7f59b00039b0 2024-04-26T20:50:38.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.729+0000 7f59c6ffd640 1 -- 172.21.15.73:0/959783239 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f59b000c260 con 0x7f59b00051c0 2024-04-26T20:50:38.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.730+0000 7f59c77fe640 1 --2- 172.21.15.73:0/959783239 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59b0004800 0x7f59b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:38.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.730+0000 7f59c7fff640 1 --2- 172.21.15.73:0/959783239 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59b00039b0 0x7f59b0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:38.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.730+0000 7f59ccf84640 1 --2- 172.21.15.73:0/959783239 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f59b00051c0 0x7f59b000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:38.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.730+0000 7f59c77fe640 1 -- 172.21.15.73:0/959783239 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f59b00051c0 msgr2=0x7f59b000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:38.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.730+0000 7f59c77fe640 1 --2- 172.21.15.73:0/959783239 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f59b00051c0 0x7f59b000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:38.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.730+0000 7f59c77fe640 1 -- 172.21.15.73:0/959783239 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59b00039b0 msgr2=0x7f59b0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:38.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.730+0000 7f59c77fe640 1 --2- 172.21.15.73:0/959783239 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59b00039b0 0x7f59b0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:38.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.730+0000 7f59c77fe640 1 -- 172.21.15.73:0/959783239 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59b0003520 con 0x7f59b0004800 2024-04-26T20:50:38.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.730+0000 7f59ccf84640 1 --2- 172.21.15.73:0/959783239 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f59b00051c0 0x7f59b000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:50:38.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.730+0000 7f59ccf84640 1 -- 172.21.15.73:0/959783239 reap_dead start 2024-04-26T20:50:38.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.730+0000 7f59c77fe640 1 --2- 172.21.15.73:0/959783239 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59b0004800 0x7f59b0004c80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f59bc001750 tx=0x7f59b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d45c0ccb7d2b3fa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:38.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.731+0000 7f59c6ffd640 1 -- 172.21.15.73:0/959783239 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59bc001370 con 0x7f59b0004800 2024-04-26T20:50:38.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.731+0000 7f59c6ffd640 1 -- 172.21.15.73:0/959783239 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59bc0052d0 con 0x7f59b0004800 2024-04-26T20:50:38.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.731+0000 7f59c6ffd640 1 -- 172.21.15.73:0/959783239 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59bc02abc0 con 0x7f59b0004800 2024-04-26T20:50:38.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.731+0000 7f59ce9e7640 1 -- 172.21.15.73:0/959783239 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59b0004800 msgr2=0x7f59b0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:38.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.731+0000 7f59ce9e7640 1 --2- 172.21.15.73:0/959783239 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59b0004800 0x7f59b0004c80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f59bc001750 tx=0x7f59b0005830 comp rx=0 tx=0).stop 2024-04-26T20:50:38.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.731+0000 7f59ce9e7640 1 -- 172.21.15.73:0/959783239 shutdown_connections 2024-04-26T20:50:38.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.731+0000 7f59ce9e7640 1 --2- 172.21.15.73:0/959783239 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59b0004800 0x7f59b0004c80 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:38.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.731+0000 7f59ce9e7640 1 -- 172.21.15.73:0/959783239 >> 172.21.15.73:0/959783239 conn(0x7f59c80d0be0 msgr2=0x7f59c80d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:38.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.731+0000 7f59ce9e7640 1 -- 172.21.15.73:0/959783239 shutdown_connections 2024-04-26T20:50:38.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.731+0000 7f59ce9e7640 1 -- 172.21.15.73:0/959783239 wait complete. 2024-04-26T20:50:38.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.732+0000 7f59ce9e7640 1 Processor -- start 2024-04-26T20:50:38.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.732+0000 7f59ce9e7640 1 -- start start 2024-04-26T20:50:38.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.733+0000 7f59ce9e7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59b0004800 0x7f59c814c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:38.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.733+0000 7f59ce9e7640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59c814f1c0 0x7f59c815bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:38.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.733+0000 7f59ce9e7640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f59c80cd580 0x7f59c80cda30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:38.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.733+0000 7f59ce9e7640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f59c80d0470 con 0x7f59c814f1c0 2024-04-26T20:50:38.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.733+0000 7f59ce9e7640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f59c80d02f0 con 0x7f59b0004800 2024-04-26T20:50:38.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.733+0000 7f59ce9e7640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f59c80cbbd0 con 0x7f59c80cd580 2024-04-26T20:50:38.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.733+0000 7f59c77fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59c814f1c0 0x7f59c815bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:38.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.733+0000 7f59c77fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59c814f1c0 0x7f59c815bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34864/0 (socket says 172.21.15.73:34864) 2024-04-26T20:50:38.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.733+0000 7f59c77fe640 1 -- 172.21.15.73:0/17943618 learned_addr learned my addr 172.21.15.73:0/17943618 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:38.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.733+0000 7f59c77fe640 1 -- 172.21.15.73:0/17943618 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f59c80cd580 msgr2=0x7f59c80cda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:38.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.733+0000 7f59c77fe640 1 --2- 172.21.15.73:0/17943618 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f59c80cd580 0x7f59c80cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:38.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.733+0000 7f59c77fe640 1 -- 172.21.15.73:0/17943618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59b0004800 msgr2=0x7f59c814c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:38.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.733+0000 7f59c77fe640 1 --2- 172.21.15.73:0/17943618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59b0004800 0x7f59c814c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:38.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.733+0000 7f59c77fe640 1 -- 172.21.15.73:0/17943618 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59c815c470 con 0x7f59c814f1c0 2024-04-26T20:50:38.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.734+0000 7f59c77fe640 1 --2- 172.21.15.73:0/17943618 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59c814f1c0 0x7f59c815bbe0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f59bc02a570 tx=0x7f59bc001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:38.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.734+0000 7f59c4ff9640 1 -- 172.21.15.73:0/17943618 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59bc03f030 con 0x7f59c814f1c0 2024-04-26T20:50:38.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.734+0000 7f59c4ff9640 1 -- 172.21.15.73:0/17943618 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59bc005480 con 0x7f59c814f1c0 2024-04-26T20:50:38.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.734+0000 7f59c4ff9640 1 -- 172.21.15.73:0/17943618 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59bc037040 con 0x7f59c814f1c0 2024-04-26T20:50:38.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.734+0000 7f59ce9e7640 1 -- 172.21.15.73:0/17943618 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59c815c750 con 0x7f59c814f1c0 2024-04-26T20:50:38.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.734+0000 7f59ce9e7640 1 -- 172.21.15.73:0/17943618 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59c815cd10 con 0x7f59c814f1c0 2024-04-26T20:50:38.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.735+0000 7f59ce9e7640 1 -- 172.21.15.73:0/17943618 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59c80694f0 con 0x7f59c814f1c0 2024-04-26T20:50:38.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.735+0000 7f59c4ff9640 1 -- 172.21.15.73:0/17943618 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f59bc042070 con 0x7f59c814f1c0 2024-04-26T20:50:38.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.735+0000 7f59c4ff9640 1 --2- 172.21.15.73:0/17943618 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f59b406c920 0x7f59b406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:38.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.736+0000 7f59c4ff9640 1 -- 172.21.15.73:0/17943618 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f59bc007640 con 0x7f59c814f1c0 2024-04-26T20:50:38.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.736+0000 7f59c7fff640 1 --2- 172.21.15.73:0/17943618 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f59b406c920 0x7f59b406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:38.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.736+0000 7f59c7fff640 1 --2- 172.21.15.73:0/17943618 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f59b406c920 0x7f59b406ede0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f59b8001050 tx=0x7f59b8001af0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:38.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.740+0000 7f59c4ff9640 1 -- 172.21.15.73:0/17943618 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f59bc0795c0 con 0x7f59c814f1c0 2024-04-26T20:50:38.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.953+0000 7f59ce9e7640 1 -- 172.21.15.73:0/17943618 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f59c80ce7b0 con 0x7f59c814f1c0 2024-04-26T20:50:38.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.953+0000 7f59c4ff9640 1 -- 172.21.15.73:0/17943618 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5180 (secure 0 0 0) 0x7f59bc078ec0 con 0x7f59c814f1c0 2024-04-26T20:50:38.954 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:50:38.954 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/4263167680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6833","nonce":4263167680},{"type":"v1","addr":"172.21.15.120:6835","nonce":4263167680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6834/1614342510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1614342510},{"type":"v1","addr":"172.21.15.120:6834","nonce":1614342510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2146325725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2146325725},{"type":"v1","addr":"172.21.15.73:6835","nonce":2146325725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:50:31.964317+0000","modified":"2024-04-26T20:50:37.007492+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/181202544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":181202544},{"type":"v1","addr":"172.21.15.73:6837","nonce":181202544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4281,"qdb_cluster":[4281]},"id":1}]} 2024-04-26T20:50:38.955 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 6 2024-04-26T20:50:38.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.955+0000 7f59ce9e7640 1 -- 172.21.15.73:0/17943618 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f59b406c920 msgr2=0x7f59b406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:38.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.955+0000 7f59ce9e7640 1 --2- 172.21.15.73:0/17943618 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f59b406c920 0x7f59b406ede0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f59b8001050 tx=0x7f59b8001af0 comp rx=0 tx=0).stop 2024-04-26T20:50:38.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.955+0000 7f59ce9e7640 1 -- 172.21.15.73:0/17943618 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59c814f1c0 msgr2=0x7f59c815bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:38.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.955+0000 7f59ce9e7640 1 --2- 172.21.15.73:0/17943618 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59c814f1c0 0x7f59c815bbe0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f59bc02a570 tx=0x7f59bc001120 comp rx=0 tx=0).stop 2024-04-26T20:50:38.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.955+0000 7f59ce9e7640 1 -- 172.21.15.73:0/17943618 shutdown_connections 2024-04-26T20:50:38.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.955+0000 7f59ce9e7640 1 --2- 172.21.15.73:0/17943618 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59c814f1c0 0x7f59c815bbe0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:38.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.955+0000 7f59ce9e7640 1 --2- 172.21.15.73:0/17943618 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f59b406c920 0x7f59b406ede0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:38.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.955+0000 7f59ce9e7640 1 --2- 172.21.15.73:0/17943618 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f59c80cd580 0x7f59c80cda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:38.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.955+0000 7f59ce9e7640 1 --2- 172.21.15.73:0/17943618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f59b0004800 0x7f59c814c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:38.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.955+0000 7f59ce9e7640 1 -- 172.21.15.73:0/17943618 >> 172.21.15.73:0/17943618 conn(0x7f59c80d0be0 msgr2=0x7f59c815a670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:38.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.955+0000 7f59ce9e7640 1 -- 172.21.15.73:0/17943618 shutdown_connections 2024-04-26T20:50:38.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:38.955+0000 7f59ce9e7640 1 -- 172.21.15.73:0/17943618 wait complete. 2024-04-26T20:50:38.967 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:50:39.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.062+0000 7f3aa7133640 1 Processor -- start 2024-04-26T20:50:39.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.062+0000 7f3aa7133640 1 -- start start 2024-04-26T20:50:39.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.063+0000 7f3aa7133640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3aa00b2340 0x7f3aa00cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:39.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.063+0000 7f3aa7133640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f3aa00cd930 0x7f3aa00d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:39.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.063+0000 7f3aa7133640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3aa00d10d0 con 0x7f3aa0162810 2024-04-26T20:50:39.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.063+0000 7f3aa7133640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f3aa00d0750 con 0x7f3aa00b2340 2024-04-26T20:50:39.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.063+0000 7f3aa7133640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f3aa00d16a0 con 0x7f3aa00cd930 2024-04-26T20:50:39.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.063+0000 7f3a9ffff640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3aa00b2340 0x7f3aa00cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:39.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.063+0000 7f3aa4ecf640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f3aa0162810 0x7f3aa0070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:52042/0 (socket says 172.21.15.73:52042) 2024-04-26T20:50:39.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.063+0000 7f3a9ffff640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3aa00b2340 0x7f3aa00cd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34876/0 (socket says 172.21.15.73:34876) 2024-04-26T20:50:39.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.063+0000 7f3aa4ecf640 1 -- 172.21.15.73:0/1783620606 learned_addr learned my addr 172.21.15.73:0/1783620606 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:50:39.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.063+0000 7f3aa56d0640 1 --2- 172.21.15.73:0/1783620606 >> v2:172.21.15.120:3300/0 conn(0x7f3aa00cd930 0x7f3aa00d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:39.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.063+0000 7f3a9ffff640 1 -- 172.21.15.73:0/1783620606 >> v2:172.21.15.120:3300/0 conn(0x7f3aa00cd930 msgr2=0x7f3aa00d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:39.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.063+0000 7f3a9ffff640 1 --2- 172.21.15.73:0/1783620606 >> v2:172.21.15.120:3300/0 conn(0x7f3aa00cd930 0x7f3aa00d1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:39.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.063+0000 7f3a9ffff640 1 -- 172.21.15.73:0/1783620606 >> v1:172.21.15.73:6789/0 conn(0x7f3aa0162810 legacy=0x7f3aa0070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:39.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.063+0000 7f3a9f7fe640 1 -- 172.21.15.73:0/1783620606 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2011083287 0 0) 0x7f3aa00d10d0 con 0x7f3aa0162810 2024-04-26T20:50:39.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.063+0000 7f3a9ffff640 1 -- 172.21.15.73:0/1783620606 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aa00d1370 con 0x7f3aa00b2340 2024-04-26T20:50:39.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.064+0000 7f3a9ffff640 1 --2- 172.21.15.73:0/1783620606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3aa00b2340 0x7f3aa00cd3f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3aa00b2280 tx=0x7f3a8c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18c4b9c653eea1f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:39.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.064+0000 7f3a9f7fe640 1 -- 172.21.15.73:0/1783620606 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c037070 con 0x7f3aa00b2340 2024-04-26T20:50:39.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.064+0000 7f3a9f7fe640 1 -- 172.21.15.73:0/1783620606 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a8c032440 con 0x7f3aa00b2340 2024-04-26T20:50:39.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.064+0000 7f3a9f7fe640 1 -- 172.21.15.73:0/1783620606 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c032810 con 0x7f3aa00b2340 2024-04-26T20:50:39.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.064+0000 7f3aa7133640 1 -- 172.21.15.73:0/1783620606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3aa00b2340 msgr2=0x7f3aa00cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:39.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.064+0000 7f3aa7133640 1 --2- 172.21.15.73:0/1783620606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3aa00b2340 0x7f3aa00cd3f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3aa00b2280 tx=0x7f3a8c02a1a0 comp rx=0 tx=0).stop 2024-04-26T20:50:39.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.064+0000 7f3aa7133640 1 -- 172.21.15.73:0/1783620606 shutdown_connections 2024-04-26T20:50:39.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.064+0000 7f3aa7133640 1 --2- 172.21.15.73:0/1783620606 >> v2:172.21.15.120:3300/0 conn(0x7f3aa00cd930 0x7f3aa00d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:39.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.064+0000 7f3aa7133640 1 --2- 172.21.15.73:0/1783620606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3aa00b2340 0x7f3aa00cd3f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:39.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.064+0000 7f3aa7133640 1 -- 172.21.15.73:0/1783620606 >> 172.21.15.73:0/1783620606 conn(0x7f3aa00d2da0 msgr2=0x7f3aa00d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:39.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.064+0000 7f3aa7133640 1 -- 172.21.15.73:0/1783620606 shutdown_connections 2024-04-26T20:50:39.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.064+0000 7f3aa7133640 1 -- 172.21.15.73:0/1783620606 wait complete. 2024-04-26T20:50:39.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.066+0000 7f3aa7133640 1 Processor -- start 2024-04-26T20:50:39.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.066+0000 7f3aa7133640 1 -- start start 2024-04-26T20:50:39.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.066+0000 7f3aa7133640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3aa00b2340 0x7f3aa00c6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:39.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.066+0000 7f3aa7133640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3aa00cd930 0x7f3aa00c9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:39.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.066+0000 7f3aa7133640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3aa0162810 0x7f3aa00b0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:39.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.066+0000 7f3aa7133640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa00d2070 con 0x7f3aa00b2340 2024-04-26T20:50:39.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.066+0000 7f3aa7133640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa00d0470 con 0x7f3aa0162810 2024-04-26T20:50:39.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.066+0000 7f3aa7133640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f3aa00d21f0 con 0x7f3aa00cd930 2024-04-26T20:50:39.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.066+0000 7f3aa4ecf640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3aa00b2340 0x7f3aa00c6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:39.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.066+0000 7f3aa4ecf640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3aa00b2340 0x7f3aa00c6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34892/0 (socket says 172.21.15.73:34892) 2024-04-26T20:50:39.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.066+0000 7f3aa4ecf640 1 -- 172.21.15.73:0/866540685 learned_addr learned my addr 172.21.15.73:0/866540685 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:39.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.067+0000 7f3a9ffff640 1 --2- 172.21.15.73:0/866540685 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3aa00cd930 0x7f3aa00c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:39.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.067+0000 7f3aa56d0640 1 --2- 172.21.15.73:0/866540685 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3aa0162810 0x7f3aa00b0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:39.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.067+0000 7f3aa4ecf640 1 -- 172.21.15.73:0/866540685 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3aa00cd930 msgr2=0x7f3aa00c9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:39.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.067+0000 7f3aa4ecf640 1 --2- 172.21.15.73:0/866540685 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3aa00cd930 0x7f3aa00c9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:39.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.067+0000 7f3aa4ecf640 1 -- 172.21.15.73:0/866540685 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3aa0162810 msgr2=0x7f3aa00b0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:39.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.067+0000 7f3aa4ecf640 1 --2- 172.21.15.73:0/866540685 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3aa0162810 0x7f3aa00b0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:39.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.067+0000 7f3aa4ecf640 1 -- 172.21.15.73:0/866540685 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aa015a2a0 con 0x7f3aa00b2340 2024-04-26T20:50:39.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.067+0000 7f3aa56d0640 1 --2- 172.21.15.73:0/866540685 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3aa0162810 0x7f3aa00b0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:50:39.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.067+0000 7f3aa4ecf640 1 --2- 172.21.15.73:0/866540685 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3aa00b2340 0x7f3aa00c6e70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3a9400c460 tx=0x7f3a9400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:39.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.067+0000 7f3a9d7fa640 1 -- 172.21.15.73:0/866540685 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a94018070 con 0x7f3aa00b2340 2024-04-26T20:50:39.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.067+0000 7f3a9d7fa640 1 -- 172.21.15.73:0/866540685 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a94007500 con 0x7f3aa00b2340 2024-04-26T20:50:39.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.067+0000 7f3aa7133640 1 -- 172.21.15.73:0/866540685 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3aa014cb40 con 0x7f3aa00b2340 2024-04-26T20:50:39.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.067+0000 7f3a9d7fa640 1 -- 172.21.15.73:0/866540685 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a94004a20 con 0x7f3aa00b2340 2024-04-26T20:50:39.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.067+0000 7f3aa7133640 1 -- 172.21.15.73:0/866540685 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3aa00c8a20 con 0x7f3aa00b2340 2024-04-26T20:50:39.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.068+0000 7f3aa7133640 1 -- 172.21.15.73:0/866540685 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3aa00720a0 con 0x7f3aa00b2340 2024-04-26T20:50:39.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.068+0000 7f3a9d7fa640 1 -- 172.21.15.73:0/866540685 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3a9401b070 con 0x7f3aa00b2340 2024-04-26T20:50:39.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.069+0000 7f3a9d7fa640 1 --2- 172.21.15.73:0/866540685 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f3a8406c8f0 0x7f3a8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:39.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.069+0000 7f3a9ffff640 1 --2- 172.21.15.73:0/866540685 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f3a8406c8f0 0x7f3a8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:39.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.069+0000 7f3a9d7fa640 1 -- 172.21.15.73:0/866540685 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f3a9408b0f0 con 0x7f3aa00b2340 2024-04-26T20:50:39.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.069+0000 7f3a9ffff640 1 --2- 172.21.15.73:0/866540685 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f3a8406c8f0 0x7f3a8406edb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c034000 tx=0x7f3a8c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:39.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.073+0000 7f3a9d7fa640 1 -- 172.21.15.73:0/866540685 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3aa00720a0 con 0x7f3aa00b2340 2024-04-26T20:50:39.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.249+0000 7f3aa7133640 1 -- 172.21.15.73:0/866540685 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3aa00694f0 con 0x7f3aa00b2340 2024-04-26T20:50:39.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.251+0000 7f3a9d7fa640 1 -- 172.21.15.73:0/866540685 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18514 (secure 0 0 0) 0x7f3aa015a2a0 con 0x7f3aa00b2340 2024-04-26T20:50:39.252 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:50:39.252 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":25,"fsid":"fbd3f548-b547-4713-856b-09f98f5d1764","created":"2024-04-26T20:50:07.876786+0000","modified":"2024-04-26T20:50:38.657510+0000","last_up_change":"2024-04-26T20:50:19.082741+0000","last_in_change":"2024-04-26T20:50:10.018400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:50:19.993731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:50:29.964420+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:50:30.920082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:50:34.650207+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"72d6f4f3-214a-4392-a02f-c33cbcab3c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6809","nonce":3461705124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6811","nonce":3461705124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6815","nonce":3461705124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6813","nonce":3461705124}]},"public_addr":"172.21.15.73:6809/3461705124","cluster_addr":"172.21.15.73:6811/3461705124","heartbeat_back_addr":"172.21.15.73:6815/3461705124","heartbeat_front_addr":"172.21.15.73:6813/3461705124","state":["exists","up"]},{"osd":1,"uuid":"786bf8ca-6ddc-4b3c-be6c-9152398bd5fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6817","nonce":2827619403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6819","nonce":2827619403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6825","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6827","nonce":2827619403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6821","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6823","nonce":2827619403}]},"public_addr":"172.21.15.73:6817/2827619403","cluster_addr":"172.21.15.73:6819/2827619403","heartbeat_back_addr":"172.21.15.73:6827/2827619403","heartbeat_front_addr":"172.21.15.73:6823/2827619403","state":["exists","up"]},{"osd":2,"uuid":"90963b1a-5437-4ef8-ad8b-6b74c534eeb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6801","nonce":2542681064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6803","nonce":2542681064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6808","nonce":2542681064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6805","nonce":2542681064}]},"public_addr":"172.21.15.73:6801/2542681064","cluster_addr":"172.21.15.73:6803/2542681064","heartbeat_back_addr":"172.21.15.73:6808/2542681064","heartbeat_front_addr":"172.21.15.73:6805/2542681064","state":["exists","up"]},{"osd":3,"uuid":"bd3d2148-6e38-41b4-a453-fe6e3b8b4ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6822","nonce":1604042718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6826","nonce":1604042718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6831","nonce":1604042718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6829","nonce":1604042718}]},"public_addr":"172.21.15.73:6822/1604042718","cluster_addr":"172.21.15.73:6826/1604042718","heartbeat_back_addr":"172.21.15.73:6831/1604042718","heartbeat_front_addr":"172.21.15.73:6829/1604042718","state":["exists","up"]},{"osd":4,"uuid":"e8668dc8-a23f-46d2-8311-cb882da08e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6817","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6819","nonce":3591055561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6824","nonce":3591055561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6831","nonce":3591055561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6828","nonce":3591055561}]},"public_addr":"172.21.15.120:6819/3591055561","cluster_addr":"172.21.15.120:6824/3591055561","heartbeat_back_addr":"172.21.15.120:6831/3591055561","heartbeat_front_addr":"172.21.15.120:6828/3591055561","state":["exists","up"]},{"osd":5,"uuid":"cbac0bdb-96bd-4852-bcce-e564495c3fcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6809","nonce":819450433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6811","nonce":819450433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6815","nonce":819450433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6813","nonce":819450433}]},"public_addr":"172.21.15.120:6809/819450433","cluster_addr":"172.21.15.120:6811/819450433","heartbeat_back_addr":"172.21.15.120:6815/819450433","heartbeat_front_addr":"172.21.15.120:6813/819450433","state":["exists","up"]},{"osd":6,"uuid":"1884b2a8-569f-4272-96a9-f25a09364dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6801","nonce":3946210727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6803","nonce":3946210727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6807","nonce":3946210727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6805","nonce":3946210727}]},"public_addr":"172.21.15.120:6801/3946210727","cluster_addr":"172.21.15.120:6803/3946210727","heartbeat_back_addr":"172.21.15.120:6807/3946210727","heartbeat_front_addr":"172.21.15.120:6805/3946210727","state":["exists","up"]},{"osd":7,"uuid":"c0c3bf6d-a4c5-4f1e-aa2d-330951830666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6818","nonce":2915574674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6821","nonce":2915574674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6827","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6829","nonce":2915574674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6823","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6825","nonce":2915574674}]},"public_addr":"172.21.15.120:6818/2915574674","cluster_addr":"172.21.15.120:6821/2915574674","heartbeat_back_addr":"172.21.15.120:6829/2915574674","heartbeat_front_addr":"172.21.15.120:6825/2915574674","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.218216+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.208710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.112705+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.182789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.164324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.174212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.225396+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.261559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:50:39.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.252+0000 7f3aa7133640 1 -- 172.21.15.73:0/866540685 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f3a8406c8f0 msgr2=0x7f3a8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:39.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.252+0000 7f3aa7133640 1 --2- 172.21.15.73:0/866540685 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f3a8406c8f0 0x7f3a8406edb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c034000 tx=0x7f3a8c02e000 comp rx=0 tx=0).stop 2024-04-26T20:50:39.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.253+0000 7f3aa7133640 1 -- 172.21.15.73:0/866540685 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3aa00b2340 msgr2=0x7f3aa00c6e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:39.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.253+0000 7f3aa7133640 1 --2- 172.21.15.73:0/866540685 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3aa00b2340 0x7f3aa00c6e70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3a9400c460 tx=0x7f3a9400c920 comp rx=0 tx=0).stop 2024-04-26T20:50:39.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.253+0000 7f3aa7133640 1 -- 172.21.15.73:0/866540685 shutdown_connections 2024-04-26T20:50:39.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.253+0000 7f3aa7133640 1 --2- 172.21.15.73:0/866540685 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f3a8406c8f0 0x7f3a8406edb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:39.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.253+0000 7f3aa7133640 1 --2- 172.21.15.73:0/866540685 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3aa0162810 0x7f3aa00b0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:39.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.253+0000 7f3aa7133640 1 --2- 172.21.15.73:0/866540685 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3aa00cd930 0x7f3aa00c9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:39.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.253+0000 7f3aa7133640 1 --2- 172.21.15.73:0/866540685 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3aa00b2340 0x7f3aa00c6e70 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:39.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.253+0000 7f3aa7133640 1 -- 172.21.15.73:0/866540685 >> 172.21.15.73:0/866540685 conn(0x7f3aa00d2da0 msgr2=0x7f3aa00b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:39.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.253+0000 7f3aa7133640 1 -- 172.21.15.73:0/866540685 shutdown_connections 2024-04-26T20:50:39.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.253+0000 7f3aa7133640 1 -- 172.21.15.73:0/866540685 wait complete. 2024-04-26T20:50:39.669 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T20:50:31.964317+0000', 'modified': '2024-04-26T20:50:37.007492+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4281}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4281': {'gid': 4281, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.73:6837/181202544', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6836', 'nonce': 181202544}, {'type': 'v1', 'addr': '172.21.15.73:6837', 'nonce': 181202544}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4281, 'qdb_cluster': [4281]} 2024-04-26T20:50:39.669 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T20:50:39.669 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-26T20:50:39.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.767+0000 7fd3fa6cf640 1 Processor -- start 2024-04-26T20:50:39.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.767+0000 7fd3fa6cf640 1 -- start start 2024-04-26T20:50:39.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.768+0000 7fd3fa6cf640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd3f4162880 0x7fd3f40d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:39.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.768+0000 7fd3fa6cf640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd3f40d3bc0 0x7fd3f40b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:39.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.768+0000 7fd3fa6cf640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3f40b7e50 con 0x7fd3f40b5f20 2024-04-26T20:50:39.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.768+0000 7fd3fa6cf640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7fd3f40d0580 con 0x7fd3f4162880 2024-04-26T20:50:39.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.768+0000 7fd3fa6cf640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fd3f40d1800 con 0x7fd3f40d3bc0 2024-04-26T20:50:39.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.768+0000 7fd3f3fff640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd3f4162880 0x7fd3f40d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:39.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.768+0000 7fd3f3fff640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd3f4162880 0x7fd3f40d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34794/0 (socket says 172.21.15.73:34794) 2024-04-26T20:50:39.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.768+0000 7fd3f3fff640 1 -- 172.21.15.73:0/1173591812 learned_addr learned my addr 172.21.15.73:0/1173591812 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:39.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.768+0000 7fd3f8c6c640 1 --2- 172.21.15.73:0/1173591812 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd3f40d3bc0 0x7fd3f40b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:39.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.768+0000 7fd3f3fff640 1 -- 172.21.15.73:0/1173591812 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd3f40d3bc0 msgr2=0x7fd3f40b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:39.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.768+0000 7fd3f3fff640 1 --2- 172.21.15.73:0/1173591812 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd3f40d3bc0 0x7fd3f40b8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:39.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.768+0000 7fd3f3fff640 1 -- 172.21.15.73:0/1173591812 >> v1:172.21.15.120:6789/0 conn(0x7fd3f40b5f20 legacy=0x7fd3f40d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:39.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.768+0000 7fd3f3fff640 1 -- 172.21.15.73:0/1173591812 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3f40d07d0 con 0x7fd3f4162880 2024-04-26T20:50:39.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.768+0000 7fd3f8c6c640 1 --2- 172.21.15.73:0/1173591812 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd3f40d3bc0 0x7fd3f40b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:50:39.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.769+0000 7fd3f3fff640 1 --2- 172.21.15.73:0/1173591812 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3f4162880 0x7fd3f40d2cb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8009870 tx=0x7fd3e802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a7896539bc92f8f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:39.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.769+0000 7fd3f2ffd640 1 -- 172.21.15.73:0/1173591812 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e8037070 con 0x7fd3f4162880 2024-04-26T20:50:39.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.769+0000 7fd3f2ffd640 1 -- 172.21.15.73:0/1173591812 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3e8032440 con 0x7fd3f4162880 2024-04-26T20:50:39.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.769+0000 7fd3f2ffd640 1 -- 172.21.15.73:0/1173591812 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e8032810 con 0x7fd3f4162880 2024-04-26T20:50:39.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.769+0000 7fd3fa6cf640 1 -- 172.21.15.73:0/1173591812 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3f4162880 msgr2=0x7fd3f40d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:39.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.769+0000 7fd3fa6cf640 1 --2- 172.21.15.73:0/1173591812 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3f4162880 0x7fd3f40d2cb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8009870 tx=0x7fd3e802a210 comp rx=0 tx=0).stop 2024-04-26T20:50:39.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.769+0000 7fd3fa6cf640 1 -- 172.21.15.73:0/1173591812 shutdown_connections 2024-04-26T20:50:39.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.769+0000 7fd3fa6cf640 1 --2- 172.21.15.73:0/1173591812 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd3f40d3bc0 0x7fd3f40b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:39.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.769+0000 7fd3fa6cf640 1 --2- 172.21.15.73:0/1173591812 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3f4162880 0x7fd3f40d2cb0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:39.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.769+0000 7fd3fa6cf640 1 -- 172.21.15.73:0/1173591812 >> 172.21.15.73:0/1173591812 conn(0x7fd3f40d4f60 msgr2=0x7fd3f40d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:39.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.769+0000 7fd3fa6cf640 1 -- 172.21.15.73:0/1173591812 shutdown_connections 2024-04-26T20:50:39.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.769+0000 7fd3fa6cf640 1 -- 172.21.15.73:0/1173591812 wait complete. 2024-04-26T20:50:39.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.771+0000 7fd3fa6cf640 1 Processor -- start 2024-04-26T20:50:39.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.771+0000 7fd3fa6cf640 1 -- start start 2024-04-26T20:50:39.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.771+0000 7fd3fa6cf640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd3f40b5f20 0x7fd3f415c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:39.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.771+0000 7fd3fa6cf640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3f40d3bc0 0x7fd3f4150320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:39.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.772+0000 7fd3fa6cf640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd3f4162880 0x7fd3f40cd890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:39.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.772+0000 7fd3fa6cf640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f4162c60 con 0x7fd3f40d3bc0 2024-04-26T20:50:39.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.772+0000 7fd3fa6cf640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f40d0d90 con 0x7fd3f4162880 2024-04-26T20:50:39.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.772+0000 7fd3fa6cf640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fd3f4162de0 con 0x7fd3f40b5f20 2024-04-26T20:50:39.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.772+0000 7fd3f37fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3f40d3bc0 0x7fd3f4150320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:39.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.772+0000 7fd3f37fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3f40d3bc0 0x7fd3f4150320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34804/0 (socket says 172.21.15.73:34804) 2024-04-26T20:50:39.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.772+0000 7fd3f37fe640 1 -- 172.21.15.73:0/713347774 learned_addr learned my addr 172.21.15.73:0/713347774 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:39.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.772+0000 7fd3f8c6c640 1 --2- 172.21.15.73:0/713347774 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd3f4162880 0x7fd3f40cd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:39.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.772+0000 7fd3f3fff640 1 --2- 172.21.15.73:0/713347774 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd3f40b5f20 0x7fd3f415c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:39.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.772+0000 7fd3f37fe640 1 -- 172.21.15.73:0/713347774 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd3f40b5f20 msgr2=0x7fd3f415c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:39.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.772+0000 7fd3f37fe640 1 --2- 172.21.15.73:0/713347774 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd3f40b5f20 0x7fd3f415c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:39.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.772+0000 7fd3f37fe640 1 -- 172.21.15.73:0/713347774 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd3f4162880 msgr2=0x7fd3f40cd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:39.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.772+0000 7fd3f37fe640 1 --2- 172.21.15.73:0/713347774 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd3f4162880 0x7fd3f40cd890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:39.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.772+0000 7fd3f37fe640 1 -- 172.21.15.73:0/713347774 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3f415a700 con 0x7fd3f40d3bc0 2024-04-26T20:50:39.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.772+0000 7fd3f37fe640 1 --2- 172.21.15.73:0/713347774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3f40d3bc0 0x7fd3f4150320 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd3e400b420 tx=0x7fd3e400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:39.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.772+0000 7fd3f0ff9640 1 -- 172.21.15.73:0/713347774 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e4011830 con 0x7fd3f40d3bc0 2024-04-26T20:50:39.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.772+0000 7fd3f0ff9640 1 -- 172.21.15.73:0/713347774 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3e4016070 con 0x7fd3f40d3bc0 2024-04-26T20:50:39.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.772+0000 7fd3f0ff9640 1 -- 172.21.15.73:0/713347774 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e4011c10 con 0x7fd3f40d3bc0 2024-04-26T20:50:39.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.772+0000 7fd3fa6cf640 1 -- 172.21.15.73:0/713347774 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3f40cd520 con 0x7fd3f40d3bc0 2024-04-26T20:50:39.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.772+0000 7fd3fa6cf640 1 -- 172.21.15.73:0/713347774 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3f415aca0 con 0x7fd3f40d3bc0 2024-04-26T20:50:39.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.774+0000 7fd3f0ff9640 1 -- 172.21.15.73:0/713347774 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd3e401b460 con 0x7fd3f40d3bc0 2024-04-26T20:50:39.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.774+0000 7fd3f0ff9640 1 --2- 172.21.15.73:0/713347774 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd3e006c8f0 0x7fd3e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:39.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.774+0000 7fd3f0ff9640 1 -- 172.21.15.73:0/713347774 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd3e408b950 con 0x7fd3f40d3bc0 2024-04-26T20:50:39.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.774+0000 7fd3f3fff640 1 --2- 172.21.15.73:0/713347774 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd3e006c8f0 0x7fd3e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:39.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.775+0000 7fd3f3fff640 1 --2- 172.21.15.73:0/713347774 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd3e006c8f0 0x7fd3e006edb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8034000 tx=0x7fd3e802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:39.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.775+0000 7fd3fd3e2740 1 Processor -- start 2024-04-26T20:50:39.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.775+0000 7fd3fd3e2740 1 -- start start 2024-04-26T20:50:39.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.776+0000 7fd3fd3e2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561cb8f3440 0x5561cb8f3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:39.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.776+0000 7fd3fd3e2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561cb910ce0 0x5561cb911230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:39.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.776+0000 7fd3fd3e2740 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x5561cb8c9120 0x5561cb90f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:39.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.776+0000 7fd3fd3e2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5561cb82d740 con 0x5561cb8f3440 2024-04-26T20:50:39.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.776+0000 7fd3fd3e2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5561cb7c93a0 con 0x5561cb910ce0 2024-04-26T20:50:39.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.776+0000 7fd3fd3e2740 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x5561cb829700 con 0x5561cb8c9120 2024-04-26T20:50:39.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.776+0000 7fd3f3fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561cb8f3440 0x5561cb8f3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:39.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.776+0000 7fd3f3fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561cb8f3440 0x5561cb8f3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34812/0 (socket says 172.21.15.73:34812) 2024-04-26T20:50:39.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.776+0000 7fd3f3fff640 1 -- 172.21.15.73:0/1913663837 learned_addr learned my addr 172.21.15.73:0/1913663837 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:39.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.776+0000 7fd3f8c6c640 1 --2- 172.21.15.73:0/1913663837 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x5561cb8c9120 0x5561cb90f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:39.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.776+0000 7fd3f37fe640 1 --2- 172.21.15.73:0/1913663837 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561cb910ce0 0x5561cb911230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:39.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.776+0000 7fd3f3fff640 1 -- 172.21.15.73:0/1913663837 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x5561cb8c9120 msgr2=0x5561cb90f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:39.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.776+0000 7fd3f3fff640 1 --2- 172.21.15.73:0/1913663837 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x5561cb8c9120 0x5561cb90f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:39.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.776+0000 7fd3f3fff640 1 -- 172.21.15.73:0/1913663837 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561cb910ce0 msgr2=0x5561cb911230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:39.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.776+0000 7fd3f3fff640 1 --2- 172.21.15.73:0/1913663837 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561cb910ce0 0x5561cb911230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:39.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.776+0000 7fd3f3fff640 1 -- 172.21.15.73:0/1913663837 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561cb7e5750 con 0x5561cb8f3440 2024-04-26T20:50:39.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.776+0000 7fd3f3fff640 1 --2- 172.21.15.73:0/1913663837 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561cb8f3440 0x5561cb8f3820 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8035040 tx=0x7fd3e8008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd0556dcdf0101f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:39.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.776+0000 7fd3d9ffb640 1 -- 172.21.15.73:0/1913663837 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e804e070 con 0x5561cb8f3440 2024-04-26T20:50:39.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.776+0000 7fd3d9ffb640 1 -- 172.21.15.73:0/1913663837 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3e8032d70 con 0x5561cb8f3440 2024-04-26T20:50:39.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.776+0000 7fd3d9ffb640 1 -- 172.21.15.73:0/1913663837 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e8037040 con 0x5561cb8f3440 2024-04-26T20:50:39.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.777+0000 7fd3fd3e2740 1 -- 172.21.15.73:0/1913663837 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561cb8f3440 msgr2=0x5561cb8f3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:39.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.777+0000 7fd3fd3e2740 1 --2- 172.21.15.73:0/1913663837 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561cb8f3440 0x5561cb8f3820 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8035040 tx=0x7fd3e8008030 comp rx=0 tx=0).stop 2024-04-26T20:50:39.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.777+0000 7fd3fd3e2740 1 -- 172.21.15.73:0/1913663837 shutdown_connections 2024-04-26T20:50:39.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.777+0000 7fd3fd3e2740 1 --2- 172.21.15.73:0/1913663837 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x5561cb8c9120 0x5561cb90f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:39.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.777+0000 7fd3fd3e2740 1 --2- 172.21.15.73:0/1913663837 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561cb910ce0 0x5561cb911230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:39.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.777+0000 7fd3fd3e2740 1 --2- 172.21.15.73:0/1913663837 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561cb8f3440 0x5561cb8f3820 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:39.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.777+0000 7fd3fd3e2740 1 -- 172.21.15.73:0/1913663837 >> 172.21.15.73:0/1913663837 conn(0x5561cb8c5a50 msgr2=0x5561cb90e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:39.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.777+0000 7fd3fd3e2740 1 -- 172.21.15.73:0/1913663837 shutdown_connections 2024-04-26T20:50:39.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.777+0000 7fd3fd3e2740 1 -- 172.21.15.73:0/1913663837 wait complete. 2024-04-26T20:50:39.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.777+0000 7fd3fd3e2740 1 -- start start 2024-04-26T20:50:39.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.778+0000 7fd3fd3e2740 1 Processor -- start 2024-04-26T20:50:39.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.778+0000 7fd3fd3e2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561cb8f3440 0x5561cb9069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:39.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.778+0000 7fd3fd3e2740 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x5561cb910ce0 0x5561cb906f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:39.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.778+0000 7fd3fd3e2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561cb905cc0 0x5561cb8f66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:39.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.778+0000 7fd3fd3e2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5561cb8c9ec0 con 0x5561cb8f3440 2024-04-26T20:50:39.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.778+0000 7fd3fd3e2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5561cb7b4fe0 con 0x5561cb905cc0 2024-04-26T20:50:39.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.778+0000 7fd3fd3e2740 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x5561cb8c3980 con 0x5561cb910ce0 2024-04-26T20:50:39.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.778+0000 7fd3fd3e2740 10 client.0 _ll_register_callbacks cb 0x5561cb7cb150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:50:39.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.778+0000 7fd3f3fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561cb8f3440 0x5561cb9069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:39.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.778+0000 7fd3f3fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561cb8f3440 0x5561cb9069e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34820/0 (socket says 172.21.15.73:34820) 2024-04-26T20:50:39.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.778+0000 7fd3f3fff640 1 -- 172.21.15.73:0/1707869525 learned_addr learned my addr 172.21.15.73:0/1707869525 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:39.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.779+0000 7fd3f37fe640 1 --2- 172.21.15.73:0/1707869525 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x5561cb910ce0 0x5561cb906f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:39.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.779+0000 7fd3f8c6c640 1 --2- 172.21.15.73:0/1707869525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561cb905cc0 0x5561cb8f66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:39.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.779+0000 7fd3f3fff640 1 -- 172.21.15.73:0/1707869525 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x5561cb910ce0 msgr2=0x5561cb906f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:39.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.779+0000 7fd3f3fff640 1 --2- 172.21.15.73:0/1707869525 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x5561cb910ce0 0x5561cb906f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:39.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.779+0000 7fd3f3fff640 1 -- 172.21.15.73:0/1707869525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561cb905cc0 msgr2=0x5561cb8f66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:39.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.779+0000 7fd3f3fff640 1 --2- 172.21.15.73:0/1707869525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561cb905cc0 0x5561cb8f66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:39.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.779+0000 7fd3f3fff640 1 -- 172.21.15.73:0/1707869525 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561cb53f2d0 con 0x5561cb8f3440 2024-04-26T20:50:39.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.779+0000 7fd3f8c6c640 1 --2- 172.21.15.73:0/1707869525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561cb905cc0 0x5561cb8f66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:50:39.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.779+0000 7fd3f3fff640 1 --2- 172.21.15.73:0/1707869525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561cb8f3440 0x5561cb9069e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8009990 tx=0x7fd3e8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46d0c41bccb411af server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:39.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.779+0000 7fd3d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-04-26T20:50:39.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.779+0000 7fd3d2ffd640 1 -- 172.21.15.73:0/1707869525 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e8057060 con 0x5561cb8f3440 2024-04-26T20:50:39.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.779+0000 7fd3d2ffd640 1 -- 172.21.15.73:0/1707869525 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd3e8003610 con 0x5561cb8f3440 2024-04-26T20:50:39.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.779+0000 7fd3fd3e2740 1 -- 172.21.15.73:0/1707869525 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3e8032d70 con 0x5561cb8f3440 2024-04-26T20:50:39.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.779+0000 7fd3d2ffd640 1 -- 172.21.15.73:0/1707869525 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) ==== 24+0+0 (secure 0 0 0) 0x7fd3e8037030 con 0x5561cb8f3440 2024-04-26T20:50:39.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.779+0000 7fd3fd3e2740 10 client.4437 fetch_fsmap learned FSMap version 6 2024-04-26T20:50:39.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.779+0000 7fd3fd3e2740 1 -- 172.21.15.73:0/1707869525 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) -- 0x5561cb6096b0 con 0x5561cb8f3440 2024-04-26T20:50:39.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.779+0000 7fd3d2ffd640 1 -- 172.21.15.73:0/1707869525 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3e8032d70 con 0x5561cb8f3440 2024-04-26T20:50:39.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.779+0000 7fd3d2ffd640 1 -- 172.21.15.73:0/1707869525 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 6) ==== 3502+0+0 (secure 0 0 0) 0x7fd3e8003810 con 0x5561cb8f3440 2024-04-26T20:50:39.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.779+0000 7fd3fd3e2740 10 client.4437 fetch_fsmap finished waiting for FSMap version 6 2024-04-26T20:50:39.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.779+0000 7fd3fd3e2740 10 client.4437 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T20:50:39.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.779+0000 7fd3fd3e2740 20 client.4437 populate_metadata read hostname 'smithi073' 2024-04-26T20:50:39.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.779+0000 7fd3fd3e2740 1 --2- 172.21.15.73:0/1707869525 >> [v2:172.21.15.73:6836/181202544,v1:172.21.15.73:6837/181202544] conn(0x5561cb8f4c40 0x5561cb8f5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:39.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.779+0000 7fd3fd3e2740 4 client.4437 mds_command: new command op to 4281 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:50:39.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.779+0000 7fd3fd3e2740 1 -- 172.21.15.73:0/1707869525 --> [v2:172.21.15.73:6836/181202544,v1:172.21.15.73:6837/181202544] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561cb83b350 con 0x5561cb8f4c40 2024-04-26T20:50:39.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.780+0000 7fd3f37fe640 1 --2- 172.21.15.73:0/1707869525 >> [v2:172.21.15.73:6836/181202544,v1:172.21.15.73:6837/181202544] conn(0x5561cb8f4c40 0x5561cb8f5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:39.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.780+0000 7fd3f37fe640 1 --2- 172.21.15.73:0/1707869525 >> [v2:172.21.15.73:6836/181202544,v1:172.21.15.73:6837/181202544] conn(0x5561cb8f4c40 0x5561cb8f5020 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=236a56bba71fac53 server_cookie=86dd7c6352aef37f in_seq=0 out_seq=0 2024-04-26T20:50:39.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.780+0000 7fd3d2ffd640 10 client.4437 ms_handle_connect on v2:172.21.15.73:6836/181202544 2024-04-26T20:50:39.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.780+0000 7fd3d2ffd640 1 -- 172.21.15.73:0/1707869525 <== mds.0 v2:172.21.15.73:6836/181202544 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5561cb83b350 con 0x5561cb8f4c40 2024-04-26T20:50:39.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.780+0000 7fd3d2ffd640 10 client.4437 handle_command_reply: tid=0 2024-04-26T20:50:39.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.780+0000 7fd3fd3e2740 1 client.4437 shutdown 2024-04-26T20:50:39.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.781+0000 7fd3fd3e2740 1 -- 172.21.15.73:0/1707869525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561cb8f3440 msgr2=0x5561cb9069e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:39.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.781+0000 7fd3fd3e2740 1 --2- 172.21.15.73:0/1707869525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561cb8f3440 0x5561cb9069e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8009990 tx=0x7fd3e8004730 comp rx=0 tx=0).stop 2024-04-26T20:50:39.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.781+0000 7fd3fd3e2740 1 -- 172.21.15.73:0/1707869525 shutdown_connections 2024-04-26T20:50:39.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.781+0000 7fd3fd3e2740 1 --2- 172.21.15.73:0/1707869525 >> [v2:172.21.15.73:6836/181202544,v1:172.21.15.73:6837/181202544] conn(0x5561cb8f4c40 0x5561cb8f5020 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:39.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.781+0000 7fd3fd3e2740 1 --2- 172.21.15.73:0/1707869525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561cb905cc0 0x5561cb8f66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:39.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.781+0000 7fd3fd3e2740 1 --2- 172.21.15.73:0/1707869525 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x5561cb910ce0 0x5561cb906f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:39.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.781+0000 7fd3d2ffd640 0 client.4437 ms_handle_reset on v2:172.21.15.73:6836/181202544 2024-04-26T20:50:39.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.781+0000 7fd3fd3e2740 1 --2- 172.21.15.73:0/1707869525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561cb8f3440 0x5561cb9069e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:39.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.781+0000 7fd3fd3e2740 1 -- 172.21.15.73:0/1707869525 >> 172.21.15.73:0/1707869525 conn(0x5561cb8c9120 msgr2=0x5561cb9117d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:39.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.781+0000 7fd3fd3e2740 1 -- 172.21.15.73:0/1707869525 shutdown_connections 2024-04-26T20:50:39.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.781+0000 7fd3fd3e2740 1 -- 172.21.15.73:0/1707869525 wait complete. 2024-04-26T20:50:39.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.781+0000 7fd3fd3e2740 20 client.4437 trim_cache size 0 max 16384 2024-04-26T20:50:39.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.799+0000 7fd3fd3e2740 1 Processor -- start 2024-04-26T20:50:39.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.799+0000 7fd3fd3e2740 1 -- start start 2024-04-26T20:50:39.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.799+0000 7fd3fd3e2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561cb8f3440 0x5561cb91d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:39.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.799+0000 7fd3fd3e2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561cb905cc0 0x5561cb8c3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:39.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.800+0000 7fd3fd3e2740 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x5561cb910ce0 0x5561cb8c7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:39.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.800+0000 7fd3fd3e2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f4201bf0 con 0x5561cb8f3440 2024-04-26T20:50:39.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.800+0000 7fd3fd3e2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f40cd1e0 con 0x5561cb905cc0 2024-04-26T20:50:39.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.800+0000 7fd3fd3e2740 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fd3f42015a0 con 0x5561cb910ce0 2024-04-26T20:50:39.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.800+0000 7fd3f3fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561cb8f3440 0x5561cb91d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:39.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.800+0000 7fd3f3fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561cb8f3440 0x5561cb91d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34830/0 (socket says 172.21.15.73:34830) 2024-04-26T20:50:39.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.800+0000 7fd3f3fff640 1 -- 172.21.15.73:0/795907956 learned_addr learned my addr 172.21.15.73:0/795907956 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:39.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.800+0000 7fd3f3fff640 1 -- 172.21.15.73:0/795907956 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x5561cb910ce0 msgr2=0x5561cb8c7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:39.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.800+0000 7fd3f8c6c640 1 --2- 172.21.15.73:0/795907956 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x5561cb910ce0 0x5561cb8c7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:39.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.800+0000 7fd3f37fe640 1 --2- 172.21.15.73:0/795907956 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561cb905cc0 0x5561cb8c3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:39.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.800+0000 7fd3f3fff640 1 --2- 172.21.15.73:0/795907956 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x5561cb910ce0 0x5561cb8c7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:39.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.800+0000 7fd3f3fff640 1 -- 172.21.15.73:0/795907956 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561cb905cc0 msgr2=0x5561cb8c3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:39.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.800+0000 7fd3f3fff640 1 --2- 172.21.15.73:0/795907956 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561cb905cc0 0x5561cb8c3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:39.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.800+0000 7fd3f3fff640 1 -- 172.21.15.73:0/795907956 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561cb90e050 con 0x5561cb8f3440 2024-04-26T20:50:39.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.800+0000 7fd3f8c6c640 1 --2- 172.21.15.73:0/795907956 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x5561cb910ce0 0x5561cb8c7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:50:39.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.800+0000 7fd3f37fe640 1 --2- 172.21.15.73:0/795907956 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561cb905cc0 0x5561cb8c3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:50:39.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.800+0000 7fd3f3fff640 1 --2- 172.21.15.73:0/795907956 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561cb8f3440 0x5561cb91d860 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd3e80043d0 tx=0x7fd3e800b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=617e198c3587b2b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:39.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.800+0000 7fd3d97fa640 1 -- 172.21.15.73:0/795907956 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e804e040 con 0x5561cb8f3440 2024-04-26T20:50:39.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.800+0000 7fd3d97fa640 1 -- 172.21.15.73:0/795907956 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3e8032d70 con 0x5561cb8f3440 2024-04-26T20:50:39.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.800+0000 7fd3d97fa640 1 -- 172.21.15.73:0/795907956 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e8003810 con 0x5561cb8f3440 2024-04-26T20:50:39.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.801+0000 7fd3fd3e2740 1 -- 172.21.15.73:0/795907956 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561cb8f3440 msgr2=0x5561cb91d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:39.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.801+0000 7fd3fd3e2740 1 --2- 172.21.15.73:0/795907956 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561cb8f3440 0x5561cb91d860 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd3e80043d0 tx=0x7fd3e800b050 comp rx=0 tx=0).stop 2024-04-26T20:50:39.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.801+0000 7fd3fd3e2740 1 -- 172.21.15.73:0/795907956 shutdown_connections 2024-04-26T20:50:39.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.801+0000 7fd3fd3e2740 1 --2- 172.21.15.73:0/795907956 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x5561cb910ce0 0x5561cb8c7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:39.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.801+0000 7fd3fd3e2740 1 --2- 172.21.15.73:0/795907956 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561cb905cc0 0x5561cb8c3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:39.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.801+0000 7fd3fd3e2740 1 --2- 172.21.15.73:0/795907956 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561cb8f3440 0x5561cb91d860 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:39.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.801+0000 7fd3fd3e2740 1 -- 172.21.15.73:0/795907956 >> 172.21.15.73:0/795907956 conn(0x5561cb8c9120 msgr2=0x5561cb907e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:39.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.801+0000 7fd3fd3e2740 1 -- 172.21.15.73:0/795907956 shutdown_connections 2024-04-26T20:50:39.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.801+0000 7fd3fd3e2740 1 -- 172.21.15.73:0/795907956 wait complete. 2024-04-26T20:50:39.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.801+0000 7fd3fd3e2740 1 -- start start 2024-04-26T20:50:39.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.802+0000 7fd3fd3e2740 1 Processor -- start 2024-04-26T20:50:39.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.802+0000 7fd3fd3e2740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561cb8f3440 0x5561cb8ed040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:39.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.802+0000 7fd3fd3e2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561cb905cc0 0x5561cb8f5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:39.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.802+0000 7fd3fd3e2740 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x5561cb910ce0 0x5561cb906980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:39.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.802+0000 7fd3fd3e2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5561cb90db90 con 0x5561cb905cc0 2024-04-26T20:50:39.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.802+0000 7fd3fd3e2740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x5561cb8f43b0 con 0x5561cb8f3440 2024-04-26T20:50:39.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.802+0000 7fd3fd3e2740 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x5561cb8f6be0 con 0x5561cb910ce0 2024-04-26T20:50:39.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.802+0000 7fd3fd3e2740 10 client.0 _ll_register_callbacks cb 0x7fd3e804c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:50:39.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.802+0000 7fd3f37fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561cb905cc0 0x5561cb8f5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:39.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.802+0000 7fd3f37fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561cb905cc0 0x5561cb8f5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34836/0 (socket says 172.21.15.73:34836) 2024-04-26T20:50:39.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.802+0000 7fd3f37fe640 1 -- 172.21.15.73:0/2557529037 learned_addr learned my addr 172.21.15.73:0/2557529037 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:39.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.802+0000 7fd3f3fff640 1 --2- 172.21.15.73:0/2557529037 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561cb8f3440 0x5561cb8ed040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:39.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.802+0000 7fd3f8c6c640 1 --2- 172.21.15.73:0/2557529037 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x5561cb910ce0 0x5561cb906980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:39.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.803+0000 7fd3f37fe640 1 -- 172.21.15.73:0/2557529037 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x5561cb910ce0 msgr2=0x5561cb906980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:39.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.803+0000 7fd3f37fe640 1 --2- 172.21.15.73:0/2557529037 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x5561cb910ce0 0x5561cb906980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:39.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.803+0000 7fd3f37fe640 1 -- 172.21.15.73:0/2557529037 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561cb8f3440 msgr2=0x5561cb8ed040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:39.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.803+0000 7fd3f37fe640 1 --2- 172.21.15.73:0/2557529037 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561cb8f3440 0x5561cb8ed040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:39.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.803+0000 7fd3f37fe640 1 -- 172.21.15.73:0/2557529037 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561cb90e2f0 con 0x5561cb905cc0 2024-04-26T20:50:39.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.803+0000 7fd3f3fff640 1 --2- 172.21.15.73:0/2557529037 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561cb8f3440 0x5561cb8ed040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:50:39.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.803+0000 7fd3f37fe640 1 --2- 172.21.15.73:0/2557529037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561cb905cc0 0x5561cb8f5f00 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd3e40619f0 tx=0x7fd3e4059240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21b346c47b09a82b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:39.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.803+0000 7fd3d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-04-26T20:50:39.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.803+0000 7fd3d2ffd640 1 -- 172.21.15.73:0/2557529037 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e40559b0 con 0x5561cb905cc0 2024-04-26T20:50:39.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.803+0000 7fd3d2ffd640 1 -- 172.21.15.73:0/2557529037 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd3e4058790 con 0x5561cb905cc0 2024-04-26T20:50:39.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.803+0000 7fd3fd3e2740 1 -- 172.21.15.73:0/2557529037 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3e8032d70 con 0x5561cb905cc0 2024-04-26T20:50:39.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.803+0000 7fd3d2ffd640 1 -- 172.21.15.73:0/2557529037 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) ==== 24+0+0 (secure 0 0 0) 0x7fd3e4076050 con 0x5561cb905cc0 2024-04-26T20:50:39.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.803+0000 7fd3fd3e2740 10 client.4443 fetch_fsmap learned FSMap version 6 2024-04-26T20:50:39.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.803+0000 7fd3fd3e2740 1 -- 172.21.15.73:0/2557529037 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) -- 0x5561cb8ed670 con 0x5561cb905cc0 2024-04-26T20:50:39.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.803+0000 7fd3d2ffd640 1 -- 172.21.15.73:0/2557529037 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3e8032d70 con 0x5561cb905cc0 2024-04-26T20:50:39.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.803+0000 7fd3d2ffd640 1 -- 172.21.15.73:0/2557529037 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 6) ==== 3502+0+0 (secure 0 0 0) 0x7fd3e4071b20 con 0x5561cb905cc0 2024-04-26T20:50:39.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.803+0000 7fd3fd3e2740 10 client.4443 fetch_fsmap finished waiting for FSMap version 6 2024-04-26T20:50:39.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.803+0000 7fd3fd3e2740 10 client.4443 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T20:50:39.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.803+0000 7fd3fd3e2740 20 client.4443 populate_metadata read hostname 'smithi073' 2024-04-26T20:50:39.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.803+0000 7fd3fd3e2740 1 --2- 172.21.15.73:0/2557529037 >> [v2:172.21.15.73:6836/181202544,v1:172.21.15.73:6837/181202544] conn(0x5561cb8ecad0 0x5561cb903be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:39.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.803+0000 7fd3fd3e2740 4 client.4443 mds_command: new command op to 4281 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T20:50:39.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.803+0000 7fd3fd3e2740 1 -- 172.21.15.73:0/2557529037 --> [v2:172.21.15.73:6836/181202544,v1:172.21.15.73:6837/181202544] -- command(tid 0: {"prefix": "status"}) -- 0x5561cb8351c0 con 0x5561cb8ecad0 2024-04-26T20:50:39.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.803+0000 7fd3f3fff640 1 --2- 172.21.15.73:0/2557529037 >> [v2:172.21.15.73:6836/181202544,v1:172.21.15.73:6837/181202544] conn(0x5561cb8ecad0 0x5561cb903be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:39.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.804+0000 7fd3d2ffd640 10 client.4443 ms_handle_connect on v2:172.21.15.73:6836/181202544 2024-04-26T20:50:39.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.804+0000 7fd3f3fff640 1 --2- 172.21.15.73:0/2557529037 >> [v2:172.21.15.73:6836/181202544,v1:172.21.15.73:6837/181202544] conn(0x5561cb8ecad0 0x5561cb903be0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db9760de70574d8d server_cookie=a48074f148fbd697 in_seq=0 out_seq=0 2024-04-26T20:50:39.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.804+0000 7fd3d2ffd640 1 -- 172.21.15.73:0/2557529037 <== mds.0 v2:172.21.15.73:6836/181202544 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x5561cb8351c0 con 0x5561cb8ecad0 2024-04-26T20:50:39.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.804+0000 7fd3d2ffd640 10 client.4443 handle_command_reply: tid=0 2024-04-26T20:50:39.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.804+0000 7fd3fd3e2740 1 client.4443 shutdown 2024-04-26T20:50:39.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.804+0000 7fd3fd3e2740 1 -- 172.21.15.73:0/2557529037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561cb905cc0 msgr2=0x5561cb8f5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:39.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.804+0000 7fd3fd3e2740 1 --2- 172.21.15.73:0/2557529037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561cb905cc0 0x5561cb8f5f00 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd3e40619f0 tx=0x7fd3e4059240 comp rx=0 tx=0).stop 2024-04-26T20:50:39.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.804+0000 7fd3fd3e2740 1 -- 172.21.15.73:0/2557529037 shutdown_connections 2024-04-26T20:50:39.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.804+0000 7fd3fd3e2740 1 --2- 172.21.15.73:0/2557529037 >> [v2:172.21.15.73:6836/181202544,v1:172.21.15.73:6837/181202544] conn(0x5561cb8ecad0 0x5561cb903be0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:39.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.804+0000 7fd3fd3e2740 1 --2- 172.21.15.73:0/2557529037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561cb905cc0 0x5561cb8f5f00 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:39.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.804+0000 7fd3d2ffd640 0 client.4443 ms_handle_reset on v2:172.21.15.73:6836/181202544 2024-04-26T20:50:39.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.804+0000 7fd3fd3e2740 1 --2- 172.21.15.73:0/2557529037 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x5561cb910ce0 0x5561cb906980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:39.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.804+0000 7fd3fd3e2740 1 --2- 172.21.15.73:0/2557529037 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x5561cb8f3440 0x5561cb8ed040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:39.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.804+0000 7fd3fd3e2740 1 -- 172.21.15.73:0/2557529037 >> 172.21.15.73:0/2557529037 conn(0x5561cb8c9120 msgr2=0x5561cb8c3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:39.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.804+0000 7fd3fd3e2740 1 -- 172.21.15.73:0/2557529037 shutdown_connections 2024-04-26T20:50:39.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.804+0000 7fd3fd3e2740 1 -- 172.21.15.73:0/2557529037 wait complete. 2024-04-26T20:50:39.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.804+0000 7fd3fd3e2740 20 client.4443 trim_cache size 0 max 16384 2024-04-26T20:50:39.813 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-26T20:50:39.813 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_fsid": "fbd3f548-b547-4713-856b-09f98f5d1764", 2024-04-26T20:50:39.813 INFO:teuthology.orchestra.run.smithi073.stdout: "whoami": 0, 2024-04-26T20:50:39.813 INFO:teuthology.orchestra.run.smithi073.stdout: "id": 4281, 2024-04-26T20:50:39.813 INFO:teuthology.orchestra.run.smithi073.stdout: "want_state": "up:active", 2024-04-26T20:50:39.813 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "up:active", 2024-04-26T20:50:39.813 INFO:teuthology.orchestra.run.smithi073.stdout: "fs_name": "cephfs", 2024-04-26T20:50:39.813 INFO:teuthology.orchestra.run.smithi073.stdout: "rank_uptime": 7.1730901249999999, 2024-04-26T20:50:39.814 INFO:teuthology.orchestra.run.smithi073.stdout: "mdsmap_epoch": 6, 2024-04-26T20:50:39.814 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch": 25, 2024-04-26T20:50:39.814 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch_barrier": 23, 2024-04-26T20:50:39.814 INFO:teuthology.orchestra.run.smithi073.stdout: "uptime": 14.243539793 2024-04-26T20:50:39.814 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-26T20:50:39.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.805+0000 7fd3fa6cf640 1 -- 172.21.15.73:0/713347774 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd3e006c8f0 msgr2=0x7fd3e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:39.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.805+0000 7fd3fa6cf640 1 --2- 172.21.15.73:0/713347774 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd3e006c8f0 0x7fd3e006edb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8034000 tx=0x7fd3e802e000 comp rx=0 tx=0).stop 2024-04-26T20:50:39.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.805+0000 7fd3fa6cf640 1 -- 172.21.15.73:0/713347774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3f40d3bc0 msgr2=0x7fd3f4150320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:39.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.805+0000 7fd3fa6cf640 1 --2- 172.21.15.73:0/713347774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3f40d3bc0 0x7fd3f4150320 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd3e400b420 tx=0x7fd3e400b8e0 comp rx=0 tx=0).stop 2024-04-26T20:50:39.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.805+0000 7fd3fa6cf640 1 -- 172.21.15.73:0/713347774 shutdown_connections 2024-04-26T20:50:39.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.805+0000 7fd3fa6cf640 1 --2- 172.21.15.73:0/713347774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3f40d3bc0 0x7fd3f4150320 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:39.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.805+0000 7fd3fa6cf640 1 --2- 172.21.15.73:0/713347774 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd3e006c8f0 0x7fd3e006edb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:39.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.805+0000 7fd3fa6cf640 1 --2- 172.21.15.73:0/713347774 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd3f4162880 0x7fd3f40cd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:39.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.805+0000 7fd3fa6cf640 1 --2- 172.21.15.73:0/713347774 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd3f40b5f20 0x7fd3f415c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:39.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.805+0000 7fd3fa6cf640 1 -- 172.21.15.73:0/713347774 >> 172.21.15.73:0/713347774 conn(0x7fd3f40d4f60 msgr2=0x7fd3f415f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:39.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.805+0000 7fd3fa6cf640 1 -- 172.21.15.73:0/713347774 shutdown_connections 2024-04-26T20:50:39.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.805+0000 7fd3fa6cf640 1 -- 172.21.15.73:0/713347774 wait complete. 2024-04-26T20:50:39.817 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T20:50:39.817 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-26T20:50:39.817 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T20:50:39.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.913+0000 7fb40bff0640 1 Processor -- start 2024-04-26T20:50:39.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.913+0000 7fb40bff0640 1 -- start start 2024-04-26T20:50:39.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.914+0000 7fb40bff0640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb4040b2340 0x7fb4040cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:39.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.914+0000 7fb40bff0640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fb4040cd930 0x7fb4040d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:39.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.914+0000 7fb40bff0640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4040d10d0 con 0x7fb404162810 2024-04-26T20:50:39.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.914+0000 7fb40bff0640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7fb4040d0750 con 0x7fb4040b2340 2024-04-26T20:50:39.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.914+0000 7fb40bff0640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fb4040d16a0 con 0x7fb4040cd930 2024-04-26T20:50:39.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.914+0000 7fb40958b640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb4040b2340 0x7fb4040cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:39.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.914+0000 7fb409d8c640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fb404162810 0x7fb404070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40952/0 (socket says 172.21.15.73:40952) 2024-04-26T20:50:39.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.914+0000 7fb40958b640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb4040b2340 0x7fb4040cd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34850/0 (socket says 172.21.15.73:34850) 2024-04-26T20:50:39.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.914+0000 7fb409d8c640 1 -- 172.21.15.73:0/389412890 learned_addr learned my addr 172.21.15.73:0/389412890 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:50:39.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.914+0000 7fb40a58d640 1 --2- 172.21.15.73:0/389412890 >> v2:172.21.15.120:3300/0 conn(0x7fb4040cd930 0x7fb4040d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:39.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.914+0000 7fb40958b640 1 -- 172.21.15.73:0/389412890 >> v2:172.21.15.120:3300/0 conn(0x7fb4040cd930 msgr2=0x7fb4040d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:39.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.914+0000 7fb40958b640 1 --2- 172.21.15.73:0/389412890 >> v2:172.21.15.120:3300/0 conn(0x7fb4040cd930 0x7fb4040d1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:39.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.914+0000 7fb40958b640 1 -- 172.21.15.73:0/389412890 >> v1:172.21.15.73:6789/0 conn(0x7fb404162810 legacy=0x7fb404070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:39.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.914+0000 7fb40958b640 1 -- 172.21.15.73:0/389412890 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4040d1370 con 0x7fb4040b2340 2024-04-26T20:50:39.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.915+0000 7fb40958b640 1 --2- 172.21.15.73:0/389412890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4040b2340 0x7fb4040cd3f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb4040b2280 tx=0x7fb3f402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd0bfa0cc66dbddd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:39.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.915+0000 7fb408d8a640 1 -- 172.21.15.73:0/389412890 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4037070 con 0x7fb4040b2340 2024-04-26T20:50:39.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.915+0000 7fb408d8a640 1 -- 172.21.15.73:0/389412890 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3f4032440 con 0x7fb4040b2340 2024-04-26T20:50:39.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.915+0000 7fb408d8a640 1 -- 172.21.15.73:0/389412890 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4032810 con 0x7fb4040b2340 2024-04-26T20:50:39.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.915+0000 7fb40bff0640 1 -- 172.21.15.73:0/389412890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4040b2340 msgr2=0x7fb4040cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:39.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.915+0000 7fb40bff0640 1 --2- 172.21.15.73:0/389412890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4040b2340 0x7fb4040cd3f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb4040b2280 tx=0x7fb3f402a1a0 comp rx=0 tx=0).stop 2024-04-26T20:50:39.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.915+0000 7fb40bff0640 1 -- 172.21.15.73:0/389412890 shutdown_connections 2024-04-26T20:50:39.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.915+0000 7fb40bff0640 1 --2- 172.21.15.73:0/389412890 >> v2:172.21.15.120:3300/0 conn(0x7fb4040cd930 0x7fb4040d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:39.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.915+0000 7fb40bff0640 1 --2- 172.21.15.73:0/389412890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4040b2340 0x7fb4040cd3f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:39.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.915+0000 7fb40bff0640 1 -- 172.21.15.73:0/389412890 >> 172.21.15.73:0/389412890 conn(0x7fb4040d2da0 msgr2=0x7fb4040d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:39.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.915+0000 7fb40bff0640 1 -- 172.21.15.73:0/389412890 shutdown_connections 2024-04-26T20:50:39.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.915+0000 7fb40bff0640 1 -- 172.21.15.73:0/389412890 wait complete. 2024-04-26T20:50:39.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.916+0000 7fb40bff0640 1 Processor -- start 2024-04-26T20:50:39.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.917+0000 7fb40bff0640 1 -- start start 2024-04-26T20:50:39.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.917+0000 7fb40bff0640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4040b2340 0x7fb4040cca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:39.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.917+0000 7fb40bff0640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb4040cd930 0x7fb404157960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:39.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.917+0000 7fb40bff0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb404162810 0x7fb4041f4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:39.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.917+0000 7fb40bff0640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb4040d2070 con 0x7fb4040b2340 2024-04-26T20:50:39.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.917+0000 7fb40bff0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb4040d0470 con 0x7fb404162810 2024-04-26T20:50:39.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.917+0000 7fb40bff0640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fb4040d21f0 con 0x7fb4040cd930 2024-04-26T20:50:39.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.917+0000 7fb409d8c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4040b2340 0x7fb4040cca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:39.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.917+0000 7fb409d8c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4040b2340 0x7fb4040cca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34858/0 (socket says 172.21.15.73:34858) 2024-04-26T20:50:39.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.917+0000 7fb409d8c640 1 -- 172.21.15.73:0/3659609333 learned_addr learned my addr 172.21.15.73:0/3659609333 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:39.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.918+0000 7fb409d8c640 1 -- 172.21.15.73:0/3659609333 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb4040cd930 msgr2=0x7fb404157960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:39.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.918+0000 7fb409d8c640 1 --2- 172.21.15.73:0/3659609333 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb4040cd930 0x7fb404157960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:39.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.918+0000 7fb409d8c640 1 -- 172.21.15.73:0/3659609333 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb404162810 msgr2=0x7fb4041f4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:39.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.918+0000 7fb409d8c640 1 --2- 172.21.15.73:0/3659609333 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb404162810 0x7fb4041f4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:39.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.918+0000 7fb409d8c640 1 -- 172.21.15.73:0/3659609333 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4041600b0 con 0x7fb4040b2340 2024-04-26T20:50:39.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.918+0000 7fb409d8c640 1 --2- 172.21.15.73:0/3659609333 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4040b2340 0x7fb4040cca30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb40000c3f0 tx=0x7fb40000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:39.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.918+0000 7fb3fa7fc640 1 -- 172.21.15.73:0/3659609333 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb400017070 con 0x7fb4040b2340 2024-04-26T20:50:39.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.918+0000 7fb3fa7fc640 1 -- 172.21.15.73:0/3659609333 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb400005ce0 con 0x7fb4040b2340 2024-04-26T20:50:39.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.918+0000 7fb40bff0640 1 -- 172.21.15.73:0/3659609333 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb404160390 con 0x7fb4040b2340 2024-04-26T20:50:39.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.918+0000 7fb3fa7fc640 1 -- 172.21.15.73:0/3659609333 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4000072f0 con 0x7fb4040b2340 2024-04-26T20:50:39.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.918+0000 7fb40bff0640 1 -- 172.21.15.73:0/3659609333 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb404160950 con 0x7fb4040b2340 2024-04-26T20:50:39.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.919+0000 7fb40bff0640 1 -- 172.21.15.73:0/3659609333 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4040cfb20 con 0x7fb4040b2340 2024-04-26T20:50:39.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.919+0000 7fb3fa7fc640 1 -- 172.21.15.73:0/3659609333 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb40001a070 con 0x7fb4040b2340 2024-04-26T20:50:39.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.920+0000 7fb3fa7fc640 1 --2- 172.21.15.73:0/3659609333 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb3e806c8f0 0x7fb3e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:39.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.920+0000 7fb40958b640 1 --2- 172.21.15.73:0/3659609333 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb3e806c8f0 0x7fb3e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:39.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.920+0000 7fb3fa7fc640 1 -- 172.21.15.73:0/3659609333 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fb40008b210 con 0x7fb4040b2340 2024-04-26T20:50:39.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.920+0000 7fb40958b640 1 --2- 172.21.15.73:0/3659609333 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb3e806c8f0 0x7fb3e806edb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4034000 tx=0x7fb3f402c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:39.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:39.924+0000 7fb3fa7fc640 1 -- 172.21.15.73:0/3659609333 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb4040cfb20 con 0x7fb4040b2340 2024-04-26T20:50:40.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:40.139+0000 7fb40bff0640 1 -- 172.21.15.73:0/3659609333 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb4040694f0 con 0x7fb4040b2340 2024-04-26T20:50:40.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:40.140+0000 7fb3fa7fc640 1 -- 172.21.15.73:0/3659609333 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) ==== 53+0+88 (secure 0 0 0) 0x7fb400059300 con 0x7fb4040b2340 2024-04-26T20:50:40.141 INFO:teuthology.orchestra.run.smithi073.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-26T20:50:40.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:40.141+0000 7fb40bff0640 1 -- 172.21.15.73:0/3659609333 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb3e806c8f0 msgr2=0x7fb3e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:40.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:40.141+0000 7fb40bff0640 1 --2- 172.21.15.73:0/3659609333 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb3e806c8f0 0x7fb3e806edb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4034000 tx=0x7fb3f402c000 comp rx=0 tx=0).stop 2024-04-26T20:50:40.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:40.141+0000 7fb40bff0640 1 -- 172.21.15.73:0/3659609333 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4040b2340 msgr2=0x7fb4040cca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:40.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:40.141+0000 7fb40bff0640 1 --2- 172.21.15.73:0/3659609333 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4040b2340 0x7fb4040cca30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb40000c3f0 tx=0x7fb40000c8b0 comp rx=0 tx=0).stop 2024-04-26T20:50:40.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:40.141+0000 7fb40bff0640 1 -- 172.21.15.73:0/3659609333 shutdown_connections 2024-04-26T20:50:40.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:40.141+0000 7fb40bff0640 1 --2- 172.21.15.73:0/3659609333 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb3e806c8f0 0x7fb3e806edb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:40.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:40.142+0000 7fb40bff0640 1 --2- 172.21.15.73:0/3659609333 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb404162810 0x7fb4041f4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:40.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:40.142+0000 7fb40bff0640 1 --2- 172.21.15.73:0/3659609333 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb4040cd930 0x7fb404157960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:40.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:40.142+0000 7fb40bff0640 1 --2- 172.21.15.73:0/3659609333 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4040b2340 0x7fb4040cca30 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:40.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:40.142+0000 7fb40bff0640 1 -- 172.21.15.73:0/3659609333 >> 172.21.15.73:0/3659609333 conn(0x7fb4040d2da0 msgr2=0x7fb4040cf410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:40.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:40.142+0000 7fb40bff0640 1 -- 172.21.15.73:0/3659609333 shutdown_connections 2024-04-26T20:50:40.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:40.142+0000 7fb40bff0640 1 -- 172.21.15.73:0/3659609333 wait complete. 2024-04-26T20:50:40.153 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T20:50:40.153 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T20:50:40.153 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi073.front.sepia.ceph.com 2024-04-26T20:50:40.153 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-26T20:50:40.153 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-26T20:50:40.153 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T20:50:40.153 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T20:50:40.153 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T20:50:40.154 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-26T20:50:40.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:50:40.154 DEBUG:teuthology.orchestra.run.smithi073:> ip addr 2024-04-26T20:50:40.169 INFO:teuthology.orchestra.run.smithi073.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:50:40.169 INFO:teuthology.orchestra.run.smithi073.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:50:40.169 INFO:teuthology.orchestra.run.smithi073.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:50:40.169 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:50:40.169 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 ::1/128 scope host 2024-04-26T20:50:40.169 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:50:40.169 INFO:teuthology.orchestra.run.smithi073.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:50:40.169 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:ea brd ff:ff:ff:ff:ff:ff 2024-04-26T20:50:40.169 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f0 2024-04-26T20:50:40.170 INFO:teuthology.orchestra.run.smithi073.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:50:40.170 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:eb brd ff:ff:ff:ff:ff:ff 2024-04-26T20:50:40.170 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f1 2024-04-26T20:50:40.170 INFO:teuthology.orchestra.run.smithi073.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:50:40.170 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7c brd ff:ff:ff:ff:ff:ff 2024-04-26T20:50:40.170 INFO:teuthology.orchestra.run.smithi073.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T20:50:40.170 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7d brd ff:ff:ff:ff:ff:ff 2024-04-26T20:50:40.170 INFO:teuthology.orchestra.run.smithi073.stdout: inet 172.21.15.73/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-26T20:50:40.170 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft 42747sec preferred_lft 42747sec 2024-04-26T20:50:40.170 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::b99b:8e91:ffc2:71f/64 scope link noprefixroute 2024-04-26T20:50:40.170 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:50:40.171 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T20:50:40.171 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:50:40.171 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2024-04-26T20:50:40.171 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link add name ceph-brx type bridge 2024-04-26T20:50:40.172 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip addr flush dev ceph-brx 2024-04-26T20:50:40.172 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link set ceph-brx up 2024-04-26T20:50:40.172 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T20:50:40.172 DEBUG:teuthology.orchestra.run.smithi073:> ') 2024-04-26T20:50:40.256 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:50:40.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:50:40.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:50:40.359 DEBUG:teuthology.orchestra.run.smithi073:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T20:50:40.382 INFO:teuthology.orchestra.run.smithi073.stdout:1 2024-04-26T20:50:40.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:50:40.384 DEBUG:teuthology.orchestra.run.smithi073:> route 2024-04-26T20:50:40.450 INFO:teuthology.orchestra.run.smithi073.stdout:Kernel IP routing table 2024-04-26T20:50:40.451 INFO:teuthology.orchestra.run.smithi073.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T20:50:40.451 INFO:teuthology.orchestra.run.smithi073.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T20:50:40.451 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T20:50:40.451 INFO:teuthology.orchestra.run.smithi073.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T20:50:40.451 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:50:40.451 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2024-04-26T20:50:40.451 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T20:50:40.451 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T20:50:40.451 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T20:50:40.452 DEBUG:teuthology.orchestra.run.smithi073:> ') 2024-04-26T20:50:40.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:50:40.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:50:40.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:50:40.736 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list 2024-04-26T20:50:40.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:50:40.751 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list-id 2024-04-26T20:50:40.805 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:50:40.805 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2024-04-26T20:50:40.805 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T20:50:40.805 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-26T20:50:40.805 DEBUG:teuthology.orchestra.run.smithi073:> ') 2024-04-26T20:50:40.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:50:40.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:50:40.921 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-26T20:50:40.921 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:50:40.922 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2024-04-26T20:50:40.922 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-26T20:50:40.922 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T20:50:40.922 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-26T20:50:40.922 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-26T20:50:40.922 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-26T20:50:40.922 DEBUG:teuthology.orchestra.run.smithi073:> ') 2024-04-26T20:50:40.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:50:41.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:50:41.039 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:50:41.040 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2024-04-26T20:50:41.040 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link set brx.0 up 2024-04-26T20:50:41.040 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T20:50:41.040 DEBUG:teuthology.orchestra.run.smithi073:> ') 2024-04-26T20:50:41.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:50:41.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:50:41.113 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-26T20:50:41.113 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:50:41.114 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-26T20:50:41.128 INFO:teuthology.orchestra.run.smithi073.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-26T20:50:41.128 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:50:41.128 DEBUG:teuthology.orchestra.run.smithi073:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-26T20:50:41.183 DEBUG:teuthology.orchestra.run.smithi073:> sudo modprobe fuse 2024-04-26T20:50:41.247 DEBUG:teuthology.orchestra.run.smithi073:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T20:50:41.303 INFO:teuthology.orchestra.run.smithi073.stdout:/proc 2024-04-26T20:50:41.303 INFO:teuthology.orchestra.run.smithi073.stdout:/sys 2024-04-26T20:50:41.303 INFO:teuthology.orchestra.run.smithi073.stdout:/dev 2024-04-26T20:50:41.303 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/kernel/security 2024-04-26T20:50:41.303 INFO:teuthology.orchestra.run.smithi073.stdout:/dev/shm 2024-04-26T20:50:41.303 INFO:teuthology.orchestra.run.smithi073.stdout:/dev/pts 2024-04-26T20:50:41.304 INFO:teuthology.orchestra.run.smithi073.stdout:/run 2024-04-26T20:50:41.304 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/fs/cgroup 2024-04-26T20:50:41.304 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/fs/pstore 2024-04-26T20:50:41.304 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/fs/bpf 2024-04-26T20:50:41.304 INFO:teuthology.orchestra.run.smithi073.stdout:/ 2024-04-26T20:50:41.304 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/fs/selinux 2024-04-26T20:50:41.304 INFO:teuthology.orchestra.run.smithi073.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T20:50:41.304 INFO:teuthology.orchestra.run.smithi073.stdout:/dev/hugepages 2024-04-26T20:50:41.304 INFO:teuthology.orchestra.run.smithi073.stdout:/dev/mqueue 2024-04-26T20:50:41.304 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/kernel/debug 2024-04-26T20:50:41.304 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/kernel/tracing 2024-04-26T20:50:41.304 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/kernel/config 2024-04-26T20:50:41.304 INFO:teuthology.orchestra.run.smithi073.stdout:/run/credentials/systemd-sysusers.service 2024-04-26T20:50:41.304 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/fs/fuse/connections 2024-04-26T20:50:41.305 INFO:teuthology.orchestra.run.smithi073.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T20:50:41.305 INFO:teuthology.orchestra.run.smithi073.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T20:50:41.305 INFO:teuthology.orchestra.run.smithi073.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T20:50:41.305 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T20:50:41.305 INFO:teuthology.orchestra.run.smithi073.stdout:/run/user/1000 2024-04-26T20:50:41.305 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/ceph 2024-04-26T20:50:41.305 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/ceph/osd/ceph-0 2024-04-26T20:50:41.305 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/ceph/osd/ceph-1 2024-04-26T20:50:41.305 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/ceph/osd/ceph-2 2024-04-26T20:50:41.305 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/ceph/osd/ceph-3 2024-04-26T20:50:41.305 INFO:teuthology.orchestra.run.smithi073.stdout:/run/netns 2024-04-26T20:50:41.305 INFO:teuthology.orchestra.run.smithi073.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T20:50:41.305 INFO:teuthology.orchestra.run.smithi073.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T20:50:41.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:50:41.306 DEBUG:teuthology.orchestra.run.smithi073:> ls /sys/fs/fuse/connections 2024-04-26T20:50:41.358 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-26T20:50:41.358 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-26T20:50:41.401 DEBUG:teuthology.orchestra.run.smithi073:> sudo modprobe fuse 2024-04-26T20:50:41.423 DEBUG:teuthology.orchestra.run.smithi073:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T20:50:41.479 INFO:teuthology.orchestra.run.smithi073.stdout:/proc 2024-04-26T20:50:41.479 INFO:teuthology.orchestra.run.smithi073.stdout:/sys 2024-04-26T20:50:41.479 INFO:teuthology.orchestra.run.smithi073.stdout:/dev 2024-04-26T20:50:41.479 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/kernel/security 2024-04-26T20:50:41.479 INFO:teuthology.orchestra.run.smithi073.stdout:/dev/shm 2024-04-26T20:50:41.479 INFO:teuthology.orchestra.run.smithi073.stdout:/dev/pts 2024-04-26T20:50:41.479 INFO:teuthology.orchestra.run.smithi073.stdout:/run 2024-04-26T20:50:41.479 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/fs/cgroup 2024-04-26T20:50:41.479 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/fs/pstore 2024-04-26T20:50:41.480 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/fs/bpf 2024-04-26T20:50:41.480 INFO:teuthology.orchestra.run.smithi073.stdout:/ 2024-04-26T20:50:41.480 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/fs/selinux 2024-04-26T20:50:41.480 INFO:teuthology.orchestra.run.smithi073.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T20:50:41.480 INFO:teuthology.orchestra.run.smithi073.stdout:/dev/hugepages 2024-04-26T20:50:41.480 INFO:teuthology.orchestra.run.smithi073.stdout:/dev/mqueue 2024-04-26T20:50:41.480 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/kernel/debug 2024-04-26T20:50:41.480 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/kernel/tracing 2024-04-26T20:50:41.480 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/kernel/config 2024-04-26T20:50:41.480 INFO:teuthology.orchestra.run.smithi073.stdout:/run/credentials/systemd-sysusers.service 2024-04-26T20:50:41.480 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/fs/fuse/connections 2024-04-26T20:50:41.480 INFO:teuthology.orchestra.run.smithi073.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T20:50:41.480 INFO:teuthology.orchestra.run.smithi073.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T20:50:41.480 INFO:teuthology.orchestra.run.smithi073.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T20:50:41.480 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T20:50:41.481 INFO:teuthology.orchestra.run.smithi073.stdout:/run/user/1000 2024-04-26T20:50:41.481 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/ceph 2024-04-26T20:50:41.481 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/ceph/osd/ceph-0 2024-04-26T20:50:41.481 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/ceph/osd/ceph-1 2024-04-26T20:50:41.481 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/ceph/osd/ceph-2 2024-04-26T20:50:41.481 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/ceph/osd/ceph-3 2024-04-26T20:50:41.481 INFO:teuthology.orchestra.run.smithi073.stdout:/run/netns 2024-04-26T20:50:41.481 INFO:teuthology.orchestra.run.smithi073.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T20:50:41.481 INFO:teuthology.orchestra.run.smithi073.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T20:50:41.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:50:41.481 DEBUG:teuthology.orchestra.run.smithi073:> ls /sys/fs/fuse/connections 2024-04-26T20:50:41.505 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi073.stderr:2024-04-26T20:50:41.503+0000 7f3a1d179480 -1 init, newargv = 0x563c9af05000 newargc=15 2024-04-26T20:50:41.506 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi073.stderr:ceph-fuse[41427]: starting ceph client 2024-04-26T20:50:41.510 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi073.stderr:ceph-fuse[41427]: starting fuse 2024-04-26T20:50:41.520 INFO:teuthology.orchestra.run.smithi073.stdout:43 2024-04-26T20:50:41.520 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-04-26T20:50:41.520 DEBUG:teuthology.orchestra.run.smithi073:> sudo stdin-killer -- python3 -c ' 2024-04-26T20:50:41.520 DEBUG:teuthology.orchestra.run.smithi073:> import glob 2024-04-26T20:50:41.520 DEBUG:teuthology.orchestra.run.smithi073:> import re 2024-04-26T20:50:41.520 DEBUG:teuthology.orchestra.run.smithi073:> import os 2024-04-26T20:50:41.520 DEBUG:teuthology.orchestra.run.smithi073:> import subprocess 2024-04-26T20:50:41.520 DEBUG:teuthology.orchestra.run.smithi073:> 2024-04-26T20:50:41.520 DEBUG:teuthology.orchestra.run.smithi073:> def _find_admin_socket(client_name): 2024-04-26T20:50:41.520 DEBUG:teuthology.orchestra.run.smithi073:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-26T20:50:41.521 DEBUG:teuthology.orchestra.run.smithi073:> files = glob.glob(asok_path) 2024-04-26T20:50:41.521 DEBUG:teuthology.orchestra.run.smithi073:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-26T20:50:41.521 DEBUG:teuthology.orchestra.run.smithi073:> 2024-04-26T20:50:41.521 DEBUG:teuthology.orchestra.run.smithi073:> # Given a non-glob path, it better be there 2024-04-26T20:50:41.521 DEBUG:teuthology.orchestra.run.smithi073:> if "*" not in asok_path: 2024-04-26T20:50:41.521 DEBUG:teuthology.orchestra.run.smithi073:> assert(len(files) == 1) 2024-04-26T20:50:41.521 DEBUG:teuthology.orchestra.run.smithi073:> return files[0] 2024-04-26T20:50:41.521 DEBUG:teuthology.orchestra.run.smithi073:> 2024-04-26T20:50:41.521 DEBUG:teuthology.orchestra.run.smithi073:> for f in files: 2024-04-26T20:50:41.521 DEBUG:teuthology.orchestra.run.smithi073:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-26T20:50:41.521 DEBUG:teuthology.orchestra.run.smithi073:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-26T20:50:41.521 DEBUG:teuthology.orchestra.run.smithi073:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-26T20:50:41.521 DEBUG:teuthology.orchestra.run.smithi073:> contents = proc_f.read() 2024-04-26T20:50:41.521 DEBUG:teuthology.orchestra.run.smithi073:> if mountpoint in contents: 2024-04-26T20:50:41.521 DEBUG:teuthology.orchestra.run.smithi073:> return f 2024-04-26T20:50:41.522 DEBUG:teuthology.orchestra.run.smithi073:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-26T20:50:41.522 DEBUG:teuthology.orchestra.run.smithi073:> 2024-04-26T20:50:41.522 DEBUG:teuthology.orchestra.run.smithi073:> print(_find_admin_socket("client.admin")) 2024-04-26T20:50:41.522 DEBUG:teuthology.orchestra.run.smithi073:> ' 2024-04-26T20:50:41.636 INFO:teuthology.orchestra.run.smithi073.stdout:/var/run/ceph/ceph-client.admin.41427.asok 2024-04-26T20:50:41.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:50:41.646 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.41427.asok 2024-04-26T20:50:41.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:50:41.646 DEBUG:teuthology.orchestra.run.smithi073:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.41427.asok status 2024-04-26T20:50:41.780 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-26T20:50:41.780 INFO:teuthology.orchestra.run.smithi073.stdout: "metadata": { 2024-04-26T20:50:41.780 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_sha1": "bc612ffb9c1d5bc27536bc917a705262a36d4387", 2024-04-26T20:50:41.781 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T20:50:41.781 INFO:teuthology.orchestra.run.smithi073.stdout: "entity_id": "admin", 2024-04-26T20:50:41.781 INFO:teuthology.orchestra.run.smithi073.stdout: "hostname": "smithi073", 2024-04-26T20:50:41.781 INFO:teuthology.orchestra.run.smithi073.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-26T20:50:41.781 INFO:teuthology.orchestra.run.smithi073.stdout: "pid": "41427", 2024-04-26T20:50:41.781 INFO:teuthology.orchestra.run.smithi073.stdout: "root": "/" 2024-04-26T20:50:41.781 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-26T20:50:41.781 INFO:teuthology.orchestra.run.smithi073.stdout: "dentry_count": 0, 2024-04-26T20:50:41.781 INFO:teuthology.orchestra.run.smithi073.stdout: "dentry_pinned_count": 0, 2024-04-26T20:50:41.781 INFO:teuthology.orchestra.run.smithi073.stdout: "id": 4458, 2024-04-26T20:50:41.781 INFO:teuthology.orchestra.run.smithi073.stdout: "inst": { 2024-04-26T20:50:41.781 INFO:teuthology.orchestra.run.smithi073.stdout: "name": { 2024-04-26T20:50:41.781 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "client", 2024-04-26T20:50:41.781 INFO:teuthology.orchestra.run.smithi073.stdout: "num": 4458 2024-04-26T20:50:41.781 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-26T20:50:41.782 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": { 2024-04-26T20:50:41.782 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-04-26T20:50:41.782 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "192.168.0.1:0", 2024-04-26T20:50:41.782 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2189191731 2024-04-26T20:50:41.782 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-26T20:50:41.782 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-26T20:50:41.782 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": { 2024-04-26T20:50:41.782 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-04-26T20:50:41.782 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "192.168.0.1:0", 2024-04-26T20:50:41.782 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2189191731 2024-04-26T20:50:41.782 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-26T20:50:41.782 INFO:teuthology.orchestra.run.smithi073.stdout: "inst_str": "client.4458 192.168.0.1:0/2189191731", 2024-04-26T20:50:41.782 INFO:teuthology.orchestra.run.smithi073.stdout: "addr_str": "192.168.0.1:0/2189191731", 2024-04-26T20:50:41.782 INFO:teuthology.orchestra.run.smithi073.stdout: "inode_count": 1, 2024-04-26T20:50:41.782 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_epoch": 6, 2024-04-26T20:50:41.783 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_epoch": 25, 2024-04-26T20:50:41.783 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_epoch_barrier": 0, 2024-04-26T20:50:41.783 INFO:teuthology.orchestra.run.smithi073.stdout: "blocklisted": false, 2024-04-26T20:50:41.783 INFO:teuthology.orchestra.run.smithi073.stdout: "fs_name": "cephfs" 2024-04-26T20:50:41.783 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-26T20:50:41.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:50:41.789 DEBUG:teuthology.orchestra.run.smithi073:> stat --file-system '--printf=%T 2024-04-26T20:50:41.789 DEBUG:teuthology.orchestra.run.smithi073:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-26T20:50:41.804 INFO:teuthology.orchestra.run.smithi073.stdout:fuseblk 2024-04-26T20:50:41.804 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-26T20:50:41.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:50:41.804 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-26T20:50:41.869 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-26T20:50:41.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:41 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T20:50:41.976 INFO:teuthology.orchestra.run.smithi073.stdout:# file: . 2024-04-26T20:50:41.976 INFO:teuthology.orchestra.run.smithi073.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-26T20:50:41.976 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:50:41.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:50:41.982 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-26T20:50:41.983 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:50:41.998 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:50:42.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.134+0000 7efc8a133640 1 Processor -- start 2024-04-26T20:50:42.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.134+0000 7efc8a133640 1 -- start start 2024-04-26T20:50:42.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.135+0000 7efc8a133640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efc840cf5e0 0x7efc840ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:42.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.135+0000 7efc8a133640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7efc840cbbd0 0x7efc840cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:42.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.135+0000 7efc8a133640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc8415b9e0 con 0x7efc8415b600 2024-04-26T20:50:42.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.135+0000 7efc8a133640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7efc840cb9b0 con 0x7efc840cbbd0 2024-04-26T20:50:42.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.135+0000 7efc8a133640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7efc840d0740 con 0x7efc840cf5e0 2024-04-26T20:50:42.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.135+0000 7efc837fe640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7efc8415b600 0x7efc84071f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40954/0 (socket says 172.21.15.73:40954) 2024-04-26T20:50:42.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.135+0000 7efc837fe640 1 -- 172.21.15.73:0/3537744764 learned_addr learned my addr 172.21.15.73:0/3537744764 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:50:42.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.135+0000 7efc82ffd640 1 --2- 172.21.15.73:0/3537744764 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efc840cf5e0 0x7efc840ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:42.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.135+0000 7efc83fff640 1 --2- 172.21.15.73:0/3537744764 >> v2:172.21.15.120:3300/0 conn(0x7efc840cbbd0 0x7efc840cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:42.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.135+0000 7efc827fc640 1 -- 172.21.15.73:0/3537744764 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2007914423 0 0) 0x7efc8415b9e0 con 0x7efc8415b600 2024-04-26T20:50:42.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.135+0000 7efc827fc640 1 -- 172.21.15.73:0/3537744764 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efc6c005730 con 0x7efc8415b600 2024-04-26T20:50:42.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.135+0000 7efc827fc640 1 -- 172.21.15.73:0/3537744764 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1603830906 0 0) 0x7efc6c005730 con 0x7efc8415b600 2024-04-26T20:50:42.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.135+0000 7efc827fc640 1 -- 172.21.15.73:0/3537744764 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efc840cf5e0 msgr2=0x7efc840ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:42.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.135+0000 7efc827fc640 1 --2- 172.21.15.73:0/3537744764 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efc840cf5e0 0x7efc840ce9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:42.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.135+0000 7efc827fc640 1 -- 172.21.15.73:0/3537744764 >> v2:172.21.15.120:3300/0 conn(0x7efc840cbbd0 msgr2=0x7efc840cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:42.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.135+0000 7efc827fc640 1 --2- 172.21.15.73:0/3537744764 >> v2:172.21.15.120:3300/0 conn(0x7efc840cbbd0 0x7efc840cef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:42.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.135+0000 7efc827fc640 1 -- 172.21.15.73:0/3537744764 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc840cb360 con 0x7efc8415b600 2024-04-26T20:50:42.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.135+0000 7efc827fc640 1 -- 172.21.15.73:0/3537744764 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7efc74001670 con 0x7efc8415b600 2024-04-26T20:50:42.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.135+0000 7efc827fc640 1 -- 172.21.15.73:0/3537744764 >> v1:172.21.15.73:6789/0 conn(0x7efc8415b600 legacy=0x7efc84071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:42.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.135+0000 7efc827fc640 1 --2- 172.21.15.73:0/3537744764 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc6c005bc0 0x7efc6c005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:42.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.136+0000 7efc827fc640 1 --2- 172.21.15.73:0/3537744764 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efc840cf5e0 0x7efc6c006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:42.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.136+0000 7efc827fc640 1 --2- 172.21.15.73:0/3537744764 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc6c006f50 0x7efc6c0073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:42.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.136+0000 7efc827fc640 1 -- 172.21.15.73:0/3537744764 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efc840cb9b0 con 0x7efc6c006f50 2024-04-26T20:50:42.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.136+0000 7efc827fc640 1 -- 172.21.15.73:0/3537744764 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efc840d0740 con 0x7efc6c005bc0 2024-04-26T20:50:42.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.136+0000 7efc827fc640 1 -- 172.21.15.73:0/3537744764 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7efc6c00dbd0 con 0x7efc840cf5e0 2024-04-26T20:50:42.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.136+0000 7efc83fff640 1 --2- 172.21.15.73:0/3537744764 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc6c006f50 0x7efc6c0073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:42.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.136+0000 7efc837fe640 1 --2- 172.21.15.73:0/3537744764 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc6c005bc0 0x7efc6c005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:42.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.136+0000 7efc82ffd640 1 --2- 172.21.15.73:0/3537744764 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efc840cf5e0 0x7efc6c006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:42.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.136+0000 7efc83fff640 1 -- 172.21.15.73:0/3537744764 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efc840cf5e0 msgr2=0x7efc6c006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:42.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.136+0000 7efc83fff640 1 --2- 172.21.15.73:0/3537744764 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efc840cf5e0 0x7efc6c006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:42.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.136+0000 7efc83fff640 1 -- 172.21.15.73:0/3537744764 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc6c005bc0 msgr2=0x7efc6c005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:42.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.136+0000 7efc83fff640 1 --2- 172.21.15.73:0/3537744764 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc6c005bc0 0x7efc6c005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:42.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.136+0000 7efc83fff640 1 -- 172.21.15.73:0/3537744764 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc6c005730 con 0x7efc6c006f50 2024-04-26T20:50:42.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.137+0000 7efc83fff640 1 --2- 172.21.15.73:0/3537744764 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc6c006f50 0x7efc6c0073d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7efc7c008330 tx=0x7efc6c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bdab715f8eb99c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:42.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.137+0000 7efc827fc640 1 -- 172.21.15.73:0/3537744764 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7c039070 con 0x7efc6c006f50 2024-04-26T20:50:42.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.137+0000 7efc827fc640 1 -- 172.21.15.73:0/3537744764 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc7c034680 con 0x7efc6c006f50 2024-04-26T20:50:42.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.137+0000 7efc827fc640 1 -- 172.21.15.73:0/3537744764 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7c00a2c0 con 0x7efc6c006f50 2024-04-26T20:50:42.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.137+0000 7efc8a133640 1 -- 172.21.15.73:0/3537744764 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc6c006f50 msgr2=0x7efc6c0073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:42.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.137+0000 7efc8a133640 1 --2- 172.21.15.73:0/3537744764 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc6c006f50 0x7efc6c0073d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7efc7c008330 tx=0x7efc6c007a40 comp rx=0 tx=0).stop 2024-04-26T20:50:42.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.137+0000 7efc83fff640 1 -- 172.21.15.73:0/3537744764 reap_dead start 2024-04-26T20:50:42.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.137+0000 7efc8a133640 1 -- 172.21.15.73:0/3537744764 shutdown_connections 2024-04-26T20:50:42.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.137+0000 7efc8a133640 1 -- 172.21.15.73:0/3537744764 >> 172.21.15.73:0/3537744764 conn(0x7efc840d0be0 msgr2=0x7efc840d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:42.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.137+0000 7efc8a133640 1 -- 172.21.15.73:0/3537744764 shutdown_connections 2024-04-26T20:50:42.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.137+0000 7efc8a133640 1 -- 172.21.15.73:0/3537744764 wait complete. 2024-04-26T20:50:42.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.138+0000 7efc8a133640 1 Processor -- start 2024-04-26T20:50:42.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.139+0000 7efc8a133640 1 -- start start 2024-04-26T20:50:42.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.139+0000 7efc8a133640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc6c006f50 0x7efc84156ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:42.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.139+0000 7efc8a133640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efc840c97e0 0x7efc8414f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:42.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.139+0000 7efc8a133640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc840cd580 0x7efc840cda00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:42.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.139+0000 7efc8a133640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efc840d0470 con 0x7efc840cd580 2024-04-26T20:50:42.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.139+0000 7efc8a133640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efc840d02f0 con 0x7efc6c006f50 2024-04-26T20:50:42.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.139+0000 7efc8a133640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7efc840ce9d0 con 0x7efc840c97e0 2024-04-26T20:50:42.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.139+0000 7efc83fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc840cd580 0x7efc840cda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:42.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.139+0000 7efc83fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc840cd580 0x7efc840cda00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34872/0 (socket says 172.21.15.73:34872) 2024-04-26T20:50:42.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.139+0000 7efc83fff640 1 -- 172.21.15.73:0/2006508322 learned_addr learned my addr 172.21.15.73:0/2006508322 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:42.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.140+0000 7efc837fe640 1 --2- 172.21.15.73:0/2006508322 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc6c006f50 0x7efc84156ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:42.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.140+0000 7efc82ffd640 1 --2- 172.21.15.73:0/2006508322 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efc840c97e0 0x7efc8414f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:42.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.140+0000 7efc83fff640 1 -- 172.21.15.73:0/2006508322 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efc840c97e0 msgr2=0x7efc8414f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:42.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.140+0000 7efc83fff640 1 --2- 172.21.15.73:0/2006508322 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efc840c97e0 0x7efc8414f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:42.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.140+0000 7efc83fff640 1 -- 172.21.15.73:0/2006508322 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc6c006f50 msgr2=0x7efc84156ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:42.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.140+0000 7efc83fff640 1 --2- 172.21.15.73:0/2006508322 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc6c006f50 0x7efc84156ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:42.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.140+0000 7efc83fff640 1 -- 172.21.15.73:0/2006508322 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc840c9c20 con 0x7efc840cd580 2024-04-26T20:50:42.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.140+0000 7efc837fe640 1 --2- 172.21.15.73:0/2006508322 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc6c006f50 0x7efc84156ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:50:42.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.140+0000 7efc83fff640 1 --2- 172.21.15.73:0/2006508322 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc840cd580 0x7efc840cda00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7efc7c00ada0 tx=0x7efc7c00a7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:42.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.140+0000 7efc6bfff640 1 -- 172.21.15.73:0/2006508322 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7c039040 con 0x7efc840cd580 2024-04-26T20:50:42.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.140+0000 7efc6bfff640 1 -- 172.21.15.73:0/2006508322 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc7c034830 con 0x7efc840cd580 2024-04-26T20:50:42.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.140+0000 7efc8a133640 1 -- 172.21.15.73:0/2006508322 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc840c9ec0 con 0x7efc840cd580 2024-04-26T20:50:42.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.140+0000 7efc6bfff640 1 -- 172.21.15.73:0/2006508322 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7c03d610 con 0x7efc840cd580 2024-04-26T20:50:42.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.140+0000 7efc8a133640 1 -- 172.21.15.73:0/2006508322 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc84157ae0 con 0x7efc840cd580 2024-04-26T20:50:42.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.141+0000 7efc8a133640 1 -- 172.21.15.73:0/2006508322 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc840cdf40 con 0x7efc840cd580 2024-04-26T20:50:42.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.142+0000 7efc6bfff640 1 -- 172.21.15.73:0/2006508322 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efc7c046030 con 0x7efc840cd580 2024-04-26T20:50:42.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.142+0000 7efc6bfff640 1 --2- 172.21.15.73:0/2006508322 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7efc7006c8f0 0x7efc7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:42.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.142+0000 7efc6bfff640 1 -- 172.21.15.73:0/2006508322 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7efc7c0ac260 con 0x7efc840cd580 2024-04-26T20:50:42.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.142+0000 7efc837fe640 1 --2- 172.21.15.73:0/2006508322 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7efc7006c8f0 0x7efc7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:42.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.142+0000 7efc837fe640 1 --2- 172.21.15.73:0/2006508322 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7efc7006c8f0 0x7efc7006edb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efc74000c00 tx=0x7efc74009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:42.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.146+0000 7efc6bfff640 1 -- 172.21.15.73:0/2006508322 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efc7c04d060 con 0x7efc840cd580 2024-04-26T20:50:42.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.322+0000 7efc8a133640 1 -- 172.21.15.73:0/2006508322 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efc84072770 con 0x7efc840cd580 2024-04-26T20:50:42.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.323+0000 7efc6bfff640 1 -- 172.21.15.73:0/2006508322 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18514 (secure 0 0 0) 0x7efc7c02e030 con 0x7efc840cd580 2024-04-26T20:50:42.325 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:50:42.325 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":25,"fsid":"fbd3f548-b547-4713-856b-09f98f5d1764","created":"2024-04-26T20:50:07.876786+0000","modified":"2024-04-26T20:50:38.657510+0000","last_up_change":"2024-04-26T20:50:19.082741+0000","last_in_change":"2024-04-26T20:50:10.018400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:50:19.993731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:50:29.964420+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:50:30.920082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:50:34.650207+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"72d6f4f3-214a-4392-a02f-c33cbcab3c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6809","nonce":3461705124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6811","nonce":3461705124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6815","nonce":3461705124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6813","nonce":3461705124}]},"public_addr":"172.21.15.73:6809/3461705124","cluster_addr":"172.21.15.73:6811/3461705124","heartbeat_back_addr":"172.21.15.73:6815/3461705124","heartbeat_front_addr":"172.21.15.73:6813/3461705124","state":["exists","up"]},{"osd":1,"uuid":"786bf8ca-6ddc-4b3c-be6c-9152398bd5fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6817","nonce":2827619403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6819","nonce":2827619403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6825","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6827","nonce":2827619403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6821","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6823","nonce":2827619403}]},"public_addr":"172.21.15.73:6817/2827619403","cluster_addr":"172.21.15.73:6819/2827619403","heartbeat_back_addr":"172.21.15.73:6827/2827619403","heartbeat_front_addr":"172.21.15.73:6823/2827619403","state":["exists","up"]},{"osd":2,"uuid":"90963b1a-5437-4ef8-ad8b-6b74c534eeb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6801","nonce":2542681064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6803","nonce":2542681064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6808","nonce":2542681064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6805","nonce":2542681064}]},"public_addr":"172.21.15.73:6801/2542681064","cluster_addr":"172.21.15.73:6803/2542681064","heartbeat_back_addr":"172.21.15.73:6808/2542681064","heartbeat_front_addr":"172.21.15.73:6805/2542681064","state":["exists","up"]},{"osd":3,"uuid":"bd3d2148-6e38-41b4-a453-fe6e3b8b4ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6822","nonce":1604042718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6826","nonce":1604042718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6831","nonce":1604042718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6829","nonce":1604042718}]},"public_addr":"172.21.15.73:6822/1604042718","cluster_addr":"172.21.15.73:6826/1604042718","heartbeat_back_addr":"172.21.15.73:6831/1604042718","heartbeat_front_addr":"172.21.15.73:6829/1604042718","state":["exists","up"]},{"osd":4,"uuid":"e8668dc8-a23f-46d2-8311-cb882da08e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6817","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6819","nonce":3591055561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6824","nonce":3591055561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6831","nonce":3591055561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6828","nonce":3591055561}]},"public_addr":"172.21.15.120:6819/3591055561","cluster_addr":"172.21.15.120:6824/3591055561","heartbeat_back_addr":"172.21.15.120:6831/3591055561","heartbeat_front_addr":"172.21.15.120:6828/3591055561","state":["exists","up"]},{"osd":5,"uuid":"cbac0bdb-96bd-4852-bcce-e564495c3fcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6809","nonce":819450433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6811","nonce":819450433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6815","nonce":819450433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6813","nonce":819450433}]},"public_addr":"172.21.15.120:6809/819450433","cluster_addr":"172.21.15.120:6811/819450433","heartbeat_back_addr":"172.21.15.120:6815/819450433","heartbeat_front_addr":"172.21.15.120:6813/819450433","state":["exists","up"]},{"osd":6,"uuid":"1884b2a8-569f-4272-96a9-f25a09364dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6801","nonce":3946210727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6803","nonce":3946210727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6807","nonce":3946210727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6805","nonce":3946210727}]},"public_addr":"172.21.15.120:6801/3946210727","cluster_addr":"172.21.15.120:6803/3946210727","heartbeat_back_addr":"172.21.15.120:6807/3946210727","heartbeat_front_addr":"172.21.15.120:6805/3946210727","state":["exists","up"]},{"osd":7,"uuid":"c0c3bf6d-a4c5-4f1e-aa2d-330951830666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6818","nonce":2915574674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6821","nonce":2915574674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6827","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6829","nonce":2915574674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6823","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6825","nonce":2915574674}]},"public_addr":"172.21.15.120:6818/2915574674","cluster_addr":"172.21.15.120:6821/2915574674","heartbeat_back_addr":"172.21.15.120:6829/2915574674","heartbeat_front_addr":"172.21.15.120:6825/2915574674","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.218216+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.208710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.112705+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.182789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.164324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.174212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.225396+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.261559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:50:42.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.325+0000 7efc8a133640 1 -- 172.21.15.73:0/2006508322 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7efc7006c8f0 msgr2=0x7efc7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:42.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.325+0000 7efc8a133640 1 --2- 172.21.15.73:0/2006508322 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7efc7006c8f0 0x7efc7006edb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efc74000c00 tx=0x7efc74009040 comp rx=0 tx=0).stop 2024-04-26T20:50:42.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.325+0000 7efc8a133640 1 -- 172.21.15.73:0/2006508322 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc840cd580 msgr2=0x7efc840cda00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:42.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.325+0000 7efc8a133640 1 --2- 172.21.15.73:0/2006508322 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc840cd580 0x7efc840cda00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7efc7c00ada0 tx=0x7efc7c00a7b0 comp rx=0 tx=0).stop 2024-04-26T20:50:42.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.326+0000 7efc8a133640 1 -- 172.21.15.73:0/2006508322 shutdown_connections 2024-04-26T20:50:42.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.326+0000 7efc8a133640 1 --2- 172.21.15.73:0/2006508322 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc840cd580 0x7efc840cda00 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:42.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.326+0000 7efc8a133640 1 --2- 172.21.15.73:0/2006508322 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7efc7006c8f0 0x7efc7006edb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:42.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.326+0000 7efc8a133640 1 --2- 172.21.15.73:0/2006508322 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efc840c97e0 0x7efc8414f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:42.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.326+0000 7efc8a133640 1 --2- 172.21.15.73:0/2006508322 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efc6c006f50 0x7efc84156ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:42.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.326+0000 7efc8a133640 1 -- 172.21.15.73:0/2006508322 >> 172.21.15.73:0/2006508322 conn(0x7efc840d0be0 msgr2=0x7efc840b1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:42.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.326+0000 7efc8a133640 1 -- 172.21.15.73:0/2006508322 shutdown_connections 2024-04-26T20:50:42.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.326+0000 7efc8a133640 1 -- 172.21.15.73:0/2006508322 wait complete. 2024-04-26T20:50:42.337 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:50:19.993731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:50:29.964420+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:50:30.920082+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T20:50:34.650207+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T20:50:42.338 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:50:42.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.438+0000 7ff7b30d1640 1 Processor -- start 2024-04-26T20:50:42.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.438+0000 7ff7b30d1640 1 -- start start 2024-04-26T20:50:42.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.439+0000 7ff7b30d1640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff7ac0cfde0 0x7ff7ac0d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:42.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.439+0000 7ff7b30d1640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7ff7ac071260 0x7ff7ac15e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:42.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.439+0000 7ff7b30d1640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7ac0d2390 con 0x7ff7ac070350 2024-04-26T20:50:42.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.439+0000 7ff7b30d1640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7ff7ac0afab0 con 0x7ff7ac0cfde0 2024-04-26T20:50:42.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.439+0000 7ff7b30d1640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7ff7ac06dd20 con 0x7ff7ac071260 2024-04-26T20:50:42.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.439+0000 7ff7b0e6d640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff7ac0cfde0 0x7ff7ac0d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:42.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.439+0000 7ff7b0e6d640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff7ac0cfde0 0x7ff7ac0d1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34886/0 (socket says 172.21.15.73:34886) 2024-04-26T20:50:42.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.439+0000 7ff7b0e6d640 1 -- 172.21.15.73:0/2408561980 learned_addr learned my addr 172.21.15.73:0/2408561980 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:42.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.439+0000 7ff7b166e640 1 --2- 172.21.15.73:0/2408561980 >> v2:172.21.15.120:3300/0 conn(0x7ff7ac071260 0x7ff7ac15e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:42.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.439+0000 7ff7b0e6d640 1 -- 172.21.15.73:0/2408561980 >> v2:172.21.15.120:3300/0 conn(0x7ff7ac071260 msgr2=0x7ff7ac15e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:42.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.439+0000 7ff7b0e6d640 1 --2- 172.21.15.73:0/2408561980 >> v2:172.21.15.120:3300/0 conn(0x7ff7ac071260 0x7ff7ac15e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:42.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.439+0000 7ff7b0e6d640 1 -- 172.21.15.73:0/2408561980 >> v1:172.21.15.120:6789/0 conn(0x7ff7ac070350 legacy=0x7ff7ac070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:42.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.439+0000 7ff7b0e6d640 1 -- 172.21.15.73:0/2408561980 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7ac06d3f0 con 0x7ff7ac0cfde0 2024-04-26T20:50:42.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.440+0000 7ff7b0e6d640 1 --2- 172.21.15.73:0/2408561980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7ac0cfde0 0x7ff7ac0d1b60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac0c6860 tx=0x7ff7a802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=316ef52911878edb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:42.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.440+0000 7ff7a37fe640 1 -- 172.21.15.73:0/2408561980 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a8037070 con 0x7ff7ac0cfde0 2024-04-26T20:50:42.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.440+0000 7ff7a37fe640 1 -- 172.21.15.73:0/2408561980 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7a8032440 con 0x7ff7ac0cfde0 2024-04-26T20:50:42.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.440+0000 7ff7a37fe640 1 -- 172.21.15.73:0/2408561980 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a8032810 con 0x7ff7ac0cfde0 2024-04-26T20:50:42.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.440+0000 7ff7b30d1640 1 -- 172.21.15.73:0/2408561980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7ac0cfde0 msgr2=0x7ff7ac0d1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:42.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.440+0000 7ff7b30d1640 1 --2- 172.21.15.73:0/2408561980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7ac0cfde0 0x7ff7ac0d1b60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac0c6860 tx=0x7ff7a802a1a0 comp rx=0 tx=0).stop 2024-04-26T20:50:42.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.440+0000 7ff7b30d1640 1 -- 172.21.15.73:0/2408561980 shutdown_connections 2024-04-26T20:50:42.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.440+0000 7ff7b30d1640 1 --2- 172.21.15.73:0/2408561980 >> v2:172.21.15.120:3300/0 conn(0x7ff7ac071260 0x7ff7ac15e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:42.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.440+0000 7ff7b30d1640 1 --2- 172.21.15.73:0/2408561980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7ac0cfde0 0x7ff7ac0d1b60 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:42.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.440+0000 7ff7b30d1640 1 -- 172.21.15.73:0/2408561980 >> 172.21.15.73:0/2408561980 conn(0x7ff7ac0d2d80 msgr2=0x7ff7ac0d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:42.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.440+0000 7ff7b30d1640 1 -- 172.21.15.73:0/2408561980 shutdown_connections 2024-04-26T20:50:42.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.440+0000 7ff7b30d1640 1 -- 172.21.15.73:0/2408561980 wait complete. 2024-04-26T20:50:42.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.441+0000 7ff7b30d1640 1 Processor -- start 2024-04-26T20:50:42.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.442+0000 7ff7b30d1640 1 -- start start 2024-04-26T20:50:42.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.442+0000 7ff7b30d1640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7ac070350 0x7ff7ac09f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:42.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.442+0000 7ff7b30d1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7ac071260 0x7ff7ac09f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:42.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.442+0000 7ff7b30d1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7ac0cfde0 0x7ff7ac1608b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:42.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.442+0000 7ff7b30d1640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ac0d20a0 con 0x7ff7ac0cfde0 2024-04-26T20:50:42.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.442+0000 7ff7b30d1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ac06d9b0 con 0x7ff7ac071260 2024-04-26T20:50:42.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.442+0000 7ff7b30d1640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7ff7ac15ef30 con 0x7ff7ac070350 2024-04-26T20:50:42.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.443+0000 7ff7b166e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7ac0cfde0 0x7ff7ac1608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:42.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.443+0000 7ff7b166e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7ac0cfde0 0x7ff7ac1608b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34894/0 (socket says 172.21.15.73:34894) 2024-04-26T20:50:42.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.443+0000 7ff7b166e640 1 -- 172.21.15.73:0/3463780820 learned_addr learned my addr 172.21.15.73:0/3463780820 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:42.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.443+0000 7ff7b0e6d640 1 --2- 172.21.15.73:0/3463780820 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7ac070350 0x7ff7ac09f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:42.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.443+0000 7ff7a3fff640 1 --2- 172.21.15.73:0/3463780820 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7ac071260 0x7ff7ac09f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:42.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.443+0000 7ff7b166e640 1 -- 172.21.15.73:0/3463780820 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7ac070350 msgr2=0x7ff7ac09f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:42.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.443+0000 7ff7b166e640 1 --2- 172.21.15.73:0/3463780820 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7ac070350 0x7ff7ac09f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:42.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.443+0000 7ff7b166e640 1 -- 172.21.15.73:0/3463780820 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7ac071260 msgr2=0x7ff7ac09f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:42.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.443+0000 7ff7b166e640 1 --2- 172.21.15.73:0/3463780820 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7ac071260 0x7ff7ac09f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:42.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.443+0000 7ff7b166e640 1 -- 172.21.15.73:0/3463780820 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7ac161c50 con 0x7ff7ac0cfde0 2024-04-26T20:50:42.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.443+0000 7ff7b166e640 1 --2- 172.21.15.73:0/3463780820 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7ac0cfde0 0x7ff7ac1608b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4007e30 tx=0x7ff7a400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:42.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.443+0000 7ff7a17fa640 1 -- 172.21.15.73:0/3463780820 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a4014620 con 0x7ff7ac0cfde0 2024-04-26T20:50:42.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.443+0000 7ff7a17fa640 1 -- 172.21.15.73:0/3463780820 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7a4019070 con 0x7ff7ac0cfde0 2024-04-26T20:50:42.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.443+0000 7ff7b30d1640 1 -- 172.21.15.73:0/3463780820 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7ac161690 con 0x7ff7ac0cfde0 2024-04-26T20:50:42.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.443+0000 7ff7a17fa640 1 -- 172.21.15.73:0/3463780820 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a4014a00 con 0x7ff7ac0cfde0 2024-04-26T20:50:42.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.443+0000 7ff7b30d1640 1 -- 172.21.15.73:0/3463780820 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7ac14e080 con 0x7ff7ac0cfde0 2024-04-26T20:50:42.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.444+0000 7ff7b30d1640 1 -- 172.21.15.73:0/3463780820 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7ac0694f0 con 0x7ff7ac0cfde0 2024-04-26T20:50:42.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.444+0000 7ff7a17fa640 1 -- 172.21.15.73:0/3463780820 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff7a4014ba0 con 0x7ff7ac0cfde0 2024-04-26T20:50:42.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.445+0000 7ff7a17fa640 1 --2- 172.21.15.73:0/3463780820 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff79806c8f0 0x7ff79806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:42.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.445+0000 7ff7a17fa640 1 -- 172.21.15.73:0/3463780820 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff7a408c6d0 con 0x7ff7ac0cfde0 2024-04-26T20:50:42.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.445+0000 7ff7b0e6d640 1 --2- 172.21.15.73:0/3463780820 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff79806c8f0 0x7ff79806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:42.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.445+0000 7ff7b0e6d640 1 --2- 172.21.15.73:0/3463780820 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff79806c8f0 0x7ff79806edb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8034000 tx=0x7ff7a802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:42.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.449+0000 7ff7a17fa640 1 -- 172.21.15.73:0/3463780820 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff7a405a850 con 0x7ff7ac0cfde0 2024-04-26T20:50:42.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.623+0000 7ff7b30d1640 1 -- 172.21.15.73:0/3463780820 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff7ac0afde0 con 0x7ff7ac0cfde0 2024-04-26T20:50:42.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.624+0000 7ff7a17fa640 1 -- 172.21.15.73:0/3463780820 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7ff7a405a150 con 0x7ff7ac0cfde0 2024-04-26T20:50:42.625 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-26T20:50:42.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.626+0000 7ff7b30d1640 1 -- 172.21.15.73:0/3463780820 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff79806c8f0 msgr2=0x7ff79806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:42.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.626+0000 7ff7b30d1640 1 --2- 172.21.15.73:0/3463780820 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff79806c8f0 0x7ff79806edb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8034000 tx=0x7ff7a802e000 comp rx=0 tx=0).stop 2024-04-26T20:50:42.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.626+0000 7ff7b30d1640 1 -- 172.21.15.73:0/3463780820 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7ac0cfde0 msgr2=0x7ff7ac1608b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:42.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.626+0000 7ff7b30d1640 1 --2- 172.21.15.73:0/3463780820 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7ac0cfde0 0x7ff7ac1608b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4007e30 tx=0x7ff7a400e4b0 comp rx=0 tx=0).stop 2024-04-26T20:50:42.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.626+0000 7ff7b30d1640 1 -- 172.21.15.73:0/3463780820 shutdown_connections 2024-04-26T20:50:42.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.626+0000 7ff7b30d1640 1 --2- 172.21.15.73:0/3463780820 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7ac0cfde0 0x7ff7ac1608b0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:42.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.626+0000 7ff7b30d1640 1 --2- 172.21.15.73:0/3463780820 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff79806c8f0 0x7ff79806edb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:42.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.626+0000 7ff7b30d1640 1 --2- 172.21.15.73:0/3463780820 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7ac071260 0x7ff7ac09f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:42.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.626+0000 7ff7b30d1640 1 --2- 172.21.15.73:0/3463780820 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7ac070350 0x7ff7ac09f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:42.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.626+0000 7ff7b30d1640 1 -- 172.21.15.73:0/3463780820 >> 172.21.15.73:0/3463780820 conn(0x7ff7ac0d2d80 msgr2=0x7ff7ac0b37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:42.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.626+0000 7ff7b30d1640 1 -- 172.21.15.73:0/3463780820 shutdown_connections 2024-04-26T20:50:42.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.626+0000 7ff7b30d1640 1 -- 172.21.15.73:0/3463780820 wait complete. 2024-04-26T20:50:42.638 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:50:42.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.737+0000 7f42f3a5b640 1 Processor -- start 2024-04-26T20:50:42.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.737+0000 7f42f3a5b640 1 -- start start 2024-04-26T20:50:42.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.738+0000 7f42f3a5b640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f42ec0d1720 0x7f42ec0d0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:42.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.738+0000 7f42f3a5b640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f42ec0d39f0 0x7f42ec070970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:42.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.738+0000 7f42f3a5b640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42ec0701e0 con 0x7f42ec15d250 2024-04-26T20:50:42.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.738+0000 7f42f3a5b640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f42ec15d630 con 0x7f42ec0d39f0 2024-04-26T20:50:42.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.738+0000 7f42f3a5b640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f42ec0704e0 con 0x7f42ec0d1720 2024-04-26T20:50:42.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.738+0000 7f42f17f7640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f42ec15d250 0x7f42ec0d1010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40966/0 (socket says 172.21.15.73:40966) 2024-04-26T20:50:42.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.738+0000 7f42f17f7640 1 -- 172.21.15.73:0/1193108174 learned_addr learned my addr 172.21.15.73:0/1193108174 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:50:42.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.738+0000 7f42f0ff6640 1 --2- 172.21.15.73:0/1193108174 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f42ec0d1720 0x7f42ec0d0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:42.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.738+0000 7f42e3fff640 1 -- 172.21.15.73:0/1193108174 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3373706278 0 0) 0x7f42ec0701e0 con 0x7f42ec15d250 2024-04-26T20:50:42.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.738+0000 7f42f1ff8640 1 --2- 172.21.15.73:0/1193108174 >> v2:172.21.15.120:3300/0 conn(0x7f42ec0d39f0 0x7f42ec070970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:42.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.738+0000 7f42e3fff640 1 -- 172.21.15.73:0/1193108174 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f42d8003520 con 0x7f42ec15d250 2024-04-26T20:50:42.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.739+0000 7f42e3fff640 1 -- 172.21.15.73:0/1193108174 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2413744433 0 0) 0x7f42d8003520 con 0x7f42ec15d250 2024-04-26T20:50:42.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.739+0000 7f42e3fff640 1 -- 172.21.15.73:0/1193108174 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f42ec0d1720 msgr2=0x7f42ec0d0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:42.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.739+0000 7f42e3fff640 1 --2- 172.21.15.73:0/1193108174 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f42ec0d1720 0x7f42ec0d0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:42.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.739+0000 7f42e3fff640 1 -- 172.21.15.73:0/1193108174 >> v2:172.21.15.120:3300/0 conn(0x7f42ec0d39f0 msgr2=0x7f42ec070970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:42.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.739+0000 7f42e3fff640 1 --2- 172.21.15.73:0/1193108174 >> v2:172.21.15.120:3300/0 conn(0x7f42ec0d39f0 0x7f42ec070970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:42.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.739+0000 7f42e3fff640 1 -- 172.21.15.73:0/1193108174 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42ec0ce370 con 0x7f42ec15d250 2024-04-26T20:50:42.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.739+0000 7f42e3fff640 1 -- 172.21.15.73:0/1193108174 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7f42e8001670 con 0x7f42ec15d250 2024-04-26T20:50:42.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.739+0000 7f42e3fff640 1 -- 172.21.15.73:0/1193108174 >> v1:172.21.15.73:6789/0 conn(0x7f42ec15d250 legacy=0x7f42ec0d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:42.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.739+0000 7f42e3fff640 1 --2- 172.21.15.73:0/1193108174 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42d80039b0 0x7f42d8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:42.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.739+0000 7f42e3fff640 1 --2- 172.21.15.73:0/1193108174 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f42ec0d1720 0x7f42d8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:42.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.739+0000 7f42e3fff640 1 --2- 172.21.15.73:0/1193108174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42d8004d40 0x7f42d80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:42.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.739+0000 7f42e3fff640 1 -- 172.21.15.73:0/1193108174 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f42ec15d630 con 0x7f42d8004d40 2024-04-26T20:50:42.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.739+0000 7f42e3fff640 1 -- 172.21.15.73:0/1193108174 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f42ec0704e0 con 0x7f42d80039b0 2024-04-26T20:50:42.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.739+0000 7f42e3fff640 1 -- 172.21.15.73:0/1193108174 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f42d800b9c0 con 0x7f42ec0d1720 2024-04-26T20:50:42.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.739+0000 7f42f1ff8640 1 --2- 172.21.15.73:0/1193108174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42d8004d40 0x7f42d80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:42.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.739+0000 7f42f17f7640 1 --2- 172.21.15.73:0/1193108174 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42d80039b0 0x7f42d8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:42.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.739+0000 7f42f0ff6640 1 --2- 172.21.15.73:0/1193108174 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f42ec0d1720 0x7f42d8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:42.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.739+0000 7f42f1ff8640 1 -- 172.21.15.73:0/1193108174 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f42ec0d1720 msgr2=0x7f42d8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:42.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.739+0000 7f42f1ff8640 1 --2- 172.21.15.73:0/1193108174 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f42ec0d1720 0x7f42d8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:42.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.739+0000 7f42f1ff8640 1 -- 172.21.15.73:0/1193108174 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42d80039b0 msgr2=0x7f42d8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:42.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.739+0000 7f42f1ff8640 1 --2- 172.21.15.73:0/1193108174 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42d80039b0 0x7f42d8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:42.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.739+0000 7f42f1ff8640 1 -- 172.21.15.73:0/1193108174 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42d8003520 con 0x7f42d8004d40 2024-04-26T20:50:42.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.740+0000 7f42f1ff8640 1 --2- 172.21.15.73:0/1193108174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42d8004d40 0x7f42d80051c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f42e4009fc0 tx=0x7f42d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c6fb5a562409e59 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:42.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.740+0000 7f42e3fff640 1 -- 172.21.15.73:0/1193108174 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e4009c10 con 0x7f42d8004d40 2024-04-26T20:50:42.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.740+0000 7f42e3fff640 1 -- 172.21.15.73:0/1193108174 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42e4036070 con 0x7f42d8004d40 2024-04-26T20:50:42.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.740+0000 7f42e3fff640 1 -- 172.21.15.73:0/1193108174 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e402d070 con 0x7f42d8004d40 2024-04-26T20:50:42.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.740+0000 7f42f3a5b640 1 -- 172.21.15.73:0/1193108174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42d8004d40 msgr2=0x7f42d80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:42.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.740+0000 7f42f3a5b640 1 --2- 172.21.15.73:0/1193108174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42d8004d40 0x7f42d80051c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f42e4009fc0 tx=0x7f42d8005830 comp rx=0 tx=0).stop 2024-04-26T20:50:42.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.740+0000 7f42f1ff8640 1 -- 172.21.15.73:0/1193108174 reap_dead start 2024-04-26T20:50:42.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.740+0000 7f42f3a5b640 1 -- 172.21.15.73:0/1193108174 shutdown_connections 2024-04-26T20:50:42.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.740+0000 7f42f3a5b640 1 -- 172.21.15.73:0/1193108174 >> 172.21.15.73:0/1193108174 conn(0x7f42ec0d2d80 msgr2=0x7f42ec0d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:42.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.741+0000 7f42f3a5b640 1 -- 172.21.15.73:0/1193108174 shutdown_connections 2024-04-26T20:50:42.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.741+0000 7f42f3a5b640 1 -- 172.21.15.73:0/1193108174 wait complete. 2024-04-26T20:50:42.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.742+0000 7f42f3a5b640 1 Processor -- start 2024-04-26T20:50:42.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.742+0000 7f42f3a5b640 1 -- start start 2024-04-26T20:50:42.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.742+0000 7f42f3a5b640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f42ec1521f0 0x7f42ec06e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:42.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.742+0000 7f42f3a5b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42ec1525d0 0x7f42ec155be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:42.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.742+0000 7f42f3a5b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42ec156120 0x7f42ec153030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:42.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.742+0000 7f42f3a5b640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f42ec0cebb0 con 0x7f42ec1525d0 2024-04-26T20:50:42.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.742+0000 7f42f3a5b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f42ec0ce8d0 con 0x7f42ec156120 2024-04-26T20:50:42.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.742+0000 7f42f3a5b640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f42ec0d0950 con 0x7f42ec1521f0 2024-04-26T20:50:42.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.743+0000 7f42f0ff6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42ec1525d0 0x7f42ec155be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:42.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.743+0000 7f42f0ff6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42ec1525d0 0x7f42ec155be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34898/0 (socket says 172.21.15.73:34898) 2024-04-26T20:50:42.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.743+0000 7f42f0ff6640 1 -- 172.21.15.73:0/1921392136 learned_addr learned my addr 172.21.15.73:0/1921392136 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:42.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.743+0000 7f42f17f7640 1 --2- 172.21.15.73:0/1921392136 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f42ec1521f0 0x7f42ec06e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:42.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.743+0000 7f42f1ff8640 1 --2- 172.21.15.73:0/1921392136 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42ec156120 0x7f42ec153030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:42.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.743+0000 7f42f0ff6640 1 -- 172.21.15.73:0/1921392136 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f42ec1521f0 msgr2=0x7f42ec06e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:42.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.743+0000 7f42f0ff6640 1 --2- 172.21.15.73:0/1921392136 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f42ec1521f0 0x7f42ec06e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:42.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.743+0000 7f42f0ff6640 1 -- 172.21.15.73:0/1921392136 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42ec156120 msgr2=0x7f42ec153030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:42.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.743+0000 7f42f0ff6640 1 --2- 172.21.15.73:0/1921392136 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42ec156120 0x7f42ec153030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:42.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.743+0000 7f42f0ff6640 1 -- 172.21.15.73:0/1921392136 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42ec06d730 con 0x7f42ec1525d0 2024-04-26T20:50:42.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.743+0000 7f42f0ff6640 1 --2- 172.21.15.73:0/1921392136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42ec1525d0 0x7f42ec155be0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f42dc0119d0 tx=0x7f42dc011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:42.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.743+0000 7f42e1ffb640 1 -- 172.21.15.73:0/1921392136 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42dc00fb50 con 0x7f42ec1525d0 2024-04-26T20:50:42.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.743+0000 7f42e1ffb640 1 -- 172.21.15.73:0/1921392136 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42dc00fcf0 con 0x7f42ec1525d0 2024-04-26T20:50:42.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.743+0000 7f42f3a5b640 1 -- 172.21.15.73:0/1921392136 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42ec06da10 con 0x7f42ec1525d0 2024-04-26T20:50:42.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.743+0000 7f42e1ffb640 1 -- 172.21.15.73:0/1921392136 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42dc00fea0 con 0x7f42ec1525d0 2024-04-26T20:50:42.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.743+0000 7f42f3a5b640 1 -- 172.21.15.73:0/1921392136 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42ec06dfd0 con 0x7f42ec1525d0 2024-04-26T20:50:42.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.744+0000 7f42f3a5b640 1 -- 172.21.15.73:0/1921392136 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42ec0694f0 con 0x7f42ec1525d0 2024-04-26T20:50:42.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.745+0000 7f42e1ffb640 1 -- 172.21.15.73:0/1921392136 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f42dc012040 con 0x7f42ec1525d0 2024-04-26T20:50:42.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.745+0000 7f42e1ffb640 1 --2- 172.21.15.73:0/1921392136 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f42d006c8f0 0x7f42d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:42.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.745+0000 7f42e1ffb640 1 -- 172.21.15.73:0/1921392136 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f42dc08c560 con 0x7f42ec1525d0 2024-04-26T20:50:42.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.745+0000 7f42f17f7640 1 --2- 172.21.15.73:0/1921392136 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f42d006c8f0 0x7f42d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:42.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.746+0000 7f42f17f7640 1 --2- 172.21.15.73:0/1921392136 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f42d006c8f0 0x7f42d006edb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f42e8001eb0 tx=0x7f42e8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:42.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.749+0000 7f42e1ffb640 1 -- 172.21.15.73:0/1921392136 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f42dc019030 con 0x7f42ec1525d0 2024-04-26T20:50:42.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.924+0000 7f42f3a5b640 1 -- 172.21.15.73:0/1921392136 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f42ec15b5f0 con 0x7f42ec1525d0 2024-04-26T20:50:42.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.925+0000 7f42e1ffb640 1 -- 172.21.15.73:0/1921392136 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f42dc05a6e0 con 0x7f42ec1525d0 2024-04-26T20:50:42.926 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-26T20:50:42.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.927+0000 7f42f3a5b640 1 -- 172.21.15.73:0/1921392136 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f42d006c8f0 msgr2=0x7f42d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:42.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.927+0000 7f42f3a5b640 1 --2- 172.21.15.73:0/1921392136 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f42d006c8f0 0x7f42d006edb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f42e8001eb0 tx=0x7f42e8009040 comp rx=0 tx=0).stop 2024-04-26T20:50:42.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.927+0000 7f42f3a5b640 1 -- 172.21.15.73:0/1921392136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42ec1525d0 msgr2=0x7f42ec155be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:42.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.927+0000 7f42f3a5b640 1 --2- 172.21.15.73:0/1921392136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42ec1525d0 0x7f42ec155be0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f42dc0119d0 tx=0x7f42dc011e90 comp rx=0 tx=0).stop 2024-04-26T20:50:42.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.927+0000 7f42f3a5b640 1 -- 172.21.15.73:0/1921392136 shutdown_connections 2024-04-26T20:50:42.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.927+0000 7f42f3a5b640 1 --2- 172.21.15.73:0/1921392136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42ec1525d0 0x7f42ec155be0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:42.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.927+0000 7f42f3a5b640 1 --2- 172.21.15.73:0/1921392136 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f42d006c8f0 0x7f42d006edb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:42.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.927+0000 7f42f3a5b640 1 --2- 172.21.15.73:0/1921392136 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f42ec156120 0x7f42ec153030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:42.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.927+0000 7f42f3a5b640 1 --2- 172.21.15.73:0/1921392136 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f42ec1521f0 0x7f42ec06e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:42.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.927+0000 7f42f3a5b640 1 -- 172.21.15.73:0/1921392136 >> 172.21.15.73:0/1921392136 conn(0x7f42ec0d2d80 msgr2=0x7f42ec0c9ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:42.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.927+0000 7f42f3a5b640 1 -- 172.21.15.73:0/1921392136 shutdown_connections 2024-04-26T20:50:42.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:42.927+0000 7f42f3a5b640 1 -- 172.21.15.73:0/1921392136 wait complete. 2024-04-26T20:50:42.938 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:50:43.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.039+0000 7fcd01d6a640 1 Processor -- start 2024-04-26T20:50:43.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.040+0000 7fcd01d6a640 1 -- start start 2024-04-26T20:50:43.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.040+0000 7fcd01d6a640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fccfc072fb0 0x7fccfc0b56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:43.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.040+0000 7fcd01d6a640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fccfc0b06f0 0x7fccfc06fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:43.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.040+0000 7fcd01d6a640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccfc0b5e30 con 0x7fccfc073390 2024-04-26T20:50:43.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.040+0000 7fcd01d6a640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7fccfc0b0b80 con 0x7fccfc072fb0 2024-04-26T20:50:43.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.040+0000 7fcd01d6a640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fccfc0b5c00 con 0x7fccfc0b06f0 2024-04-26T20:50:43.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.040+0000 7fccfb7fe640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fccfc072fb0 0x7fccfc0b56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:43.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.041+0000 7fccfb7fe640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fccfc072fb0 0x7fccfc0b56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34908/0 (socket says 172.21.15.73:34908) 2024-04-26T20:50:43.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.041+0000 7fccfb7fe640 1 -- 172.21.15.73:0/311567175 learned_addr learned my addr 172.21.15.73:0/311567175 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:43.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.041+0000 7fccfbfff640 1 --2- 172.21.15.73:0/311567175 >> v2:172.21.15.120:3300/0 conn(0x7fccfc0b06f0 0x7fccfc06fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:43.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.041+0000 7fccfb7fe640 1 -- 172.21.15.73:0/311567175 >> v2:172.21.15.120:3300/0 conn(0x7fccfc0b06f0 msgr2=0x7fccfc06fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:43.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.041+0000 7fccfb7fe640 1 --2- 172.21.15.73:0/311567175 >> v2:172.21.15.120:3300/0 conn(0x7fccfc0b06f0 0x7fccfc06fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:43.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.041+0000 7fccfb7fe640 1 -- 172.21.15.73:0/311567175 >> v1:172.21.15.120:6789/0 conn(0x7fccfc073390 legacy=0x7fccfc06f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:43.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.041+0000 7fccfb7fe640 1 -- 172.21.15.73:0/311567175 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccfc0b6110 con 0x7fccfc072fb0 2024-04-26T20:50:43.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.041+0000 7fccfb7fe640 1 --2- 172.21.15.73:0/311567175 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccfc072fb0 0x7fccfc0b56c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fccfc071b40 tx=0x7fccec02c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b7f51b9f9a121ce server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:43.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.042+0000 7fccfa7fc640 1 -- 172.21.15.73:0/311567175 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccec0325a0 con 0x7fccfc072fb0 2024-04-26T20:50:43.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.042+0000 7fccfa7fc640 1 -- 172.21.15.73:0/311567175 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccec038070 con 0x7fccfc072fb0 2024-04-26T20:50:43.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.042+0000 7fccfa7fc640 1 -- 172.21.15.73:0/311567175 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccec032960 con 0x7fccfc072fb0 2024-04-26T20:50:43.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.042+0000 7fcd01d6a640 1 -- 172.21.15.73:0/311567175 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccfc072fb0 msgr2=0x7fccfc0b56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:43.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.042+0000 7fcd01d6a640 1 --2- 172.21.15.73:0/311567175 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccfc072fb0 0x7fccfc0b56c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fccfc071b40 tx=0x7fccec02c8e0 comp rx=0 tx=0).stop 2024-04-26T20:50:43.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.042+0000 7fcd01d6a640 1 -- 172.21.15.73:0/311567175 shutdown_connections 2024-04-26T20:50:43.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.042+0000 7fcd01d6a640 1 --2- 172.21.15.73:0/311567175 >> v2:172.21.15.120:3300/0 conn(0x7fccfc0b06f0 0x7fccfc06fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:43.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.042+0000 7fcd01d6a640 1 --2- 172.21.15.73:0/311567175 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccfc072fb0 0x7fccfc0b56c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:43.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.042+0000 7fcd01d6a640 1 -- 172.21.15.73:0/311567175 >> 172.21.15.73:0/311567175 conn(0x7fccfc0b39b0 msgr2=0x7fccfc0b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:43.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.042+0000 7fcd01d6a640 1 -- 172.21.15.73:0/311567175 shutdown_connections 2024-04-26T20:50:43.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.042+0000 7fcd01d6a640 1 -- 172.21.15.73:0/311567175 wait complete. 2024-04-26T20:50:43.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.043+0000 7fcd01d6a640 1 Processor -- start 2024-04-26T20:50:43.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.043+0000 7fcd01d6a640 1 -- start start 2024-04-26T20:50:43.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.044+0000 7fcd01d6a640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fccfc072fb0 0x7fccfc15eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:43.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.044+0000 7fcd01d6a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fccfc073390 0x7fccfc150f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:43.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.044+0000 7fcd01d6a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccfc0b06f0 0x7fccfc15c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:43.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.044+0000 7fcd01d6a640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fccfc0737f0 con 0x7fccfc0b06f0 2024-04-26T20:50:43.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.044+0000 7fcd01d6a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fccfc06f2b0 con 0x7fccfc073390 2024-04-26T20:50:43.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.044+0000 7fcd01d6a640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fccfc15c160 con 0x7fccfc072fb0 2024-04-26T20:50:43.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.044+0000 7fccfbfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccfc0b06f0 0x7fccfc15c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:43.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.044+0000 7fccfbfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccfc0b06f0 0x7fccfc15c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34922/0 (socket says 172.21.15.73:34922) 2024-04-26T20:50:43.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.044+0000 7fccfbfff640 1 -- 172.21.15.73:0/3974612991 learned_addr learned my addr 172.21.15.73:0/3974612991 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:43.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.044+0000 7fccfb7fe640 1 --2- 172.21.15.73:0/3974612991 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fccfc072fb0 0x7fccfc15eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:43.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.044+0000 7fccfaffd640 1 --2- 172.21.15.73:0/3974612991 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fccfc073390 0x7fccfc150f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:43.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.045+0000 7fccfbfff640 1 -- 172.21.15.73:0/3974612991 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fccfc072fb0 msgr2=0x7fccfc15eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:43.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.045+0000 7fccfbfff640 1 --2- 172.21.15.73:0/3974612991 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fccfc072fb0 0x7fccfc15eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:43.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.045+0000 7fccfbfff640 1 -- 172.21.15.73:0/3974612991 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fccfc073390 msgr2=0x7fccfc150f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:43.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.045+0000 7fccfbfff640 1 --2- 172.21.15.73:0/3974612991 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fccfc073390 0x7fccfc150f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:43.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.045+0000 7fccfbfff640 1 -- 172.21.15.73:0/3974612991 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccfc1e45a0 con 0x7fccfc0b06f0 2024-04-26T20:50:43.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.045+0000 7fccfbfff640 1 --2- 172.21.15.73:0/3974612991 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccfc0b06f0 0x7fccfc15c4f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fccf400cc30 tx=0x7fccf400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:43.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.045+0000 7fcce3fff640 1 -- 172.21.15.73:0/3974612991 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf400ce40 con 0x7fccfc0b06f0 2024-04-26T20:50:43.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.045+0000 7fcce3fff640 1 -- 172.21.15.73:0/3974612991 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccf4003990 con 0x7fccfc0b06f0 2024-04-26T20:50:43.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.045+0000 7fcd01d6a640 1 -- 172.21.15.73:0/3974612991 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccfc154500 con 0x7fccfc0b06f0 2024-04-26T20:50:43.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.045+0000 7fcce3fff640 1 -- 172.21.15.73:0/3974612991 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf4003d40 con 0x7fccfc0b06f0 2024-04-26T20:50:43.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.045+0000 7fcd01d6a640 1 -- 172.21.15.73:0/3974612991 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccfc1547e0 con 0x7fccfc0b06f0 2024-04-26T20:50:43.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.046+0000 7fcd01d6a640 1 -- 172.21.15.73:0/3974612991 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccfc0ce3b0 con 0x7fccfc0b06f0 2024-04-26T20:50:43.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.046+0000 7fcce3fff640 1 -- 172.21.15.73:0/3974612991 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fccf4004000 con 0x7fccfc0b06f0 2024-04-26T20:50:43.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.047+0000 7fcce3fff640 1 --2- 172.21.15.73:0/3974612991 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fcce806c8f0 0x7fcce806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:43.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.047+0000 7fcce3fff640 1 -- 172.21.15.73:0/3974612991 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fccf408c310 con 0x7fccfc0b06f0 2024-04-26T20:50:43.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.047+0000 7fccfb7fe640 1 --2- 172.21.15.73:0/3974612991 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fcce806c8f0 0x7fcce806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:43.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.047+0000 7fccfb7fe640 1 --2- 172.21.15.73:0/3974612991 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fcce806c8f0 0x7fcce806edb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fccec037000 tx=0x7fccec03e040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:43.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.051+0000 7fcce3fff640 1 -- 172.21.15.73:0/3974612991 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fccfc0ce3b0 con 0x7fccfc0b06f0 2024-04-26T20:50:43.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.229+0000 7fcd01d6a640 1 -- 172.21.15.73:0/3974612991 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fccfc0694f0 con 0x7fccfc0b06f0 2024-04-26T20:50:43.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.229+0000 7fcce3fff640 1 -- 172.21.15.73:0/3974612991 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7fccf4014030 con 0x7fccfc0b06f0 2024-04-26T20:50:43.230 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-26T20:50:43.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.231+0000 7fcd01d6a640 1 -- 172.21.15.73:0/3974612991 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fcce806c8f0 msgr2=0x7fcce806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:43.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.231+0000 7fcd01d6a640 1 --2- 172.21.15.73:0/3974612991 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fcce806c8f0 0x7fcce806edb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fccec037000 tx=0x7fccec03e040 comp rx=0 tx=0).stop 2024-04-26T20:50:43.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.231+0000 7fcd01d6a640 1 -- 172.21.15.73:0/3974612991 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccfc0b06f0 msgr2=0x7fccfc15c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:43.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.231+0000 7fcd01d6a640 1 --2- 172.21.15.73:0/3974612991 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccfc0b06f0 0x7fccfc15c4f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fccf400cc30 tx=0x7fccf400a5c0 comp rx=0 tx=0).stop 2024-04-26T20:50:43.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.231+0000 7fcd01d6a640 1 -- 172.21.15.73:0/3974612991 shutdown_connections 2024-04-26T20:50:43.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.231+0000 7fcd01d6a640 1 --2- 172.21.15.73:0/3974612991 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccfc0b06f0 0x7fccfc15c4f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:43.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.231+0000 7fcd01d6a640 1 --2- 172.21.15.73:0/3974612991 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fcce806c8f0 0x7fcce806edb0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:43.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.231+0000 7fcd01d6a640 1 --2- 172.21.15.73:0/3974612991 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fccfc073390 0x7fccfc150f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:43.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.231+0000 7fcd01d6a640 1 --2- 172.21.15.73:0/3974612991 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fccfc072fb0 0x7fccfc15eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:43.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.231+0000 7fcd01d6a640 1 -- 172.21.15.73:0/3974612991 >> 172.21.15.73:0/3974612991 conn(0x7fccfc0b39b0 msgr2=0x7fccfc157b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:43.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.232+0000 7fcd01d6a640 1 -- 172.21.15.73:0/3974612991 shutdown_connections 2024-04-26T20:50:43.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.232+0000 7fcd01d6a640 1 -- 172.21.15.73:0/3974612991 wait complete. 2024-04-26T20:50:43.243 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T20:50:43.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.342+0000 7f36d4662640 1 Processor -- start 2024-04-26T20:50:43.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.342+0000 7f36d4662640 1 -- start start 2024-04-26T20:50:43.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.343+0000 7f36d4662640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f36cc071e60 0x7f36cc06f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:43.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.343+0000 7f36d4662640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f36cc06fe20 0x7f36cc072240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:43.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.343+0000 7f36d4662640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36cc15d7c0 con 0x7f36cc072780 2024-04-26T20:50:43.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.343+0000 7f36d4662640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f36cc15daa0 con 0x7f36cc06fe20 2024-04-26T20:50:43.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.343+0000 7f36d4662640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f36cc15dd80 con 0x7f36cc071e60 2024-04-26T20:50:43.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.343+0000 7f36d1bfd640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f36cc06fe20 0x7f36cc072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:43.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.343+0000 7f36d2bff640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f36cc072780 0x7f36cc0cb470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40968/0 (socket says 172.21.15.73:40968) 2024-04-26T20:50:43.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.343+0000 7f36d2bff640 1 -- 172.21.15.73:0/3521016657 learned_addr learned my addr 172.21.15.73:0/3521016657 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:50:43.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.343+0000 7f36d23fe640 1 --2- 172.21.15.73:0/3521016657 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f36cc071e60 0x7f36cc06f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:43.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.343+0000 7f36d1bfd640 1 -- 172.21.15.73:0/3521016657 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f36cc071e60 msgr2=0x7f36cc06f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:43.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.343+0000 7f36d1bfd640 1 --2- 172.21.15.73:0/3521016657 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f36cc071e60 0x7f36cc06f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:43.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.343+0000 7f36d1bfd640 1 -- 172.21.15.73:0/3521016657 >> v1:172.21.15.73:6789/0 conn(0x7f36cc072780 legacy=0x7f36cc0cb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:43.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.343+0000 7f36d1bfd640 1 -- 172.21.15.73:0/3521016657 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36cc15e060 con 0x7f36cc06fe20 2024-04-26T20:50:43.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.343+0000 7f36d23fe640 1 --2- 172.21.15.73:0/3521016657 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f36cc071e60 0x7f36cc06f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:50:43.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.344+0000 7f36d1bfd640 1 --2- 172.21.15.73:0/3521016657 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36cc06fe20 0x7f36cc072240 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f36bc0097a0 tx=0x7f36bc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13dbd9a53decf05d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:43.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.344+0000 7f36d13fc640 1 -- 172.21.15.73:0/3521016657 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36bc037070 con 0x7f36cc06fe20 2024-04-26T20:50:43.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.344+0000 7f36d13fc640 1 -- 172.21.15.73:0/3521016657 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36bc032440 con 0x7f36cc06fe20 2024-04-26T20:50:43.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.344+0000 7f36d13fc640 1 -- 172.21.15.73:0/3521016657 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36bc032810 con 0x7f36cc06fe20 2024-04-26T20:50:43.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.345+0000 7f36d4662640 1 -- 172.21.15.73:0/3521016657 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36cc06fe20 msgr2=0x7f36cc072240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:43.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.345+0000 7f36d4662640 1 --2- 172.21.15.73:0/3521016657 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36cc06fe20 0x7f36cc072240 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f36bc0097a0 tx=0x7f36bc02a180 comp rx=0 tx=0).stop 2024-04-26T20:50:43.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.345+0000 7f36d4662640 1 -- 172.21.15.73:0/3521016657 shutdown_connections 2024-04-26T20:50:43.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.345+0000 7f36d4662640 1 --2- 172.21.15.73:0/3521016657 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36cc06fe20 0x7f36cc072240 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:43.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.345+0000 7f36d4662640 1 --2- 172.21.15.73:0/3521016657 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f36cc071e60 0x7f36cc06f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:43.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.345+0000 7f36d4662640 1 -- 172.21.15.73:0/3521016657 >> 172.21.15.73:0/3521016657 conn(0x7f36cc0d0be0 msgr2=0x7f36cc0d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:43.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.345+0000 7f36d4662640 1 -- 172.21.15.73:0/3521016657 shutdown_connections 2024-04-26T20:50:43.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.345+0000 7f36d4662640 1 -- 172.21.15.73:0/3521016657 wait complete. 2024-04-26T20:50:43.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.346+0000 7f36d4662640 1 Processor -- start 2024-04-26T20:50:43.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.346+0000 7f36d4662640 1 -- start start 2024-04-26T20:50:43.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.347+0000 7f36d4662640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f36cc06fe20 0x7f36cc1f35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:43.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.347+0000 7f36d4662640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f36cc071e60 0x7f36cc0b4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:43.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.347+0000 7f36d4662640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36cc072780 0x7f36cc149bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:43.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.347+0000 7f36d4662640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f36cc15d3f0 con 0x7f36cc072780 2024-04-26T20:50:43.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.347+0000 7f36d4662640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f36cc072c10 con 0x7f36cc06fe20 2024-04-26T20:50:43.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.347+0000 7f36d4662640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f36cc15d570 con 0x7f36cc071e60 2024-04-26T20:50:43.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.347+0000 7f36d2bff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36cc072780 0x7f36cc149bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:43.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.347+0000 7f36d2bff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36cc072780 0x7f36cc149bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34950/0 (socket says 172.21.15.73:34950) 2024-04-26T20:50:43.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.347+0000 7f36d2bff640 1 -- 172.21.15.73:0/547404869 learned_addr learned my addr 172.21.15.73:0/547404869 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:43.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.347+0000 7f36d1bfd640 1 --2- 172.21.15.73:0/547404869 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f36cc071e60 0x7f36cc0b4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:43.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.347+0000 7f36d2bff640 1 -- 172.21.15.73:0/547404869 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f36cc071e60 msgr2=0x7f36cc0b4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:43.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.347+0000 7f36d2bff640 1 --2- 172.21.15.73:0/547404869 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f36cc071e60 0x7f36cc0b4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:43.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.347+0000 7f36d2bff640 1 -- 172.21.15.73:0/547404869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f36cc06fe20 msgr2=0x7f36cc1f35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:43.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.347+0000 7f36d23fe640 1 --2- 172.21.15.73:0/547404869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f36cc06fe20 0x7f36cc1f35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:43.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.347+0000 7f36d2bff640 1 --2- 172.21.15.73:0/547404869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f36cc06fe20 0x7f36cc1f35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:43.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.347+0000 7f36d2bff640 1 -- 172.21.15.73:0/547404869 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36cc1f3ef0 con 0x7f36cc072780 2024-04-26T20:50:43.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.347+0000 7f36d23fe640 1 --2- 172.21.15.73:0/547404869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f36cc06fe20 0x7f36cc1f35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:50:43.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.347+0000 7f36d2bff640 1 --2- 172.21.15.73:0/547404869 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36cc072780 0x7f36cc149bd0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f36c400ca80 tx=0x7f36c400cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:43.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.347+0000 7f36c2ffd640 1 -- 172.21.15.73:0/547404869 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c4017070 con 0x7f36cc072780 2024-04-26T20:50:43.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.347+0000 7f36c2ffd640 1 -- 172.21.15.73:0/547404869 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36c400abd0 con 0x7f36cc072780 2024-04-26T20:50:43.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.347+0000 7f36c2ffd640 1 -- 172.21.15.73:0/547404869 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c4012400 con 0x7f36cc072780 2024-04-26T20:50:43.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.347+0000 7f36d4662640 1 -- 172.21.15.73:0/547404869 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36cc1f41d0 con 0x7f36cc072780 2024-04-26T20:50:43.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.348+0000 7f36d4662640 1 -- 172.21.15.73:0/547404869 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36cc0c9350 con 0x7f36cc072780 2024-04-26T20:50:43.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.349+0000 7f36d4662640 1 -- 172.21.15.73:0/547404869 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36cc1f43a0 con 0x7f36cc072780 2024-04-26T20:50:43.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.349+0000 7f36c2ffd640 1 -- 172.21.15.73:0/547404869 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f36c401a070 con 0x7f36cc072780 2024-04-26T20:50:43.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.349+0000 7f36c2ffd640 1 --2- 172.21.15.73:0/547404869 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f36b006c8f0 0x7f36b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:43.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.349+0000 7f36c2ffd640 1 -- 172.21.15.73:0/547404869 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f36c408ba90 con 0x7f36cc072780 2024-04-26T20:50:43.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.349+0000 7f36d23fe640 1 --2- 172.21.15.73:0/547404869 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f36b006c8f0 0x7f36b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:43.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.350+0000 7f36d23fe640 1 --2- 172.21.15.73:0/547404869 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f36b006c8f0 0x7f36b006edb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f36c8006fd0 tx=0x7f36c8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:43.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.354+0000 7f36c2ffd640 1 -- 172.21.15.73:0/547404869 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f36c4059530 con 0x7f36cc072780 2024-04-26T20:50:43.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.529+0000 7f36d4662640 1 -- 172.21.15.73:0/547404869 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f36cc0694f0 con 0x7f36cc072780 2024-04-26T20:50:43.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.529+0000 7f36c2ffd640 1 -- 172.21.15.73:0/547404869 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7f36c405d830 con 0x7f36cc072780 2024-04-26T20:50:43.530 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-26T20:50:43.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.531+0000 7f36d4662640 1 -- 172.21.15.73:0/547404869 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f36b006c8f0 msgr2=0x7f36b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:43.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.531+0000 7f36d4662640 1 --2- 172.21.15.73:0/547404869 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f36b006c8f0 0x7f36b006edb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f36c8006fd0 tx=0x7f36c8008040 comp rx=0 tx=0).stop 2024-04-26T20:50:43.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.531+0000 7f36d4662640 1 -- 172.21.15.73:0/547404869 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36cc072780 msgr2=0x7f36cc149bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:43.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.531+0000 7f36d4662640 1 --2- 172.21.15.73:0/547404869 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36cc072780 0x7f36cc149bd0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f36c400ca80 tx=0x7f36c400cf40 comp rx=0 tx=0).stop 2024-04-26T20:50:43.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.531+0000 7f36d4662640 1 -- 172.21.15.73:0/547404869 shutdown_connections 2024-04-26T20:50:43.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.531+0000 7f36d4662640 1 --2- 172.21.15.73:0/547404869 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36cc072780 0x7f36cc149bd0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:43.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.531+0000 7f36d4662640 1 --2- 172.21.15.73:0/547404869 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f36b006c8f0 0x7f36b006edb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:43.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.531+0000 7f36d4662640 1 --2- 172.21.15.73:0/547404869 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f36cc071e60 0x7f36cc0b4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:43.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.531+0000 7f36d4662640 1 --2- 172.21.15.73:0/547404869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f36cc06fe20 0x7f36cc1f35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:43.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.531+0000 7f36d4662640 1 -- 172.21.15.73:0/547404869 >> 172.21.15.73:0/547404869 conn(0x7f36cc0d0be0 msgr2=0x7f36cc0cb370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:43.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.531+0000 7f36d4662640 1 -- 172.21.15.73:0/547404869 shutdown_connections 2024-04-26T20:50:43.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.531+0000 7f36d4662640 1 -- 172.21.15.73:0/547404869 wait complete. 2024-04-26T20:50:43.543 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T20:50:43.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.640+0000 7fd19d864640 1 Processor -- start 2024-04-26T20:50:43.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.640+0000 7fd19d864640 1 -- start start 2024-04-26T20:50:43.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.641+0000 7fd19d864640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd198160720 0x7fd1980ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:43.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.641+0000 7fd19d864640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fd1980ceb60 0x7fd1980d11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:43.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.641+0000 7fd19d864640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1980d0710 con 0x7fd1980d0d00 2024-04-26T20:50:43.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.641+0000 7fd19d864640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fd1980d01e0 con 0x7fd1980ceb60 2024-04-26T20:50:43.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.641+0000 7fd19d864640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fd1980d0430 con 0x7fd198160720 2024-04-26T20:50:43.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.641+0000 7fd196ffd640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd198160720 0x7fd1980ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:43.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.641+0000 7fd1967fc640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fd1980ceb60 0x7fd1980d11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:43.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.641+0000 7fd196ffd640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd198160720 0x7fd1980ce620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.73:53180/0 (socket says 172.21.15.73:53180) 2024-04-26T20:50:43.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.641+0000 7fd196ffd640 1 -- 172.21.15.73:0/3362280310 learned_addr learned my addr 172.21.15.73:0/3362280310 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:43.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.641+0000 7fd1977fe640 1 --1- 172.21.15.73:0/3362280310 >> v1:172.21.15.120:6789/0 conn(0x7fd1980d0d00 0x7fd1980b2570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.73:36712/0 (socket says 172.21.15.73:36712) 2024-04-26T20:50:43.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.642+0000 7fd196ffd640 1 -- 172.21.15.73:0/3362280310 >> v2:172.21.15.120:3300/0 conn(0x7fd1980ceb60 msgr2=0x7fd1980d11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:43.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.642+0000 7fd196ffd640 1 --2- 172.21.15.73:0/3362280310 >> v2:172.21.15.120:3300/0 conn(0x7fd1980ceb60 0x7fd1980d11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:43.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.642+0000 7fd196ffd640 1 -- 172.21.15.73:0/3362280310 >> v1:172.21.15.120:6789/0 conn(0x7fd1980d0d00 legacy=0x7fd1980b2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:43.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.642+0000 7fd196ffd640 1 -- 172.21.15.73:0/3362280310 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1980cdb40 con 0x7fd198160720 2024-04-26T20:50:43.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.642+0000 7fd196ffd640 1 --2- 172.21.15.73:0/3362280310 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd198160720 0x7fd1980ce620 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd188009870 tx=0x7fd18802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebe22ccc6c515cb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:43.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.642+0000 7fd195ffb640 1 -- 172.21.15.73:0/3362280310 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd188037070 con 0x7fd198160720 2024-04-26T20:50:43.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.642+0000 7fd195ffb640 1 -- 172.21.15.73:0/3362280310 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd188032440 con 0x7fd198160720 2024-04-26T20:50:43.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.643+0000 7fd195ffb640 1 -- 172.21.15.73:0/3362280310 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd188032810 con 0x7fd198160720 2024-04-26T20:50:43.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.643+0000 7fd19d864640 1 -- 172.21.15.73:0/3362280310 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd198160720 msgr2=0x7fd1980ce620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:43.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.643+0000 7fd19d864640 1 --2- 172.21.15.73:0/3362280310 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd198160720 0x7fd1980ce620 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd188009870 tx=0x7fd18802a270 comp rx=0 tx=0).stop 2024-04-26T20:50:43.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.643+0000 7fd19d864640 1 -- 172.21.15.73:0/3362280310 shutdown_connections 2024-04-26T20:50:43.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.643+0000 7fd19d864640 1 --2- 172.21.15.73:0/3362280310 >> v2:172.21.15.120:3300/0 conn(0x7fd1980ceb60 0x7fd1980d11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:43.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.643+0000 7fd19d864640 1 --2- 172.21.15.73:0/3362280310 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd198160720 0x7fd1980ce620 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:43.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.643+0000 7fd19d864640 1 -- 172.21.15.73:0/3362280310 >> 172.21.15.73:0/3362280310 conn(0x7fd1980d2e00 msgr2=0x7fd1980d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:43.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.643+0000 7fd19d864640 1 -- 172.21.15.73:0/3362280310 shutdown_connections 2024-04-26T20:50:43.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.643+0000 7fd19d864640 1 -- 172.21.15.73:0/3362280310 wait complete. 2024-04-26T20:50:43.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.644+0000 7fd19d864640 1 Processor -- start 2024-04-26T20:50:43.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.644+0000 7fd19d864640 1 -- start start 2024-04-26T20:50:43.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.645+0000 7fd19d864640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd1980ceb60 0x7fd1980c8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:43.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.645+0000 7fd19d864640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd1980d0d00 0x7fd198159950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:43.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.645+0000 7fd19d864640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd198160720 0x7fd1980b70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:43.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.645+0000 7fd19d864640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd198160b00 con 0x7fd1980d0d00 2024-04-26T20:50:43.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.645+0000 7fd19d864640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd1980ce100 con 0x7fd198160720 2024-04-26T20:50:43.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.645+0000 7fd19d864640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fd1980c8fc0 con 0x7fd1980ceb60 2024-04-26T20:50:43.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.645+0000 7fd1967fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd1980d0d00 0x7fd198159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:43.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.645+0000 7fd1967fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd1980d0d00 0x7fd198159950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34952/0 (socket says 172.21.15.73:34952) 2024-04-26T20:50:43.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.645+0000 7fd1967fc640 1 -- 172.21.15.73:0/3521349481 learned_addr learned my addr 172.21.15.73:0/3521349481 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:43.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.645+0000 7fd196ffd640 1 --2- 172.21.15.73:0/3521349481 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd1980ceb60 0x7fd1980c8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:43.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.645+0000 7fd1977fe640 1 --2- 172.21.15.73:0/3521349481 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd198160720 0x7fd1980b70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:43.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.645+0000 7fd1967fc640 1 -- 172.21.15.73:0/3521349481 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd1980ceb60 msgr2=0x7fd1980c8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:43.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.645+0000 7fd1967fc640 1 --2- 172.21.15.73:0/3521349481 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd1980ceb60 0x7fd1980c8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:43.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.645+0000 7fd1967fc640 1 -- 172.21.15.73:0/3521349481 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd198160720 msgr2=0x7fd1980b70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:43.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.645+0000 7fd1967fc640 1 --2- 172.21.15.73:0/3521349481 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd198160720 0x7fd1980b70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:43.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.645+0000 7fd1967fc640 1 -- 172.21.15.73:0/3521349481 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1981581b0 con 0x7fd1980d0d00 2024-04-26T20:50:43.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.646+0000 7fd1967fc640 1 --2- 172.21.15.73:0/3521349481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd1980d0d00 0x7fd198159950 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd18c00e980 tx=0x7fd18c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:43.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.646+0000 7fd17ffff640 1 -- 172.21.15.73:0/3521349481 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18c017070 con 0x7fd1980d0d00 2024-04-26T20:50:43.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.646+0000 7fd17ffff640 1 -- 172.21.15.73:0/3521349481 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd18c00c9a0 con 0x7fd1980d0d00 2024-04-26T20:50:43.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.646+0000 7fd17ffff640 1 -- 172.21.15.73:0/3521349481 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18c00cd70 con 0x7fd1980d0d00 2024-04-26T20:50:43.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.646+0000 7fd19d864640 1 -- 172.21.15.73:0/3521349481 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd198158490 con 0x7fd1980d0d00 2024-04-26T20:50:43.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.646+0000 7fd19d864640 1 -- 172.21.15.73:0/3521349481 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd198158a50 con 0x7fd1980d0d00 2024-04-26T20:50:43.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.647+0000 7fd19d864640 1 -- 172.21.15.73:0/3521349481 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1980694f0 con 0x7fd1980d0d00 2024-04-26T20:50:43.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.647+0000 7fd17ffff640 1 -- 172.21.15.73:0/3521349481 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd18c01a070 con 0x7fd1980d0d00 2024-04-26T20:50:43.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.647+0000 7fd17ffff640 1 --2- 172.21.15.73:0/3521349481 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd18406c8f0 0x7fd18406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:43.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.648+0000 7fd196ffd640 1 --2- 172.21.15.73:0/3521349481 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd18406c8f0 0x7fd18406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:43.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.648+0000 7fd17ffff640 1 -- 172.21.15.73:0/3521349481 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd18c08bdb0 con 0x7fd1980d0d00 2024-04-26T20:50:43.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.648+0000 7fd196ffd640 1 --2- 172.21.15.73:0/3521349481 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd18406c8f0 0x7fd18406edb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd188034000 tx=0x7fd18802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:43.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.652+0000 7fd17ffff640 1 -- 172.21.15.73:0/3521349481 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd18c090070 con 0x7fd1980d0d00 2024-04-26T20:50:43.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.840+0000 7fd19d864640 1 -- 172.21.15.73:0/3521349481 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd1980d3d30 con 0x7fd1980d0d00 2024-04-26T20:50:43.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.841+0000 7fd17ffff640 1 -- 172.21.15.73:0/3521349481 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7fd18c059840 con 0x7fd1980d0d00 2024-04-26T20:50:43.842 INFO:teuthology.orchestra.run.smithi073.stderr:listed 0 entries 2024-04-26T20:50:43.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.843+0000 7fd19d864640 1 -- 172.21.15.73:0/3521349481 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd18406c8f0 msgr2=0x7fd18406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:43.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.843+0000 7fd19d864640 1 --2- 172.21.15.73:0/3521349481 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd18406c8f0 0x7fd18406edb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd188034000 tx=0x7fd18802e000 comp rx=0 tx=0).stop 2024-04-26T20:50:43.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.843+0000 7fd19d864640 1 -- 172.21.15.73:0/3521349481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd1980d0d00 msgr2=0x7fd198159950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:43.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.843+0000 7fd19d864640 1 --2- 172.21.15.73:0/3521349481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd1980d0d00 0x7fd198159950 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd18c00e980 tx=0x7fd18c00ee40 comp rx=0 tx=0).stop 2024-04-26T20:50:43.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.843+0000 7fd19d864640 1 -- 172.21.15.73:0/3521349481 shutdown_connections 2024-04-26T20:50:43.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.843+0000 7fd19d864640 1 --2- 172.21.15.73:0/3521349481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd1980d0d00 0x7fd198159950 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:43.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.843+0000 7fd19d864640 1 --2- 172.21.15.73:0/3521349481 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd18406c8f0 0x7fd18406edb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:43.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.843+0000 7fd19d864640 1 --2- 172.21.15.73:0/3521349481 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd198160720 0x7fd1980b70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:43.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.843+0000 7fd19d864640 1 --2- 172.21.15.73:0/3521349481 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd1980ceb60 0x7fd1980c8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:43.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.843+0000 7fd19d864640 1 -- 172.21.15.73:0/3521349481 >> 172.21.15.73:0/3521349481 conn(0x7fd1980d2e00 msgr2=0x7fd198070fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:43.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.843+0000 7fd19d864640 1 -- 172.21.15.73:0/3521349481 shutdown_connections 2024-04-26T20:50:43.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:43.843+0000 7fd19d864640 1 -- 172.21.15.73:0/3521349481 wait complete. 2024-04-26T20:50:43.854 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-26T20:50:43.854 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:50:43.870 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:50:44.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.006+0000 7f65cf138640 1 Processor -- start 2024-04-26T20:50:44.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.006+0000 7f65cf138640 1 -- start start 2024-04-26T20:50:44.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.007+0000 7f65cf138640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f65c80d1b30 0x7f65c80d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:44.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.007+0000 7f65cf138640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65c80d10d0 con 0x7f65c80b2340 2024-04-26T20:50:44.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.007+0000 7f65cf138640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65c80d1370 con 0x7f65c8162810 2024-04-26T20:50:44.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.007+0000 7f65cf138640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f65c80d0750 con 0x7f65c80d1b30 2024-04-26T20:50:44.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.007+0000 7f65c7fff640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f65c80b2340 0x7f65c80cd3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40974/0 (socket says 172.21.15.73:40974) 2024-04-26T20:50:44.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.007+0000 7f65c7fff640 1 -- 172.21.15.73:0/1318038297 learned_addr learned my addr 172.21.15.73:0/1318038297 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:50:44.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.007+0000 7f65cd6d5640 1 --2- 172.21.15.73:0/1318038297 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f65c80d1b30 0x7f65c80d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:44.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.007+0000 7f65c77fe640 1 -- 172.21.15.73:0/1318038297 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2891153842 0 0) 0x7f65c80d10d0 con 0x7f65c80b2340 2024-04-26T20:50:44.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.007+0000 7f65c77fe640 1 -- 172.21.15.73:0/1318038297 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f65b0003520 con 0x7f65c80b2340 2024-04-26T20:50:44.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.008+0000 7f65c77fe640 1 -- 172.21.15.73:0/1318038297 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 987900486 0 0) 0x7f65b0003520 con 0x7f65c80b2340 2024-04-26T20:50:44.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.008+0000 7f65c77fe640 1 -- 172.21.15.73:0/1318038297 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f65c80d1b30 msgr2=0x7f65c80d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:44.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.008+0000 7f65c77fe640 1 --2- 172.21.15.73:0/1318038297 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f65c80d1b30 0x7f65c80d1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:44.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.008+0000 7f65c77fe640 1 -- 172.21.15.73:0/1318038297 >> v1:172.21.15.120:6789/0 conn(0x7f65c8162810 legacy=0x7f65c8070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:44.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.008+0000 7f65c77fe640 1 -- 172.21.15.73:0/1318038297 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c80d01b0 con 0x7f65c80b2340 2024-04-26T20:50:44.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.008+0000 7f65c77fe640 1 -- 172.21.15.73:0/1318038297 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7f65bc001670 con 0x7f65c80b2340 2024-04-26T20:50:44.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.008+0000 7f65c77fe640 1 -- 172.21.15.73:0/1318038297 >> v1:172.21.15.73:6789/0 conn(0x7f65c80b2340 legacy=0x7f65c80cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:44.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.008+0000 7f65c77fe640 1 --2- 172.21.15.73:0/1318038297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65b00039e0 0x7f65b0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:44.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.008+0000 7f65c77fe640 1 --2- 172.21.15.73:0/1318038297 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f65c80d1b30 0x7f65b0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:44.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.008+0000 7f65c77fe640 1 --2- 172.21.15.73:0/1318038297 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65b0004d40 0x7f65b00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:44.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.008+0000 7f65c77fe640 1 -- 172.21.15.73:0/1318038297 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f65c80d0750 con 0x7f65b0004d40 2024-04-26T20:50:44.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.008+0000 7f65c77fe640 1 -- 172.21.15.73:0/1318038297 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f65b000baa0 con 0x7f65b00039e0 2024-04-26T20:50:44.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.008+0000 7f65c77fe640 1 -- 172.21.15.73:0/1318038297 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f65b000bca0 con 0x7f65c80d1b30 2024-04-26T20:50:44.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.008+0000 7f65cd6d5640 1 --2- 172.21.15.73:0/1318038297 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65b0004d40 0x7f65b00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:44.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.008+0000 7f65cced4640 1 --2- 172.21.15.73:0/1318038297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65b00039e0 0x7f65b0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:44.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.008+0000 7f65c7fff640 1 --2- 172.21.15.73:0/1318038297 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f65c80d1b30 0x7f65b0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:44.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.008+0000 7f65cd6d5640 1 -- 172.21.15.73:0/1318038297 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f65c80d1b30 msgr2=0x7f65b0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:44.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.008+0000 7f65cd6d5640 1 --2- 172.21.15.73:0/1318038297 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f65c80d1b30 0x7f65b0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:44.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.008+0000 7f65cd6d5640 1 -- 172.21.15.73:0/1318038297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65b00039e0 msgr2=0x7f65b0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:44.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.008+0000 7f65cd6d5640 1 --2- 172.21.15.73:0/1318038297 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65b00039e0 0x7f65b0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:44.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.008+0000 7f65cd6d5640 1 -- 172.21.15.73:0/1318038297 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65b0003520 con 0x7f65b0004d40 2024-04-26T20:50:44.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.009+0000 7f65cd6d5640 1 --2- 172.21.15.73:0/1318038297 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65b0004d40 0x7f65b00051c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f65c0009f90 tx=0x7f65b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d5776b8dd09ae49 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:44.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.009+0000 7f65c77fe640 1 -- 172.21.15.73:0/1318038297 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c0009c10 con 0x7f65b0004d40 2024-04-26T20:50:44.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.009+0000 7f65c77fe640 1 -- 172.21.15.73:0/1318038297 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65c0036070 con 0x7f65b0004d40 2024-04-26T20:50:44.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.009+0000 7f65c77fe640 1 -- 172.21.15.73:0/1318038297 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c0031850 con 0x7f65b0004d40 2024-04-26T20:50:44.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.009+0000 7f65cf138640 1 -- 172.21.15.73:0/1318038297 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65b0004d40 msgr2=0x7f65b00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:44.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.009+0000 7f65cf138640 1 --2- 172.21.15.73:0/1318038297 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65b0004d40 0x7f65b00051c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f65c0009f90 tx=0x7f65b0005830 comp rx=0 tx=0).stop 2024-04-26T20:50:44.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.009+0000 7f65cd6d5640 1 -- 172.21.15.73:0/1318038297 reap_dead start 2024-04-26T20:50:44.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.009+0000 7f65cf138640 1 -- 172.21.15.73:0/1318038297 shutdown_connections 2024-04-26T20:50:44.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.009+0000 7f65cf138640 1 -- 172.21.15.73:0/1318038297 >> 172.21.15.73:0/1318038297 conn(0x7f65c80d2da0 msgr2=0x7f65c80d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:44.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.009+0000 7f65cf138640 1 -- 172.21.15.73:0/1318038297 shutdown_connections 2024-04-26T20:50:44.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.010+0000 7f65cf138640 1 -- 172.21.15.73:0/1318038297 wait complete. 2024-04-26T20:50:44.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.011+0000 7f65cf138640 1 Processor -- start 2024-04-26T20:50:44.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.011+0000 7f65cf138640 1 -- start start 2024-04-26T20:50:44.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.011+0000 7f65cf138640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f65b0004d40 0x7f65c80afab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:44.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.011+0000 7f65cf138640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65c80b11c0 0x7f65c80c9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:44.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.011+0000 7f65cf138640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65c80ca3b0 0x7f65c80cc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:44.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.011+0000 7f65cf138640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f65c80cdb00 con 0x7f65c80b11c0 2024-04-26T20:50:44.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.011+0000 7f65cf138640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f65c80d16a0 con 0x7f65c80ca3b0 2024-04-26T20:50:44.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.011+0000 7f65cf138640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f65c80d1fc0 con 0x7f65b0004d40 2024-04-26T20:50:44.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.012+0000 7f65c7fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65c80b11c0 0x7f65c80c9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:44.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.012+0000 7f65c7fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65c80b11c0 0x7f65c80c9e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34964/0 (socket says 172.21.15.73:34964) 2024-04-26T20:50:44.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.012+0000 7f65c7fff640 1 -- 172.21.15.73:0/3824326171 learned_addr learned my addr 172.21.15.73:0/3824326171 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:44.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.012+0000 7f65cced4640 1 --2- 172.21.15.73:0/3824326171 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f65b0004d40 0x7f65c80afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:44.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.012+0000 7f65cd6d5640 1 --2- 172.21.15.73:0/3824326171 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65c80ca3b0 0x7f65c80cc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:44.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.012+0000 7f65c7fff640 1 -- 172.21.15.73:0/3824326171 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f65b0004d40 msgr2=0x7f65c80afab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:44.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.012+0000 7f65c7fff640 1 --2- 172.21.15.73:0/3824326171 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f65b0004d40 0x7f65c80afab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:44.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.012+0000 7f65c7fff640 1 -- 172.21.15.73:0/3824326171 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65c80ca3b0 msgr2=0x7f65c80cc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:44.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.012+0000 7f65c7fff640 1 --2- 172.21.15.73:0/3824326171 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65c80ca3b0 0x7f65c80cc2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:44.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.012+0000 7f65c7fff640 1 -- 172.21.15.73:0/3824326171 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65c80b0120 con 0x7f65c80b11c0 2024-04-26T20:50:44.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.012+0000 7f65cced4640 1 --2- 172.21.15.73:0/3824326171 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f65b0004d40 0x7f65c80afab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:50:44.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.012+0000 7f65c7fff640 1 --2- 172.21.15.73:0/3824326171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65c80b11c0 0x7f65c80c9e70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f65bc0072f0 tx=0x7f65bc0077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:44.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.012+0000 7f65c57fa640 1 -- 172.21.15.73:0/3824326171 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65bc00f960 con 0x7f65c80b11c0 2024-04-26T20:50:44.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.012+0000 7f65c57fa640 1 -- 172.21.15.73:0/3824326171 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65bc018070 con 0x7f65c80b11c0 2024-04-26T20:50:44.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.012+0000 7f65c57fa640 1 -- 172.21.15.73:0/3824326171 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65bc00fd60 con 0x7f65c80b11c0 2024-04-26T20:50:44.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.012+0000 7f65cf138640 1 -- 172.21.15.73:0/3824326171 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65c80b0400 con 0x7f65c80b11c0 2024-04-26T20:50:44.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.012+0000 7f65cf138640 1 -- 172.21.15.73:0/3824326171 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65c80b09c0 con 0x7f65c80b11c0 2024-04-26T20:50:44.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.013+0000 7f65cf138640 1 -- 172.21.15.73:0/3824326171 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65c80ca7c0 con 0x7f65c80b11c0 2024-04-26T20:50:44.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.013+0000 7f65c57fa640 1 -- 172.21.15.73:0/3824326171 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f65bc01d460 con 0x7f65c80b11c0 2024-04-26T20:50:44.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.014+0000 7f65c57fa640 1 --2- 172.21.15.73:0/3824326171 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f65b406c920 0x7f65b406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:44.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.014+0000 7f65cced4640 1 --2- 172.21.15.73:0/3824326171 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f65b406c920 0x7f65b406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:44.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.014+0000 7f65c57fa640 1 -- 172.21.15.73:0/3824326171 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f65bc08bf30 con 0x7f65c80b11c0 2024-04-26T20:50:44.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.014+0000 7f65cced4640 1 --2- 172.21.15.73:0/3824326171 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f65b406c920 0x7f65b406ede0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f65c80c6a40 tx=0x7f65b8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:44.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.018+0000 7f65c57fa640 1 -- 172.21.15.73:0/3824326171 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f65c80ca7c0 con 0x7f65c80b11c0 2024-04-26T20:50:44.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.195+0000 7f65cf138640 1 -- 172.21.15.73:0/3824326171 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f65c80ce3c0 con 0x7f65c80b11c0 2024-04-26T20:50:44.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.196+0000 7f65c57fa640 1 -- 172.21.15.73:0/3824326171 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18514 (secure 0 0 0) 0x7f65bc05a0b0 con 0x7f65c80b11c0 2024-04-26T20:50:44.197 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:50:44.197 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":25,"fsid":"fbd3f548-b547-4713-856b-09f98f5d1764","created":"2024-04-26T20:50:07.876786+0000","modified":"2024-04-26T20:50:38.657510+0000","last_up_change":"2024-04-26T20:50:19.082741+0000","last_in_change":"2024-04-26T20:50:10.018400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:50:19.993731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:50:29.964420+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:50:30.920082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:50:34.650207+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"72d6f4f3-214a-4392-a02f-c33cbcab3c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6809","nonce":3461705124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6811","nonce":3461705124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6815","nonce":3461705124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6813","nonce":3461705124}]},"public_addr":"172.21.15.73:6809/3461705124","cluster_addr":"172.21.15.73:6811/3461705124","heartbeat_back_addr":"172.21.15.73:6815/3461705124","heartbeat_front_addr":"172.21.15.73:6813/3461705124","state":["exists","up"]},{"osd":1,"uuid":"786bf8ca-6ddc-4b3c-be6c-9152398bd5fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6817","nonce":2827619403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6819","nonce":2827619403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6825","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6827","nonce":2827619403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6821","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6823","nonce":2827619403}]},"public_addr":"172.21.15.73:6817/2827619403","cluster_addr":"172.21.15.73:6819/2827619403","heartbeat_back_addr":"172.21.15.73:6827/2827619403","heartbeat_front_addr":"172.21.15.73:6823/2827619403","state":["exists","up"]},{"osd":2,"uuid":"90963b1a-5437-4ef8-ad8b-6b74c534eeb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6801","nonce":2542681064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6803","nonce":2542681064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6808","nonce":2542681064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6805","nonce":2542681064}]},"public_addr":"172.21.15.73:6801/2542681064","cluster_addr":"172.21.15.73:6803/2542681064","heartbeat_back_addr":"172.21.15.73:6808/2542681064","heartbeat_front_addr":"172.21.15.73:6805/2542681064","state":["exists","up"]},{"osd":3,"uuid":"bd3d2148-6e38-41b4-a453-fe6e3b8b4ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6822","nonce":1604042718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6826","nonce":1604042718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6831","nonce":1604042718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6829","nonce":1604042718}]},"public_addr":"172.21.15.73:6822/1604042718","cluster_addr":"172.21.15.73:6826/1604042718","heartbeat_back_addr":"172.21.15.73:6831/1604042718","heartbeat_front_addr":"172.21.15.73:6829/1604042718","state":["exists","up"]},{"osd":4,"uuid":"e8668dc8-a23f-46d2-8311-cb882da08e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6817","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6819","nonce":3591055561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6824","nonce":3591055561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6831","nonce":3591055561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6828","nonce":3591055561}]},"public_addr":"172.21.15.120:6819/3591055561","cluster_addr":"172.21.15.120:6824/3591055561","heartbeat_back_addr":"172.21.15.120:6831/3591055561","heartbeat_front_addr":"172.21.15.120:6828/3591055561","state":["exists","up"]},{"osd":5,"uuid":"cbac0bdb-96bd-4852-bcce-e564495c3fcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6809","nonce":819450433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6811","nonce":819450433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6815","nonce":819450433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6813","nonce":819450433}]},"public_addr":"172.21.15.120:6809/819450433","cluster_addr":"172.21.15.120:6811/819450433","heartbeat_back_addr":"172.21.15.120:6815/819450433","heartbeat_front_addr":"172.21.15.120:6813/819450433","state":["exists","up"]},{"osd":6,"uuid":"1884b2a8-569f-4272-96a9-f25a09364dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6801","nonce":3946210727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6803","nonce":3946210727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6807","nonce":3946210727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6805","nonce":3946210727}]},"public_addr":"172.21.15.120:6801/3946210727","cluster_addr":"172.21.15.120:6803/3946210727","heartbeat_back_addr":"172.21.15.120:6807/3946210727","heartbeat_front_addr":"172.21.15.120:6805/3946210727","state":["exists","up"]},{"osd":7,"uuid":"c0c3bf6d-a4c5-4f1e-aa2d-330951830666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6818","nonce":2915574674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6821","nonce":2915574674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6827","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6829","nonce":2915574674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6823","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6825","nonce":2915574674}]},"public_addr":"172.21.15.120:6818/2915574674","cluster_addr":"172.21.15.120:6821/2915574674","heartbeat_back_addr":"172.21.15.120:6829/2915574674","heartbeat_front_addr":"172.21.15.120:6825/2915574674","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.218216+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.208710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.112705+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.182789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.164324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.174212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.225396+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.261559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:50:44.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.198+0000 7f65cf138640 1 -- 172.21.15.73:0/3824326171 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f65b406c920 msgr2=0x7f65b406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:44.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.198+0000 7f65cf138640 1 --2- 172.21.15.73:0/3824326171 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f65b406c920 0x7f65b406ede0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f65c80c6a40 tx=0x7f65b8009040 comp rx=0 tx=0).stop 2024-04-26T20:50:44.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.198+0000 7f65cf138640 1 -- 172.21.15.73:0/3824326171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65c80b11c0 msgr2=0x7f65c80c9e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:44.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.198+0000 7f65cf138640 1 --2- 172.21.15.73:0/3824326171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65c80b11c0 0x7f65c80c9e70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f65bc0072f0 tx=0x7f65bc0077b0 comp rx=0 tx=0).stop 2024-04-26T20:50:44.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.198+0000 7f65cf138640 1 -- 172.21.15.73:0/3824326171 shutdown_connections 2024-04-26T20:50:44.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.198+0000 7f65cf138640 1 --2- 172.21.15.73:0/3824326171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65c80b11c0 0x7f65c80c9e70 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:44.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.198+0000 7f65cf138640 1 --2- 172.21.15.73:0/3824326171 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f65b406c920 0x7f65b406ede0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:44.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.198+0000 7f65cf138640 1 --2- 172.21.15.73:0/3824326171 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f65c80ca3b0 0x7f65c80cc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:44.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.198+0000 7f65cf138640 1 --2- 172.21.15.73:0/3824326171 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f65b0004d40 0x7f65c80afab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:44.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.198+0000 7f65cf138640 1 -- 172.21.15.73:0/3824326171 >> 172.21.15.73:0/3824326171 conn(0x7f65c80d2da0 msgr2=0x7f65c80d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:44.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.198+0000 7f65cf138640 1 -- 172.21.15.73:0/3824326171 shutdown_connections 2024-04-26T20:50:44.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.198+0000 7f65cf138640 1 -- 172.21.15.73:0/3824326171 wait complete. 2024-04-26T20:50:44.210 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:50:19.993731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:50:29.964420+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:50:30.920082+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T20:50:34.650207+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T20:50:44.210 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:50:44.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.309+0000 7f221f22a640 1 Processor -- start 2024-04-26T20:50:44.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.309+0000 7f221f22a640 1 -- start start 2024-04-26T20:50:44.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.310+0000 7f221f22a640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f221815f550 0x7f22180d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:44.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.310+0000 7f221f22a640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f22180d3bc0 0x7f22180d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:44.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.310+0000 7f221f22a640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22180d1040 con 0x7f221815f170 2024-04-26T20:50:44.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.310+0000 7f221f22a640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f22180c9430 con 0x7f221815f550 2024-04-26T20:50:44.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.310+0000 7f221f22a640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f22180c9680 con 0x7f22180d3bc0 2024-04-26T20:50:44.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.310+0000 7f2217fff640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f221815f550 0x7f22180d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:44.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.310+0000 7f2217fff640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f221815f550 0x7f22180d5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34974/0 (socket says 172.21.15.73:34974) 2024-04-26T20:50:44.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.310+0000 7f221cfc6640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f221815f170 0x7f22180d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40978/0 (socket says 172.21.15.73:40978) 2024-04-26T20:50:44.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.310+0000 7f2217fff640 1 -- 172.21.15.73:0/329991345 learned_addr learned my addr 172.21.15.73:0/329991345 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:44.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.310+0000 7f221d7c7640 1 --2- 172.21.15.73:0/329991345 >> v2:172.21.15.120:3300/0 conn(0x7f22180d3bc0 0x7f22180d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:44.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.310+0000 7f2217fff640 1 -- 172.21.15.73:0/329991345 >> v2:172.21.15.120:3300/0 conn(0x7f22180d3bc0 msgr2=0x7f22180d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:44.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.310+0000 7f2217fff640 1 --2- 172.21.15.73:0/329991345 >> v2:172.21.15.120:3300/0 conn(0x7f22180d3bc0 0x7f22180d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:44.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.310+0000 7f2217fff640 1 -- 172.21.15.73:0/329991345 >> v1:172.21.15.73:6789/0 conn(0x7f221815f170 legacy=0x7f22180d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:44.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.310+0000 7f2217fff640 1 -- 172.21.15.73:0/329991345 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22180c9960 con 0x7f221815f550 2024-04-26T20:50:44.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.310+0000 7f22177fe640 1 -- 172.21.15.73:0/329991345 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3788655796 0 0) 0x7f220c001110 con 0x7f221815f170 2024-04-26T20:50:44.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.311+0000 7f2217fff640 1 --2- 172.21.15.73:0/329991345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f221815f550 0x7f22180d5a80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2204009870 tx=0x7f220402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edb7a0942e43ecc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:44.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.311+0000 7f22177fe640 1 -- 172.21.15.73:0/329991345 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2204037070 con 0x7f221815f550 2024-04-26T20:50:44.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.311+0000 7f22177fe640 1 -- 172.21.15.73:0/329991345 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2204032440 con 0x7f221815f550 2024-04-26T20:50:44.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.311+0000 7f22177fe640 1 -- 172.21.15.73:0/329991345 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2204032810 con 0x7f221815f550 2024-04-26T20:50:44.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.311+0000 7f221f22a640 1 -- 172.21.15.73:0/329991345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f221815f550 msgr2=0x7f22180d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:44.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.311+0000 7f221f22a640 1 --2- 172.21.15.73:0/329991345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f221815f550 0x7f22180d5a80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2204009870 tx=0x7f220402a2a0 comp rx=0 tx=0).stop 2024-04-26T20:50:44.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.311+0000 7f221f22a640 1 -- 172.21.15.73:0/329991345 shutdown_connections 2024-04-26T20:50:44.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.311+0000 7f221f22a640 1 --2- 172.21.15.73:0/329991345 >> v2:172.21.15.120:3300/0 conn(0x7f22180d3bc0 0x7f22180d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:44.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.311+0000 7f221f22a640 1 --2- 172.21.15.73:0/329991345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f221815f550 0x7f22180d5a80 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:44.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.311+0000 7f221f22a640 1 -- 172.21.15.73:0/329991345 >> 172.21.15.73:0/329991345 conn(0x7f22180d4f60 msgr2=0x7f22180d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:44.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.311+0000 7f221f22a640 1 -- 172.21.15.73:0/329991345 shutdown_connections 2024-04-26T20:50:44.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.311+0000 7f221f22a640 1 -- 172.21.15.73:0/329991345 wait complete. 2024-04-26T20:50:44.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.313+0000 7f221f22a640 1 Processor -- start 2024-04-26T20:50:44.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.313+0000 7f221f22a640 1 -- start start 2024-04-26T20:50:44.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.313+0000 7f221f22a640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f22180d3bc0 0x7f22180cee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:44.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.313+0000 7f221f22a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f221815f170 0x7f22180cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:44.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.313+0000 7f221f22a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2218152210 0x7f221815a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:44.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.313+0000 7f221f22a640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f22180ca5c0 con 0x7f221815f170 2024-04-26T20:50:44.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.313+0000 7f221f22a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f22180ca440 con 0x7f2218152210 2024-04-26T20:50:44.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.313+0000 7f221f22a640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f22180ca740 con 0x7f22180d3bc0 2024-04-26T20:50:44.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.314+0000 7f2217fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f221815f170 0x7f22180cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:44.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.314+0000 7f2217fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f221815f170 0x7f22180cbfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34976/0 (socket says 172.21.15.73:34976) 2024-04-26T20:50:44.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.314+0000 7f2217fff640 1 -- 172.21.15.73:0/2108448535 learned_addr learned my addr 172.21.15.73:0/2108448535 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:44.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.314+0000 7f221cfc6640 1 --2- 172.21.15.73:0/2108448535 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f22180d3bc0 0x7f22180cee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:44.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.314+0000 7f221d7c7640 1 --2- 172.21.15.73:0/2108448535 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2218152210 0x7f221815a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:44.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.314+0000 7f2217fff640 1 -- 172.21.15.73:0/2108448535 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f22180d3bc0 msgr2=0x7f22180cee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:44.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.314+0000 7f2217fff640 1 --2- 172.21.15.73:0/2108448535 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f22180d3bc0 0x7f22180cee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:44.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.314+0000 7f2217fff640 1 -- 172.21.15.73:0/2108448535 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2218152210 msgr2=0x7f221815a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:44.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.314+0000 7f2217fff640 1 --2- 172.21.15.73:0/2108448535 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2218152210 0x7f221815a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:44.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.314+0000 7f2217fff640 1 -- 172.21.15.73:0/2108448535 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22180cc710 con 0x7f221815f170 2024-04-26T20:50:44.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.314+0000 7f221cfc6640 1 --2- 172.21.15.73:0/2108448535 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f22180d3bc0 0x7f22180cee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:50:44.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.314+0000 7f2217fff640 1 --2- 172.21.15.73:0/2108448535 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f221815f170 0x7f22180cbfa0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f220402a2e0 tx=0x7f22040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:44.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.314+0000 7f22157fa640 1 -- 172.21.15.73:0/2108448535 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2204037040 con 0x7f221815f170 2024-04-26T20:50:44.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.314+0000 7f22157fa640 1 -- 172.21.15.73:0/2108448535 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2204042070 con 0x7f221815f170 2024-04-26T20:50:44.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.314+0000 7f22157fa640 1 -- 172.21.15.73:0/2108448535 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2204030880 con 0x7f221815f170 2024-04-26T20:50:44.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.314+0000 7f221f22a640 1 -- 172.21.15.73:0/2108448535 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22180cc9f0 con 0x7f221815f170 2024-04-26T20:50:44.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.314+0000 7f221f22a640 1 -- 172.21.15.73:0/2108448535 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22180ccfb0 con 0x7f221815f170 2024-04-26T20:50:44.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.315+0000 7f221f22a640 1 -- 172.21.15.73:0/2108448535 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22180694f0 con 0x7f221815f170 2024-04-26T20:50:44.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.315+0000 7f22157fa640 1 -- 172.21.15.73:0/2108448535 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2204030a20 con 0x7f221815f170 2024-04-26T20:50:44.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.316+0000 7f22157fa640 1 --2- 172.21.15.73:0/2108448535 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f21fc06c8f0 0x7f21fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:44.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.316+0000 7f22157fa640 1 -- 172.21.15.73:0/2108448535 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f22040ac6c0 con 0x7f221815f170 2024-04-26T20:50:44.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.316+0000 7f221cfc6640 1 --2- 172.21.15.73:0/2108448535 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f21fc06c8f0 0x7f21fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:44.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.316+0000 7f221cfc6640 1 --2- 172.21.15.73:0/2108448535 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f21fc06c8f0 0x7f21fc06edb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f220c000f60 tx=0x7f220c007a30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:44.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.320+0000 7f22157fa640 1 -- 172.21.15.73:0/2108448535 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2204045360 con 0x7f221815f170 2024-04-26T20:50:44.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.492+0000 7f221f22a640 1 -- 172.21.15.73:0/2108448535 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f221815ea90 con 0x7f221815f170 2024-04-26T20:50:44.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.492+0000 7f22157fa640 1 -- 172.21.15.73:0/2108448535 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f220407a840 con 0x7f221815f170 2024-04-26T20:50:44.493 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-26T20:50:44.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.494+0000 7f221f22a640 1 -- 172.21.15.73:0/2108448535 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f21fc06c8f0 msgr2=0x7f21fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:44.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.494+0000 7f221f22a640 1 --2- 172.21.15.73:0/2108448535 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f21fc06c8f0 0x7f21fc06edb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f220c000f60 tx=0x7f220c007a30 comp rx=0 tx=0).stop 2024-04-26T20:50:44.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.494+0000 7f221f22a640 1 -- 172.21.15.73:0/2108448535 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f221815f170 msgr2=0x7f22180cbfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:44.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.494+0000 7f221f22a640 1 --2- 172.21.15.73:0/2108448535 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f221815f170 0x7f22180cbfa0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f220402a2e0 tx=0x7f22040046c0 comp rx=0 tx=0).stop 2024-04-26T20:50:44.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.494+0000 7f221f22a640 1 -- 172.21.15.73:0/2108448535 shutdown_connections 2024-04-26T20:50:44.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.494+0000 7f221f22a640 1 --2- 172.21.15.73:0/2108448535 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f221815f170 0x7f22180cbfa0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:44.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.494+0000 7f221f22a640 1 --2- 172.21.15.73:0/2108448535 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f21fc06c8f0 0x7f21fc06edb0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:44.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.494+0000 7f221f22a640 1 --2- 172.21.15.73:0/2108448535 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2218152210 0x7f221815a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:44.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.494+0000 7f221f22a640 1 --2- 172.21.15.73:0/2108448535 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f22180d3bc0 0x7f22180cee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:44.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.494+0000 7f221f22a640 1 -- 172.21.15.73:0/2108448535 >> 172.21.15.73:0/2108448535 conn(0x7f22180d4f60 msgr2=0x7f22180d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:44.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.494+0000 7f221f22a640 1 -- 172.21.15.73:0/2108448535 shutdown_connections 2024-04-26T20:50:44.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.494+0000 7f221f22a640 1 -- 172.21.15.73:0/2108448535 wait complete. 2024-04-26T20:50:44.506 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:50:44.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.604+0000 7f3024546640 1 Processor -- start 2024-04-26T20:50:44.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.604+0000 7f3024546640 1 -- start start 2024-04-26T20:50:44.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.605+0000 7f3024546640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f301c1628f0 0x7f301c0d1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:44.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.605+0000 7f3024546640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f301c0d1550 0x7f301c0d1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:44.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.605+0000 7f3024546640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f301c0d0890 con 0x7f301c162cd0 2024-04-26T20:50:44.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.605+0000 7f3024546640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f301c0d1130 con 0x7f301c0d1550 2024-04-26T20:50:44.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.605+0000 7f3024546640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f301c0d0b70 con 0x7f301c1628f0 2024-04-26T20:50:44.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.605+0000 7f3022ae3640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f301c0d1550 0x7f301c0d1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:44.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.605+0000 7f3021ae1640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f301c162cd0 0x7f301c0d2ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40986/0 (socket says 172.21.15.73:40986) 2024-04-26T20:50:44.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.605+0000 7f3021ae1640 1 -- 172.21.15.73:0/2839589358 learned_addr learned my addr 172.21.15.73:0/2839589358 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:50:44.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.606+0000 7f30222e2640 1 --2- 172.21.15.73:0/2839589358 >> v2:172.21.15.120:3300/0 conn(0x7f301c1628f0 0x7f301c0d1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:44.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.606+0000 7f3022ae3640 1 -- 172.21.15.73:0/2839589358 >> v2:172.21.15.120:3300/0 conn(0x7f301c1628f0 msgr2=0x7f301c0d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:44.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.606+0000 7f3022ae3640 1 --2- 172.21.15.73:0/2839589358 >> v2:172.21.15.120:3300/0 conn(0x7f301c1628f0 0x7f301c0d1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:44.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.606+0000 7f3022ae3640 1 -- 172.21.15.73:0/2839589358 >> v1:172.21.15.73:6789/0 conn(0x7f301c162cd0 legacy=0x7f301c0d2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:44.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.606+0000 7f3022ae3640 1 -- 172.21.15.73:0/2839589358 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f301c0d0e50 con 0x7f301c0d1550 2024-04-26T20:50:44.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.606+0000 7f3022ae3640 1 --2- 172.21.15.73:0/2839589358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f301c0d1550 0x7f301c0d1f40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f30140097a0 tx=0x7f301402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f916475466f4ad90 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:44.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.606+0000 7f30212e0640 1 -- 172.21.15.73:0/2839589358 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3014037070 con 0x7f301c0d1550 2024-04-26T20:50:44.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.606+0000 7f30212e0640 1 -- 172.21.15.73:0/2839589358 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f301402ae50 con 0x7f301c0d1550 2024-04-26T20:50:44.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.606+0000 7f30212e0640 1 -- 172.21.15.73:0/2839589358 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3014032660 con 0x7f301c0d1550 2024-04-26T20:50:44.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.607+0000 7f3024546640 1 -- 172.21.15.73:0/2839589358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f301c0d1550 msgr2=0x7f301c0d1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:44.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.607+0000 7f3024546640 1 --2- 172.21.15.73:0/2839589358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f301c0d1550 0x7f301c0d1f40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f30140097a0 tx=0x7f301402a140 comp rx=0 tx=0).stop 2024-04-26T20:50:44.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.607+0000 7f3024546640 1 -- 172.21.15.73:0/2839589358 shutdown_connections 2024-04-26T20:50:44.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.607+0000 7f3024546640 1 --2- 172.21.15.73:0/2839589358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f301c0d1550 0x7f301c0d1f40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:44.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.607+0000 7f3024546640 1 --2- 172.21.15.73:0/2839589358 >> v2:172.21.15.120:3300/0 conn(0x7f301c1628f0 0x7f301c0d1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:44.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.607+0000 7f3024546640 1 -- 172.21.15.73:0/2839589358 >> 172.21.15.73:0/2839589358 conn(0x7f301c0d4ff0 msgr2=0x7f301c0d5400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:44.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.607+0000 7f3024546640 1 -- 172.21.15.73:0/2839589358 shutdown_connections 2024-04-26T20:50:44.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.607+0000 7f3024546640 1 -- 172.21.15.73:0/2839589358 wait complete. 2024-04-26T20:50:44.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.608+0000 7f3024546640 1 Processor -- start 2024-04-26T20:50:44.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.608+0000 7f3024546640 1 -- start start 2024-04-26T20:50:44.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.609+0000 7f3024546640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f301c0d1550 0x7f301c0ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:44.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.609+0000 7f3024546640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f301c1628f0 0x7f301c06e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:44.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.609+0000 7f3024546640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f301c162cd0 0x7f301c15a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:44.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.609+0000 7f3024546640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f301c0b5df0 con 0x7f301c162cd0 2024-04-26T20:50:44.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.609+0000 7f3024546640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f301c0b5c70 con 0x7f301c0d1550 2024-04-26T20:50:44.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.609+0000 7f3024546640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f301c0b5f70 con 0x7f301c1628f0 2024-04-26T20:50:44.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.609+0000 7f3022ae3640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f301c162cd0 0x7f301c15a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:44.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.609+0000 7f3022ae3640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f301c162cd0 0x7f301c15a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34986/0 (socket says 172.21.15.73:34986) 2024-04-26T20:50:44.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.609+0000 7f3022ae3640 1 -- 172.21.15.73:0/519619992 learned_addr learned my addr 172.21.15.73:0/519619992 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:44.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.609+0000 7f30222e2640 1 --2- 172.21.15.73:0/519619992 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f301c0d1550 0x7f301c0ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:44.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.609+0000 7f3021ae1640 1 --2- 172.21.15.73:0/519619992 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f301c1628f0 0x7f301c06e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:44.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.609+0000 7f3022ae3640 1 -- 172.21.15.73:0/519619992 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f301c1628f0 msgr2=0x7f301c06e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:44.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.609+0000 7f3022ae3640 1 --2- 172.21.15.73:0/519619992 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f301c1628f0 0x7f301c06e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:44.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.609+0000 7f3022ae3640 1 -- 172.21.15.73:0/519619992 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f301c0d1550 msgr2=0x7f301c0ce7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:44.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.609+0000 7f3022ae3640 1 --2- 172.21.15.73:0/519619992 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f301c0d1550 0x7f301c0ce7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:44.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.609+0000 7f3022ae3640 1 -- 172.21.15.73:0/519619992 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f301c0cc5f0 con 0x7f301c162cd0 2024-04-26T20:50:44.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.609+0000 7f30222e2640 1 --2- 172.21.15.73:0/519619992 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f301c0d1550 0x7f301c0ce7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:50:44.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.609+0000 7f3022ae3640 1 --2- 172.21.15.73:0/519619992 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f301c162cd0 0x7f301c15a140 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f30140098c0 tx=0x7f30140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:44.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.610+0000 7f3012ffd640 1 -- 172.21.15.73:0/519619992 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3014041020 con 0x7f301c162cd0 2024-04-26T20:50:44.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.610+0000 7f3012ffd640 1 -- 172.21.15.73:0/519619992 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f301403c670 con 0x7f301c162cd0 2024-04-26T20:50:44.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.610+0000 7f3024546640 1 -- 172.21.15.73:0/519619992 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f301c0cc8d0 con 0x7f301c162cd0 2024-04-26T20:50:44.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.610+0000 7f3012ffd640 1 -- 172.21.15.73:0/519619992 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3014037040 con 0x7f301c162cd0 2024-04-26T20:50:44.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.610+0000 7f3024546640 1 -- 172.21.15.73:0/519619992 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f301c1f60d0 con 0x7f301c162cd0 2024-04-26T20:50:44.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.611+0000 7f3024546640 1 -- 172.21.15.73:0/519619992 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f301c0694f0 con 0x7f301c162cd0 2024-04-26T20:50:44.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.611+0000 7f3012ffd640 1 -- 172.21.15.73:0/519619992 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3014044020 con 0x7f301c162cd0 2024-04-26T20:50:44.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.611+0000 7f3012ffd640 1 --2- 172.21.15.73:0/519619992 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f300006c8f0 0x7f300006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:44.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.611+0000 7f3012ffd640 1 -- 172.21.15.73:0/519619992 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f30140ac530 con 0x7f301c162cd0 2024-04-26T20:50:44.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.611+0000 7f30222e2640 1 --2- 172.21.15.73:0/519619992 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f300006c8f0 0x7f300006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:44.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.612+0000 7f30222e2640 1 --2- 172.21.15.73:0/519619992 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f300006c8f0 0x7f300006edb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3018009b20 tx=0x7f30180039d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:44.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.616+0000 7f3012ffd640 1 -- 172.21.15.73:0/519619992 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f301404b030 con 0x7f301c162cd0 2024-04-26T20:50:44.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.796+0000 7f3024546640 1 -- 172.21.15.73:0/519619992 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f301c0d0510 con 0x7f301c162cd0 2024-04-26T20:50:44.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.797+0000 7f3012ffd640 1 -- 172.21.15.73:0/519619992 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f301407a6b0 con 0x7f301c162cd0 2024-04-26T20:50:44.798 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-26T20:50:44.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.798+0000 7f3024546640 1 -- 172.21.15.73:0/519619992 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f300006c8f0 msgr2=0x7f300006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:44.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.798+0000 7f3024546640 1 --2- 172.21.15.73:0/519619992 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f300006c8f0 0x7f300006edb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3018009b20 tx=0x7f30180039d0 comp rx=0 tx=0).stop 2024-04-26T20:50:44.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.798+0000 7f3024546640 1 -- 172.21.15.73:0/519619992 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f301c162cd0 msgr2=0x7f301c15a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:44.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.798+0000 7f3024546640 1 --2- 172.21.15.73:0/519619992 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f301c162cd0 0x7f301c15a140 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f30140098c0 tx=0x7f30140046c0 comp rx=0 tx=0).stop 2024-04-26T20:50:44.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.799+0000 7f3024546640 1 -- 172.21.15.73:0/519619992 shutdown_connections 2024-04-26T20:50:44.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.799+0000 7f3024546640 1 --2- 172.21.15.73:0/519619992 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f301c162cd0 0x7f301c15a140 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:44.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.799+0000 7f3024546640 1 --2- 172.21.15.73:0/519619992 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f300006c8f0 0x7f300006edb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:44.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.799+0000 7f3024546640 1 --2- 172.21.15.73:0/519619992 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f301c1628f0 0x7f301c06e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:44.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.799+0000 7f3024546640 1 --2- 172.21.15.73:0/519619992 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f301c0d1550 0x7f301c0ce7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:44.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.799+0000 7f3024546640 1 -- 172.21.15.73:0/519619992 >> 172.21.15.73:0/519619992 conn(0x7f301c0d4ff0 msgr2=0x7f301c0d53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:44.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.799+0000 7f3024546640 1 -- 172.21.15.73:0/519619992 shutdown_connections 2024-04-26T20:50:44.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.799+0000 7f3024546640 1 -- 172.21.15.73:0/519619992 wait complete. 2024-04-26T20:50:44.810 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:50:44.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.909+0000 7fae6bc08640 1 Processor -- start 2024-04-26T20:50:44.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.909+0000 7fae6bc08640 1 -- start start 2024-04-26T20:50:44.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.910+0000 7fae6bc08640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fae6415f170 0x7fae640d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:44.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.910+0000 7fae6bc08640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fae6415f550 0x7fae640d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:44.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.910+0000 7fae6bc08640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae640d1040 con 0x7fae640d34b0 2024-04-26T20:50:44.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.910+0000 7fae6bc08640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7fae640c9430 con 0x7fae6415f170 2024-04-26T20:50:44.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.910+0000 7fae6bc08640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fae640c9680 con 0x7fae6415f550 2024-04-26T20:50:44.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.910+0000 7fae699a4640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fae6415f170 0x7fae640d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:44.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.910+0000 7fae6a1a5640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fae640d34b0 0x7fae640cfc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:41002/0 (socket says 172.21.15.73:41002) 2024-04-26T20:50:44.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.910+0000 7fae6a1a5640 1 -- 172.21.15.73:0/1623496395 learned_addr learned my addr 172.21.15.73:0/1623496395 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:50:44.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.910+0000 7fae691a3640 1 --2- 172.21.15.73:0/1623496395 >> v2:172.21.15.120:3300/0 conn(0x7fae6415f550 0x7fae640d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:44.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.910+0000 7fae699a4640 1 -- 172.21.15.73:0/1623496395 >> v2:172.21.15.120:3300/0 conn(0x7fae6415f550 msgr2=0x7fae640d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:44.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.910+0000 7fae699a4640 1 --2- 172.21.15.73:0/1623496395 >> v2:172.21.15.120:3300/0 conn(0x7fae6415f550 0x7fae640d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:44.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.910+0000 7fae699a4640 1 -- 172.21.15.73:0/1623496395 >> v1:172.21.15.73:6789/0 conn(0x7fae640d34b0 legacy=0x7fae640cfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:44.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.910+0000 7fae689a2640 1 -- 172.21.15.73:0/1623496395 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3595646329 0 0) 0x7fae5c001110 con 0x7fae640d34b0 2024-04-26T20:50:44.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.910+0000 7fae699a4640 1 -- 172.21.15.73:0/1623496395 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae640c9960 con 0x7fae6415f170 2024-04-26T20:50:44.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.911+0000 7fae699a4640 1 --2- 172.21.15.73:0/1623496395 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae6415f170 0x7fae640d5a80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fae60009d90 tx=0x7fae6002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94aebfbee6a48e44 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:44.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.911+0000 7fae689a2640 1 -- 172.21.15.73:0/1623496395 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae60007740 con 0x7fae6415f170 2024-04-26T20:50:44.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.911+0000 7fae689a2640 1 -- 172.21.15.73:0/1623496395 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae600078e0 con 0x7fae6415f170 2024-04-26T20:50:44.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.911+0000 7fae689a2640 1 -- 172.21.15.73:0/1623496395 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae60007cb0 con 0x7fae6415f170 2024-04-26T20:50:44.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.912+0000 7fae6bc08640 1 -- 172.21.15.73:0/1623496395 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae6415f170 msgr2=0x7fae640d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:44.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.912+0000 7fae6bc08640 1 --2- 172.21.15.73:0/1623496395 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae6415f170 0x7fae640d5a80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fae60009d90 tx=0x7fae6002a6a0 comp rx=0 tx=0).stop 2024-04-26T20:50:44.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.912+0000 7fae6bc08640 1 -- 172.21.15.73:0/1623496395 shutdown_connections 2024-04-26T20:50:44.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.912+0000 7fae6bc08640 1 --2- 172.21.15.73:0/1623496395 >> v2:172.21.15.120:3300/0 conn(0x7fae6415f550 0x7fae640d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:44.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.912+0000 7fae6bc08640 1 --2- 172.21.15.73:0/1623496395 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae6415f170 0x7fae640d5a80 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:44.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.912+0000 7fae6bc08640 1 -- 172.21.15.73:0/1623496395 >> 172.21.15.73:0/1623496395 conn(0x7fae640d4f60 msgr2=0x7fae640d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:44.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.912+0000 7fae6bc08640 1 -- 172.21.15.73:0/1623496395 shutdown_connections 2024-04-26T20:50:44.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.912+0000 7fae6bc08640 1 -- 172.21.15.73:0/1623496395 wait complete. 2024-04-26T20:50:44.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.913+0000 7fae6bc08640 1 Processor -- start 2024-04-26T20:50:44.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.913+0000 7fae6bc08640 1 -- start start 2024-04-26T20:50:44.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.914+0000 7fae6bc08640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fae640d34b0 0x7fae640ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:44.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.914+0000 7fae6bc08640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae6415f170 0x7fae64152340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:44.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.914+0000 7fae6bc08640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae640ce800 0x7fae640b9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:44.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.914+0000 7fae6bc08640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fae640cb5d0 con 0x7fae640ce800 2024-04-26T20:50:44.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.914+0000 7fae6bc08640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fae640cb450 con 0x7fae6415f170 2024-04-26T20:50:44.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.914+0000 7fae6bc08640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fae640cb750 con 0x7fae640d34b0 2024-04-26T20:50:44.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.914+0000 7fae6a1a5640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae640ce800 0x7fae640b9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:44.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.914+0000 7fae6a1a5640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae640ce800 0x7fae640b9b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35004/0 (socket says 172.21.15.73:35004) 2024-04-26T20:50:44.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.914+0000 7fae6a1a5640 1 -- 172.21.15.73:0/2346808787 learned_addr learned my addr 172.21.15.73:0/2346808787 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:44.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.914+0000 7fae699a4640 1 --2- 172.21.15.73:0/2346808787 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fae640d34b0 0x7fae640ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:44.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.914+0000 7fae691a3640 1 --2- 172.21.15.73:0/2346808787 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae6415f170 0x7fae64152340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:44.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.914+0000 7fae6a1a5640 1 -- 172.21.15.73:0/2346808787 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fae640d34b0 msgr2=0x7fae640ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:44.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.914+0000 7fae6a1a5640 1 --2- 172.21.15.73:0/2346808787 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fae640d34b0 0x7fae640ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:44.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.914+0000 7fae6a1a5640 1 -- 172.21.15.73:0/2346808787 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae6415f170 msgr2=0x7fae64152340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:44.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.914+0000 7fae6a1a5640 1 --2- 172.21.15.73:0/2346808787 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae6415f170 0x7fae64152340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:44.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.914+0000 7fae6a1a5640 1 -- 172.21.15.73:0/2346808787 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae640cd760 con 0x7fae640ce800 2024-04-26T20:50:44.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.914+0000 7fae6a1a5640 1 --2- 172.21.15.73:0/2346808787 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae640ce800 0x7fae640b9b90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fae5c00c7b0 tx=0x7fae5c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:44.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.914+0000 7fae5a7fc640 1 -- 172.21.15.73:0/2346808787 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5c018070 con 0x7fae640ce800 2024-04-26T20:50:44.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.914+0000 7fae5a7fc640 1 -- 172.21.15.73:0/2346808787 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae5c0070d0 con 0x7fae640ce800 2024-04-26T20:50:44.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.914+0000 7fae6bc08640 1 -- 172.21.15.73:0/2346808787 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae640cda40 con 0x7fae640ce800 2024-04-26T20:50:44.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.914+0000 7fae5a7fc640 1 -- 172.21.15.73:0/2346808787 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5c0074c0 con 0x7fae640ce800 2024-04-26T20:50:44.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.914+0000 7fae6bc08640 1 -- 172.21.15.73:0/2346808787 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae640ce000 con 0x7fae640ce800 2024-04-26T20:50:44.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.915+0000 7fae6bc08640 1 -- 172.21.15.73:0/2346808787 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae640694f0 con 0x7fae640ce800 2024-04-26T20:50:44.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.916+0000 7fae5a7fc640 1 -- 172.21.15.73:0/2346808787 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fae5c01b070 con 0x7fae640ce800 2024-04-26T20:50:44.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.916+0000 7fae5a7fc640 1 --2- 172.21.15.73:0/2346808787 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fae4806c8f0 0x7fae4806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:44.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.916+0000 7fae699a4640 1 --2- 172.21.15.73:0/2346808787 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fae4806c8f0 0x7fae4806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:44.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.916+0000 7fae5a7fc640 1 -- 172.21.15.73:0/2346808787 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fae5c08b180 con 0x7fae640ce800 2024-04-26T20:50:44.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.917+0000 7fae699a4640 1 --2- 172.21.15.73:0/2346808787 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fae4806c8f0 0x7fae4806edb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fae60034000 tx=0x7fae6002f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:44.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:44.920+0000 7fae5a7fc640 1 -- 172.21.15.73:0/2346808787 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fae640d1040 con 0x7fae640ce800 2024-04-26T20:50:45.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.097+0000 7fae6bc08640 1 -- 172.21.15.73:0/2346808787 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fae64152880 con 0x7fae640ce800 2024-04-26T20:50:45.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.098+0000 7fae5a7fc640 1 -- 172.21.15.73:0/2346808787 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7fae5c059270 con 0x7fae640ce800 2024-04-26T20:50:45.099 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-26T20:50:45.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.099+0000 7fae6bc08640 1 -- 172.21.15.73:0/2346808787 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fae4806c8f0 msgr2=0x7fae4806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:45.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.099+0000 7fae6bc08640 1 --2- 172.21.15.73:0/2346808787 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fae4806c8f0 0x7fae4806edb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fae60034000 tx=0x7fae6002f000 comp rx=0 tx=0).stop 2024-04-26T20:50:45.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.100+0000 7fae6bc08640 1 -- 172.21.15.73:0/2346808787 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae640ce800 msgr2=0x7fae640b9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:45.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.100+0000 7fae6bc08640 1 --2- 172.21.15.73:0/2346808787 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae640ce800 0x7fae640b9b90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fae5c00c7b0 tx=0x7fae5c00cc70 comp rx=0 tx=0).stop 2024-04-26T20:50:45.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.100+0000 7fae6bc08640 1 -- 172.21.15.73:0/2346808787 shutdown_connections 2024-04-26T20:50:45.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.100+0000 7fae6bc08640 1 --2- 172.21.15.73:0/2346808787 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae640ce800 0x7fae640b9b90 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:45.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.100+0000 7fae6bc08640 1 --2- 172.21.15.73:0/2346808787 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fae4806c8f0 0x7fae4806edb0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:45.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.100+0000 7fae6bc08640 1 --2- 172.21.15.73:0/2346808787 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fae6415f170 0x7fae64152340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:45.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.100+0000 7fae6bc08640 1 --2- 172.21.15.73:0/2346808787 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fae640d34b0 0x7fae640ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:45.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.100+0000 7fae6bc08640 1 -- 172.21.15.73:0/2346808787 >> 172.21.15.73:0/2346808787 conn(0x7fae640d4f60 msgr2=0x7fae6406d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:45.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.100+0000 7fae6bc08640 1 -- 172.21.15.73:0/2346808787 shutdown_connections 2024-04-26T20:50:45.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.100+0000 7fae6bc08640 1 -- 172.21.15.73:0/2346808787 wait complete. 2024-04-26T20:50:45.111 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T20:50:45.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.210+0000 7f6a52a52640 1 Processor -- start 2024-04-26T20:50:45.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.210+0000 7f6a52a52640 1 -- start start 2024-04-26T20:50:45.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.211+0000 7f6a52a52640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6a4c06c950 0x7f6a4c0ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:45.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.211+0000 7f6a52a52640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a4c0cfd80 con 0x7f6a4c0b10b0 2024-04-26T20:50:45.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.211+0000 7f6a52a52640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a4c0d0020 con 0x7f6a4c0b0380 2024-04-26T20:50:45.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.211+0000 7f6a52a52640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f6a4c0af8e0 con 0x7f6a4c06c950 2024-04-26T20:50:45.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.211+0000 7f6a4b7fe640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f6a4c0b10b0 0x7f6a4c15e970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:41010/0 (socket says 172.21.15.73:41010) 2024-04-26T20:50:45.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.211+0000 7f6a4b7fe640 1 -- 172.21.15.73:0/2039062415 learned_addr learned my addr 172.21.15.73:0/2039062415 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:50:45.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.211+0000 7f6a50fef640 1 --2- 172.21.15.73:0/2039062415 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6a4c06c950 0x7f6a4c0ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:45.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.211+0000 7f6a4affd640 1 -- 172.21.15.73:0/2039062415 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2194408853 0 0) 0x7f6a4c0cfd80 con 0x7f6a4c0b10b0 2024-04-26T20:50:45.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.211+0000 7f6a4affd640 1 -- 172.21.15.73:0/2039062415 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a34003520 con 0x7f6a4c0b10b0 2024-04-26T20:50:45.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.211+0000 7f6a4affd640 1 -- 172.21.15.73:0/2039062415 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2386624277 0 0) 0x7f6a4c0d0020 con 0x7f6a4c0b0380 2024-04-26T20:50:45.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.211+0000 7f6a4affd640 1 -- 172.21.15.73:0/2039062415 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a4c0cfd80 con 0x7f6a4c0b0380 2024-04-26T20:50:45.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.212+0000 7f6a4affd640 1 -- 172.21.15.73:0/2039062415 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2013924037 0 0) 0x7f6a34003520 con 0x7f6a4c0b10b0 2024-04-26T20:50:45.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.212+0000 7f6a4affd640 1 -- 172.21.15.73:0/2039062415 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6a4c06c950 msgr2=0x7f6a4c0ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:45.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.212+0000 7f6a4affd640 1 --2- 172.21.15.73:0/2039062415 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6a4c06c950 0x7f6a4c0ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:45.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.212+0000 7f6a4affd640 1 -- 172.21.15.73:0/2039062415 >> v1:172.21.15.120:6789/0 conn(0x7f6a4c0b0380 legacy=0x7f6a4c06c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:45.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.212+0000 7f6a4affd640 1 -- 172.21.15.73:0/2039062415 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a4c0d0300 con 0x7f6a4c0b10b0 2024-04-26T20:50:45.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.212+0000 7f6a4affd640 1 -- 172.21.15.73:0/2039062415 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7f6a40001690 con 0x7f6a4c0b10b0 2024-04-26T20:50:45.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.212+0000 7f6a4affd640 1 -- 172.21.15.73:0/2039062415 >> v1:172.21.15.73:6789/0 conn(0x7f6a4c0b10b0 legacy=0x7f6a4c15e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:45.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.212+0000 7f6a4affd640 1 --2- 172.21.15.73:0/2039062415 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6a34003b70 0x7f6a34003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:45.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.212+0000 7f6a4affd640 1 --2- 172.21.15.73:0/2039062415 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a34004800 0x7f6a34004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:45.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.212+0000 7f6a4affd640 1 --2- 172.21.15.73:0/2039062415 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6a4c06c950 0x7f6a34005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:45.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.212+0000 7f6a4affd640 1 -- 172.21.15.73:0/2039062415 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6a4c0af8e0 con 0x7f6a34004800 2024-04-26T20:50:45.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.212+0000 7f6a4affd640 1 -- 172.21.15.73:0/2039062415 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3400bb50 con 0x7f6a34003b70 2024-04-26T20:50:45.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.212+0000 7f6a4affd640 1 -- 172.21.15.73:0/2039062415 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f6a3400bd50 con 0x7f6a4c06c950 2024-04-26T20:50:45.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.212+0000 7f6a4b7fe640 1 --2- 172.21.15.73:0/2039062415 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a34004800 0x7f6a34004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:45.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.212+0000 7f6a50fef640 1 --2- 172.21.15.73:0/2039062415 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6a4c06c950 0x7f6a34005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:45.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.212+0000 7f6a4bfff640 1 --2- 172.21.15.73:0/2039062415 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6a34003b70 0x7f6a34003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:45.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.212+0000 7f6a4b7fe640 1 -- 172.21.15.73:0/2039062415 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6a4c06c950 msgr2=0x7f6a34005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:45.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.212+0000 7f6a4b7fe640 1 --2- 172.21.15.73:0/2039062415 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6a4c06c950 0x7f6a34005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:45.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.212+0000 7f6a4b7fe640 1 -- 172.21.15.73:0/2039062415 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6a34003b70 msgr2=0x7f6a34003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:45.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.212+0000 7f6a4b7fe640 1 --2- 172.21.15.73:0/2039062415 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6a34003b70 0x7f6a34003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:45.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.212+0000 7f6a4b7fe640 1 -- 172.21.15.73:0/2039062415 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a34003520 con 0x7f6a34004800 2024-04-26T20:50:45.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.213+0000 7f6a4b7fe640 1 --2- 172.21.15.73:0/2039062415 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a34004800 0x7f6a34004c80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6a40007fb0 tx=0x7f6a34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d18355e5c4eeede2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:45.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.213+0000 7f6a4affd640 1 -- 172.21.15.73:0/2039062415 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a40007c00 con 0x7f6a34004800 2024-04-26T20:50:45.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.213+0000 7f6a4affd640 1 -- 172.21.15.73:0/2039062415 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a40005ad0 con 0x7f6a34004800 2024-04-26T20:50:45.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.213+0000 7f6a4affd640 1 -- 172.21.15.73:0/2039062415 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a40007070 con 0x7f6a34004800 2024-04-26T20:50:45.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.213+0000 7f6a52a52640 1 -- 172.21.15.73:0/2039062415 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a34004800 msgr2=0x7f6a34004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:45.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.213+0000 7f6a52a52640 1 --2- 172.21.15.73:0/2039062415 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a34004800 0x7f6a34004c80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6a40007fb0 tx=0x7f6a34005830 comp rx=0 tx=0).stop 2024-04-26T20:50:45.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.213+0000 7f6a50fef640 1 -- 172.21.15.73:0/2039062415 reap_dead start 2024-04-26T20:50:45.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.213+0000 7f6a52a52640 1 -- 172.21.15.73:0/2039062415 shutdown_connections 2024-04-26T20:50:45.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.213+0000 7f6a52a52640 1 -- 172.21.15.73:0/2039062415 >> 172.21.15.73:0/2039062415 conn(0x7f6a4c0d1cf0 msgr2=0x7f6a4c0d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:45.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.213+0000 7f6a52a52640 1 -- 172.21.15.73:0/2039062415 shutdown_connections 2024-04-26T20:50:45.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.213+0000 7f6a52a52640 1 -- 172.21.15.73:0/2039062415 wait complete. 2024-04-26T20:50:45.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.215+0000 7f6a52a52640 1 Processor -- start 2024-04-26T20:50:45.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.215+0000 7f6a52a52640 1 -- start start 2024-04-26T20:50:45.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.215+0000 7f6a52a52640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a34004800 0x7f6a4c15d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:45.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.215+0000 7f6a52a52640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6a4c0ca350 0x7f6a4c0ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:45.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.215+0000 7f6a52a52640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6a4c0b2b50 0x7f6a4c0b2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:45.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.216+0000 7f6a52a52640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6a4c0b0760 con 0x7f6a34004800 2024-04-26T20:50:45.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.216+0000 7f6a52a52640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6a4c0d1980 con 0x7f6a4c0b2b50 2024-04-26T20:50:45.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.216+0000 7f6a52a52640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f6a4c0d15c0 con 0x7f6a4c0ca350 2024-04-26T20:50:45.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.216+0000 7f6a4bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a34004800 0x7f6a4c15d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:45.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.216+0000 7f6a4bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a34004800 0x7f6a4c15d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35022/0 (socket says 172.21.15.73:35022) 2024-04-26T20:50:45.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.216+0000 7f6a4bfff640 1 -- 172.21.15.73:0/3758523759 learned_addr learned my addr 172.21.15.73:0/3758523759 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:45.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.216+0000 7f6a4b7fe640 1 --2- 172.21.15.73:0/3758523759 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6a4c0ca350 0x7f6a4c0ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:45.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.216+0000 7f6a50fef640 1 --2- 172.21.15.73:0/3758523759 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6a4c0b2b50 0x7f6a4c0b2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:45.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.216+0000 7f6a4bfff640 1 -- 172.21.15.73:0/3758523759 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6a4c0ca350 msgr2=0x7f6a4c0ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:45.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.216+0000 7f6a4bfff640 1 --2- 172.21.15.73:0/3758523759 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6a4c0ca350 0x7f6a4c0ca7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:45.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.216+0000 7f6a4bfff640 1 -- 172.21.15.73:0/3758523759 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6a4c0b2b50 msgr2=0x7f6a4c0b2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:45.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.216+0000 7f6a4bfff640 1 --2- 172.21.15.73:0/3758523759 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6a4c0b2b50 0x7f6a4c0b2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:45.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.216+0000 7f6a4bfff640 1 -- 172.21.15.73:0/3758523759 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a4c0cada0 con 0x7f6a34004800 2024-04-26T20:50:45.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.216+0000 7f6a4b7fe640 1 --2- 172.21.15.73:0/3758523759 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6a4c0ca350 0x7f6a4c0ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:50:45.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.216+0000 7f6a50fef640 1 --2- 172.21.15.73:0/3758523759 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6a4c0b2b50 0x7f6a4c0b2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:50:45.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.216+0000 7f6a4bfff640 1 --2- 172.21.15.73:0/3758523759 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a34004800 0x7f6a4c15d290 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c00f7a0 tx=0x7f6a3c00fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:45.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.216+0000 7f6a48ff9640 1 -- 172.21.15.73:0/3758523759 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c01a070 con 0x7f6a34004800 2024-04-26T20:50:45.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.216+0000 7f6a48ff9640 1 -- 172.21.15.73:0/3758523759 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a3c00fe50 con 0x7f6a34004800 2024-04-26T20:50:45.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.216+0000 7f6a52a52640 1 -- 172.21.15.73:0/3758523759 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a4c0cb080 con 0x7f6a34004800 2024-04-26T20:50:45.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.216+0000 7f6a48ff9640 1 -- 172.21.15.73:0/3758523759 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c007050 con 0x7f6a34004800 2024-04-26T20:50:45.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.216+0000 7f6a52a52640 1 -- 172.21.15.73:0/3758523759 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a4c0cb640 con 0x7f6a34004800 2024-04-26T20:50:45.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.217+0000 7f6a52a52640 1 -- 172.21.15.73:0/3758523759 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a4c0694f0 con 0x7f6a34004800 2024-04-26T20:50:45.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.218+0000 7f6a48ff9640 1 -- 172.21.15.73:0/3758523759 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6a3c01d070 con 0x7f6a34004800 2024-04-26T20:50:45.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.218+0000 7f6a48ff9640 1 --2- 172.21.15.73:0/3758523759 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f6a3806c8f0 0x7f6a3806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:45.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.218+0000 7f6a48ff9640 1 -- 172.21.15.73:0/3758523759 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f6a3c08b730 con 0x7f6a34004800 2024-04-26T20:50:45.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.218+0000 7f6a4b7fe640 1 --2- 172.21.15.73:0/3758523759 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f6a3806c8f0 0x7f6a3806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:45.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.219+0000 7f6a4b7fe640 1 --2- 172.21.15.73:0/3758523759 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f6a3806c8f0 0x7f6a3806edb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6a40007fb0 tx=0x7f6a40009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:45.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.222+0000 7f6a48ff9640 1 -- 172.21.15.73:0/3758523759 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6a3c059820 con 0x7f6a34004800 2024-04-26T20:50:45.399 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.399+0000 7f6a52a52640 1 -- 172.21.15.73:0/3758523759 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f6a4c0cf100 con 0x7f6a34004800 2024-04-26T20:50:45.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.399+0000 7f6a48ff9640 1 -- 172.21.15.73:0/3758523759 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7f6a3c059120 con 0x7f6a34004800 2024-04-26T20:50:45.400 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-26T20:50:45.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.401+0000 7f6a52a52640 1 -- 172.21.15.73:0/3758523759 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f6a3806c8f0 msgr2=0x7f6a3806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:45.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.401+0000 7f6a52a52640 1 --2- 172.21.15.73:0/3758523759 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f6a3806c8f0 0x7f6a3806edb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6a40007fb0 tx=0x7f6a40009030 comp rx=0 tx=0).stop 2024-04-26T20:50:45.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.401+0000 7f6a52a52640 1 -- 172.21.15.73:0/3758523759 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a34004800 msgr2=0x7f6a4c15d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:45.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.401+0000 7f6a52a52640 1 --2- 172.21.15.73:0/3758523759 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a34004800 0x7f6a4c15d290 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c00f7a0 tx=0x7f6a3c00fc60 comp rx=0 tx=0).stop 2024-04-26T20:50:45.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.401+0000 7f6a52a52640 1 -- 172.21.15.73:0/3758523759 shutdown_connections 2024-04-26T20:50:45.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.401+0000 7f6a52a52640 1 --2- 172.21.15.73:0/3758523759 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f6a3806c8f0 0x7f6a3806edb0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:45.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.401+0000 7f6a52a52640 1 --2- 172.21.15.73:0/3758523759 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6a4c0b2b50 0x7f6a4c0b2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:45.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.401+0000 7f6a52a52640 1 --2- 172.21.15.73:0/3758523759 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6a4c0ca350 0x7f6a4c0ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:45.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.401+0000 7f6a52a52640 1 --2- 172.21.15.73:0/3758523759 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a34004800 0x7f6a4c15d290 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:45.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.401+0000 7f6a52a52640 1 -- 172.21.15.73:0/3758523759 >> 172.21.15.73:0/3758523759 conn(0x7f6a4c0d1cf0 msgr2=0x7f6a4c072f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:45.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.401+0000 7f6a52a52640 1 -- 172.21.15.73:0/3758523759 shutdown_connections 2024-04-26T20:50:45.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.401+0000 7f6a52a52640 1 -- 172.21.15.73:0/3758523759 wait complete. 2024-04-26T20:50:45.413 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T20:50:45.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.509+0000 7f7c74989640 1 Processor -- start 2024-04-26T20:50:45.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.509+0000 7f7c74989640 1 -- start start 2024-04-26T20:50:45.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.510+0000 7f7c74989640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7c70162810 0x7f7c70070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:45.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.510+0000 7f7c74989640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f7c700b2340 0x7f7c700cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:45.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.510+0000 7f7c74989640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c700d10d0 con 0x7f7c700cd930 2024-04-26T20:50:45.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.510+0000 7f7c74989640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f7c700d0750 con 0x7f7c70162810 2024-04-26T20:50:45.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.510+0000 7f7c74989640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f7c700d16a0 con 0x7f7c700b2340 2024-04-26T20:50:45.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.510+0000 7f7c6e59c640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7c70162810 0x7f7c70070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:45.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.511+0000 7f7c6ed9d640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f7c700cd930 0x7f7c700d1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:41020/0 (socket says 172.21.15.73:41020) 2024-04-26T20:50:45.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.511+0000 7f7c6ed9d640 1 -- 172.21.15.73:0/2951978089 learned_addr learned my addr 172.21.15.73:0/2951978089 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:50:45.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.511+0000 7f7c6dd9b640 1 --2- 172.21.15.73:0/2951978089 >> v2:172.21.15.120:3300/0 conn(0x7f7c700b2340 0x7f7c700cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:45.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.511+0000 7f7c6e59c640 1 -- 172.21.15.73:0/2951978089 >> v2:172.21.15.120:3300/0 conn(0x7f7c700b2340 msgr2=0x7f7c700cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:45.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.511+0000 7f7c6e59c640 1 --2- 172.21.15.73:0/2951978089 >> v2:172.21.15.120:3300/0 conn(0x7f7c700b2340 0x7f7c700cd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:45.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.511+0000 7f7c6e59c640 1 -- 172.21.15.73:0/2951978089 >> v1:172.21.15.73:6789/0 conn(0x7f7c700cd930 legacy=0x7f7c700d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:45.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.511+0000 7f7c6e59c640 1 -- 172.21.15.73:0/2951978089 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c700d1370 con 0x7f7c70162810 2024-04-26T20:50:45.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.511+0000 7f7c6d59a640 1 -- 172.21.15.73:0/2951978089 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4039223720 0 0) 0x7f7c700d10d0 con 0x7f7c700cd930 2024-04-26T20:50:45.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.511+0000 7f7c6e59c640 1 --2- 172.21.15.73:0/2951978089 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c70162810 0x7f7c70070350 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7c700cd0a0 tx=0x7f7c6802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad9fa9be64064bda server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:45.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.512+0000 7f7c6d59a640 1 -- 172.21.15.73:0/2951978089 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c68037070 con 0x7f7c70162810 2024-04-26T20:50:45.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.512+0000 7f7c6d59a640 1 -- 172.21.15.73:0/2951978089 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c68032440 con 0x7f7c70162810 2024-04-26T20:50:45.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.512+0000 7f7c6d59a640 1 -- 172.21.15.73:0/2951978089 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c68032810 con 0x7f7c70162810 2024-04-26T20:50:45.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.512+0000 7f7c74989640 1 -- 172.21.15.73:0/2951978089 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c70162810 msgr2=0x7f7c70070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:45.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.512+0000 7f7c74989640 1 --2- 172.21.15.73:0/2951978089 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c70162810 0x7f7c70070350 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7c700cd0a0 tx=0x7f7c6802a1a0 comp rx=0 tx=0).stop 2024-04-26T20:50:45.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.512+0000 7f7c74989640 1 -- 172.21.15.73:0/2951978089 shutdown_connections 2024-04-26T20:50:45.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.512+0000 7f7c74989640 1 --2- 172.21.15.73:0/2951978089 >> v2:172.21.15.120:3300/0 conn(0x7f7c700b2340 0x7f7c700cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:45.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.512+0000 7f7c74989640 1 --2- 172.21.15.73:0/2951978089 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c70162810 0x7f7c70070350 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:45.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.512+0000 7f7c74989640 1 -- 172.21.15.73:0/2951978089 >> 172.21.15.73:0/2951978089 conn(0x7f7c700d2da0 msgr2=0x7f7c700d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:45.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.512+0000 7f7c74989640 1 -- 172.21.15.73:0/2951978089 shutdown_connections 2024-04-26T20:50:45.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.512+0000 7f7c74989640 1 -- 172.21.15.73:0/2951978089 wait complete. 2024-04-26T20:50:45.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.513+0000 7f7c74989640 1 Processor -- start 2024-04-26T20:50:45.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.514+0000 7f7c74989640 1 -- start start 2024-04-26T20:50:45.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.514+0000 7f7c74989640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7c700b2340 0x7f7c7015c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:45.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.514+0000 7f7c74989640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c700cd930 0x7f7c7015e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:45.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.514+0000 7f7c74989640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c70162810 0x7f7c700ca850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:45.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.514+0000 7f7c74989640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7c70070890 con 0x7f7c700cd930 2024-04-26T20:50:45.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.514+0000 7f7c74989640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7c700d0470 con 0x7f7c70162810 2024-04-26T20:50:45.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.514+0000 7f7c74989640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f7c700d2240 con 0x7f7c700b2340 2024-04-26T20:50:45.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.514+0000 7f7c6dd9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c700cd930 0x7f7c7015e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:45.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.514+0000 7f7c6dd9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c700cd930 0x7f7c7015e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35054/0 (socket says 172.21.15.73:35054) 2024-04-26T20:50:45.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.514+0000 7f7c6dd9b640 1 -- 172.21.15.73:0/1160312967 learned_addr learned my addr 172.21.15.73:0/1160312967 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:45.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.515+0000 7f7c6dd9b640 1 -- 172.21.15.73:0/1160312967 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7c700b2340 msgr2=0x7f7c7015c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:45.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.515+0000 7f7c6dd9b640 1 --2- 172.21.15.73:0/1160312967 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7c700b2340 0x7f7c7015c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:45.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.515+0000 7f7c6ed9d640 1 --2- 172.21.15.73:0/1160312967 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c70162810 0x7f7c700ca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:45.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.515+0000 7f7c6dd9b640 1 -- 172.21.15.73:0/1160312967 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c70162810 msgr2=0x7f7c700ca850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:45.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.515+0000 7f7c6dd9b640 1 --2- 172.21.15.73:0/1160312967 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c70162810 0x7f7c700ca850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:45.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.515+0000 7f7c6dd9b640 1 -- 172.21.15.73:0/1160312967 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c700cba10 con 0x7f7c700cd930 2024-04-26T20:50:45.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.515+0000 7f7c6dd9b640 1 --2- 172.21.15.73:0/1160312967 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c700cd930 0x7f7c7015e580 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7c6400ce10 tx=0x7f7c6400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:45.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.515+0000 7f7c5effd640 1 -- 172.21.15.73:0/1160312967 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c64003b90 con 0x7f7c700cd930 2024-04-26T20:50:45.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.515+0000 7f7c5effd640 1 -- 172.21.15.73:0/1160312967 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c64003d30 con 0x7f7c700cd930 2024-04-26T20:50:45.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.515+0000 7f7c74989640 1 -- 172.21.15.73:0/1160312967 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c700c6b20 con 0x7f7c700cd930 2024-04-26T20:50:45.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.515+0000 7f7c5effd640 1 -- 172.21.15.73:0/1160312967 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c64004240 con 0x7f7c700cd930 2024-04-26T20:50:45.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.515+0000 7f7c74989640 1 -- 172.21.15.73:0/1160312967 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c700c70e0 con 0x7f7c700cd930 2024-04-26T20:50:45.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.516+0000 7f7c74989640 1 -- 172.21.15.73:0/1160312967 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c700694f0 con 0x7f7c700cd930 2024-04-26T20:50:45.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.516+0000 7f7c5effd640 1 -- 172.21.15.73:0/1160312967 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7c640043e0 con 0x7f7c700cd930 2024-04-26T20:50:45.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.517+0000 7f7c5effd640 1 --2- 172.21.15.73:0/1160312967 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f7c5006c8f0 0x7f7c5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:45.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.517+0000 7f7c5effd640 1 -- 172.21.15.73:0/1160312967 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f7c6408c970 con 0x7f7c700cd930 2024-04-26T20:50:45.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.517+0000 7f7c6e59c640 1 --2- 172.21.15.73:0/1160312967 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f7c5006c8f0 0x7f7c5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:45.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.517+0000 7f7c6e59c640 1 --2- 172.21.15.73:0/1160312967 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f7c5006c8f0 0x7f7c5006edb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7c68034000 tx=0x7f7c6802c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:45.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.521+0000 7f7c5effd640 1 -- 172.21.15.73:0/1160312967 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7c6401d340 con 0x7f7c700cd930 2024-04-26T20:50:45.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.709+0000 7f7c74989640 1 -- 172.21.15.73:0/1160312967 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7c700ce0e0 con 0x7f7c700cd930 2024-04-26T20:50:45.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.709+0000 7f7c5effd640 1 -- 172.21.15.73:0/1160312967 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7f7c6405aa60 con 0x7f7c700cd930 2024-04-26T20:50:45.710 INFO:teuthology.orchestra.run.smithi073.stderr:listed 0 entries 2024-04-26T20:50:45.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.711+0000 7f7c74989640 1 -- 172.21.15.73:0/1160312967 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f7c5006c8f0 msgr2=0x7f7c5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:45.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.711+0000 7f7c74989640 1 --2- 172.21.15.73:0/1160312967 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f7c5006c8f0 0x7f7c5006edb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7c68034000 tx=0x7f7c6802c000 comp rx=0 tx=0).stop 2024-04-26T20:50:45.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.711+0000 7f7c74989640 1 -- 172.21.15.73:0/1160312967 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c700cd930 msgr2=0x7f7c7015e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:45.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.711+0000 7f7c74989640 1 --2- 172.21.15.73:0/1160312967 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c700cd930 0x7f7c7015e580 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7c6400ce10 tx=0x7f7c6400a660 comp rx=0 tx=0).stop 2024-04-26T20:50:45.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.711+0000 7f7c74989640 1 -- 172.21.15.73:0/1160312967 shutdown_connections 2024-04-26T20:50:45.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.711+0000 7f7c74989640 1 --2- 172.21.15.73:0/1160312967 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c700cd930 0x7f7c7015e580 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:45.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.711+0000 7f7c74989640 1 --2- 172.21.15.73:0/1160312967 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f7c5006c8f0 0x7f7c5006edb0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:45.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.711+0000 7f7c74989640 1 --2- 172.21.15.73:0/1160312967 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7c70162810 0x7f7c700ca850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:45.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.711+0000 7f7c74989640 1 --2- 172.21.15.73:0/1160312967 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7c700b2340 0x7f7c7015c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:45.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.711+0000 7f7c74989640 1 -- 172.21.15.73:0/1160312967 >> 172.21.15.73:0/1160312967 conn(0x7f7c700d2da0 msgr2=0x7f7c700ce600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:45.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.711+0000 7f7c74989640 1 -- 172.21.15.73:0/1160312967 shutdown_connections 2024-04-26T20:50:45.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:45.711+0000 7f7c74989640 1 -- 172.21.15.73:0/1160312967 wait complete. 2024-04-26T20:50:45.723 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi073.front.sepia.ceph.com... 2024-04-26T20:50:45.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:50:45.723 DEBUG:teuthology.orchestra.run.smithi073:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-26T20:50:45.745 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi073.stderr:ceph-fuse[41427]: fuse finished with error 0 and tester_r 0 2024-04-26T20:50:45.747 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:50:51.849 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi073.front.sepia.ceph.com 2024-04-26T20:50:51.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:50:51.849 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-26T20:50:51.887 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:50:52.027 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:50:52.029 DEBUG:teuthology.orchestra.run.smithi120:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:50:52.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.042+0000 7f97011a4640 1 Processor -- start 2024-04-26T20:50:52.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.042+0000 7f97011a4640 1 -- start start 2024-04-26T20:50:52.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.043+0000 7f97011a4640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f96fc160720 0x7f96fc0ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:52.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.043+0000 7f97011a4640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f96fc0d18f0 0x7f96fc0b2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:52.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.043+0000 7f97011a4640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96fc0d0710 con 0x7f96fc0ceb60 2024-04-26T20:50:52.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.043+0000 7f97011a4640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f96fc0d01e0 con 0x7f96fc160720 2024-04-26T20:50:52.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.043+0000 7f97011a4640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f96fc0d0430 con 0x7f96fc0d18f0 2024-04-26T20:50:52.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.043+0000 7f96fad9d640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f96fc160720 0x7f96fc0ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:52.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.043+0000 7f96fad9d640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f96fc160720 0x7f96fc0ce620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49248/0 (socket says 172.21.15.73:49248) 2024-04-26T20:50:52.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.043+0000 7f96fad9d640 1 -- 172.21.15.73:0/860119050 learned_addr learned my addr 172.21.15.73:0/860119050 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:52.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.043+0000 7f96fb59e640 1 --2- 172.21.15.73:0/860119050 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f96fc0d18f0 0x7f96fc0b2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:52.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.044+0000 7f96fad9d640 1 -- 172.21.15.73:0/860119050 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f96fc0d18f0 msgr2=0x7f96fc0b2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:52.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.044+0000 7f96fad9d640 1 --2- 172.21.15.73:0/860119050 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f96fc0d18f0 0x7f96fc0b2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:52.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.044+0000 7f96fad9d640 1 -- 172.21.15.73:0/860119050 >> v1:172.21.15.120:6789/0 conn(0x7f96fc0ceb60 legacy=0x7f96fc0d11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:52.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.044+0000 7f96fad9d640 1 -- 172.21.15.73:0/860119050 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96fc0cdb40 con 0x7f96fc160720 2024-04-26T20:50:52.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.044+0000 7f96fad9d640 1 --2- 172.21.15.73:0/860119050 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96fc160720 0x7f96fc0ce620 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f96f0009870 tx=0x7f96f002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3734261b2493d260 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:52.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.045+0000 7f96f9d9b640 1 -- 172.21.15.73:0/860119050 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f0037070 con 0x7f96fc160720 2024-04-26T20:50:52.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.045+0000 7f96f9d9b640 1 -- 172.21.15.73:0/860119050 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96f0032440 con 0x7f96fc160720 2024-04-26T20:50:52.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.045+0000 7f96f9d9b640 1 -- 172.21.15.73:0/860119050 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f0032810 con 0x7f96fc160720 2024-04-26T20:50:52.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.045+0000 7f97011a4640 1 -- 172.21.15.73:0/860119050 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96fc160720 msgr2=0x7f96fc0ce620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:52.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.045+0000 7f97011a4640 1 --2- 172.21.15.73:0/860119050 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96fc160720 0x7f96fc0ce620 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f96f0009870 tx=0x7f96f002a270 comp rx=0 tx=0).stop 2024-04-26T20:50:52.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.045+0000 7f97011a4640 1 -- 172.21.15.73:0/860119050 shutdown_connections 2024-04-26T20:50:52.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.045+0000 7f97011a4640 1 --2- 172.21.15.73:0/860119050 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f96fc0d18f0 0x7f96fc0b2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:52.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.045+0000 7f97011a4640 1 --2- 172.21.15.73:0/860119050 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96fc160720 0x7f96fc0ce620 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:52.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.045+0000 7f97011a4640 1 -- 172.21.15.73:0/860119050 >> 172.21.15.73:0/860119050 conn(0x7f96fc0d2e00 msgr2=0x7f96fc0d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:52.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.045+0000 7f97011a4640 1 -- 172.21.15.73:0/860119050 shutdown_connections 2024-04-26T20:50:52.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.045+0000 7f97011a4640 1 -- 172.21.15.73:0/860119050 wait complete. 2024-04-26T20:50:52.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.047+0000 7f97011a4640 1 Processor -- start 2024-04-26T20:50:52.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.047+0000 7f97011a4640 1 -- start start 2024-04-26T20:50:52.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.048+0000 7f97011a4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96fc0ceb60 0x7f96fc159720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:52.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.048+0000 7f97011a4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96fc0d18f0 0x7f96fc0d0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:52.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.048+0000 7f97011a4640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f96fc160720 0x7f96fc0d1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:52.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.048+0000 7f97011a4640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f96fc160b00 con 0x7f96fc0d18f0 2024-04-26T20:50:52.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.048+0000 7f97011a4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f96fc0ce100 con 0x7f96fc0ceb60 2024-04-26T20:50:52.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.048+0000 7f97011a4640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f96fc0cb210 con 0x7f96fc160720 2024-04-26T20:50:52.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.048+0000 7f96fa59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96fc0d18f0 0x7f96fc0d0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:52.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.048+0000 7f96fa59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96fc0d18f0 0x7f96fc0d0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49254/0 (socket says 172.21.15.73:49254) 2024-04-26T20:50:52.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.048+0000 7f96fa59c640 1 -- 172.21.15.73:0/4117748408 learned_addr learned my addr 172.21.15.73:0/4117748408 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:52.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.048+0000 7f96fad9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96fc0ceb60 0x7f96fc159720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:52.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.048+0000 7f96fb59e640 1 --2- 172.21.15.73:0/4117748408 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f96fc160720 0x7f96fc0d1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:52.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.048+0000 7f96fa59c640 1 -- 172.21.15.73:0/4117748408 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f96fc160720 msgr2=0x7f96fc0d1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:52.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.048+0000 7f96fa59c640 1 --2- 172.21.15.73:0/4117748408 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f96fc160720 0x7f96fc0d1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:52.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.048+0000 7f96fa59c640 1 -- 172.21.15.73:0/4117748408 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96fc0ceb60 msgr2=0x7f96fc159720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:52.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.048+0000 7f96fa59c640 1 --2- 172.21.15.73:0/4117748408 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96fc0ceb60 0x7f96fc159720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:52.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.048+0000 7f96fa59c640 1 -- 172.21.15.73:0/4117748408 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96fc15d7d0 con 0x7f96fc0d18f0 2024-04-26T20:50:52.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.049+0000 7f96fa59c640 1 --2- 172.21.15.73:0/4117748408 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96fc0d18f0 0x7f96fc0d0d00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f96ec00c440 tx=0x7f96ec001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:52.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.049+0000 7f96e37fe640 1 -- 172.21.15.73:0/4117748408 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ec0052d0 con 0x7f96fc0d18f0 2024-04-26T20:50:52.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.049+0000 7f96e37fe640 1 -- 172.21.15.73:0/4117748408 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96ec005470 con 0x7f96fc0d18f0 2024-04-26T20:50:52.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.049+0000 7f97011a4640 1 -- 172.21.15.73:0/4117748408 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96fc15dab0 con 0x7f96fc0d18f0 2024-04-26T20:50:52.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.049+0000 7f96e37fe640 1 -- 172.21.15.73:0/4117748408 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ec007260 con 0x7f96fc0d18f0 2024-04-26T20:50:52.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.049+0000 7f97011a4640 1 -- 172.21.15.73:0/4117748408 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96fc0cb790 con 0x7f96fc0d18f0 2024-04-26T20:50:52.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.050+0000 7f97011a4640 1 -- 172.21.15.73:0/4117748408 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96fc15dc80 con 0x7f96fc0d18f0 2024-04-26T20:50:52.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.051+0000 7f96e37fe640 1 -- 172.21.15.73:0/4117748408 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f96ec007400 con 0x7f96fc0d18f0 2024-04-26T20:50:52.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.051+0000 7f96e37fe640 1 --2- 172.21.15.73:0/4117748408 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f96e806c8f0 0x7f96e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:52.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.051+0000 7f96e37fe640 1 -- 172.21.15.73:0/4117748408 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f96ec08b330 con 0x7f96fc0d18f0 2024-04-26T20:50:52.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.051+0000 7f96fad9d640 1 --2- 172.21.15.73:0/4117748408 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f96e806c8f0 0x7f96e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:52.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.052+0000 7f96fad9d640 1 --2- 172.21.15.73:0/4117748408 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f96e806c8f0 0x7f96e806edb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f96f0034000 tx=0x7f96f002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:52.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.056+0000 7f96e37fe640 1 -- 172.21.15.73:0/4117748408 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f96ec058dd0 con 0x7f96fc0d18f0 2024-04-26T20:50:52.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.236+0000 7f97011a4640 1 -- 172.21.15.73:0/4117748408 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f96fc0694f0 con 0x7f96fc0d18f0 2024-04-26T20:50:52.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.237+0000 7f96e37fe640 1 -- 172.21.15.73:0/4117748408 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18514 (secure 0 0 0) 0x7f96ec05d0d0 con 0x7f96fc0d18f0 2024-04-26T20:50:52.238 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:50:52.238 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":25,"fsid":"fbd3f548-b547-4713-856b-09f98f5d1764","created":"2024-04-26T20:50:07.876786+0000","modified":"2024-04-26T20:50:38.657510+0000","last_up_change":"2024-04-26T20:50:19.082741+0000","last_in_change":"2024-04-26T20:50:10.018400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:50:19.993731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:50:29.964420+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:50:30.920082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:50:34.650207+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"72d6f4f3-214a-4392-a02f-c33cbcab3c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6809","nonce":3461705124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6811","nonce":3461705124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6815","nonce":3461705124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6813","nonce":3461705124}]},"public_addr":"172.21.15.73:6809/3461705124","cluster_addr":"172.21.15.73:6811/3461705124","heartbeat_back_addr":"172.21.15.73:6815/3461705124","heartbeat_front_addr":"172.21.15.73:6813/3461705124","state":["exists","up"]},{"osd":1,"uuid":"786bf8ca-6ddc-4b3c-be6c-9152398bd5fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6817","nonce":2827619403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6819","nonce":2827619403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6825","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6827","nonce":2827619403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6821","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6823","nonce":2827619403}]},"public_addr":"172.21.15.73:6817/2827619403","cluster_addr":"172.21.15.73:6819/2827619403","heartbeat_back_addr":"172.21.15.73:6827/2827619403","heartbeat_front_addr":"172.21.15.73:6823/2827619403","state":["exists","up"]},{"osd":2,"uuid":"90963b1a-5437-4ef8-ad8b-6b74c534eeb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6801","nonce":2542681064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6803","nonce":2542681064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6808","nonce":2542681064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6805","nonce":2542681064}]},"public_addr":"172.21.15.73:6801/2542681064","cluster_addr":"172.21.15.73:6803/2542681064","heartbeat_back_addr":"172.21.15.73:6808/2542681064","heartbeat_front_addr":"172.21.15.73:6805/2542681064","state":["exists","up"]},{"osd":3,"uuid":"bd3d2148-6e38-41b4-a453-fe6e3b8b4ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6822","nonce":1604042718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6826","nonce":1604042718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6831","nonce":1604042718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6829","nonce":1604042718}]},"public_addr":"172.21.15.73:6822/1604042718","cluster_addr":"172.21.15.73:6826/1604042718","heartbeat_back_addr":"172.21.15.73:6831/1604042718","heartbeat_front_addr":"172.21.15.73:6829/1604042718","state":["exists","up"]},{"osd":4,"uuid":"e8668dc8-a23f-46d2-8311-cb882da08e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6817","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6819","nonce":3591055561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6824","nonce":3591055561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6831","nonce":3591055561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6828","nonce":3591055561}]},"public_addr":"172.21.15.120:6819/3591055561","cluster_addr":"172.21.15.120:6824/3591055561","heartbeat_back_addr":"172.21.15.120:6831/3591055561","heartbeat_front_addr":"172.21.15.120:6828/3591055561","state":["exists","up"]},{"osd":5,"uuid":"cbac0bdb-96bd-4852-bcce-e564495c3fcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6809","nonce":819450433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6811","nonce":819450433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6815","nonce":819450433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6813","nonce":819450433}]},"public_addr":"172.21.15.120:6809/819450433","cluster_addr":"172.21.15.120:6811/819450433","heartbeat_back_addr":"172.21.15.120:6815/819450433","heartbeat_front_addr":"172.21.15.120:6813/819450433","state":["exists","up"]},{"osd":6,"uuid":"1884b2a8-569f-4272-96a9-f25a09364dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6801","nonce":3946210727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6803","nonce":3946210727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6807","nonce":3946210727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6805","nonce":3946210727}]},"public_addr":"172.21.15.120:6801/3946210727","cluster_addr":"172.21.15.120:6803/3946210727","heartbeat_back_addr":"172.21.15.120:6807/3946210727","heartbeat_front_addr":"172.21.15.120:6805/3946210727","state":["exists","up"]},{"osd":7,"uuid":"c0c3bf6d-a4c5-4f1e-aa2d-330951830666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6818","nonce":2915574674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6821","nonce":2915574674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6827","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6829","nonce":2915574674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6823","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6825","nonce":2915574674}]},"public_addr":"172.21.15.120:6818/2915574674","cluster_addr":"172.21.15.120:6821/2915574674","heartbeat_back_addr":"172.21.15.120:6829/2915574674","heartbeat_front_addr":"172.21.15.120:6825/2915574674","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.218216+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.208710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.112705+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.182789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.164324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.174212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.225396+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.261559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:50:52.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.239+0000 7f97011a4640 1 -- 172.21.15.73:0/4117748408 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f96e806c8f0 msgr2=0x7f96e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:52.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.239+0000 7f97011a4640 1 --2- 172.21.15.73:0/4117748408 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f96e806c8f0 0x7f96e806edb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f96f0034000 tx=0x7f96f002e000 comp rx=0 tx=0).stop 2024-04-26T20:50:52.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.239+0000 7f97011a4640 1 -- 172.21.15.73:0/4117748408 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96fc0d18f0 msgr2=0x7f96fc0d0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:52.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.239+0000 7f97011a4640 1 --2- 172.21.15.73:0/4117748408 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96fc0d18f0 0x7f96fc0d0d00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f96ec00c440 tx=0x7f96ec001e70 comp rx=0 tx=0).stop 2024-04-26T20:50:52.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.239+0000 7f97011a4640 1 -- 172.21.15.73:0/4117748408 shutdown_connections 2024-04-26T20:50:52.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.239+0000 7f97011a4640 1 --2- 172.21.15.73:0/4117748408 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96fc0d18f0 0x7f96fc0d0d00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:52.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.239+0000 7f97011a4640 1 --2- 172.21.15.73:0/4117748408 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f96e806c8f0 0x7f96e806edb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:52.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.239+0000 7f97011a4640 1 --2- 172.21.15.73:0/4117748408 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f96fc160720 0x7f96fc0d1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:52.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.239+0000 7f97011a4640 1 --2- 172.21.15.73:0/4117748408 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96fc0ceb60 0x7f96fc159720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:52.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.239+0000 7f97011a4640 1 -- 172.21.15.73:0/4117748408 >> 172.21.15.73:0/4117748408 conn(0x7f96fc0d2e00 msgr2=0x7f96fc0b5f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:52.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.239+0000 7f97011a4640 1 -- 172.21.15.73:0/4117748408 shutdown_connections 2024-04-26T20:50:52.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.239+0000 7f97011a4640 1 -- 172.21.15.73:0/4117748408 wait complete. 2024-04-26T20:50:52.251 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:50:52.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.348+0000 7fc1847bd640 1 Processor -- start 2024-04-26T20:50:52.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.348+0000 7fc1847bd640 1 -- start start 2024-04-26T20:50:52.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.349+0000 7fc1847bd640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc17c0d38f0 0x7fc17c0ce890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:52.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.349+0000 7fc1847bd640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc17c0d09a0 0x7fc17c0d1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:52.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.349+0000 7fc1847bd640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc17c0cdb10 con 0x7fc17c1606f0 2024-04-26T20:50:52.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.349+0000 7fc1847bd640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7fc17c0d1d40 con 0x7fc17c0d09a0 2024-04-26T20:50:52.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.349+0000 7fc1847bd640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fc17c0d0e30 con 0x7fc17c0d38f0 2024-04-26T20:50:52.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.349+0000 7fc182559640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fc17c1606f0 0x7fc17c0d10f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33028/0 (socket says 172.21.15.73:33028) 2024-04-26T20:50:52.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.349+0000 7fc182559640 1 -- 172.21.15.73:0/2684891552 learned_addr learned my addr 172.21.15.73:0/2684891552 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:50:52.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.349+0000 7fc182d5a640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc17c0d09a0 0x7fc17c0d1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:52.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.349+0000 7fc181d58640 1 --2- 172.21.15.73:0/2684891552 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc17c0d38f0 0x7fc17c0ce890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:52.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.349+0000 7fc181557640 1 -- 172.21.15.73:0/2684891552 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3863780269 0 0) 0x7fc17c0cdb10 con 0x7fc17c1606f0 2024-04-26T20:50:52.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.349+0000 7fc181557640 1 -- 172.21.15.73:0/2684891552 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc168003520 con 0x7fc17c1606f0 2024-04-26T20:50:52.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.349+0000 7fc182d5a640 1 -- 172.21.15.73:0/2684891552 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc17c0d38f0 msgr2=0x7fc17c0ce890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:52.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.349+0000 7fc182d5a640 1 --2- 172.21.15.73:0/2684891552 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc17c0d38f0 0x7fc17c0ce890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:52.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.349+0000 7fc182d5a640 1 -- 172.21.15.73:0/2684891552 >> v1:172.21.15.73:6789/0 conn(0x7fc17c1606f0 legacy=0x7fc17c0d10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:52.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.349+0000 7fc182d5a640 1 -- 172.21.15.73:0/2684891552 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc17c0cddb0 con 0x7fc17c0d09a0 2024-04-26T20:50:52.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.350+0000 7fc182d5a640 1 --2- 172.21.15.73:0/2684891552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc17c0d09a0 0x7fc17c0d1800 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc174009dc0 tx=0x7fc17402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef5afacf3ea8347d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:52.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.350+0000 7fc181557640 1 -- 172.21.15.73:0/2684891552 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc174007740 con 0x7fc17c0d09a0 2024-04-26T20:50:52.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.350+0000 7fc181557640 1 -- 172.21.15.73:0/2684891552 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1740078e0 con 0x7fc17c0d09a0 2024-04-26T20:50:52.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.350+0000 7fc181557640 1 -- 172.21.15.73:0/2684891552 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc174007c90 con 0x7fc17c0d09a0 2024-04-26T20:50:52.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.351+0000 7fc1847bd640 1 -- 172.21.15.73:0/2684891552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc17c0d09a0 msgr2=0x7fc17c0d1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:52.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.351+0000 7fc1847bd640 1 --2- 172.21.15.73:0/2684891552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc17c0d09a0 0x7fc17c0d1800 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc174009dc0 tx=0x7fc17402a5f0 comp rx=0 tx=0).stop 2024-04-26T20:50:52.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.351+0000 7fc1847bd640 1 -- 172.21.15.73:0/2684891552 shutdown_connections 2024-04-26T20:50:52.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.351+0000 7fc1847bd640 1 --2- 172.21.15.73:0/2684891552 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc17c0d38f0 0x7fc17c0ce890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:52.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.351+0000 7fc1847bd640 1 --2- 172.21.15.73:0/2684891552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc17c0d09a0 0x7fc17c0d1800 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:52.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.351+0000 7fc1847bd640 1 -- 172.21.15.73:0/2684891552 >> 172.21.15.73:0/2684891552 conn(0x7fc17c0d2dd0 msgr2=0x7fc17c0d31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:52.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.351+0000 7fc1847bd640 1 -- 172.21.15.73:0/2684891552 shutdown_connections 2024-04-26T20:50:52.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.351+0000 7fc1847bd640 1 -- 172.21.15.73:0/2684891552 wait complete. 2024-04-26T20:50:52.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.352+0000 7fc1847bd640 1 Processor -- start 2024-04-26T20:50:52.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.352+0000 7fc1847bd640 1 -- start start 2024-04-26T20:50:52.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.353+0000 7fc1847bd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc17c0d09a0 0x7fc17c14ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:52.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.353+0000 7fc1847bd640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc17c0d38f0 0x7fc17c0b5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:52.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.353+0000 7fc1847bd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc17c1606f0 0x7fc17c0c6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:52.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.353+0000 7fc1847bd640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc17c0b5c20 con 0x7fc17c1606f0 2024-04-26T20:50:52.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.353+0000 7fc1847bd640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc17c0ce370 con 0x7fc17c0d09a0 2024-04-26T20:50:52.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.353+0000 7fc1847bd640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fc17c0b5da0 con 0x7fc17c0d38f0 2024-04-26T20:50:52.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.353+0000 7fc182d5a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc17c1606f0 0x7fc17c0c6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:52.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.353+0000 7fc182d5a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc17c1606f0 0x7fc17c0c6900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49274/0 (socket says 172.21.15.73:49274) 2024-04-26T20:50:52.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.353+0000 7fc182d5a640 1 -- 172.21.15.73:0/4167979810 learned_addr learned my addr 172.21.15.73:0/4167979810 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:52.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.353+0000 7fc181d58640 1 --2- 172.21.15.73:0/4167979810 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc17c0d38f0 0x7fc17c0b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:52.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.353+0000 7fc182559640 1 --2- 172.21.15.73:0/4167979810 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc17c0d09a0 0x7fc17c14ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:52.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.353+0000 7fc182d5a640 1 -- 172.21.15.73:0/4167979810 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc17c0d38f0 msgr2=0x7fc17c0b5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:52.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.353+0000 7fc182d5a640 1 --2- 172.21.15.73:0/4167979810 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc17c0d38f0 0x7fc17c0b5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:52.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.353+0000 7fc182d5a640 1 -- 172.21.15.73:0/4167979810 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc17c0d09a0 msgr2=0x7fc17c14ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:52.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.353+0000 7fc182d5a640 1 --2- 172.21.15.73:0/4167979810 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc17c0d09a0 0x7fc17c14ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:52.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.353+0000 7fc182d5a640 1 -- 172.21.15.73:0/4167979810 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc17c0c9520 con 0x7fc17c1606f0 2024-04-26T20:50:52.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.353+0000 7fc182d5a640 1 --2- 172.21.15.73:0/4167979810 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc17c1606f0 0x7fc17c0c6900 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc174009eb0 tx=0x7fc174004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:52.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.354+0000 7fc172ffd640 1 -- 172.21.15.73:0/4167979810 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc174009450 con 0x7fc17c1606f0 2024-04-26T20:50:52.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.354+0000 7fc172ffd640 1 -- 172.21.15.73:0/4167979810 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc174004260 con 0x7fc17c1606f0 2024-04-26T20:50:52.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.354+0000 7fc1847bd640 1 -- 172.21.15.73:0/4167979810 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc17c14f1b0 con 0x7fc17c1606f0 2024-04-26T20:50:52.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.354+0000 7fc172ffd640 1 -- 172.21.15.73:0/4167979810 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc174004650 con 0x7fc17c1606f0 2024-04-26T20:50:52.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.354+0000 7fc1847bd640 1 -- 172.21.15.73:0/4167979810 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc17c14cca0 con 0x7fc17c1606f0 2024-04-26T20:50:52.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.355+0000 7fc1847bd640 1 -- 172.21.15.73:0/4167979810 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc17c0694f0 con 0x7fc17c1606f0 2024-04-26T20:50:52.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.355+0000 7fc172ffd640 1 -- 172.21.15.73:0/4167979810 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc174039040 con 0x7fc17c1606f0 2024-04-26T20:50:52.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.355+0000 7fc172ffd640 1 --2- 172.21.15.73:0/4167979810 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc16006c8f0 0x7fc16006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:52.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.356+0000 7fc172ffd640 1 -- 172.21.15.73:0/4167979810 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc1740ad360 con 0x7fc17c1606f0 2024-04-26T20:50:52.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.356+0000 7fc182559640 1 --2- 172.21.15.73:0/4167979810 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc16006c8f0 0x7fc16006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:52.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.356+0000 7fc182559640 1 --2- 172.21.15.73:0/4167979810 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc16006c8f0 0x7fc16006edb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc16c005880 tx=0x7fc16c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:52.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.360+0000 7fc172ffd640 1 -- 172.21.15.73:0/4167979810 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc174040030 con 0x7fc17c1606f0 2024-04-26T20:50:52.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.544+0000 7fc1847bd640 1 -- 172.21.15.73:0/4167979810 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc17c0b8d50 con 0x7fc17c1606f0 2024-04-26T20:50:52.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.545+0000 7fc172ffd640 1 -- 172.21.15.73:0/4167979810 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18514 (secure 0 0 0) 0x7fc17407aec0 con 0x7fc17c1606f0 2024-04-26T20:50:52.547 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:50:52.547 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":25,"fsid":"fbd3f548-b547-4713-856b-09f98f5d1764","created":"2024-04-26T20:50:07.876786+0000","modified":"2024-04-26T20:50:38.657510+0000","last_up_change":"2024-04-26T20:50:19.082741+0000","last_in_change":"2024-04-26T20:50:10.018400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:50:19.993731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:50:29.964420+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:50:30.920082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:50:34.650207+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"72d6f4f3-214a-4392-a02f-c33cbcab3c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6809","nonce":3461705124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6811","nonce":3461705124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6815","nonce":3461705124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6813","nonce":3461705124}]},"public_addr":"172.21.15.73:6809/3461705124","cluster_addr":"172.21.15.73:6811/3461705124","heartbeat_back_addr":"172.21.15.73:6815/3461705124","heartbeat_front_addr":"172.21.15.73:6813/3461705124","state":["exists","up"]},{"osd":1,"uuid":"786bf8ca-6ddc-4b3c-be6c-9152398bd5fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6817","nonce":2827619403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6819","nonce":2827619403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6825","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6827","nonce":2827619403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6821","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6823","nonce":2827619403}]},"public_addr":"172.21.15.73:6817/2827619403","cluster_addr":"172.21.15.73:6819/2827619403","heartbeat_back_addr":"172.21.15.73:6827/2827619403","heartbeat_front_addr":"172.21.15.73:6823/2827619403","state":["exists","up"]},{"osd":2,"uuid":"90963b1a-5437-4ef8-ad8b-6b74c534eeb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6801","nonce":2542681064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6803","nonce":2542681064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6808","nonce":2542681064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6805","nonce":2542681064}]},"public_addr":"172.21.15.73:6801/2542681064","cluster_addr":"172.21.15.73:6803/2542681064","heartbeat_back_addr":"172.21.15.73:6808/2542681064","heartbeat_front_addr":"172.21.15.73:6805/2542681064","state":["exists","up"]},{"osd":3,"uuid":"bd3d2148-6e38-41b4-a453-fe6e3b8b4ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6822","nonce":1604042718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6826","nonce":1604042718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6831","nonce":1604042718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6829","nonce":1604042718}]},"public_addr":"172.21.15.73:6822/1604042718","cluster_addr":"172.21.15.73:6826/1604042718","heartbeat_back_addr":"172.21.15.73:6831/1604042718","heartbeat_front_addr":"172.21.15.73:6829/1604042718","state":["exists","up"]},{"osd":4,"uuid":"e8668dc8-a23f-46d2-8311-cb882da08e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6817","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6819","nonce":3591055561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6824","nonce":3591055561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6831","nonce":3591055561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6828","nonce":3591055561}]},"public_addr":"172.21.15.120:6819/3591055561","cluster_addr":"172.21.15.120:6824/3591055561","heartbeat_back_addr":"172.21.15.120:6831/3591055561","heartbeat_front_addr":"172.21.15.120:6828/3591055561","state":["exists","up"]},{"osd":5,"uuid":"cbac0bdb-96bd-4852-bcce-e564495c3fcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6809","nonce":819450433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6811","nonce":819450433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6815","nonce":819450433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6813","nonce":819450433}]},"public_addr":"172.21.15.120:6809/819450433","cluster_addr":"172.21.15.120:6811/819450433","heartbeat_back_addr":"172.21.15.120:6815/819450433","heartbeat_front_addr":"172.21.15.120:6813/819450433","state":["exists","up"]},{"osd":6,"uuid":"1884b2a8-569f-4272-96a9-f25a09364dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6801","nonce":3946210727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6803","nonce":3946210727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6807","nonce":3946210727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6805","nonce":3946210727}]},"public_addr":"172.21.15.120:6801/3946210727","cluster_addr":"172.21.15.120:6803/3946210727","heartbeat_back_addr":"172.21.15.120:6807/3946210727","heartbeat_front_addr":"172.21.15.120:6805/3946210727","state":["exists","up"]},{"osd":7,"uuid":"c0c3bf6d-a4c5-4f1e-aa2d-330951830666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6818","nonce":2915574674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6821","nonce":2915574674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6827","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6829","nonce":2915574674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6823","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6825","nonce":2915574674}]},"public_addr":"172.21.15.120:6818/2915574674","cluster_addr":"172.21.15.120:6821/2915574674","heartbeat_back_addr":"172.21.15.120:6829/2915574674","heartbeat_front_addr":"172.21.15.120:6825/2915574674","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.218216+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.208710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.112705+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.182789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.164324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.174212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.225396+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.261559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:50:52.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.547+0000 7fc1847bd640 1 -- 172.21.15.73:0/4167979810 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc16006c8f0 msgr2=0x7fc16006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:52.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.547+0000 7fc1847bd640 1 --2- 172.21.15.73:0/4167979810 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc16006c8f0 0x7fc16006edb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc16c005880 tx=0x7fc16c009040 comp rx=0 tx=0).stop 2024-04-26T20:50:52.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.547+0000 7fc1847bd640 1 -- 172.21.15.73:0/4167979810 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc17c1606f0 msgr2=0x7fc17c0c6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:52.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.547+0000 7fc1847bd640 1 --2- 172.21.15.73:0/4167979810 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc17c1606f0 0x7fc17c0c6900 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc174009eb0 tx=0x7fc174004a40 comp rx=0 tx=0).stop 2024-04-26T20:50:52.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.547+0000 7fc1847bd640 1 -- 172.21.15.73:0/4167979810 shutdown_connections 2024-04-26T20:50:52.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.547+0000 7fc1847bd640 1 --2- 172.21.15.73:0/4167979810 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc17c1606f0 0x7fc17c0c6900 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:52.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.547+0000 7fc1847bd640 1 --2- 172.21.15.73:0/4167979810 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc16006c8f0 0x7fc16006edb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:52.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.547+0000 7fc1847bd640 1 --2- 172.21.15.73:0/4167979810 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc17c0d38f0 0x7fc17c0b5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:52.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.547+0000 7fc1847bd640 1 --2- 172.21.15.73:0/4167979810 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc17c0d09a0 0x7fc17c14ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:52.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.547+0000 7fc1847bd640 1 -- 172.21.15.73:0/4167979810 >> 172.21.15.73:0/4167979810 conn(0x7fc17c0d2dd0 msgr2=0x7fc17c071530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:52.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.548+0000 7fc1847bd640 1 -- 172.21.15.73:0/4167979810 shutdown_connections 2024-04-26T20:50:52.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.548+0000 7fc1847bd640 1 -- 172.21.15.73:0/4167979810 wait complete. 2024-04-26T20:50:52.560 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T20:50:52.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.662+0000 7f3744cfc640 1 Processor -- start 2024-04-26T20:50:52.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.662+0000 7f3744cfc640 1 -- start start 2024-04-26T20:50:52.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.663+0000 7f3744cfc640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f37400cf5e0 0x7f37400ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:52.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.663+0000 7f3744cfc640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f37400cbbd0 0x7f37400cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:52.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.663+0000 7f3744cfc640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f374015b9e0 con 0x7f374015b600 2024-04-26T20:50:52.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.663+0000 7f3744cfc640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f37400cb9b0 con 0x7f37400cf5e0 2024-04-26T20:50:52.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.663+0000 7f3744cfc640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f37400d0740 con 0x7f37400cbbd0 2024-04-26T20:50:52.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.663+0000 7f373dd9b640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f37400cf5e0 0x7f37400ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:52.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.663+0000 7f373dd9b640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f37400cf5e0 0x7f37400ce9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49288/0 (socket says 172.21.15.73:49288) 2024-04-26T20:50:52.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.663+0000 7f373dd9b640 1 -- 172.21.15.73:0/4182532911 learned_addr learned my addr 172.21.15.73:0/4182532911 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:52.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.663+0000 7f373ed9d640 1 --2- 172.21.15.73:0/4182532911 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f37400cbbd0 0x7f37400cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:52.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.663+0000 7f373dd9b640 1 -- 172.21.15.73:0/4182532911 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f37400cbbd0 msgr2=0x7f37400cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:52.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.663+0000 7f373dd9b640 1 --2- 172.21.15.73:0/4182532911 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f37400cbbd0 0x7f37400cef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:52.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.663+0000 7f373dd9b640 1 -- 172.21.15.73:0/4182532911 >> v1:172.21.15.120:6789/0 conn(0x7f374015b600 legacy=0x7f3740071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:52.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.663+0000 7f373dd9b640 1 -- 172.21.15.73:0/4182532911 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37400cb360 con 0x7f37400cf5e0 2024-04-26T20:50:52.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.664+0000 7f373dd9b640 1 --2- 172.21.15.73:0/4182532911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37400cf5e0 0x7f37400ce9d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f373400bb10 tx=0x7f373402c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39068d06560fd52c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:52.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.664+0000 7f373d59a640 1 -- 172.21.15.73:0/4182532911 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3734032790 con 0x7f37400cf5e0 2024-04-26T20:50:52.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.664+0000 7f373d59a640 1 -- 172.21.15.73:0/4182532911 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3734038070 con 0x7f37400cf5e0 2024-04-26T20:50:52.667 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.664+0000 7f373d59a640 1 -- 172.21.15.73:0/4182532911 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3734032b50 con 0x7f37400cf5e0 2024-04-26T20:50:52.667 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.665+0000 7f3744cfc640 1 -- 172.21.15.73:0/4182532911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37400cf5e0 msgr2=0x7f37400ce9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:52.667 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.665+0000 7f3744cfc640 1 --2- 172.21.15.73:0/4182532911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37400cf5e0 0x7f37400ce9d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f373400bb10 tx=0x7f373402c8d0 comp rx=0 tx=0).stop 2024-04-26T20:50:52.667 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.665+0000 7f3744cfc640 1 -- 172.21.15.73:0/4182532911 shutdown_connections 2024-04-26T20:50:52.667 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.665+0000 7f3744cfc640 1 --2- 172.21.15.73:0/4182532911 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f37400cbbd0 0x7f37400cef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:52.667 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.665+0000 7f3744cfc640 1 --2- 172.21.15.73:0/4182532911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37400cf5e0 0x7f37400ce9d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:52.667 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.665+0000 7f3744cfc640 1 -- 172.21.15.73:0/4182532911 >> 172.21.15.73:0/4182532911 conn(0x7f37400d0be0 msgr2=0x7f37400d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:52.667 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.665+0000 7f3744cfc640 1 -- 172.21.15.73:0/4182532911 shutdown_connections 2024-04-26T20:50:52.667 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.665+0000 7f3744cfc640 1 -- 172.21.15.73:0/4182532911 wait complete. 2024-04-26T20:50:52.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.666+0000 7f3744cfc640 1 Processor -- start 2024-04-26T20:50:52.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.666+0000 7f3744cfc640 1 -- start start 2024-04-26T20:50:52.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.667+0000 7f3744cfc640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f37400cbbd0 0x7f37401550a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:52.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.667+0000 7f3744cfc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37400cf5e0 0x7f37401f4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:52.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.667+0000 7f3744cfc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f374015b600 0x7f374015ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:52.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.667+0000 7f3744cfc640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f37400d0470 con 0x7f37400cf5e0 2024-04-26T20:50:52.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.667+0000 7f3744cfc640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f37400d02f0 con 0x7f374015b600 2024-04-26T20:50:52.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.667+0000 7f3744cfc640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f374006ddf0 con 0x7f37400cbbd0 2024-04-26T20:50:52.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.667+0000 7f373dd9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37400cf5e0 0x7f37401f4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:52.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.667+0000 7f373dd9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37400cf5e0 0x7f37401f4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49294/0 (socket says 172.21.15.73:49294) 2024-04-26T20:50:52.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.667+0000 7f373dd9b640 1 -- 172.21.15.73:0/1685911998 learned_addr learned my addr 172.21.15.73:0/1685911998 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:52.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.667+0000 7f373e59c640 1 --2- 172.21.15.73:0/1685911998 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f37400cbbd0 0x7f37401550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:52.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.667+0000 7f373ed9d640 1 --2- 172.21.15.73:0/1685911998 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f374015b600 0x7f374015ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:52.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.667+0000 7f373dd9b640 1 -- 172.21.15.73:0/1685911998 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f37400cbbd0 msgr2=0x7f37401550a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:52.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.667+0000 7f373dd9b640 1 --2- 172.21.15.73:0/1685911998 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f37400cbbd0 0x7f37401550a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:52.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.667+0000 7f373dd9b640 1 -- 172.21.15.73:0/1685911998 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f374015b600 msgr2=0x7f374015ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:52.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.667+0000 7f373dd9b640 1 --2- 172.21.15.73:0/1685911998 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f374015b600 0x7f374015ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:52.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.667+0000 7f373dd9b640 1 -- 172.21.15.73:0/1685911998 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37401f5f20 con 0x7f37400cf5e0 2024-04-26T20:50:52.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.667+0000 7f373ed9d640 1 --2- 172.21.15.73:0/1685911998 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f374015b600 0x7f374015ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:50:52.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.668+0000 7f373dd9b640 1 --2- 172.21.15.73:0/1685911998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37400cf5e0 0x7f37401f4ff0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f373400bb10 tx=0x7f3734004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:52.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.668+0000 7f372effd640 1 -- 172.21.15.73:0/1685911998 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3734043070 con 0x7f37400cf5e0 2024-04-26T20:50:52.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.668+0000 7f372effd640 1 -- 172.21.15.73:0/1685911998 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f373403c620 con 0x7f37400cf5e0 2024-04-26T20:50:52.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.668+0000 7f372effd640 1 -- 172.21.15.73:0/1685911998 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373403ca10 con 0x7f37400cf5e0 2024-04-26T20:50:52.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.668+0000 7f3744cfc640 1 -- 172.21.15.73:0/1685911998 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37401f5960 con 0x7f37400cf5e0 2024-04-26T20:50:52.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.668+0000 7f3744cfc640 1 -- 172.21.15.73:0/1685911998 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37401f6270 con 0x7f37400cf5e0 2024-04-26T20:50:52.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.669+0000 7f3744cfc640 1 -- 172.21.15.73:0/1685911998 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37400694f0 con 0x7f37400cf5e0 2024-04-26T20:50:52.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.670+0000 7f372effd640 1 -- 172.21.15.73:0/1685911998 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3734046070 con 0x7f37400cf5e0 2024-04-26T20:50:52.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.670+0000 7f372effd640 1 --2- 172.21.15.73:0/1685911998 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f372006c8f0 0x7f372006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:52.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.670+0000 7f372effd640 1 -- 172.21.15.73:0/1685911998 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f37340ac4b0 con 0x7f37400cf5e0 2024-04-26T20:50:52.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.670+0000 7f373e59c640 1 --2- 172.21.15.73:0/1685911998 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f372006c8f0 0x7f372006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:52.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.671+0000 7f373e59c640 1 --2- 172.21.15.73:0/1685911998 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f372006c8f0 0x7f372006edb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3730006fd0 tx=0x7f3730008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:52.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.675+0000 7f372effd640 1 -- 172.21.15.73:0/1685911998 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f373407a750 con 0x7f37400cf5e0 2024-04-26T20:50:52.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:52.899+0000 7f3744cfc640 1 -- 172.21.15.73:0/1685911998 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f37400ce7b0 con 0x7f37400cf5e0 2024-04-26T20:50:53.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.709+0000 7f372effd640 1 -- 172.21.15.73:0/1685911998 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7f373407a050 con 0x7f37400cf5e0 2024-04-26T20:50:53.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.711+0000 7f3744cfc640 1 -- 172.21.15.73:0/1685911998 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f372006c8f0 msgr2=0x7f372006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:53.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.711+0000 7f3744cfc640 1 --2- 172.21.15.73:0/1685911998 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f372006c8f0 0x7f372006edb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3730006fd0 tx=0x7f3730008040 comp rx=0 tx=0).stop 2024-04-26T20:50:53.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.711+0000 7f3744cfc640 1 -- 172.21.15.73:0/1685911998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37400cf5e0 msgr2=0x7f37401f4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:53.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.711+0000 7f3744cfc640 1 --2- 172.21.15.73:0/1685911998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37400cf5e0 0x7f37401f4ff0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f373400bb10 tx=0x7f3734004730 comp rx=0 tx=0).stop 2024-04-26T20:50:53.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.711+0000 7f3744cfc640 1 -- 172.21.15.73:0/1685911998 shutdown_connections 2024-04-26T20:50:53.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.711+0000 7f3744cfc640 1 --2- 172.21.15.73:0/1685911998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37400cf5e0 0x7f37401f4ff0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:53.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.711+0000 7f3744cfc640 1 --2- 172.21.15.73:0/1685911998 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f372006c8f0 0x7f372006edb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:53.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.711+0000 7f3744cfc640 1 --2- 172.21.15.73:0/1685911998 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f374015b600 0x7f374015ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:53.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.711+0000 7f3744cfc640 1 --2- 172.21.15.73:0/1685911998 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f37400cbbd0 0x7f37401550a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:53.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.711+0000 7f3744cfc640 1 -- 172.21.15.73:0/1685911998 >> 172.21.15.73:0/1685911998 conn(0x7f37400d0be0 msgr2=0x7f3740156180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:53.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.711+0000 7f3744cfc640 1 -- 172.21.15.73:0/1685911998 shutdown_connections 2024-04-26T20:50:53.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.711+0000 7f3744cfc640 1 -- 172.21.15.73:0/1685911998 wait complete. 2024-04-26T20:50:53.723 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-04-26T20:50:53.723 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T20:50:53.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.823+0000 7f0c7bc50640 1 Processor -- start 2024-04-26T20:50:53.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.823+0000 7f0c7bc50640 1 -- start start 2024-04-26T20:50:53.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.824+0000 7f0c7bc50640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0c74071e60 0x7f0c7406f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:53.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.824+0000 7f0c7bc50640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c7415d7c0 con 0x7f0c7406fe20 2024-04-26T20:50:53.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.824+0000 7f0c7bc50640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c7415daa0 con 0x7f0c740cc480 2024-04-26T20:50:53.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.824+0000 7f0c7bc50640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f0c7415dd80 con 0x7f0c74071e60 2024-04-26T20:50:53.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.824+0000 7f0c791eb640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f0c7406fe20 0x7f0c74072240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33032/0 (socket says 172.21.15.73:33032) 2024-04-26T20:50:53.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.824+0000 7f0c791eb640 1 -- 172.21.15.73:0/2120319801 learned_addr learned my addr 172.21.15.73:0/2120319801 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:50:53.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.824+0000 7f0c789ea640 1 -- 172.21.15.73:0/2120319801 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4097143164 0 0) 0x7f0c7415d7c0 con 0x7f0c7406fe20 2024-04-26T20:50:53.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.824+0000 7f0c799ec640 1 --2- 172.21.15.73:0/2120319801 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0c74071e60 0x7f0c7406f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:53.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.824+0000 7f0c789ea640 1 -- 172.21.15.73:0/2120319801 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0c60003520 con 0x7f0c7406fe20 2024-04-26T20:50:53.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.825+0000 7f0c789ea640 1 -- 172.21.15.73:0/2120319801 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3991177523 0 0) 0x7f0c60003520 con 0x7f0c7406fe20 2024-04-26T20:50:53.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.825+0000 7f0c789ea640 1 -- 172.21.15.73:0/2120319801 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0c74071e60 msgr2=0x7f0c7406f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:53.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.825+0000 7f0c789ea640 1 --2- 172.21.15.73:0/2120319801 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0c74071e60 0x7f0c7406f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:53.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.825+0000 7f0c789ea640 1 -- 172.21.15.73:0/2120319801 >> v1:172.21.15.120:6789/0 conn(0x7f0c740cc480 legacy=0x7f0c74166ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:53.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.825+0000 7f0c789ea640 1 -- 172.21.15.73:0/2120319801 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c7415e060 con 0x7f0c7406fe20 2024-04-26T20:50:53.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.825+0000 7f0c789ea640 1 -- 172.21.15.73:0/2120319801 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7f0c64001670 con 0x7f0c7406fe20 2024-04-26T20:50:53.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.825+0000 7f0c789ea640 1 -- 172.21.15.73:0/2120319801 >> v1:172.21.15.73:6789/0 conn(0x7f0c7406fe20 legacy=0x7f0c74072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:53.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.825+0000 7f0c789ea640 1 --2- 172.21.15.73:0/2120319801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0c600039e0 0x7f0c60003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:53.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.825+0000 7f0c789ea640 1 --2- 172.21.15.73:0/2120319801 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0c74071e60 0x7f0c60004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:53.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.825+0000 7f0c789ea640 1 --2- 172.21.15.73:0/2120319801 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0c60004d40 0x7f0c600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:53.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.825+0000 7f0c789ea640 1 -- 172.21.15.73:0/2120319801 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0c7415dd80 con 0x7f0c600039e0 2024-04-26T20:50:53.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.825+0000 7f0c789ea640 1 -- 172.21.15.73:0/2120319801 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6000ba70 con 0x7f0c60004d40 2024-04-26T20:50:53.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.825+0000 7f0c789ea640 1 -- 172.21.15.73:0/2120319801 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f0c6000bc70 con 0x7f0c74071e60 2024-04-26T20:50:53.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.825+0000 7f0c799ec640 1 --2- 172.21.15.73:0/2120319801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0c600039e0 0x7f0c60003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:53.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.825+0000 7f0c791eb640 1 --2- 172.21.15.73:0/2120319801 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0c74071e60 0x7f0c60004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:53.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.825+0000 7f0c7a1ed640 1 --2- 172.21.15.73:0/2120319801 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0c60004d40 0x7f0c600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:53.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.825+0000 7f0c799ec640 1 -- 172.21.15.73:0/2120319801 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0c74071e60 msgr2=0x7f0c60004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:53.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.825+0000 7f0c799ec640 1 --2- 172.21.15.73:0/2120319801 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0c74071e60 0x7f0c60004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:53.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.825+0000 7f0c799ec640 1 -- 172.21.15.73:0/2120319801 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0c60004d40 msgr2=0x7f0c600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:53.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.825+0000 7f0c799ec640 1 --2- 172.21.15.73:0/2120319801 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0c60004d40 0x7f0c600051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:53.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.825+0000 7f0c799ec640 1 -- 172.21.15.73:0/2120319801 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c60003520 con 0x7f0c600039e0 2024-04-26T20:50:53.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.826+0000 7f0c799ec640 1 --2- 172.21.15.73:0/2120319801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0c600039e0 0x7f0c60003e00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0c70009a90 tx=0x7f0c60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f21cdb0d407304e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:53.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.826+0000 7f0c789ea640 1 -- 172.21.15.73:0/2120319801 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c70036070 con 0x7f0c600039e0 2024-04-26T20:50:53.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.826+0000 7f0c789ea640 1 -- 172.21.15.73:0/2120319801 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c70031400 con 0x7f0c600039e0 2024-04-26T20:50:53.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.826+0000 7f0c789ea640 1 -- 172.21.15.73:0/2120319801 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c70009690 con 0x7f0c600039e0 2024-04-26T20:50:53.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.826+0000 7f0c7bc50640 1 -- 172.21.15.73:0/2120319801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0c600039e0 msgr2=0x7f0c60003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:53.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.826+0000 7f0c7bc50640 1 --2- 172.21.15.73:0/2120319801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0c600039e0 0x7f0c60003e00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0c70009a90 tx=0x7f0c60005830 comp rx=0 tx=0).stop 2024-04-26T20:50:53.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.826+0000 7f0c7a1ed640 1 -- 172.21.15.73:0/2120319801 reap_dead start 2024-04-26T20:50:53.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.826+0000 7f0c7bc50640 1 -- 172.21.15.73:0/2120319801 shutdown_connections 2024-04-26T20:50:53.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.826+0000 7f0c7bc50640 1 -- 172.21.15.73:0/2120319801 >> 172.21.15.73:0/2120319801 conn(0x7f0c740d0be0 msgr2=0x7f0c740d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:53.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.827+0000 7f0c7bc50640 1 -- 172.21.15.73:0/2120319801 shutdown_connections 2024-04-26T20:50:53.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.827+0000 7f0c7bc50640 1 -- 172.21.15.73:0/2120319801 wait complete. 2024-04-26T20:50:53.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.828+0000 7f0c7bc50640 1 Processor -- start 2024-04-26T20:50:53.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.828+0000 7f0c7bc50640 1 -- start start 2024-04-26T20:50:53.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.828+0000 7f0c7bc50640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0c600039e0 0x7f0c74148bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:53.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.828+0000 7f0c7bc50640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0c741493f0 0x7f0c7414b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:53.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.828+0000 7f0c7bc50640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0c74149870 0x7f0c7415e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:53.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.828+0000 7f0c7bc50640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0c7415d570 con 0x7f0c74149870 2024-04-26T20:50:53.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.828+0000 7f0c7bc50640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0c7415d3f0 con 0x7f0c741493f0 2024-04-26T20:50:53.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.828+0000 7f0c7bc50640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f0c74072950 con 0x7f0c600039e0 2024-04-26T20:50:53.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.829+0000 7f0c7a1ed640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0c74149870 0x7f0c7415e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:53.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.829+0000 7f0c7a1ed640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0c74149870 0x7f0c7415e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49322/0 (socket says 172.21.15.73:49322) 2024-04-26T20:50:53.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.829+0000 7f0c7a1ed640 1 -- 172.21.15.73:0/122479375 learned_addr learned my addr 172.21.15.73:0/122479375 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:53.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.829+0000 7f0c791eb640 1 --2- 172.21.15.73:0/122479375 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0c741493f0 0x7f0c7414b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:53.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.829+0000 7f0c799ec640 1 --2- 172.21.15.73:0/122479375 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0c600039e0 0x7f0c74148bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:53.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.829+0000 7f0c7a1ed640 1 -- 172.21.15.73:0/122479375 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0c600039e0 msgr2=0x7f0c74148bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:53.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.829+0000 7f0c7a1ed640 1 --2- 172.21.15.73:0/122479375 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0c600039e0 0x7f0c74148bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:53.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.829+0000 7f0c7a1ed640 1 -- 172.21.15.73:0/122479375 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0c741493f0 msgr2=0x7f0c7414b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:53.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.829+0000 7f0c7a1ed640 1 --2- 172.21.15.73:0/122479375 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0c741493f0 0x7f0c7414b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:53.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.829+0000 7f0c7a1ed640 1 -- 172.21.15.73:0/122479375 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c741490f0 con 0x7f0c74149870 2024-04-26T20:50:53.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.829+0000 7f0c7a1ed640 1 --2- 172.21.15.73:0/122479375 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0c74149870 0x7f0c7415e580 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0c740b06b0 tx=0x7f0c6c00ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:53.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.829+0000 7f0c6a7fc640 1 -- 172.21.15.73:0/122479375 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c003970 con 0x7f0c74149870 2024-04-26T20:50:53.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.829+0000 7f0c6a7fc640 1 -- 172.21.15.73:0/122479375 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c6c018070 con 0x7f0c74149870 2024-04-26T20:50:53.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.829+0000 7f0c7bc50640 1 -- 172.21.15.73:0/122479375 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c7414bf40 con 0x7f0c74149870 2024-04-26T20:50:53.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.829+0000 7f0c6a7fc640 1 -- 172.21.15.73:0/122479375 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c003d50 con 0x7f0c74149870 2024-04-26T20:50:53.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.829+0000 7f0c7bc50640 1 -- 172.21.15.73:0/122479375 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c7415b3c0 con 0x7f0c74149870 2024-04-26T20:50:53.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.830+0000 7f0c7bc50640 1 -- 172.21.15.73:0/122479375 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c740b04e0 con 0x7f0c74149870 2024-04-26T20:50:53.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.831+0000 7f0c6a7fc640 1 -- 172.21.15.73:0/122479375 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0c6c0050d0 con 0x7f0c74149870 2024-04-26T20:50:53.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.831+0000 7f0c6a7fc640 1 --2- 172.21.15.73:0/122479375 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f0c5806c920 0x7f0c5806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:53.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.831+0000 7f0c6a7fc640 1 -- 172.21.15.73:0/122479375 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f0c6c08bb00 con 0x7f0c74149870 2024-04-26T20:50:53.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.831+0000 7f0c799ec640 1 --2- 172.21.15.73:0/122479375 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f0c5806c920 0x7f0c5806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:53.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.831+0000 7f0c799ec640 1 --2- 172.21.15.73:0/122479375 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f0c5806c920 0x7f0c5806ede0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0c70033000 tx=0x7f0c70008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:53.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:53.835+0000 7f0c6a7fc640 1 -- 172.21.15.73:0/122479375 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0c6c059bf0 con 0x7f0c74149870 2024-04-26T20:50:54.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.047+0000 7f0c7bc50640 1 -- 172.21.15.73:0/122479375 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f0c740694f0 con 0x7f0c74149870 2024-04-26T20:50:54.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.714+0000 7f0c6a7fc640 1 -- 172.21.15.73:0/122479375 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7f0c6c0594f0 con 0x7f0c74149870 2024-04-26T20:50:54.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.717+0000 7f0c7bc50640 1 -- 172.21.15.73:0/122479375 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f0c5806c920 msgr2=0x7f0c5806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:54.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.717+0000 7f0c7bc50640 1 --2- 172.21.15.73:0/122479375 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f0c5806c920 0x7f0c5806ede0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0c70033000 tx=0x7f0c70008030 comp rx=0 tx=0).stop 2024-04-26T20:50:54.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.717+0000 7f0c7bc50640 1 -- 172.21.15.73:0/122479375 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0c74149870 msgr2=0x7f0c7415e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:54.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.717+0000 7f0c7bc50640 1 --2- 172.21.15.73:0/122479375 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0c74149870 0x7f0c7415e580 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0c740b06b0 tx=0x7f0c6c00ec10 comp rx=0 tx=0).stop 2024-04-26T20:50:54.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.718+0000 7f0c7bc50640 1 -- 172.21.15.73:0/122479375 shutdown_connections 2024-04-26T20:50:54.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.718+0000 7f0c7bc50640 1 --2- 172.21.15.73:0/122479375 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0c74149870 0x7f0c7415e580 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:54.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.718+0000 7f0c7bc50640 1 --2- 172.21.15.73:0/122479375 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f0c5806c920 0x7f0c5806ede0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:54.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.718+0000 7f0c7bc50640 1 --2- 172.21.15.73:0/122479375 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0c741493f0 0x7f0c7414b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:54.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.718+0000 7f0c7bc50640 1 --2- 172.21.15.73:0/122479375 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0c600039e0 0x7f0c74148bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:54.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.718+0000 7f0c7bc50640 1 -- 172.21.15.73:0/122479375 >> 172.21.15.73:0/122479375 conn(0x7f0c740d0be0 msgr2=0x7f0c7415f860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:54.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.718+0000 7f0c7bc50640 1 -- 172.21.15.73:0/122479375 shutdown_connections 2024-04-26T20:50:54.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.718+0000 7f0c7bc50640 1 -- 172.21.15.73:0/122479375 wait complete. 2024-04-26T20:50:54.730 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T20:50:54.731 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:50:54.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.828+0000 7f544be59640 1 Processor -- start 2024-04-26T20:50:54.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.828+0000 7f544be59640 1 -- start start 2024-04-26T20:50:54.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.829+0000 7f544be59640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f54440d1b30 0x7f54440d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:54.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.829+0000 7f544be59640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54440d10d0 con 0x7f5444162810 2024-04-26T20:50:54.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.829+0000 7f544be59640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54440d1370 con 0x7f54440b2340 2024-04-26T20:50:54.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.829+0000 7f544be59640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f54440d0750 con 0x7f54440d1b30 2024-04-26T20:50:54.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.829+0000 7f5449bf5640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f5444162810 0x7f5444070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33044/0 (socket says 172.21.15.73:33044) 2024-04-26T20:50:54.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.829+0000 7f5449bf5640 1 -- 172.21.15.73:0/3508263788 learned_addr learned my addr 172.21.15.73:0/3508263788 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:50:54.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.829+0000 7f544a3f6640 1 --2- 172.21.15.73:0/3508263788 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f54440d1b30 0x7f54440d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:54.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.829+0000 7f5448bf3640 1 -- 172.21.15.73:0/3508263788 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3060629416 0 0) 0x7f54440d10d0 con 0x7f5444162810 2024-04-26T20:50:54.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.829+0000 7f5448bf3640 1 -- 172.21.15.73:0/3508263788 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5430003520 con 0x7f5444162810 2024-04-26T20:50:54.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.829+0000 7f5448bf3640 1 -- 172.21.15.73:0/3508263788 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3242934032 0 0) 0x7f5430003520 con 0x7f5444162810 2024-04-26T20:50:54.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.829+0000 7f5448bf3640 1 -- 172.21.15.73:0/3508263788 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f54440d1b30 msgr2=0x7f54440d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:54.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.829+0000 7f5448bf3640 1 --2- 172.21.15.73:0/3508263788 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f54440d1b30 0x7f54440d1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:54.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.829+0000 7f5448bf3640 1 -- 172.21.15.73:0/3508263788 >> v1:172.21.15.120:6789/0 conn(0x7f54440b2340 legacy=0x7f54440cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:54.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.829+0000 7f5448bf3640 1 -- 172.21.15.73:0/3508263788 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54440d01b0 con 0x7f5444162810 2024-04-26T20:50:54.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.829+0000 7f5448bf3640 1 -- 172.21.15.73:0/3508263788 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7f5440001670 con 0x7f5444162810 2024-04-26T20:50:54.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.829+0000 7f5448bf3640 1 -- 172.21.15.73:0/3508263788 >> v1:172.21.15.73:6789/0 conn(0x7f5444162810 legacy=0x7f5444070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:54.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.829+0000 7f5448bf3640 1 --2- 172.21.15.73:0/3508263788 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f54440d1b30 0x7f54300039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:54.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.829+0000 7f5448bf3640 1 --2- 172.21.15.73:0/3508263788 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5430003f20 0x7f5430004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:54.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.829+0000 7f5448bf3640 1 --2- 172.21.15.73:0/3508263788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5430004d40 0x7f54300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:54.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.829+0000 7f5448bf3640 1 -- 172.21.15.73:0/3508263788 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f54440d0750 con 0x7f5430004d40 2024-04-26T20:50:54.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.829+0000 7f5448bf3640 1 -- 172.21.15.73:0/3508263788 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f543000ba20 con 0x7f5430003f20 2024-04-26T20:50:54.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.829+0000 7f5448bf3640 1 -- 172.21.15.73:0/3508263788 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f543000bc20 con 0x7f54440d1b30 2024-04-26T20:50:54.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.830+0000 7f544a3f6640 1 --2- 172.21.15.73:0/3508263788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5430004d40 0x7f54300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:54.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.830+0000 7f5449bf5640 1 --2- 172.21.15.73:0/3508263788 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f54440d1b30 0x7f54300039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:54.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.830+0000 7f54493f4640 1 --2- 172.21.15.73:0/3508263788 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5430003f20 0x7f5430004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:54.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.830+0000 7f544a3f6640 1 -- 172.21.15.73:0/3508263788 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f54440d1b30 msgr2=0x7f54300039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:54.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.830+0000 7f544a3f6640 1 --2- 172.21.15.73:0/3508263788 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f54440d1b30 0x7f54300039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:54.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.830+0000 7f544a3f6640 1 -- 172.21.15.73:0/3508263788 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5430003f20 msgr2=0x7f5430004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:54.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.830+0000 7f544a3f6640 1 --2- 172.21.15.73:0/3508263788 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5430003f20 0x7f5430004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:54.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.830+0000 7f544a3f6640 1 -- 172.21.15.73:0/3508263788 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5430003520 con 0x7f5430004d40 2024-04-26T20:50:54.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.830+0000 7f54493f4640 1 --2- 172.21.15.73:0/3508263788 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5430003f20 0x7f5430004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:50:54.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.830+0000 7f544a3f6640 1 --2- 172.21.15.73:0/3508263788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5430004d40 0x7f54300051c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f543c009fc0 tx=0x7f5430005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bf5b1f70f189dcb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:54.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.831+0000 7f5448bf3640 1 -- 172.21.15.73:0/3508263788 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543c009c10 con 0x7f5430004d40 2024-04-26T20:50:54.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.831+0000 7f5448bf3640 1 -- 172.21.15.73:0/3508263788 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f543c036070 con 0x7f5430004d40 2024-04-26T20:50:54.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.831+0000 7f5448bf3640 1 -- 172.21.15.73:0/3508263788 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543c031880 con 0x7f5430004d40 2024-04-26T20:50:54.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.831+0000 7f544be59640 1 -- 172.21.15.73:0/3508263788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5430004d40 msgr2=0x7f54300051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:54.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.831+0000 7f544be59640 1 --2- 172.21.15.73:0/3508263788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5430004d40 0x7f54300051c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f543c009fc0 tx=0x7f5430005830 comp rx=0 tx=0).stop 2024-04-26T20:50:54.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.831+0000 7f544a3f6640 1 -- 172.21.15.73:0/3508263788 reap_dead start 2024-04-26T20:50:54.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.831+0000 7f544be59640 1 -- 172.21.15.73:0/3508263788 shutdown_connections 2024-04-26T20:50:54.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.831+0000 7f544be59640 1 -- 172.21.15.73:0/3508263788 >> 172.21.15.73:0/3508263788 conn(0x7f54440d2da0 msgr2=0x7f54440d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:54.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.831+0000 7f544be59640 1 -- 172.21.15.73:0/3508263788 shutdown_connections 2024-04-26T20:50:54.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.831+0000 7f544be59640 1 -- 172.21.15.73:0/3508263788 wait complete. 2024-04-26T20:50:54.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.832+0000 7f544be59640 1 Processor -- start 2024-04-26T20:50:54.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.832+0000 7f544be59640 1 -- start start 2024-04-26T20:50:54.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.833+0000 7f544be59640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5430004d40 0x7f54440c6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:54.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.833+0000 7f544be59640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5444152e80 0x7f54440c90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:54.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.833+0000 7f544be59640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f544415d3b0 0x7f544415d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:54.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.833+0000 7f544be59640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f54440cdb00 con 0x7f5444152e80 2024-04-26T20:50:54.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.833+0000 7f544be59640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f54440d16a0 con 0x7f5430004d40 2024-04-26T20:50:54.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.833+0000 7f544be59640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f54440d1fc0 con 0x7f544415d3b0 2024-04-26T20:50:54.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.833+0000 7f54493f4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5444152e80 0x7f54440c90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:54.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.833+0000 7f54493f4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5444152e80 0x7f54440c90c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49352/0 (socket says 172.21.15.73:49352) 2024-04-26T20:50:54.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.833+0000 7f54493f4640 1 -- 172.21.15.73:0/1262012766 learned_addr learned my addr 172.21.15.73:0/1262012766 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:54.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.833+0000 7f544a3f6640 1 --2- 172.21.15.73:0/1262012766 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f544415d3b0 0x7f544415d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:54.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.833+0000 7f5449bf5640 1 --2- 172.21.15.73:0/1262012766 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5430004d40 0x7f54440c6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:54.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.834+0000 7f54493f4640 1 -- 172.21.15.73:0/1262012766 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f544415d3b0 msgr2=0x7f544415d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:54.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.834+0000 7f54493f4640 1 --2- 172.21.15.73:0/1262012766 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f544415d3b0 0x7f544415d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:54.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.834+0000 7f54493f4640 1 -- 172.21.15.73:0/1262012766 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5430004d40 msgr2=0x7f54440c6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:54.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.834+0000 7f54493f4640 1 --2- 172.21.15.73:0/1262012766 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5430004d40 0x7f54440c6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:54.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.834+0000 7f54493f4640 1 -- 172.21.15.73:0/1262012766 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54440c99c0 con 0x7f5444152e80 2024-04-26T20:50:54.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.834+0000 7f54493f4640 1 --2- 172.21.15.73:0/1262012766 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5444152e80 0x7f54440c90c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f543400e7d0 tx=0x7f543400ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:54.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.834+0000 7f543a7fc640 1 -- 172.21.15.73:0/1262012766 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54340050d0 con 0x7f5444152e80 2024-04-26T20:50:54.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.834+0000 7f543a7fc640 1 -- 172.21.15.73:0/1262012766 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5434018070 con 0x7f5444152e80 2024-04-26T20:50:54.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.834+0000 7f544be59640 1 -- 172.21.15.73:0/1262012766 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54440ca3d0 con 0x7f5444152e80 2024-04-26T20:50:54.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.834+0000 7f543a7fc640 1 -- 172.21.15.73:0/1262012766 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543400c6c0 con 0x7f5444152e80 2024-04-26T20:50:54.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.834+0000 7f544be59640 1 -- 172.21.15.73:0/1262012766 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54440ca990 con 0x7f5444152e80 2024-04-26T20:50:54.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.835+0000 7f544be59640 1 -- 172.21.15.73:0/1262012766 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f544406dde0 con 0x7f5444152e80 2024-04-26T20:50:54.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.835+0000 7f543a7fc640 1 -- 172.21.15.73:0/1262012766 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f543400c860 con 0x7f5444152e80 2024-04-26T20:50:54.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.836+0000 7f543a7fc640 1 --2- 172.21.15.73:0/1262012766 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f542806c920 0x7f542806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:54.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.836+0000 7f5449bf5640 1 --2- 172.21.15.73:0/1262012766 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f542806c920 0x7f542806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:54.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.836+0000 7f543a7fc640 1 -- 172.21.15.73:0/1262012766 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f543408bc90 con 0x7f5444152e80 2024-04-26T20:50:54.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.836+0000 7f5449bf5640 1 --2- 172.21.15.73:0/1262012766 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f542806c920 0x7f542806ede0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f54440b2f00 tx=0x7f5440009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:54.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:54.840+0000 7f543a7fc640 1 -- 172.21.15.73:0/1262012766 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f544406dde0 con 0x7f5444152e80 2024-04-26T20:50:55.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.056+0000 7f544be59640 1 -- 172.21.15.73:0/1262012766 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f54440694f0 con 0x7f5444152e80 2024-04-26T20:50:55.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.057+0000 7f543a7fc640 1 -- 172.21.15.73:0/1262012766 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5180 (secure 0 0 0) 0x7f5434059d80 con 0x7f5444152e80 2024-04-26T20:50:55.058 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:50:55.058 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/4263167680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6833","nonce":4263167680},{"type":"v1","addr":"172.21.15.120:6835","nonce":4263167680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6834/1614342510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1614342510},{"type":"v1","addr":"172.21.15.120:6834","nonce":1614342510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2146325725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2146325725},{"type":"v1","addr":"172.21.15.73:6835","nonce":2146325725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:50:31.964317+0000","modified":"2024-04-26T20:50:54.049029+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/181202544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":181202544},{"type":"v1","addr":"172.21.15.73:6837","nonce":181202544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4281,"qdb_cluster":[4281]},"id":1}]} 2024-04-26T20:50:55.059 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 8 2024-04-26T20:50:55.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.059+0000 7f544be59640 1 -- 172.21.15.73:0/1262012766 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f542806c920 msgr2=0x7f542806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:55.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.059+0000 7f544be59640 1 --2- 172.21.15.73:0/1262012766 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f542806c920 0x7f542806ede0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f54440b2f00 tx=0x7f5440009040 comp rx=0 tx=0).stop 2024-04-26T20:50:55.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.059+0000 7f544be59640 1 -- 172.21.15.73:0/1262012766 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5444152e80 msgr2=0x7f54440c90c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:55.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.059+0000 7f544be59640 1 --2- 172.21.15.73:0/1262012766 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5444152e80 0x7f54440c90c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f543400e7d0 tx=0x7f543400ec90 comp rx=0 tx=0).stop 2024-04-26T20:50:55.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.059+0000 7f544be59640 1 -- 172.21.15.73:0/1262012766 shutdown_connections 2024-04-26T20:50:55.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.059+0000 7f544be59640 1 --2- 172.21.15.73:0/1262012766 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5444152e80 0x7f54440c90c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:55.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.059+0000 7f544be59640 1 --2- 172.21.15.73:0/1262012766 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f542806c920 0x7f542806ede0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:55.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.059+0000 7f544be59640 1 --2- 172.21.15.73:0/1262012766 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f544415d3b0 0x7f544415d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:55.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.059+0000 7f544be59640 1 --2- 172.21.15.73:0/1262012766 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5430004d40 0x7f54440c6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:55.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.059+0000 7f544be59640 1 -- 172.21.15.73:0/1262012766 >> 172.21.15.73:0/1262012766 conn(0x7f54440d2da0 msgr2=0x7f544406e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:55.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.059+0000 7f544be59640 1 -- 172.21.15.73:0/1262012766 shutdown_connections 2024-04-26T20:50:55.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.059+0000 7f544be59640 1 -- 172.21.15.73:0/1262012766 wait complete. 2024-04-26T20:50:55.071 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:50:55.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.167+0000 7f2fccd0a640 1 Processor -- start 2024-04-26T20:50:55.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.167+0000 7f2fccd0a640 1 -- start start 2024-04-26T20:50:55.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.168+0000 7f2fccd0a640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2fc80d0d00 0x7f2fc80d11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:55.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.168+0000 7f2fccd0a640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fc80d0710 con 0x7f2fc80cf180 2024-04-26T20:50:55.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.168+0000 7f2fccd0a640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fc80d0430 con 0x7f2fc8160720 2024-04-26T20:50:55.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.168+0000 7f2fccd0a640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f2fc80d01e0 con 0x7f2fc80d0d00 2024-04-26T20:50:55.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.168+0000 7f2fc5d9b640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2fc80d0d00 0x7f2fc80d11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:55.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.168+0000 7f2fc5d9b640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2fc80d0d00 0x7f2fc80d11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49358/0 (socket says 172.21.15.73:49358) 2024-04-26T20:50:55.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.168+0000 7f2fc6d9d640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f2fc80cf180 0x7f2fc80b2570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33046/0 (socket says 172.21.15.73:33046) 2024-04-26T20:50:55.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.169+0000 7f2fc5d9b640 1 -- 172.21.15.73:0/3432708398 learned_addr learned my addr 172.21.15.73:0/3432708398 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:55.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.169+0000 7f2fc5d9b640 1 -- 172.21.15.73:0/3432708398 >> v1:172.21.15.120:6789/0 conn(0x7f2fc8160720 legacy=0x7f2fc80ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:55.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.169+0000 7f2fc5d9b640 1 -- 172.21.15.73:0/3432708398 >> v1:172.21.15.73:6789/0 conn(0x7f2fc80cf180 legacy=0x7f2fc80b2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:55.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.169+0000 7f2fc5d9b640 1 -- 172.21.15.73:0/3432708398 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fc80cdb40 con 0x7f2fc80d0d00 2024-04-26T20:50:55.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.169+0000 7f2fc5d9b640 1 --2- 172.21.15.73:0/3432708398 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2fc80d0d00 0x7f2fc80d11e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc009870 tx=0x7f2fbc02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb37c29a06b6d1ed server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:55.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.170+0000 7f2fc559a640 1 -- 172.21.15.73:0/3432708398 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc037070 con 0x7f2fc80d0d00 2024-04-26T20:50:55.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.170+0000 7f2fc559a640 1 -- 172.21.15.73:0/3432708398 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fbc02ae50 con 0x7f2fc80d0d00 2024-04-26T20:50:55.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.170+0000 7f2fc559a640 1 -- 172.21.15.73:0/3432708398 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc032860 con 0x7f2fc80d0d00 2024-04-26T20:50:55.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.170+0000 7f2fccd0a640 1 -- 172.21.15.73:0/3432708398 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2fc80d0d00 msgr2=0x7f2fc80d11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:55.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.170+0000 7f2fccd0a640 1 --2- 172.21.15.73:0/3432708398 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2fc80d0d00 0x7f2fc80d11e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc009870 tx=0x7f2fbc02a300 comp rx=0 tx=0).stop 2024-04-26T20:50:55.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.170+0000 7f2fccd0a640 1 -- 172.21.15.73:0/3432708398 shutdown_connections 2024-04-26T20:50:55.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.170+0000 7f2fccd0a640 1 --2- 172.21.15.73:0/3432708398 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2fc80d0d00 0x7f2fc80d11e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:55.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.170+0000 7f2fccd0a640 1 -- 172.21.15.73:0/3432708398 >> 172.21.15.73:0/3432708398 conn(0x7f2fc80d2e00 msgr2=0x7f2fc80d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:55.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.170+0000 7f2fccd0a640 1 -- 172.21.15.73:0/3432708398 shutdown_connections 2024-04-26T20:50:55.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.170+0000 7f2fccd0a640 1 -- 172.21.15.73:0/3432708398 wait complete. 2024-04-26T20:50:55.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.171+0000 7f2fccd0a640 1 Processor -- start 2024-04-26T20:50:55.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.171+0000 7f2fccd0a640 1 -- start start 2024-04-26T20:50:55.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.172+0000 7f2fccd0a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2fc80cf180 0x7f2fc8154070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:55.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.172+0000 7f2fccd0a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2fc80d0d00 0x7f2fc8151310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:55.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.172+0000 7f2fccd0a640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2fc8160720 0x7f2fc806e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:55.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.172+0000 7f2fccd0a640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc80ced30 con 0x7f2fc80cf180 2024-04-26T20:50:55.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.172+0000 7f2fccd0a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc80ce100 con 0x7f2fc80d0d00 2024-04-26T20:50:55.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.172+0000 7f2fccd0a640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f2fc8160b00 con 0x7f2fc8160720 2024-04-26T20:50:55.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.172+0000 7f2fc659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2fc80cf180 0x7f2fc8154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:55.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.172+0000 7f2fc659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2fc80cf180 0x7f2fc8154070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49372/0 (socket says 172.21.15.73:49372) 2024-04-26T20:50:55.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.172+0000 7f2fc659c640 1 -- 172.21.15.73:0/2260236234 learned_addr learned my addr 172.21.15.73:0/2260236234 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:55.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.173+0000 7f2fc659c640 1 -- 172.21.15.73:0/2260236234 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2fc8160720 msgr2=0x7f2fc806e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:55.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.173+0000 7f2fc659c640 1 --2- 172.21.15.73:0/2260236234 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2fc8160720 0x7f2fc806e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:55.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.173+0000 7f2fc5d9b640 1 --2- 172.21.15.73:0/2260236234 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2fc80d0d00 0x7f2fc8151310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:55.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.173+0000 7f2fc659c640 1 -- 172.21.15.73:0/2260236234 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2fc80d0d00 msgr2=0x7f2fc8151310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:55.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.173+0000 7f2fc659c640 1 --2- 172.21.15.73:0/2260236234 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2fc80d0d00 0x7f2fc8151310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:55.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.173+0000 7f2fc659c640 1 -- 172.21.15.73:0/2260236234 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fc806d3f0 con 0x7f2fc80cf180 2024-04-26T20:50:55.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.173+0000 7f2fc659c640 1 --2- 172.21.15.73:0/2260236234 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2fc80cf180 0x7f2fc8154070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2fb800c790 tx=0x7f2fb800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:55.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.173+0000 7f2fb6ffd640 1 -- 172.21.15.73:0/2260236234 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb8016020 con 0x7f2fc80cf180 2024-04-26T20:50:55.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.173+0000 7f2fb6ffd640 1 -- 172.21.15.73:0/2260236234 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fb8006d50 con 0x7f2fc80cf180 2024-04-26T20:50:55.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.173+0000 7f2fccd0a640 1 -- 172.21.15.73:0/2260236234 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fc806d6d0 con 0x7f2fc80cf180 2024-04-26T20:50:55.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.173+0000 7f2fb6ffd640 1 -- 172.21.15.73:0/2260236234 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb8003bb0 con 0x7f2fc80cf180 2024-04-26T20:50:55.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.173+0000 7f2fccd0a640 1 -- 172.21.15.73:0/2260236234 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fc806dc90 con 0x7f2fc80cf180 2024-04-26T20:50:55.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.174+0000 7f2fccd0a640 1 -- 172.21.15.73:0/2260236234 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fc80694f0 con 0x7f2fc80cf180 2024-04-26T20:50:55.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.174+0000 7f2fb6ffd640 1 -- 172.21.15.73:0/2260236234 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2fb800a7f0 con 0x7f2fc80cf180 2024-04-26T20:50:55.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.175+0000 7f2fb6ffd640 1 --2- 172.21.15.73:0/2260236234 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f2fa806c8f0 0x7f2fa806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:55.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.175+0000 7f2fb6ffd640 1 -- 172.21.15.73:0/2260236234 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f2fb808ba70 con 0x7f2fc80cf180 2024-04-26T20:50:55.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.175+0000 7f2fc5d9b640 1 --2- 172.21.15.73:0/2260236234 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f2fa806c8f0 0x7f2fa806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:55.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.175+0000 7f2fc5d9b640 1 --2- 172.21.15.73:0/2260236234 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f2fa806c8f0 0x7f2fa806edb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc034000 tx=0x7f2fbc02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:55.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.179+0000 7f2fb6ffd640 1 -- 172.21.15.73:0/2260236234 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2fb8020020 con 0x7f2fc80cf180 2024-04-26T20:50:55.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.355+0000 7f2fccd0a640 1 -- 172.21.15.73:0/2260236234 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2fc80b8d80 con 0x7f2fc80cf180 2024-04-26T20:50:55.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.356+0000 7f2fb6ffd640 1 -- 172.21.15.73:0/2260236234 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18514 (secure 0 0 0) 0x7f2fb8059bf0 con 0x7f2fc80cf180 2024-04-26T20:50:55.357 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:50:55.357 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":25,"fsid":"fbd3f548-b547-4713-856b-09f98f5d1764","created":"2024-04-26T20:50:07.876786+0000","modified":"2024-04-26T20:50:38.657510+0000","last_up_change":"2024-04-26T20:50:19.082741+0000","last_in_change":"2024-04-26T20:50:10.018400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:50:19.993731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:50:29.964420+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:50:30.920082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:50:34.650207+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"72d6f4f3-214a-4392-a02f-c33cbcab3c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6809","nonce":3461705124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6811","nonce":3461705124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6815","nonce":3461705124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6813","nonce":3461705124}]},"public_addr":"172.21.15.73:6809/3461705124","cluster_addr":"172.21.15.73:6811/3461705124","heartbeat_back_addr":"172.21.15.73:6815/3461705124","heartbeat_front_addr":"172.21.15.73:6813/3461705124","state":["exists","up"]},{"osd":1,"uuid":"786bf8ca-6ddc-4b3c-be6c-9152398bd5fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6817","nonce":2827619403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6819","nonce":2827619403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6825","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6827","nonce":2827619403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6821","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6823","nonce":2827619403}]},"public_addr":"172.21.15.73:6817/2827619403","cluster_addr":"172.21.15.73:6819/2827619403","heartbeat_back_addr":"172.21.15.73:6827/2827619403","heartbeat_front_addr":"172.21.15.73:6823/2827619403","state":["exists","up"]},{"osd":2,"uuid":"90963b1a-5437-4ef8-ad8b-6b74c534eeb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6801","nonce":2542681064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6803","nonce":2542681064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6808","nonce":2542681064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6805","nonce":2542681064}]},"public_addr":"172.21.15.73:6801/2542681064","cluster_addr":"172.21.15.73:6803/2542681064","heartbeat_back_addr":"172.21.15.73:6808/2542681064","heartbeat_front_addr":"172.21.15.73:6805/2542681064","state":["exists","up"]},{"osd":3,"uuid":"bd3d2148-6e38-41b4-a453-fe6e3b8b4ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6822","nonce":1604042718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6826","nonce":1604042718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6831","nonce":1604042718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6829","nonce":1604042718}]},"public_addr":"172.21.15.73:6822/1604042718","cluster_addr":"172.21.15.73:6826/1604042718","heartbeat_back_addr":"172.21.15.73:6831/1604042718","heartbeat_front_addr":"172.21.15.73:6829/1604042718","state":["exists","up"]},{"osd":4,"uuid":"e8668dc8-a23f-46d2-8311-cb882da08e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6817","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6819","nonce":3591055561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6824","nonce":3591055561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6831","nonce":3591055561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6828","nonce":3591055561}]},"public_addr":"172.21.15.120:6819/3591055561","cluster_addr":"172.21.15.120:6824/3591055561","heartbeat_back_addr":"172.21.15.120:6831/3591055561","heartbeat_front_addr":"172.21.15.120:6828/3591055561","state":["exists","up"]},{"osd":5,"uuid":"cbac0bdb-96bd-4852-bcce-e564495c3fcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6809","nonce":819450433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6811","nonce":819450433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6815","nonce":819450433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6813","nonce":819450433}]},"public_addr":"172.21.15.120:6809/819450433","cluster_addr":"172.21.15.120:6811/819450433","heartbeat_back_addr":"172.21.15.120:6815/819450433","heartbeat_front_addr":"172.21.15.120:6813/819450433","state":["exists","up"]},{"osd":6,"uuid":"1884b2a8-569f-4272-96a9-f25a09364dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6801","nonce":3946210727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6803","nonce":3946210727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6807","nonce":3946210727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6805","nonce":3946210727}]},"public_addr":"172.21.15.120:6801/3946210727","cluster_addr":"172.21.15.120:6803/3946210727","heartbeat_back_addr":"172.21.15.120:6807/3946210727","heartbeat_front_addr":"172.21.15.120:6805/3946210727","state":["exists","up"]},{"osd":7,"uuid":"c0c3bf6d-a4c5-4f1e-aa2d-330951830666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6818","nonce":2915574674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6821","nonce":2915574674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6827","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6829","nonce":2915574674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6823","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6825","nonce":2915574674}]},"public_addr":"172.21.15.120:6818/2915574674","cluster_addr":"172.21.15.120:6821/2915574674","heartbeat_back_addr":"172.21.15.120:6829/2915574674","heartbeat_front_addr":"172.21.15.120:6825/2915574674","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.218216+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.208710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.112705+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.182789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.164324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.174212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.225396+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.261559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:50:55.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.358+0000 7f2fccd0a640 1 -- 172.21.15.73:0/2260236234 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f2fa806c8f0 msgr2=0x7f2fa806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:55.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.358+0000 7f2fccd0a640 1 --2- 172.21.15.73:0/2260236234 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f2fa806c8f0 0x7f2fa806edb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc034000 tx=0x7f2fbc02e000 comp rx=0 tx=0).stop 2024-04-26T20:50:55.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.358+0000 7f2fccd0a640 1 -- 172.21.15.73:0/2260236234 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2fc80cf180 msgr2=0x7f2fc8154070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:55.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.358+0000 7f2fccd0a640 1 --2- 172.21.15.73:0/2260236234 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2fc80cf180 0x7f2fc8154070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2fb800c790 tx=0x7f2fb800cc50 comp rx=0 tx=0).stop 2024-04-26T20:50:55.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.358+0000 7f2fccd0a640 1 -- 172.21.15.73:0/2260236234 shutdown_connections 2024-04-26T20:50:55.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.358+0000 7f2fccd0a640 1 --2- 172.21.15.73:0/2260236234 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f2fa806c8f0 0x7f2fa806edb0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:55.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.358+0000 7f2fccd0a640 1 --2- 172.21.15.73:0/2260236234 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2fc8160720 0x7f2fc806e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:55.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.358+0000 7f2fccd0a640 1 --2- 172.21.15.73:0/2260236234 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2fc80d0d00 0x7f2fc8151310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:55.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.358+0000 7f2fccd0a640 1 --2- 172.21.15.73:0/2260236234 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2fc80cf180 0x7f2fc8154070 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:55.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.358+0000 7f2fccd0a640 1 -- 172.21.15.73:0/2260236234 >> 172.21.15.73:0/2260236234 conn(0x7f2fc80d2e00 msgr2=0x7f2fc80b1630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:55.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.358+0000 7f2fccd0a640 1 -- 172.21.15.73:0/2260236234 shutdown_connections 2024-04-26T20:50:55.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.358+0000 7f2fccd0a640 1 -- 172.21.15.73:0/2260236234 wait complete. 2024-04-26T20:50:55.370 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T20:50:55.371 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:50:55.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.466+0000 7fc1bedb4640 1 Processor -- start 2024-04-26T20:50:55.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.466+0000 7fc1bedb4640 1 -- start start 2024-04-26T20:50:55.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.467+0000 7fc1bedb4640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc1b8162880 0x7fc1b80d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:55.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.467+0000 7fc1bedb4640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fc1b80b5f20 0x7fc1b80d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:55.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.467+0000 7fc1bedb4640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1b80b7e50 con 0x7fc1b80d39f0 2024-04-26T20:50:55.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.467+0000 7fc1bedb4640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7fc1b80d3e40 con 0x7fc1b8162880 2024-04-26T20:50:55.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.467+0000 7fc1bedb4640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fc1b80d0580 con 0x7fc1b80b5f20 2024-04-26T20:50:55.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.467+0000 7fc1bcb50640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc1b8162880 0x7fc1b80d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:55.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.467+0000 7fc1bcb50640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc1b8162880 0x7fc1b80d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49378/0 (socket says 172.21.15.73:49378) 2024-04-26T20:50:55.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.467+0000 7fc1bcb50640 1 -- 172.21.15.73:0/3867082801 learned_addr learned my addr 172.21.15.73:0/3867082801 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:55.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.467+0000 7fc1b7fff640 1 --2- 172.21.15.73:0/3867082801 >> v2:172.21.15.120:3300/0 conn(0x7fc1b80b5f20 0x7fc1b80d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:55.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.468+0000 7fc1bcb50640 1 -- 172.21.15.73:0/3867082801 >> v2:172.21.15.120:3300/0 conn(0x7fc1b80b5f20 msgr2=0x7fc1b80d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:55.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.468+0000 7fc1bcb50640 1 --2- 172.21.15.73:0/3867082801 >> v2:172.21.15.120:3300/0 conn(0x7fc1b80b5f20 0x7fc1b80d34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:55.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.468+0000 7fc1bcb50640 1 -- 172.21.15.73:0/3867082801 >> v1:172.21.15.120:6789/0 conn(0x7fc1b80d39f0 legacy=0x7fc1b80b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:55.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.468+0000 7fc1bcb50640 1 -- 172.21.15.73:0/3867082801 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1b80d1800 con 0x7fc1b8162880 2024-04-26T20:50:55.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.468+0000 7fc1bcb50640 1 --2- 172.21.15.73:0/3867082801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1b8162880 0x7fc1b80d2cb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc1b808d3d0 tx=0x7fc1a802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a4a0776fb39709c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:55.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.468+0000 7fc1b77fe640 1 -- 172.21.15.73:0/3867082801 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a8037070 con 0x7fc1b8162880 2024-04-26T20:50:55.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.468+0000 7fc1b77fe640 1 -- 172.21.15.73:0/3867082801 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1a8032440 con 0x7fc1b8162880 2024-04-26T20:50:55.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.468+0000 7fc1b77fe640 1 -- 172.21.15.73:0/3867082801 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a8032810 con 0x7fc1b8162880 2024-04-26T20:50:55.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.469+0000 7fc1bedb4640 1 -- 172.21.15.73:0/3867082801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1b8162880 msgr2=0x7fc1b80d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:55.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.469+0000 7fc1bedb4640 1 --2- 172.21.15.73:0/3867082801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1b8162880 0x7fc1b80d2cb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc1b808d3d0 tx=0x7fc1a802a230 comp rx=0 tx=0).stop 2024-04-26T20:50:55.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.469+0000 7fc1bedb4640 1 -- 172.21.15.73:0/3867082801 shutdown_connections 2024-04-26T20:50:55.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.469+0000 7fc1bedb4640 1 --2- 172.21.15.73:0/3867082801 >> v2:172.21.15.120:3300/0 conn(0x7fc1b80b5f20 0x7fc1b80d34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:55.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.469+0000 7fc1bedb4640 1 --2- 172.21.15.73:0/3867082801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1b8162880 0x7fc1b80d2cb0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:55.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.469+0000 7fc1bedb4640 1 -- 172.21.15.73:0/3867082801 >> 172.21.15.73:0/3867082801 conn(0x7fc1b80d4f60 msgr2=0x7fc1b80d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:55.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.469+0000 7fc1bedb4640 1 -- 172.21.15.73:0/3867082801 shutdown_connections 2024-04-26T20:50:55.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.469+0000 7fc1bedb4640 1 -- 172.21.15.73:0/3867082801 wait complete. 2024-04-26T20:50:55.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.470+0000 7fc1bedb4640 1 Processor -- start 2024-04-26T20:50:55.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.470+0000 7fc1bedb4640 1 -- start start 2024-04-26T20:50:55.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.471+0000 7fc1bedb4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1b80b5f20 0x7fc1b814d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:55.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.471+0000 7fc1bedb4640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc1b80d39f0 0x7fc1b80ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:55.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.471+0000 7fc1bedb4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1b8162880 0x7fc1b8160420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:55.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.471+0000 7fc1bedb4640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b80d0d90 con 0x7fc1b80b5f20 2024-04-26T20:50:55.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.471+0000 7fc1bedb4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b80d0ab0 con 0x7fc1b8162880 2024-04-26T20:50:55.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.471+0000 7fc1bedb4640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fc1b8162c60 con 0x7fc1b80d39f0 2024-04-26T20:50:55.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.471+0000 7fc1bcb50640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1b80b5f20 0x7fc1b814d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:55.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.471+0000 7fc1bcb50640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1b80b5f20 0x7fc1b814d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49380/0 (socket says 172.21.15.73:49380) 2024-04-26T20:50:55.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.471+0000 7fc1bcb50640 1 -- 172.21.15.73:0/3682486504 learned_addr learned my addr 172.21.15.73:0/3682486504 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:55.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.471+0000 7fc1bd351640 1 --2- 172.21.15.73:0/3682486504 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1b8162880 0x7fc1b8160420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:55.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.471+0000 7fc1b7fff640 1 --2- 172.21.15.73:0/3682486504 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc1b80d39f0 0x7fc1b80ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:55.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.471+0000 7fc1bcb50640 1 -- 172.21.15.73:0/3682486504 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc1b80d39f0 msgr2=0x7fc1b80ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:55.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.471+0000 7fc1bcb50640 1 --2- 172.21.15.73:0/3682486504 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc1b80d39f0 0x7fc1b80ce5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:55.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.471+0000 7fc1bcb50640 1 -- 172.21.15.73:0/3682486504 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1b8162880 msgr2=0x7fc1b8160420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:55.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.471+0000 7fc1bcb50640 1 --2- 172.21.15.73:0/3682486504 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1b8162880 0x7fc1b8160420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:55.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.471+0000 7fc1bcb50640 1 -- 172.21.15.73:0/3682486504 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1b8159a10 con 0x7fc1b80b5f20 2024-04-26T20:50:55.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.471+0000 7fc1b7fff640 1 --2- 172.21.15.73:0/3682486504 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc1b80d39f0 0x7fc1b80ce5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:50:55.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.471+0000 7fc1bcb50640 1 --2- 172.21.15.73:0/3682486504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1b80b5f20 0x7fc1b814d3f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8000c00 tx=0x7fc1a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:55.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.472+0000 7fc1b57fa640 1 -- 172.21.15.73:0/3682486504 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a8041020 con 0x7fc1b80b5f20 2024-04-26T20:50:55.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.472+0000 7fc1b57fa640 1 -- 172.21.15.73:0/3682486504 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1a80306c0 con 0x7fc1b80b5f20 2024-04-26T20:50:55.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.472+0000 7fc1bedb4640 1 -- 172.21.15.73:0/3682486504 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1b815a0d0 con 0x7fc1b80b5f20 2024-04-26T20:50:55.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.472+0000 7fc1b57fa640 1 -- 172.21.15.73:0/3682486504 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a8037040 con 0x7fc1b80b5f20 2024-04-26T20:50:55.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.472+0000 7fc1bedb4640 1 -- 172.21.15.73:0/3682486504 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1b815a690 con 0x7fc1b80b5f20 2024-04-26T20:50:55.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.473+0000 7fc1bedb4640 1 -- 172.21.15.73:0/3682486504 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1b80694f0 con 0x7fc1b80b5f20 2024-04-26T20:50:55.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.473+0000 7fc1b57fa640 1 -- 172.21.15.73:0/3682486504 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc1a8044020 con 0x7fc1b80b5f20 2024-04-26T20:50:55.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.473+0000 7fc1b57fa640 1 --2- 172.21.15.73:0/3682486504 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc1a406c8f0 0x7fc1a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:55.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.474+0000 7fc1b7fff640 1 --2- 172.21.15.73:0/3682486504 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc1a406c8f0 0x7fc1a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:55.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.474+0000 7fc1b57fa640 1 -- 172.21.15.73:0/3682486504 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc1a80ac6c0 con 0x7fc1b80b5f20 2024-04-26T20:50:55.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.474+0000 7fc1b7fff640 1 --2- 172.21.15.73:0/3682486504 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc1a406c8f0 0x7fc1a406edb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac006fd0 tx=0x7fc1ac008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:55.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.478+0000 7fc1b57fa640 1 -- 172.21.15.73:0/3682486504 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc1a804b030 con 0x7fc1b80b5f20 2024-04-26T20:50:55.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.653+0000 7fc1bedb4640 1 -- 172.21.15.73:0/3682486504 --> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc1b8159e80 con 0x7fc1a406c8f0 2024-04-26T20:50:55.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.661+0000 7fc1b57fa640 1 -- 172.21.15.73:0/3682486504 <== mgr.4107 v2:172.21.15.73:6832/666905549 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7fc1b8159e80 con 0x7fc1a406c8f0 2024-04-26T20:50:55.666 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:50:55.668 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-26T20:50:55.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.664+0000 7fc1bedb4640 1 -- 172.21.15.73:0/3682486504 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc1a406c8f0 msgr2=0x7fc1a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:55.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.664+0000 7fc1bedb4640 1 --2- 172.21.15.73:0/3682486504 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc1a406c8f0 0x7fc1a406edb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac006fd0 tx=0x7fc1ac008040 comp rx=0 tx=0).stop 2024-04-26T20:50:55.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.664+0000 7fc1bedb4640 1 -- 172.21.15.73:0/3682486504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1b80b5f20 msgr2=0x7fc1b814d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:55.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.664+0000 7fc1bedb4640 1 --2- 172.21.15.73:0/3682486504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1b80b5f20 0x7fc1b814d3f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8000c00 tx=0x7fc1a80046c0 comp rx=0 tx=0).stop 2024-04-26T20:50:55.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.665+0000 7fc1bedb4640 1 -- 172.21.15.73:0/3682486504 shutdown_connections 2024-04-26T20:50:55.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.665+0000 7fc1bedb4640 1 --2- 172.21.15.73:0/3682486504 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc1a406c8f0 0x7fc1a406edb0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:55.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.665+0000 7fc1bedb4640 1 --2- 172.21.15.73:0/3682486504 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1b8162880 0x7fc1b8160420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:55.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.665+0000 7fc1bedb4640 1 --2- 172.21.15.73:0/3682486504 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc1b80d39f0 0x7fc1b80ce5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:55.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.665+0000 7fc1bedb4640 1 --2- 172.21.15.73:0/3682486504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1b80b5f20 0x7fc1b814d3f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:55.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.665+0000 7fc1bedb4640 1 -- 172.21.15.73:0/3682486504 >> 172.21.15.73:0/3682486504 conn(0x7fc1b80d4f60 msgr2=0x7fc1b80b69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:55.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.665+0000 7fc1bedb4640 1 -- 172.21.15.73:0/3682486504 shutdown_connections 2024-04-26T20:50:55.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.665+0000 7fc1bedb4640 1 -- 172.21.15.73:0/3682486504 wait complete. 2024-04-26T20:50:55.677 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2024-04-26T20:50:53.983620+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219468,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755252,"statfs":{"total":773094113280,"available":772869378048,"internally_reserved":0,"allocated":3383296,"data_stored":2049160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002786"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803145+0000","last_change":"2024-04-26T20:50:31.630770+0000","last_active":"2024-04-26T20:50:38.803145+0000","last_peered":"2024-04-26T20:50:38.803145+0000","last_clean":"2024-04-26T20:50:38.803145+0000","last_became_active":"2024-04-26T20:50:31.630518+0000","last_became_peered":"2024-04-26T20:50:31.630518+0000","last_unstale":"2024-04-26T20:50:38.803145+0000","last_undegraded":"2024-04-26T20:50:38.803145+0000","last_fullsized":"2024-04-26T20:50:38.803145+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:20:49.853735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803210+0000","last_change":"2024-04-26T20:50:32.649145+0000","last_active":"2024-04-26T20:50:38.803210+0000","last_peered":"2024-04-26T20:50:38.803210+0000","last_clean":"2024-04-26T20:50:38.803210+0000","last_became_active":"2024-04-26T20:50:32.648873+0000","last_became_peered":"2024-04-26T20:50:32.648873+0000","last_unstale":"2024-04-26T20:50:38.803210+0000","last_undegraded":"2024-04-26T20:50:38.803210+0000","last_fullsized":"2024-04-26T20:50:38.803210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:16:25.308786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802643+0000","last_change":"2024-04-26T20:50:37.296160+0000","last_active":"2024-04-26T20:50:38.802643+0000","last_peered":"2024-04-26T20:50:38.802643+0000","last_clean":"2024-04-26T20:50:38.802643+0000","last_became_active":"2024-04-26T20:50:37.295441+0000","last_became_peered":"2024-04-26T20:50:37.295441+0000","last_unstale":"2024-04-26T20:50:38.802643+0000","last_undegraded":"2024-04-26T20:50:38.802643+0000","last_fullsized":"2024-04-26T20:50:38.802643+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:20:48.160586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803389+0000","last_change":"2024-04-26T20:50:31.634354+0000","last_active":"2024-04-26T20:50:38.803389+0000","last_peered":"2024-04-26T20:50:38.803389+0000","last_clean":"2024-04-26T20:50:38.803389+0000","last_became_active":"2024-04-26T20:50:31.634084+0000","last_became_peered":"2024-04-26T20:50:31.634084+0000","last_unstale":"2024-04-26T20:50:38.803389+0000","last_undegraded":"2024-04-26T20:50:38.803389+0000","last_fullsized":"2024-04-26T20:50:38.803389+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:49:28.099373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802875+0000","last_change":"2024-04-26T20:50:32.645829+0000","last_active":"2024-04-26T20:50:38.802875+0000","last_peered":"2024-04-26T20:50:38.802875+0000","last_clean":"2024-04-26T20:50:38.802875+0000","last_became_active":"2024-04-26T20:50:32.645589+0000","last_became_peered":"2024-04-26T20:50:32.645589+0000","last_unstale":"2024-04-26T20:50:38.802875+0000","last_undegraded":"2024-04-26T20:50:38.802875+0000","last_fullsized":"2024-04-26T20:50:38.802875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:36.651960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661856+0000","last_change":"2024-04-26T20:50:36.675389+0000","last_active":"2024-04-26T20:50:38.661856+0000","last_peered":"2024-04-26T20:50:38.661856+0000","last_clean":"2024-04-26T20:50:38.661856+0000","last_became_active":"2024-04-26T20:50:36.675027+0000","last_became_peered":"2024-04-26T20:50:36.675027+0000","last_unstale":"2024-04-26T20:50:38.661856+0000","last_undegraded":"2024-04-26T20:50:38.661856+0000","last_fullsized":"2024-04-26T20:50:38.661856+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:13:14.645783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800942+0000","last_change":"2024-04-26T20:50:31.631388+0000","last_active":"2024-04-26T20:50:38.800942+0000","last_peered":"2024-04-26T20:50:38.800942+0000","last_clean":"2024-04-26T20:50:38.800942+0000","last_became_active":"2024-04-26T20:50:31.631121+0000","last_became_peered":"2024-04-26T20:50:31.631121+0000","last_unstale":"2024-04-26T20:50:38.800942+0000","last_undegraded":"2024-04-26T20:50:38.800942+0000","last_fullsized":"2024-04-26T20:50:38.800942+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:46:19.858595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800306+0000","last_change":"2024-04-26T20:50:32.644708+0000","last_active":"2024-04-26T20:50:38.800306+0000","last_peered":"2024-04-26T20:50:38.800306+0000","last_clean":"2024-04-26T20:50:38.800306+0000","last_became_active":"2024-04-26T20:50:32.644440+0000","last_became_peered":"2024-04-26T20:50:32.644440+0000","last_unstale":"2024-04-26T20:50:38.800306+0000","last_undegraded":"2024-04-26T20:50:38.800306+0000","last_fullsized":"2024-04-26T20:50:38.800306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:40:19.711175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.662138+0000","last_change":"2024-04-26T20:50:36.676949+0000","last_active":"2024-04-26T20:50:38.662138+0000","last_peered":"2024-04-26T20:50:38.662138+0000","last_clean":"2024-04-26T20:50:38.662138+0000","last_became_active":"2024-04-26T20:50:36.676533+0000","last_became_peered":"2024-04-26T20:50:36.676533+0000","last_unstale":"2024-04-26T20:50:38.662138+0000","last_undegraded":"2024-04-26T20:50:38.662138+0000","last_fullsized":"2024-04-26T20:50:38.662138+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:59:08.998553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800828+0000","last_change":"2024-04-26T20:50:31.632017+0000","last_active":"2024-04-26T20:50:38.800828+0000","last_peered":"2024-04-26T20:50:38.800828+0000","last_clean":"2024-04-26T20:50:38.800828+0000","last_became_active":"2024-04-26T20:50:31.631717+0000","last_became_peered":"2024-04-26T20:50:31.631717+0000","last_unstale":"2024-04-26T20:50:38.800828+0000","last_undegraded":"2024-04-26T20:50:38.800828+0000","last_fullsized":"2024-04-26T20:50:38.800828+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:32:17.604576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800366+0000","last_change":"2024-04-26T20:50:32.644081+0000","last_active":"2024-04-26T20:50:38.800366+0000","last_peered":"2024-04-26T20:50:38.800366+0000","last_clean":"2024-04-26T20:50:38.800366+0000","last_became_active":"2024-04-26T20:50:32.643809+0000","last_became_peered":"2024-04-26T20:50:32.643809+0000","last_unstale":"2024-04-26T20:50:38.800366+0000","last_undegraded":"2024-04-26T20:50:38.800366+0000","last_fullsized":"2024-04-26T20:50:38.800366+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:16:25.308786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.662248+0000","last_change":"2024-04-26T20:50:36.679187+0000","last_active":"2024-04-26T20:50:38.662248+0000","last_peered":"2024-04-26T20:50:38.662248+0000","last_clean":"2024-04-26T20:50:38.662248+0000","last_became_active":"2024-04-26T20:50:36.678782+0000","last_became_peered":"2024-04-26T20:50:36.678782+0000","last_unstale":"2024-04-26T20:50:38.662248+0000","last_undegraded":"2024-04-26T20:50:38.662248+0000","last_fullsized":"2024-04-26T20:50:38.662248+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:12.140650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.717057+0000","last_change":"2024-04-26T20:50:32.646429+0000","last_active":"2024-04-26T20:50:38.717057+0000","last_peered":"2024-04-26T20:50:38.717057+0000","last_clean":"2024-04-26T20:50:38.717057+0000","last_became_active":"2024-04-26T20:50:32.646134+0000","last_became_peered":"2024-04-26T20:50:32.646134+0000","last_unstale":"2024-04-26T20:50:38.717057+0000","last_undegraded":"2024-04-26T20:50:38.717057+0000","last_fullsized":"2024-04-26T20:50:38.717057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:09:36.737379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800514+0000","last_change":"2024-04-26T20:50:31.631250+0000","last_active":"2024-04-26T20:50:38.800514+0000","last_peered":"2024-04-26T20:50:38.800514+0000","last_clean":"2024-04-26T20:50:38.800514+0000","last_became_active":"2024-04-26T20:50:31.630277+0000","last_became_peered":"2024-04-26T20:50:31.630277+0000","last_unstale":"2024-04-26T20:50:38.800514+0000","last_undegraded":"2024-04-26T20:50:38.800514+0000","last_fullsized":"2024-04-26T20:50:38.800514+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:06:39.630339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.662616+0000","last_change":"2024-04-26T20:50:36.676056+0000","last_active":"2024-04-26T20:50:38.662616+0000","last_peered":"2024-04-26T20:50:38.662616+0000","last_clean":"2024-04-26T20:50:38.662616+0000","last_became_active":"2024-04-26T20:50:36.675706+0000","last_became_peered":"2024-04-26T20:50:36.675706+0000","last_unstale":"2024-04-26T20:50:38.662616+0000","last_undegraded":"2024-04-26T20:50:38.662616+0000","last_fullsized":"2024-04-26T20:50:38.662616+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:13:08.850173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802373+0000","last_change":"2024-04-26T20:50:32.650473+0000","last_active":"2024-04-26T20:50:38.802373+0000","last_peered":"2024-04-26T20:50:38.802373+0000","last_clean":"2024-04-26T20:50:38.802373+0000","last_became_active":"2024-04-26T20:50:32.649711+0000","last_became_peered":"2024-04-26T20:50:32.649711+0000","last_unstale":"2024-04-26T20:50:38.802373+0000","last_undegraded":"2024-04-26T20:50:38.802373+0000","last_fullsized":"2024-04-26T20:50:38.802373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:36.651960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661208+0000","last_change":"2024-04-26T20:50:31.629572+0000","last_active":"2024-04-26T20:50:38.661208+0000","last_peered":"2024-04-26T20:50:38.661208+0000","last_clean":"2024-04-26T20:50:38.661208+0000","last_became_active":"2024-04-26T20:50:31.629287+0000","last_became_peered":"2024-04-26T20:50:31.629287+0000","last_unstale":"2024-04-26T20:50:38.661208+0000","last_undegraded":"2024-04-26T20:50:38.661208+0000","last_fullsized":"2024-04-26T20:50:38.661208+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:36.499651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.662538+0000","last_change":"2024-04-26T20:50:36.681910+0000","last_active":"2024-04-26T20:50:38.662538+0000","last_peered":"2024-04-26T20:50:38.662538+0000","last_clean":"2024-04-26T20:50:38.662538+0000","last_became_active":"2024-04-26T20:50:36.681432+0000","last_became_peered":"2024-04-26T20:50:36.681432+0000","last_unstale":"2024-04-26T20:50:38.662538+0000","last_undegraded":"2024-04-26T20:50:38.662538+0000","last_fullsized":"2024-04-26T20:50:38.662538+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:27:18.035795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802910+0000","last_change":"2024-04-26T20:50:32.647018+0000","last_active":"2024-04-26T20:50:38.802910+0000","last_peered":"2024-04-26T20:50:38.802910+0000","last_clean":"2024-04-26T20:50:38.802910+0000","last_became_active":"2024-04-26T20:50:32.646748+0000","last_became_peered":"2024-04-26T20:50:32.646748+0000","last_unstale":"2024-04-26T20:50:38.802910+0000","last_undegraded":"2024-04-26T20:50:38.802910+0000","last_fullsized":"2024-04-26T20:50:38.802910+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:56:51.926886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.660717+0000","last_change":"2024-04-26T20:50:31.631760+0000","last_active":"2024-04-26T20:50:38.660717+0000","last_peered":"2024-04-26T20:50:38.660717+0000","last_clean":"2024-04-26T20:50:38.660717+0000","last_became_active":"2024-04-26T20:50:31.629437+0000","last_became_peered":"2024-04-26T20:50:31.629437+0000","last_unstale":"2024-04-26T20:50:38.660717+0000","last_undegraded":"2024-04-26T20:50:38.660717+0000","last_fullsized":"2024-04-26T20:50:38.660717+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:23:29.866794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661242+0000","last_change":"2024-04-26T20:50:37.295810+0000","last_active":"2024-04-26T20:50:38.661242+0000","last_peered":"2024-04-26T20:50:38.661242+0000","last_clean":"2024-04-26T20:50:38.661242+0000","last_became_active":"2024-04-26T20:50:37.295425+0000","last_became_peered":"2024-04-26T20:50:37.295425+0000","last_unstale":"2024-04-26T20:50:38.661242+0000","last_undegraded":"2024-04-26T20:50:38.661242+0000","last_fullsized":"2024-04-26T20:50:38.661242+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:31:21.850102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.662187+0000","last_change":"2024-04-26T20:50:32.638584+0000","last_active":"2024-04-26T20:50:38.662187+0000","last_peered":"2024-04-26T20:50:38.662187+0000","last_clean":"2024-04-26T20:50:38.662187+0000","last_became_active":"2024-04-26T20:50:32.637734+0000","last_became_peered":"2024-04-26T20:50:32.637734+0000","last_unstale":"2024-04-26T20:50:38.662187+0000","last_undegraded":"2024-04-26T20:50:38.662187+0000","last_fullsized":"2024-04-26T20:50:38.662187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:37.504098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800397+0000","last_change":"2024-04-26T20:50:31.634926+0000","last_active":"2024-04-26T20:50:38.800397+0000","last_peered":"2024-04-26T20:50:38.800397+0000","last_clean":"2024-04-26T20:50:38.800397+0000","last_became_active":"2024-04-26T20:50:31.634278+0000","last_became_peered":"2024-04-26T20:50:31.634278+0000","last_unstale":"2024-04-26T20:50:38.800397+0000","last_undegraded":"2024-04-26T20:50:38.800397+0000","last_fullsized":"2024-04-26T20:50:38.800397+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:36.499651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.662287+0000","last_change":"2024-04-26T20:50:36.678377+0000","last_active":"2024-04-26T20:50:38.662287+0000","last_peered":"2024-04-26T20:50:38.662287+0000","last_clean":"2024-04-26T20:50:38.662287+0000","last_became_active":"2024-04-26T20:50:36.677946+0000","last_became_peered":"2024-04-26T20:50:36.677946+0000","last_unstale":"2024-04-26T20:50:38.662287+0000","last_undegraded":"2024-04-26T20:50:38.662287+0000","last_fullsized":"2024-04-26T20:50:38.662287+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:31:21.850102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803961+0000","last_change":"2024-04-26T20:50:32.637345+0000","last_active":"2024-04-26T20:50:38.803961+0000","last_peered":"2024-04-26T20:50:38.803961+0000","last_clean":"2024-04-26T20:50:38.803961+0000","last_became_active":"2024-04-26T20:50:32.637071+0000","last_became_peered":"2024-04-26T20:50:32.637071+0000","last_unstale":"2024-04-26T20:50:38.803961+0000","last_undegraded":"2024-04-26T20:50:38.803961+0000","last_fullsized":"2024-04-26T20:50:38.803961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:47:14.715739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.717764+0000","last_change":"2024-04-26T20:50:31.631119+0000","last_active":"2024-04-26T20:50:38.717764+0000","last_peered":"2024-04-26T20:50:38.717764+0000","last_clean":"2024-04-26T20:50:38.717764+0000","last_became_active":"2024-04-26T20:50:31.630825+0000","last_became_peered":"2024-04-26T20:50:31.630825+0000","last_unstale":"2024-04-26T20:50:38.717764+0000","last_undegraded":"2024-04-26T20:50:38.717764+0000","last_fullsized":"2024-04-26T20:50:38.717764+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:26.601021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661961+0000","last_change":"2024-04-26T20:50:37.295783+0000","last_active":"2024-04-26T20:50:38.661961+0000","last_peered":"2024-04-26T20:50:38.661961+0000","last_clean":"2024-04-26T20:50:38.661961+0000","last_became_active":"2024-04-26T20:50:37.295382+0000","last_became_peered":"2024-04-26T20:50:37.295382+0000","last_unstale":"2024-04-26T20:50:38.661961+0000","last_undegraded":"2024-04-26T20:50:38.661961+0000","last_fullsized":"2024-04-26T20:50:38.661961+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:58:22.587813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803938+0000","last_change":"2024-04-26T20:50:32.649792+0000","last_active":"2024-04-26T20:50:38.803938+0000","last_peered":"2024-04-26T20:50:38.803938+0000","last_clean":"2024-04-26T20:50:38.803938+0000","last_became_active":"2024-04-26T20:50:32.649510+0000","last_became_peered":"2024-04-26T20:50:32.649510+0000","last_unstale":"2024-04-26T20:50:38.803938+0000","last_undegraded":"2024-04-26T20:50:38.803938+0000","last_fullsized":"2024-04-26T20:50:38.803938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:35.639518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661244+0000","last_change":"2024-04-26T20:50:31.631623+0000","last_active":"2024-04-26T20:50:38.661244+0000","last_peered":"2024-04-26T20:50:38.661244+0000","last_clean":"2024-04-26T20:50:38.661244+0000","last_became_active":"2024-04-26T20:50:31.631312+0000","last_became_peered":"2024-04-26T20:50:31.631312+0000","last_unstale":"2024-04-26T20:50:38.661244+0000","last_undegraded":"2024-04-26T20:50:38.661244+0000","last_fullsized":"2024-04-26T20:50:38.661244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:46:19.858595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802802+0000","last_change":"2024-04-26T20:50:37.295508+0000","last_active":"2024-04-26T20:50:38.802802+0000","last_peered":"2024-04-26T20:50:38.802802+0000","last_clean":"2024-04-26T20:50:38.802802+0000","last_became_active":"2024-04-26T20:50:37.294858+0000","last_became_peered":"2024-04-26T20:50:37.294858+0000","last_unstale":"2024-04-26T20:50:38.802802+0000","last_undegraded":"2024-04-26T20:50:38.802802+0000","last_fullsized":"2024-04-26T20:50:38.802802+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:42:14.026063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661292+0000","last_change":"2024-04-26T20:50:32.640053+0000","last_active":"2024-04-26T20:50:38.661292+0000","last_peered":"2024-04-26T20:50:38.661292+0000","last_clean":"2024-04-26T20:50:38.661292+0000","last_became_active":"2024-04-26T20:50:32.639778+0000","last_became_peered":"2024-04-26T20:50:32.639778+0000","last_unstale":"2024-04-26T20:50:38.661292+0000","last_undegraded":"2024-04-26T20:50:38.661292+0000","last_fullsized":"2024-04-26T20:50:38.661292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:09:36.737379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.801056+0000","last_change":"2024-04-26T20:50:31.634409+0000","last_active":"2024-04-26T20:50:38.801056+0000","last_peered":"2024-04-26T20:50:38.801056+0000","last_clean":"2024-04-26T20:50:38.801056+0000","last_became_active":"2024-04-26T20:50:31.634103+0000","last_became_peered":"2024-04-26T20:50:31.634103+0000","last_unstale":"2024-04-26T20:50:38.801056+0000","last_undegraded":"2024-04-26T20:50:38.801056+0000","last_fullsized":"2024-04-26T20:50:38.801056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:40:15.520084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803602+0000","last_change":"2024-04-26T20:50:37.292899+0000","last_active":"2024-04-26T20:50:38.803602+0000","last_peered":"2024-04-26T20:50:38.803602+0000","last_clean":"2024-04-26T20:50:38.803602+0000","last_became_active":"2024-04-26T20:50:37.292379+0000","last_became_peered":"2024-04-26T20:50:37.292379+0000","last_unstale":"2024-04-26T20:50:38.803602+0000","last_undegraded":"2024-04-26T20:50:38.803602+0000","last_fullsized":"2024-04-26T20:50:38.803602+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:28:04.995069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800475+0000","last_change":"2024-04-26T20:50:32.649145+0000","last_active":"2024-04-26T20:50:38.800475+0000","last_peered":"2024-04-26T20:50:38.800475+0000","last_clean":"2024-04-26T20:50:38.800475+0000","last_became_active":"2024-04-26T20:50:32.648860+0000","last_became_peered":"2024-04-26T20:50:32.648860+0000","last_unstale":"2024-04-26T20:50:38.800475+0000","last_undegraded":"2024-04-26T20:50:38.800475+0000","last_fullsized":"2024-04-26T20:50:38.800475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:45:44.950496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661412+0000","last_change":"2024-04-26T20:50:31.627675+0000","last_active":"2024-04-26T20:50:38.661412+0000","last_peered":"2024-04-26T20:50:38.661412+0000","last_clean":"2024-04-26T20:50:38.661412+0000","last_became_active":"2024-04-26T20:50:31.627373+0000","last_became_peered":"2024-04-26T20:50:31.627373+0000","last_unstale":"2024-04-26T20:50:38.661412+0000","last_undegraded":"2024-04-26T20:50:38.661412+0000","last_fullsized":"2024-04-26T20:50:38.661412+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:26.601021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.660956+0000","last_change":"2024-04-26T20:50:36.679728+0000","last_active":"2024-04-26T20:50:38.660956+0000","last_peered":"2024-04-26T20:50:38.660956+0000","last_clean":"2024-04-26T20:50:38.660956+0000","last_became_active":"2024-04-26T20:50:36.678881+0000","last_became_peered":"2024-04-26T20:50:36.678881+0000","last_unstale":"2024-04-26T20:50:38.660956+0000","last_undegraded":"2024-04-26T20:50:38.660956+0000","last_fullsized":"2024-04-26T20:50:38.660956+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:59:08.998553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.662645+0000","last_change":"2024-04-26T20:50:32.633552+0000","last_active":"2024-04-26T20:50:38.662645+0000","last_peered":"2024-04-26T20:50:38.662645+0000","last_clean":"2024-04-26T20:50:38.662645+0000","last_became_active":"2024-04-26T20:50:32.632649+0000","last_became_peered":"2024-04-26T20:50:32.632649+0000","last_unstale":"2024-04-26T20:50:38.662645+0000","last_undegraded":"2024-04-26T20:50:38.662645+0000","last_fullsized":"2024-04-26T20:50:38.662645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:32:18.609023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800518+0000","last_change":"2024-04-26T20:50:31.622898+0000","last_active":"2024-04-26T20:50:38.800518+0000","last_peered":"2024-04-26T20:50:38.800518+0000","last_clean":"2024-04-26T20:50:38.800518+0000","last_became_active":"2024-04-26T20:50:31.622583+0000","last_became_peered":"2024-04-26T20:50:31.622583+0000","last_unstale":"2024-04-26T20:50:38.800518+0000","last_undegraded":"2024-04-26T20:50:38.800518+0000","last_fullsized":"2024-04-26T20:50:38.800518+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:53:10.645804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661250+0000","last_change":"2024-04-26T20:50:36.681863+0000","last_active":"2024-04-26T20:50:38.661250+0000","last_peered":"2024-04-26T20:50:38.661250+0000","last_clean":"2024-04-26T20:50:38.661250+0000","last_became_active":"2024-04-26T20:50:36.681433+0000","last_became_peered":"2024-04-26T20:50:36.681433+0000","last_unstale":"2024-04-26T20:50:38.661250+0000","last_undegraded":"2024-04-26T20:50:38.661250+0000","last_fullsized":"2024-04-26T20:50:38.661250+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:28:04.995069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661447+0000","last_change":"2024-04-26T20:50:32.640962+0000","last_active":"2024-04-26T20:50:38.661447+0000","last_peered":"2024-04-26T20:50:38.661447+0000","last_clean":"2024-04-26T20:50:38.661447+0000","last_became_active":"2024-04-26T20:50:32.640670+0000","last_became_peered":"2024-04-26T20:50:32.640670+0000","last_unstale":"2024-04-26T20:50:38.661447+0000","last_undegraded":"2024-04-26T20:50:38.661447+0000","last_fullsized":"2024-04-26T20:50:38.661447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:54.967490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802635+0000","last_change":"2024-04-26T20:50:31.631694+0000","last_active":"2024-04-26T20:50:38.802635+0000","last_peered":"2024-04-26T20:50:38.802635+0000","last_clean":"2024-04-26T20:50:38.802635+0000","last_became_active":"2024-04-26T20:50:31.631330+0000","last_became_peered":"2024-04-26T20:50:31.631330+0000","last_unstale":"2024-04-26T20:50:38.802635+0000","last_undegraded":"2024-04-26T20:50:38.802635+0000","last_fullsized":"2024-04-26T20:50:38.802635+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:49:28.099373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661389+0000","last_change":"2024-04-26T20:50:36.682799+0000","last_active":"2024-04-26T20:50:38.661389+0000","last_peered":"2024-04-26T20:50:38.661389+0000","last_clean":"2024-04-26T20:50:38.661389+0000","last_became_active":"2024-04-26T20:50:36.682271+0000","last_became_peered":"2024-04-26T20:50:36.682271+0000","last_unstale":"2024-04-26T20:50:38.661389+0000","last_undegraded":"2024-04-26T20:50:38.661389+0000","last_fullsized":"2024-04-26T20:50:38.661389+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:27:18.035795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802689+0000","last_change":"2024-04-26T20:50:32.647241+0000","last_active":"2024-04-26T20:50:38.802689+0000","last_peered":"2024-04-26T20:50:38.802689+0000","last_clean":"2024-04-26T20:50:38.802689+0000","last_became_active":"2024-04-26T20:50:32.646907+0000","last_became_peered":"2024-04-26T20:50:32.646907+0000","last_unstale":"2024-04-26T20:50:38.802689+0000","last_undegraded":"2024-04-26T20:50:38.802689+0000","last_fullsized":"2024-04-26T20:50:38.802689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:57:40.604842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800494+0000","last_change":"2024-04-26T20:50:31.633583+0000","last_active":"2024-04-26T20:50:38.800494+0000","last_peered":"2024-04-26T20:50:38.800494+0000","last_clean":"2024-04-26T20:50:38.800494+0000","last_became_active":"2024-04-26T20:50:31.633272+0000","last_became_peered":"2024-04-26T20:50:31.633272+0000","last_unstale":"2024-04-26T20:50:38.800494+0000","last_undegraded":"2024-04-26T20:50:38.800494+0000","last_fullsized":"2024-04-26T20:50:38.800494+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:53:10.645804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803808+0000","last_change":"2024-04-26T20:50:37.291410+0000","last_active":"2024-04-26T20:50:38.803808+0000","last_peered":"2024-04-26T20:50:38.803808+0000","last_clean":"2024-04-26T20:50:38.803808+0000","last_became_active":"2024-04-26T20:50:37.291016+0000","last_became_peered":"2024-04-26T20:50:37.291016+0000","last_unstale":"2024-04-26T20:50:38.803808+0000","last_undegraded":"2024-04-26T20:50:38.803808+0000","last_fullsized":"2024-04-26T20:50:38.803808+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:13:08.850173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802348+0000","last_change":"2024-04-26T20:50:32.653314+0000","last_active":"2024-04-26T20:50:38.802348+0000","last_peered":"2024-04-26T20:50:38.802348+0000","last_clean":"2024-04-26T20:50:38.802348+0000","last_became_active":"2024-04-26T20:50:32.652980+0000","last_became_peered":"2024-04-26T20:50:32.652980+0000","last_unstale":"2024-04-26T20:50:38.802348+0000","last_undegraded":"2024-04-26T20:50:38.802348+0000","last_fullsized":"2024-04-26T20:50:38.802348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:15:55.563522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800028+0000","last_change":"2024-04-26T20:50:31.629049+0000","last_active":"2024-04-26T20:50:38.800028+0000","last_peered":"2024-04-26T20:50:38.800028+0000","last_clean":"2024-04-26T20:50:38.800028+0000","last_became_active":"2024-04-26T20:50:31.628788+0000","last_became_peered":"2024-04-26T20:50:31.628788+0000","last_unstale":"2024-04-26T20:50:38.800028+0000","last_undegraded":"2024-04-26T20:50:38.800028+0000","last_fullsized":"2024-04-26T20:50:38.800028+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:26.601021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":179,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800588+0000","last_change":"2024-04-26T20:50:21.150132+0000","last_active":"2024-04-26T20:50:38.800588+0000","last_peered":"2024-04-26T20:50:38.800588+0000","last_clean":"2024-04-26T20:50:38.800588+0000","last_became_active":"2024-04-26T20:50:21.147439+0000","last_became_peered":"2024-04-26T20:50:21.147439+0000","last_unstale":"2024-04-26T20:50:38.800588+0000","last_undegraded":"2024-04-26T20:50:38.800588+0000","last_fullsized":"2024-04-26T20:50:38.800588+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:20.084499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:20.084499+0000","last_clean_scrub_stamp":"2024-04-26T20:50:20.084499+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:19:29.395477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.717442+0000","last_change":"2024-04-26T20:50:37.295209+0000","last_active":"2024-04-26T20:50:38.717442+0000","last_peered":"2024-04-26T20:50:38.717442+0000","last_clean":"2024-04-26T20:50:38.717442+0000","last_became_active":"2024-04-26T20:50:37.294715+0000","last_became_peered":"2024-04-26T20:50:37.294715+0000","last_unstale":"2024-04-26T20:50:38.717442+0000","last_undegraded":"2024-04-26T20:50:38.717442+0000","last_fullsized":"2024-04-26T20:50:38.717442+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:13:14.645783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800810+0000","last_change":"2024-04-26T20:50:32.648081+0000","last_active":"2024-04-26T20:50:38.800810+0000","last_peered":"2024-04-26T20:50:38.800810+0000","last_clean":"2024-04-26T20:50:38.800810+0000","last_became_active":"2024-04-26T20:50:32.644812+0000","last_became_peered":"2024-04-26T20:50:32.644812+0000","last_unstale":"2024-04-26T20:50:38.800810+0000","last_undegraded":"2024-04-26T20:50:38.800810+0000","last_fullsized":"2024-04-26T20:50:38.800810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:45:44.950496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803401+0000","last_change":"2024-04-26T20:50:31.630301+0000","last_active":"2024-04-26T20:50:38.803401+0000","last_peered":"2024-04-26T20:50:38.803401+0000","last_clean":"2024-04-26T20:50:38.803401+0000","last_became_active":"2024-04-26T20:50:31.630039+0000","last_became_peered":"2024-04-26T20:50:31.630039+0000","last_unstale":"2024-04-26T20:50:38.803401+0000","last_undegraded":"2024-04-26T20:50:38.803401+0000","last_fullsized":"2024-04-26T20:50:38.803401+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:21:09.963731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.717176+0000","last_change":"2024-04-26T20:50:37.291889+0000","last_active":"2024-04-26T20:50:38.717176+0000","last_peered":"2024-04-26T20:50:38.717176+0000","last_clean":"2024-04-26T20:50:38.717176+0000","last_became_active":"2024-04-26T20:50:37.291401+0000","last_became_peered":"2024-04-26T20:50:37.291401+0000","last_unstale":"2024-04-26T20:50:38.717176+0000","last_undegraded":"2024-04-26T20:50:38.717176+0000","last_fullsized":"2024-04-26T20:50:38.717176+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:28:29.639328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800412+0000","last_change":"2024-04-26T20:50:32.655082+0000","last_active":"2024-04-26T20:50:38.800412+0000","last_peered":"2024-04-26T20:50:38.800412+0000","last_clean":"2024-04-26T20:50:38.800412+0000","last_became_active":"2024-04-26T20:50:32.654751+0000","last_became_peered":"2024-04-26T20:50:32.654751+0000","last_unstale":"2024-04-26T20:50:38.800412+0000","last_undegraded":"2024-04-26T20:50:38.800412+0000","last_fullsized":"2024-04-26T20:50:38.800412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:27:59.302826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661274+0000","last_change":"2024-04-26T20:50:31.628865+0000","last_active":"2024-04-26T20:50:38.661274+0000","last_peered":"2024-04-26T20:50:38.661274+0000","last_clean":"2024-04-26T20:50:38.661274+0000","last_became_active":"2024-04-26T20:50:31.627789+0000","last_became_peered":"2024-04-26T20:50:31.627789+0000","last_unstale":"2024-04-26T20:50:38.661274+0000","last_undegraded":"2024-04-26T20:50:38.661274+0000","last_fullsized":"2024-04-26T20:50:38.661274+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:53:10.645804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803847+0000","last_change":"2024-04-26T20:50:36.679869+0000","last_active":"2024-04-26T20:50:38.803847+0000","last_peered":"2024-04-26T20:50:38.803847+0000","last_clean":"2024-04-26T20:50:38.803847+0000","last_became_active":"2024-04-26T20:50:36.678678+0000","last_became_peered":"2024-04-26T20:50:36.678678+0000","last_unstale":"2024-04-26T20:50:38.803847+0000","last_undegraded":"2024-04-26T20:50:38.803847+0000","last_fullsized":"2024-04-26T20:50:38.803847+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:54.249138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.801078+0000","last_change":"2024-04-26T20:50:32.655912+0000","last_active":"2024-04-26T20:50:38.801078+0000","last_peered":"2024-04-26T20:50:38.801078+0000","last_clean":"2024-04-26T20:50:38.801078+0000","last_became_active":"2024-04-26T20:50:32.655649+0000","last_became_peered":"2024-04-26T20:50:32.655649+0000","last_unstale":"2024-04-26T20:50:38.801078+0000","last_undegraded":"2024-04-26T20:50:38.801078+0000","last_fullsized":"2024-04-26T20:50:38.801078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:59:04.962815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803582+0000","last_change":"2024-04-26T20:50:31.631670+0000","last_active":"2024-04-26T20:50:38.803582+0000","last_peered":"2024-04-26T20:50:38.803582+0000","last_clean":"2024-04-26T20:50:38.803582+0000","last_became_active":"2024-04-26T20:50:31.631380+0000","last_became_peered":"2024-04-26T20:50:31.631380+0000","last_unstale":"2024-04-26T20:50:38.803582+0000","last_undegraded":"2024-04-26T20:50:38.803582+0000","last_fullsized":"2024-04-26T20:50:38.803582+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:23:29.866794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.799715+0000","last_change":"2024-04-26T20:50:37.291812+0000","last_active":"2024-04-26T20:50:38.799715+0000","last_peered":"2024-04-26T20:50:38.799715+0000","last_clean":"2024-04-26T20:50:38.799715+0000","last_became_active":"2024-04-26T20:50:37.291408+0000","last_became_peered":"2024-04-26T20:50:37.291408+0000","last_unstale":"2024-04-26T20:50:38.799715+0000","last_undegraded":"2024-04-26T20:50:38.799715+0000","last_fullsized":"2024-04-26T20:50:38.799715+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:29:58.077073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800564+0000","last_change":"2024-04-26T20:50:32.649031+0000","last_active":"2024-04-26T20:50:38.800564+0000","last_peered":"2024-04-26T20:50:38.800564+0000","last_clean":"2024-04-26T20:50:38.800564+0000","last_became_active":"2024-04-26T20:50:32.648769+0000","last_became_peered":"2024-04-26T20:50:32.648769+0000","last_unstale":"2024-04-26T20:50:38.800564+0000","last_undegraded":"2024-04-26T20:50:38.800564+0000","last_fullsized":"2024-04-26T20:50:38.800564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:16:25.308786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.662578+0000","last_change":"2024-04-26T20:50:31.627514+0000","last_active":"2024-04-26T20:50:38.662578+0000","last_peered":"2024-04-26T20:50:38.662578+0000","last_clean":"2024-04-26T20:50:38.662578+0000","last_became_active":"2024-04-26T20:50:31.627177+0000","last_became_peered":"2024-04-26T20:50:31.627177+0000","last_unstale":"2024-04-26T20:50:38.662578+0000","last_undegraded":"2024-04-26T20:50:38.662578+0000","last_fullsized":"2024-04-26T20:50:38.662578+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:23:21.503395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661857+0000","last_change":"2024-04-26T20:50:36.678980+0000","last_active":"2024-04-26T20:50:38.661857+0000","last_peered":"2024-04-26T20:50:38.661857+0000","last_clean":"2024-04-26T20:50:38.661857+0000","last_became_active":"2024-04-26T20:50:36.678581+0000","last_became_peered":"2024-04-26T20:50:36.678581+0000","last_unstale":"2024-04-26T20:50:38.661857+0000","last_undegraded":"2024-04-26T20:50:38.661857+0000","last_fullsized":"2024-04-26T20:50:38.661857+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:28:29.639328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800927+0000","last_change":"2024-04-26T20:50:32.647060+0000","last_active":"2024-04-26T20:50:38.800927+0000","last_peered":"2024-04-26T20:50:38.800927+0000","last_clean":"2024-04-26T20:50:38.800927+0000","last_became_active":"2024-04-26T20:50:32.646761+0000","last_became_peered":"2024-04-26T20:50:32.646761+0000","last_unstale":"2024-04-26T20:50:38.800927+0000","last_undegraded":"2024-04-26T20:50:38.800927+0000","last_fullsized":"2024-04-26T20:50:38.800927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:35.639518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802576+0000","last_change":"2024-04-26T20:50:31.627228+0000","last_active":"2024-04-26T20:50:38.802576+0000","last_peered":"2024-04-26T20:50:38.802576+0000","last_clean":"2024-04-26T20:50:38.802576+0000","last_became_active":"2024-04-26T20:50:31.626931+0000","last_became_peered":"2024-04-26T20:50:31.626931+0000","last_unstale":"2024-04-26T20:50:38.802576+0000","last_undegraded":"2024-04-26T20:50:38.802576+0000","last_fullsized":"2024-04-26T20:50:38.802576+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:26.601021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803663+0000","last_change":"2024-04-26T20:50:36.684013+0000","last_active":"2024-04-26T20:50:38.803663+0000","last_peered":"2024-04-26T20:50:38.803663+0000","last_clean":"2024-04-26T20:50:38.803663+0000","last_became_active":"2024-04-26T20:50:36.683653+0000","last_became_peered":"2024-04-26T20:50:36.683653+0000","last_unstale":"2024-04-26T20:50:38.803663+0000","last_undegraded":"2024-04-26T20:50:38.803663+0000","last_fullsized":"2024-04-26T20:50:38.803663+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:27:18.035795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.717061+0000","last_change":"2024-04-26T20:50:32.645444+0000","last_active":"2024-04-26T20:50:38.717061+0000","last_peered":"2024-04-26T20:50:38.717061+0000","last_clean":"2024-04-26T20:50:38.717061+0000","last_became_active":"2024-04-26T20:50:32.645155+0000","last_became_peered":"2024-04-26T20:50:32.645155+0000","last_unstale":"2024-04-26T20:50:38.717061+0000","last_undegraded":"2024-04-26T20:50:38.717061+0000","last_fullsized":"2024-04-26T20:50:38.717061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:49:57.427154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.717091+0000","last_change":"2024-04-26T20:50:31.629590+0000","last_active":"2024-04-26T20:50:38.717091+0000","last_peered":"2024-04-26T20:50:38.717091+0000","last_clean":"2024-04-26T20:50:38.717091+0000","last_became_active":"2024-04-26T20:50:31.629324+0000","last_became_peered":"2024-04-26T20:50:31.629324+0000","last_unstale":"2024-04-26T20:50:38.717091+0000","last_undegraded":"2024-04-26T20:50:38.717091+0000","last_fullsized":"2024-04-26T20:50:38.717091+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:49:28.099373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803803+0000","last_change":"2024-04-26T20:50:37.291611+0000","last_active":"2024-04-26T20:50:38.803803+0000","last_peered":"2024-04-26T20:50:38.803803+0000","last_clean":"2024-04-26T20:50:38.803803+0000","last_became_active":"2024-04-26T20:50:37.291068+0000","last_became_peered":"2024-04-26T20:50:37.291068+0000","last_unstale":"2024-04-26T20:50:38.803803+0000","last_undegraded":"2024-04-26T20:50:38.803803+0000","last_fullsized":"2024-04-26T20:50:38.803803+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:31:09.384259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800427+0000","last_change":"2024-04-26T20:50:32.649304+0000","last_active":"2024-04-26T20:50:38.800427+0000","last_peered":"2024-04-26T20:50:38.800427+0000","last_clean":"2024-04-26T20:50:38.800427+0000","last_became_active":"2024-04-26T20:50:32.647714+0000","last_became_peered":"2024-04-26T20:50:32.647714+0000","last_unstale":"2024-04-26T20:50:38.800427+0000","last_undegraded":"2024-04-26T20:50:38.800427+0000","last_fullsized":"2024-04-26T20:50:38.800427+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:27:59.302826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800065+0000","last_change":"2024-04-26T20:50:31.626907+0000","last_active":"2024-04-26T20:50:38.800065+0000","last_peered":"2024-04-26T20:50:38.800065+0000","last_clean":"2024-04-26T20:50:38.800065+0000","last_became_active":"2024-04-26T20:50:31.625955+0000","last_became_peered":"2024-04-26T20:50:31.625955+0000","last_unstale":"2024-04-26T20:50:38.800065+0000","last_undegraded":"2024-04-26T20:50:38.800065+0000","last_fullsized":"2024-04-26T20:50:38.800065+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:46:19.858595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803391+0000","last_change":"2024-04-26T20:50:37.292652+0000","last_active":"2024-04-26T20:50:38.803391+0000","last_peered":"2024-04-26T20:50:38.803391+0000","last_clean":"2024-04-26T20:50:38.803391+0000","last_became_active":"2024-04-26T20:50:37.291717+0000","last_became_peered":"2024-04-26T20:50:37.291717+0000","last_unstale":"2024-04-26T20:50:38.803391+0000","last_undegraded":"2024-04-26T20:50:38.803391+0000","last_fullsized":"2024-04-26T20:50:38.803391+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:31:19.069050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803321+0000","last_change":"2024-04-26T20:50:32.643484+0000","last_active":"2024-04-26T20:50:38.803321+0000","last_peered":"2024-04-26T20:50:38.803321+0000","last_clean":"2024-04-26T20:50:38.803321+0000","last_became_active":"2024-04-26T20:50:32.643225+0000","last_became_peered":"2024-04-26T20:50:32.643225+0000","last_unstale":"2024-04-26T20:50:38.803321+0000","last_undegraded":"2024-04-26T20:50:38.803321+0000","last_fullsized":"2024-04-26T20:50:38.803321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:49:57.427154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803958+0000","last_change":"2024-04-26T20:50:31.627130+0000","last_active":"2024-04-26T20:50:38.803958+0000","last_peered":"2024-04-26T20:50:38.803958+0000","last_clean":"2024-04-26T20:50:38.803958+0000","last_became_active":"2024-04-26T20:50:31.626697+0000","last_became_peered":"2024-04-26T20:50:31.626697+0000","last_unstale":"2024-04-26T20:50:38.803958+0000","last_undegraded":"2024-04-26T20:50:38.803958+0000","last_fullsized":"2024-04-26T20:50:38.803958+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:21:09.963731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800487+0000","last_change":"2024-04-26T20:50:37.294808+0000","last_active":"2024-04-26T20:50:38.800487+0000","last_peered":"2024-04-26T20:50:38.800487+0000","last_clean":"2024-04-26T20:50:38.800487+0000","last_became_active":"2024-04-26T20:50:37.294404+0000","last_became_peered":"2024-04-26T20:50:37.294404+0000","last_unstale":"2024-04-26T20:50:38.800487+0000","last_undegraded":"2024-04-26T20:50:38.800487+0000","last_fullsized":"2024-04-26T20:50:38.800487+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:29:17.876006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800773+0000","last_change":"2024-04-26T20:50:32.649505+0000","last_active":"2024-04-26T20:50:38.800773+0000","last_peered":"2024-04-26T20:50:38.800773+0000","last_clean":"2024-04-26T20:50:38.800773+0000","last_became_active":"2024-04-26T20:50:32.648538+0000","last_became_peered":"2024-04-26T20:50:32.648538+0000","last_unstale":"2024-04-26T20:50:38.800773+0000","last_undegraded":"2024-04-26T20:50:38.800773+0000","last_fullsized":"2024-04-26T20:50:38.800773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:33:28.969815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:41.974976+0000","last_change":"2024-04-26T20:50:31.628758+0000","last_active":"2024-04-26T20:50:41.974976+0000","last_peered":"2024-04-26T20:50:41.974976+0000","last_clean":"2024-04-26T20:50:41.974976+0000","last_became_active":"2024-04-26T20:50:31.628431+0000","last_became_peered":"2024-04-26T20:50:31.628431+0000","last_unstale":"2024-04-26T20:50:41.974976+0000","last_undegraded":"2024-04-26T20:50:41.974976+0000","last_fullsized":"2024-04-26T20:50:41.974976+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:36.499651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661093+0000","last_change":"2024-04-26T20:50:36.679403+0000","last_active":"2024-04-26T20:50:38.661093+0000","last_peered":"2024-04-26T20:50:38.661093+0000","last_clean":"2024-04-26T20:50:38.661093+0000","last_became_active":"2024-04-26T20:50:36.678945+0000","last_became_peered":"2024-04-26T20:50:36.678945+0000","last_unstale":"2024-04-26T20:50:38.661093+0000","last_undegraded":"2024-04-26T20:50:38.661093+0000","last_fullsized":"2024-04-26T20:50:38.661093+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:12.140650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.662571+0000","last_change":"2024-04-26T20:50:32.636718+0000","last_active":"2024-04-26T20:50:38.662571+0000","last_peered":"2024-04-26T20:50:38.662571+0000","last_clean":"2024-04-26T20:50:38.662571+0000","last_became_active":"2024-04-26T20:50:32.636419+0000","last_became_peered":"2024-04-26T20:50:32.636419+0000","last_unstale":"2024-04-26T20:50:38.662571+0000","last_undegraded":"2024-04-26T20:50:38.662571+0000","last_fullsized":"2024-04-26T20:50:38.662571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:06:40.634786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661155+0000","last_change":"2024-04-26T20:50:31.630575+0000","last_active":"2024-04-26T20:50:38.661155+0000","last_peered":"2024-04-26T20:50:38.661155+0000","last_clean":"2024-04-26T20:50:38.661155+0000","last_became_active":"2024-04-26T20:50:31.630304+0000","last_became_peered":"2024-04-26T20:50:31.630304+0000","last_unstale":"2024-04-26T20:50:38.661155+0000","last_undegraded":"2024-04-26T20:50:38.661155+0000","last_fullsized":"2024-04-26T20:50:38.661155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:32:17.604576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.662550+0000","last_change":"2024-04-26T20:50:36.672365+0000","last_active":"2024-04-26T20:50:38.662550+0000","last_peered":"2024-04-26T20:50:38.662550+0000","last_clean":"2024-04-26T20:50:38.662550+0000","last_became_active":"2024-04-26T20:50:36.671917+0000","last_became_peered":"2024-04-26T20:50:36.671917+0000","last_unstale":"2024-04-26T20:50:38.662550+0000","last_undegraded":"2024-04-26T20:50:38.662550+0000","last_fullsized":"2024-04-26T20:50:38.662550+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:28:04.995069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803056+0000","last_change":"2024-04-26T20:50:32.645616+0000","last_active":"2024-04-26T20:50:38.803056+0000","last_peered":"2024-04-26T20:50:38.803056+0000","last_clean":"2024-04-26T20:50:38.803056+0000","last_became_active":"2024-04-26T20:50:32.645372+0000","last_became_peered":"2024-04-26T20:50:32.645372+0000","last_unstale":"2024-04-26T20:50:38.803056+0000","last_undegraded":"2024-04-26T20:50:38.803056+0000","last_fullsized":"2024-04-26T20:50:38.803056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:56:51.926886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800505+0000","last_change":"2024-04-26T20:50:31.632113+0000","last_active":"2024-04-26T20:50:38.800505+0000","last_peered":"2024-04-26T20:50:38.800505+0000","last_clean":"2024-04-26T20:50:38.800505+0000","last_became_active":"2024-04-26T20:50:31.631815+0000","last_became_peered":"2024-04-26T20:50:31.631815+0000","last_unstale":"2024-04-26T20:50:38.800505+0000","last_undegraded":"2024-04-26T20:50:38.800505+0000","last_fullsized":"2024-04-26T20:50:38.800505+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:36.499651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802860+0000","last_change":"2024-04-26T20:50:36.676688+0000","last_active":"2024-04-26T20:50:38.802860+0000","last_peered":"2024-04-26T20:50:38.802860+0000","last_clean":"2024-04-26T20:50:38.802860+0000","last_became_active":"2024-04-26T20:50:36.675819+0000","last_became_peered":"2024-04-26T20:50:36.675819+0000","last_unstale":"2024-04-26T20:50:38.802860+0000","last_undegraded":"2024-04-26T20:50:38.802860+0000","last_fullsized":"2024-04-26T20:50:38.802860+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:12:04.255812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800398+0000","last_change":"2024-04-26T20:50:32.642860+0000","last_active":"2024-04-26T20:50:38.800398+0000","last_peered":"2024-04-26T20:50:38.800398+0000","last_clean":"2024-04-26T20:50:38.800398+0000","last_became_active":"2024-04-26T20:50:32.642567+0000","last_became_peered":"2024-04-26T20:50:32.642567+0000","last_unstale":"2024-04-26T20:50:38.800398+0000","last_undegraded":"2024-04-26T20:50:38.800398+0000","last_fullsized":"2024-04-26T20:50:38.800398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:00:12.394204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.717653+0000","last_change":"2024-04-26T20:50:31.632961+0000","last_active":"2024-04-26T20:50:38.717653+0000","last_peered":"2024-04-26T20:50:38.717653+0000","last_clean":"2024-04-26T20:50:38.717653+0000","last_became_active":"2024-04-26T20:50:31.632044+0000","last_became_peered":"2024-04-26T20:50:31.632044+0000","last_unstale":"2024-04-26T20:50:38.717653+0000","last_undegraded":"2024-04-26T20:50:38.717653+0000","last_fullsized":"2024-04-26T20:50:38.717653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:23:29.866794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802614+0000","last_change":"2024-04-26T20:50:37.293243+0000","last_active":"2024-04-26T20:50:38.802614+0000","last_peered":"2024-04-26T20:50:38.802614+0000","last_clean":"2024-04-26T20:50:38.802614+0000","last_became_active":"2024-04-26T20:50:37.292840+0000","last_became_peered":"2024-04-26T20:50:37.292840+0000","last_unstale":"2024-04-26T20:50:38.802614+0000","last_undegraded":"2024-04-26T20:50:38.802614+0000","last_fullsized":"2024-04-26T20:50:38.802614+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:05:40.897275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803514+0000","last_change":"2024-04-26T20:50:37.294767+0000","last_active":"2024-04-26T20:50:38.803514+0000","last_peered":"2024-04-26T20:50:38.803514+0000","last_clean":"2024-04-26T20:50:38.803514+0000","last_became_active":"2024-04-26T20:50:37.294351+0000","last_became_peered":"2024-04-26T20:50:37.294351+0000","last_unstale":"2024-04-26T20:50:38.803514+0000","last_undegraded":"2024-04-26T20:50:38.803514+0000","last_fullsized":"2024-04-26T20:50:38.803514+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:31:21.850102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.660600+0000","last_change":"2024-04-26T20:50:32.648105+0000","last_active":"2024-04-26T20:50:38.660600+0000","last_peered":"2024-04-26T20:50:38.660600+0000","last_clean":"2024-04-26T20:50:38.660600+0000","last_became_active":"2024-04-26T20:50:32.647833+0000","last_became_peered":"2024-04-26T20:50:32.647833+0000","last_unstale":"2024-04-26T20:50:38.660600+0000","last_undegraded":"2024-04-26T20:50:38.660600+0000","last_fullsized":"2024-04-26T20:50:38.660600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:55.634765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.799847+0000","last_change":"2024-04-26T20:50:31.622923+0000","last_active":"2024-04-26T20:50:38.799847+0000","last_peered":"2024-04-26T20:50:38.799847+0000","last_clean":"2024-04-26T20:50:38.799847+0000","last_became_active":"2024-04-26T20:50:31.622533+0000","last_became_peered":"2024-04-26T20:50:31.622533+0000","last_unstale":"2024-04-26T20:50:38.799847+0000","last_undegraded":"2024-04-26T20:50:38.799847+0000","last_fullsized":"2024-04-26T20:50:38.799847+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:32:17.604576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.717573+0000","last_change":"2024-04-26T20:50:36.680804+0000","last_active":"2024-04-26T20:50:38.717573+0000","last_peered":"2024-04-26T20:50:38.717573+0000","last_clean":"2024-04-26T20:50:38.717573+0000","last_became_active":"2024-04-26T20:50:36.680438+0000","last_became_peered":"2024-04-26T20:50:36.680438+0000","last_unstale":"2024-04-26T20:50:38.717573+0000","last_undegraded":"2024-04-26T20:50:38.717573+0000","last_fullsized":"2024-04-26T20:50:38.717573+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:58:22.587813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802852+0000","last_change":"2024-04-26T20:50:32.649026+0000","last_active":"2024-04-26T20:50:38.802852+0000","last_peered":"2024-04-26T20:50:38.802852+0000","last_clean":"2024-04-26T20:50:38.802852+0000","last_became_active":"2024-04-26T20:50:32.648737+0000","last_became_peered":"2024-04-26T20:50:32.648737+0000","last_unstale":"2024-04-26T20:50:38.802852+0000","last_undegraded":"2024-04-26T20:50:38.802852+0000","last_fullsized":"2024-04-26T20:50:38.802852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:47:14.715739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802756+0000","last_change":"2024-04-26T20:50:31.631362+0000","last_active":"2024-04-26T20:50:38.802756+0000","last_peered":"2024-04-26T20:50:38.802756+0000","last_clean":"2024-04-26T20:50:38.802756+0000","last_became_active":"2024-04-26T20:50:31.631099+0000","last_became_peered":"2024-04-26T20:50:31.631099+0000","last_unstale":"2024-04-26T20:50:38.802756+0000","last_undegraded":"2024-04-26T20:50:38.802756+0000","last_fullsized":"2024-04-26T20:50:38.802756+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:20:49.853735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.717285+0000","last_change":"2024-04-26T20:50:31.629712+0000","last_active":"2024-04-26T20:50:38.717285+0000","last_peered":"2024-04-26T20:50:38.717285+0000","last_clean":"2024-04-26T20:50:38.717285+0000","last_became_active":"2024-04-26T20:50:31.629412+0000","last_became_peered":"2024-04-26T20:50:31.629412+0000","last_unstale":"2024-04-26T20:50:38.717285+0000","last_undegraded":"2024-04-26T20:50:38.717285+0000","last_fullsized":"2024-04-26T20:50:38.717285+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:32:17.604576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803849+0000","last_change":"2024-04-26T20:50:32.638185+0000","last_active":"2024-04-26T20:50:38.803849+0000","last_peered":"2024-04-26T20:50:38.803849+0000","last_clean":"2024-04-26T20:50:38.803849+0000","last_became_active":"2024-04-26T20:50:32.637917+0000","last_became_peered":"2024-04-26T20:50:32.637917+0000","last_unstale":"2024-04-26T20:50:38.803849+0000","last_undegraded":"2024-04-26T20:50:38.803849+0000","last_fullsized":"2024-04-26T20:50:38.803849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:00:12.394204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803278+0000","last_change":"2024-04-26T20:50:36.682087+0000","last_active":"2024-04-26T20:50:38.803278+0000","last_peered":"2024-04-26T20:50:38.803278+0000","last_clean":"2024-04-26T20:50:38.803278+0000","last_became_active":"2024-04-26T20:50:36.681543+0000","last_became_peered":"2024-04-26T20:50:36.681543+0000","last_unstale":"2024-04-26T20:50:38.803278+0000","last_undegraded":"2024-04-26T20:50:38.803278+0000","last_fullsized":"2024-04-26T20:50:38.803278+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:57:44.640580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802464+0000","last_change":"2024-04-26T20:50:31.632761+0000","last_active":"2024-04-26T20:50:38.802464+0000","last_peered":"2024-04-26T20:50:38.802464+0000","last_clean":"2024-04-26T20:50:38.802464+0000","last_became_active":"2024-04-26T20:50:31.632381+0000","last_became_peered":"2024-04-26T20:50:31.632381+0000","last_unstale":"2024-04-26T20:50:38.802464+0000","last_undegraded":"2024-04-26T20:50:38.802464+0000","last_fullsized":"2024-04-26T20:50:38.802464+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:21:09.963731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.662120+0000","last_change":"2024-04-26T20:50:32.637920+0000","last_active":"2024-04-26T20:50:38.662120+0000","last_peered":"2024-04-26T20:50:38.662120+0000","last_clean":"2024-04-26T20:50:38.662120+0000","last_became_active":"2024-04-26T20:50:32.637618+0000","last_became_peered":"2024-04-26T20:50:32.637618+0000","last_unstale":"2024-04-26T20:50:38.662120+0000","last_undegraded":"2024-04-26T20:50:38.662120+0000","last_fullsized":"2024-04-26T20:50:38.662120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:46:20.863042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803215+0000","last_change":"2024-04-26T20:50:36.678219+0000","last_active":"2024-04-26T20:50:38.803215+0000","last_peered":"2024-04-26T20:50:38.803215+0000","last_clean":"2024-04-26T20:50:38.803215+0000","last_became_active":"2024-04-26T20:50:36.677732+0000","last_became_peered":"2024-04-26T20:50:36.677732+0000","last_unstale":"2024-04-26T20:50:38.803215+0000","last_undegraded":"2024-04-26T20:50:38.803215+0000","last_fullsized":"2024-04-26T20:50:38.803215+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:03:12.186398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.716924+0000","last_change":"2024-04-26T20:50:32.638997+0000","last_active":"2024-04-26T20:50:38.716924+0000","last_peered":"2024-04-26T20:50:38.716924+0000","last_clean":"2024-04-26T20:50:38.716924+0000","last_became_active":"2024-04-26T20:50:32.638708+0000","last_became_peered":"2024-04-26T20:50:32.638708+0000","last_unstale":"2024-04-26T20:50:38.716924+0000","last_undegraded":"2024-04-26T20:50:38.716924+0000","last_fullsized":"2024-04-26T20:50:38.716924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:36.651960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802480+0000","last_change":"2024-04-26T20:50:32.648305+0000","last_active":"2024-04-26T20:50:38.802480+0000","last_peered":"2024-04-26T20:50:38.802480+0000","last_clean":"2024-04-26T20:50:38.802480+0000","last_became_active":"2024-04-26T20:50:32.648021+0000","last_became_peered":"2024-04-26T20:50:32.648021+0000","last_unstale":"2024-04-26T20:50:38.802480+0000","last_undegraded":"2024-04-26T20:50:38.802480+0000","last_fullsized":"2024-04-26T20:50:38.802480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:46:20.863042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800030+0000","last_change":"2024-04-26T20:50:32.649133+0000","last_active":"2024-04-26T20:50:38.800030+0000","last_peered":"2024-04-26T20:50:38.800030+0000","last_clean":"2024-04-26T20:50:38.800030+0000","last_became_active":"2024-04-26T20:50:32.648855+0000","last_became_peered":"2024-04-26T20:50:32.648855+0000","last_unstale":"2024-04-26T20:50:38.800030+0000","last_undegraded":"2024-04-26T20:50:38.800030+0000","last_fullsized":"2024-04-26T20:50:38.800030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:58:18.552075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803763+0000","last_change":"2024-04-26T20:50:32.650322+0000","last_active":"2024-04-26T20:50:38.803763+0000","last_peered":"2024-04-26T20:50:38.803763+0000","last_clean":"2024-04-26T20:50:38.803763+0000","last_became_active":"2024-04-26T20:50:32.650011+0000","last_became_peered":"2024-04-26T20:50:32.650011+0000","last_unstale":"2024-04-26T20:50:38.803763+0000","last_undegraded":"2024-04-26T20:50:38.803763+0000","last_fullsized":"2024-04-26T20:50:38.803763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:15:55.563522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803550+0000","last_change":"2024-04-26T20:50:32.649914+0000","last_active":"2024-04-26T20:50:38.803550+0000","last_peered":"2024-04-26T20:50:38.803550+0000","last_clean":"2024-04-26T20:50:38.803550+0000","last_became_active":"2024-04-26T20:50:32.649439+0000","last_became_peered":"2024-04-26T20:50:32.649439+0000","last_unstale":"2024-04-26T20:50:38.803550+0000","last_undegraded":"2024-04-26T20:50:38.803550+0000","last_fullsized":"2024-04-26T20:50:38.803550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:33:28.969815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661177+0000","last_change":"2024-04-26T20:50:32.648570+0000","last_active":"2024-04-26T20:50:38.661177+0000","last_peered":"2024-04-26T20:50:38.661177+0000","last_clean":"2024-04-26T20:50:38.661177+0000","last_became_active":"2024-04-26T20:50:32.646951+0000","last_became_peered":"2024-04-26T20:50:32.646951+0000","last_unstale":"2024-04-26T20:50:38.661177+0000","last_undegraded":"2024-04-26T20:50:38.661177+0000","last_fullsized":"2024-04-26T20:50:38.661177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:15:55.563522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.717644+0000","last_change":"2024-04-26T20:50:32.640208+0000","last_active":"2024-04-26T20:50:38.717644+0000","last_peered":"2024-04-26T20:50:38.717644+0000","last_clean":"2024-04-26T20:50:38.717644+0000","last_became_active":"2024-04-26T20:50:32.639869+0000","last_became_peered":"2024-04-26T20:50:32.639869+0000","last_unstale":"2024-04-26T20:50:38.717644+0000","last_undegraded":"2024-04-26T20:50:38.717644+0000","last_fullsized":"2024-04-26T20:50:38.717644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:15:55.563522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803850+0000","last_change":"2024-04-26T20:50:32.650173+0000","last_active":"2024-04-26T20:50:38.803850+0000","last_peered":"2024-04-26T20:50:38.803850+0000","last_clean":"2024-04-26T20:50:38.803850+0000","last_became_active":"2024-04-26T20:50:32.649876+0000","last_became_peered":"2024-04-26T20:50:32.649876+0000","last_unstale":"2024-04-26T20:50:38.803850+0000","last_undegraded":"2024-04-26T20:50:38.803850+0000","last_fullsized":"2024-04-26T20:50:38.803850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:00:12.394204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.799788+0000","last_change":"2024-04-26T20:50:32.651129+0000","last_active":"2024-04-26T20:50:38.799788+0000","last_peered":"2024-04-26T20:50:38.799788+0000","last_clean":"2024-04-26T20:50:38.799788+0000","last_became_active":"2024-04-26T20:50:32.650853+0000","last_became_peered":"2024-04-26T20:50:32.650853+0000","last_unstale":"2024-04-26T20:50:38.799788+0000","last_undegraded":"2024-04-26T20:50:38.799788+0000","last_fullsized":"2024-04-26T20:50:38.799788+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:35.639518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802726+0000","last_change":"2024-04-26T20:50:32.650192+0000","last_active":"2024-04-26T20:50:38.802726+0000","last_peered":"2024-04-26T20:50:38.802726+0000","last_clean":"2024-04-26T20:50:38.802726+0000","last_became_active":"2024-04-26T20:50:32.649911+0000","last_became_peered":"2024-04-26T20:50:32.649911+0000","last_unstale":"2024-04-26T20:50:38.802726+0000","last_undegraded":"2024-04-26T20:50:38.802726+0000","last_fullsized":"2024-04-26T20:50:38.802726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:09:36.737379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.799890+0000","last_change":"2024-04-26T20:50:32.641183+0000","last_active":"2024-04-26T20:50:38.799890+0000","last_peered":"2024-04-26T20:50:38.799890+0000","last_clean":"2024-04-26T20:50:38.799890+0000","last_became_active":"2024-04-26T20:50:32.640584+0000","last_became_peered":"2024-04-26T20:50:32.640584+0000","last_unstale":"2024-04-26T20:50:38.799890+0000","last_undegraded":"2024-04-26T20:50:38.799890+0000","last_fullsized":"2024-04-26T20:50:38.799890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:13:10.610044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803653+0000","last_change":"2024-04-26T20:50:32.651661+0000","last_active":"2024-04-26T20:50:38.803653+0000","last_peered":"2024-04-26T20:50:38.803653+0000","last_clean":"2024-04-26T20:50:38.803653+0000","last_became_active":"2024-04-26T20:50:32.651316+0000","last_became_peered":"2024-04-26T20:50:32.651316+0000","last_unstale":"2024-04-26T20:50:38.803653+0000","last_undegraded":"2024-04-26T20:50:38.803653+0000","last_fullsized":"2024-04-26T20:50:38.803653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:09:36.737379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.717426+0000","last_change":"2024-04-26T20:50:32.645619+0000","last_active":"2024-04-26T20:50:38.717426+0000","last_peered":"2024-04-26T20:50:38.717426+0000","last_clean":"2024-04-26T20:50:38.717426+0000","last_became_active":"2024-04-26T20:50:32.644645+0000","last_became_peered":"2024-04-26T20:50:32.644645+0000","last_unstale":"2024-04-26T20:50:38.717426+0000","last_undegraded":"2024-04-26T20:50:38.717426+0000","last_fullsized":"2024-04-26T20:50:38.717426+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:54.967490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.801055+0000","last_change":"2024-04-26T20:50:32.646646+0000","last_active":"2024-04-26T20:50:38.801055+0000","last_peered":"2024-04-26T20:50:38.801055+0000","last_clean":"2024-04-26T20:50:38.801055+0000","last_became_active":"2024-04-26T20:50:32.645404+0000","last_became_peered":"2024-04-26T20:50:32.645404+0000","last_unstale":"2024-04-26T20:50:38.801055+0000","last_undegraded":"2024-04-26T20:50:38.801055+0000","last_fullsized":"2024-04-26T20:50:38.801055+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:58:18.552075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803547+0000","last_change":"2024-04-26T20:50:32.646750+0000","last_active":"2024-04-26T20:50:38.803547+0000","last_peered":"2024-04-26T20:50:38.803547+0000","last_clean":"2024-04-26T20:50:38.803547+0000","last_became_active":"2024-04-26T20:50:32.646447+0000","last_became_peered":"2024-04-26T20:50:32.646447+0000","last_unstale":"2024-04-26T20:50:38.803547+0000","last_undegraded":"2024-04-26T20:50:38.803547+0000","last_fullsized":"2024-04-26T20:50:38.803547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:53:11.650251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800765+0000","last_change":"2024-04-26T20:50:32.642132+0000","last_active":"2024-04-26T20:50:38.800765+0000","last_peered":"2024-04-26T20:50:38.800765+0000","last_clean":"2024-04-26T20:50:38.800765+0000","last_became_active":"2024-04-26T20:50:32.641690+0000","last_became_peered":"2024-04-26T20:50:32.641690+0000","last_unstale":"2024-04-26T20:50:38.800765+0000","last_undegraded":"2024-04-26T20:50:38.800765+0000","last_fullsized":"2024-04-26T20:50:38.800765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:59:04.962815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.799661+0000","last_change":"2024-04-26T20:50:32.644332+0000","last_active":"2024-04-26T20:50:38.799661+0000","last_peered":"2024-04-26T20:50:38.799661+0000","last_clean":"2024-04-26T20:50:38.799661+0000","last_became_active":"2024-04-26T20:50:32.643783+0000","last_became_peered":"2024-04-26T20:50:32.643783+0000","last_unstale":"2024-04-26T20:50:38.799661+0000","last_undegraded":"2024-04-26T20:50:38.799661+0000","last_fullsized":"2024-04-26T20:50:38.799661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:56:51.926886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.660696+0000","last_change":"2024-04-26T20:50:32.638177+0000","last_active":"2024-04-26T20:50:38.660696+0000","last_peered":"2024-04-26T20:50:38.660696+0000","last_clean":"2024-04-26T20:50:38.660696+0000","last_became_active":"2024-04-26T20:50:32.637905+0000","last_became_peered":"2024-04-26T20:50:32.637905+0000","last_unstale":"2024-04-26T20:50:38.660696+0000","last_undegraded":"2024-04-26T20:50:38.660696+0000","last_fullsized":"2024-04-26T20:50:38.660696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:36.651960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803236+0000","last_change":"2024-04-26T20:50:32.637456+0000","last_active":"2024-04-26T20:50:38.803236+0000","last_peered":"2024-04-26T20:50:38.803236+0000","last_clean":"2024-04-26T20:50:38.803236+0000","last_became_active":"2024-04-26T20:50:32.637188+0000","last_became_peered":"2024-04-26T20:50:32.637188+0000","last_unstale":"2024-04-26T20:50:38.803236+0000","last_undegraded":"2024-04-26T20:50:38.803236+0000","last_fullsized":"2024-04-26T20:50:38.803236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:54.967490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802268+0000","last_change":"2024-04-26T20:50:32.649659+0000","last_active":"2024-04-26T20:50:38.802268+0000","last_peered":"2024-04-26T20:50:38.802268+0000","last_clean":"2024-04-26T20:50:38.802268+0000","last_became_active":"2024-04-26T20:50:32.645837+0000","last_became_peered":"2024-04-26T20:50:32.645837+0000","last_unstale":"2024-04-26T20:50:38.802268+0000","last_undegraded":"2024-04-26T20:50:38.802268+0000","last_fullsized":"2024-04-26T20:50:38.802268+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:55.634765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.662458+0000","last_change":"2024-04-26T20:50:32.643463+0000","last_active":"2024-04-26T20:50:38.662458+0000","last_peered":"2024-04-26T20:50:38.662458+0000","last_clean":"2024-04-26T20:50:38.662458+0000","last_became_active":"2024-04-26T20:50:32.642783+0000","last_became_peered":"2024-04-26T20:50:32.642783+0000","last_unstale":"2024-04-26T20:50:38.662458+0000","last_undegraded":"2024-04-26T20:50:38.662458+0000","last_fullsized":"2024-04-26T20:50:38.662458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:53:11.650251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.717503+0000","last_change":"2024-04-26T20:50:32.643690+0000","last_active":"2024-04-26T20:50:38.717503+0000","last_peered":"2024-04-26T20:50:38.717503+0000","last_clean":"2024-04-26T20:50:38.717503+0000","last_became_active":"2024-04-26T20:50:32.643412+0000","last_became_peered":"2024-04-26T20:50:32.643412+0000","last_unstale":"2024-04-26T20:50:38.717503+0000","last_undegraded":"2024-04-26T20:50:38.717503+0000","last_fullsized":"2024-04-26T20:50:38.717503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:55.634765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.717608+0000","last_change":"2024-04-26T20:50:32.643116+0000","last_active":"2024-04-26T20:50:38.717608+0000","last_peered":"2024-04-26T20:50:38.717608+0000","last_clean":"2024-04-26T20:50:38.717608+0000","last_became_active":"2024-04-26T20:50:32.642598+0000","last_became_peered":"2024-04-26T20:50:32.642598+0000","last_unstale":"2024-04-26T20:50:38.717608+0000","last_undegraded":"2024-04-26T20:50:38.717608+0000","last_fullsized":"2024-04-26T20:50:38.717608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:47:14.715739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.801040+0000","last_change":"2024-04-26T20:50:32.654019+0000","last_active":"2024-04-26T20:50:38.801040+0000","last_peered":"2024-04-26T20:50:38.801040+0000","last_clean":"2024-04-26T20:50:38.801040+0000","last_became_active":"2024-04-26T20:50:32.653569+0000","last_became_peered":"2024-04-26T20:50:32.653569+0000","last_unstale":"2024-04-26T20:50:38.801040+0000","last_undegraded":"2024-04-26T20:50:38.801040+0000","last_fullsized":"2024-04-26T20:50:38.801040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:13:10.610044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802660+0000","last_change":"2024-04-26T20:50:32.646837+0000","last_active":"2024-04-26T20:50:38.802660+0000","last_peered":"2024-04-26T20:50:38.802660+0000","last_clean":"2024-04-26T20:50:38.802660+0000","last_became_active":"2024-04-26T20:50:32.646528+0000","last_became_peered":"2024-04-26T20:50:32.646528+0000","last_unstale":"2024-04-26T20:50:38.802660+0000","last_undegraded":"2024-04-26T20:50:38.802660+0000","last_fullsized":"2024-04-26T20:50:38.802660+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:54.967490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800781+0000","last_change":"2024-04-26T20:50:32.654006+0000","last_active":"2024-04-26T20:50:38.800781+0000","last_peered":"2024-04-26T20:50:38.800781+0000","last_clean":"2024-04-26T20:50:38.800781+0000","last_became_active":"2024-04-26T20:50:32.653553+0000","last_became_peered":"2024-04-26T20:50:32.653553+0000","last_unstale":"2024-04-26T20:50:38.800781+0000","last_undegraded":"2024-04-26T20:50:38.800781+0000","last_fullsized":"2024-04-26T20:50:38.800781+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:00:12.394204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.799949+0000","last_change":"2024-04-26T20:50:32.641407+0000","last_active":"2024-04-26T20:50:38.799949+0000","last_peered":"2024-04-26T20:50:38.799949+0000","last_clean":"2024-04-26T20:50:38.799949+0000","last_became_active":"2024-04-26T20:50:32.640889+0000","last_became_peered":"2024-04-26T20:50:32.640889+0000","last_unstale":"2024-04-26T20:50:38.799949+0000","last_undegraded":"2024-04-26T20:50:38.799949+0000","last_fullsized":"2024-04-26T20:50:38.799949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:28:25.603590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800212+0000","last_change":"2024-04-26T20:50:32.645181+0000","last_active":"2024-04-26T20:50:38.800212+0000","last_peered":"2024-04-26T20:50:38.800212+0000","last_clean":"2024-04-26T20:50:38.800212+0000","last_became_active":"2024-04-26T20:50:32.644919+0000","last_became_peered":"2024-04-26T20:50:32.644919+0000","last_unstale":"2024-04-26T20:50:38.800212+0000","last_undegraded":"2024-04-26T20:50:38.800212+0000","last_fullsized":"2024-04-26T20:50:38.800212+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:40:19.711175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800200+0000","last_change":"2024-04-26T20:50:32.655095+0000","last_active":"2024-04-26T20:50:38.800200+0000","last_peered":"2024-04-26T20:50:38.800200+0000","last_clean":"2024-04-26T20:50:38.800200+0000","last_became_active":"2024-04-26T20:50:32.654821+0000","last_became_peered":"2024-04-26T20:50:32.654821+0000","last_unstale":"2024-04-26T20:50:38.800200+0000","last_undegraded":"2024-04-26T20:50:38.800200+0000","last_fullsized":"2024-04-26T20:50:38.800200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:33:28.969815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802460+0000","last_change":"2024-04-26T20:50:32.642769+0000","last_active":"2024-04-26T20:50:38.802460+0000","last_peered":"2024-04-26T20:50:38.802460+0000","last_clean":"2024-04-26T20:50:38.802460+0000","last_became_active":"2024-04-26T20:50:32.642506+0000","last_became_peered":"2024-04-26T20:50:32.642506+0000","last_unstale":"2024-04-26T20:50:38.802460+0000","last_undegraded":"2024-04-26T20:50:38.802460+0000","last_fullsized":"2024-04-26T20:50:38.802460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:55.634765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661146+0000","last_change":"2024-04-26T20:50:32.641048+0000","last_active":"2024-04-26T20:50:38.661146+0000","last_peered":"2024-04-26T20:50:38.661146+0000","last_clean":"2024-04-26T20:50:38.661146+0000","last_became_active":"2024-04-26T20:50:32.640572+0000","last_became_peered":"2024-04-26T20:50:32.640572+0000","last_unstale":"2024-04-26T20:50:38.661146+0000","last_undegraded":"2024-04-26T20:50:38.661146+0000","last_fullsized":"2024-04-26T20:50:38.661146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:47:14.715739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800925+0000","last_change":"2024-04-26T20:50:32.650971+0000","last_active":"2024-04-26T20:50:38.800925+0000","last_peered":"2024-04-26T20:50:38.800925+0000","last_clean":"2024-04-26T20:50:38.800925+0000","last_became_active":"2024-04-26T20:50:32.650713+0000","last_became_peered":"2024-04-26T20:50:32.650713+0000","last_unstale":"2024-04-26T20:50:38.800925+0000","last_undegraded":"2024-04-26T20:50:38.800925+0000","last_fullsized":"2024-04-26T20:50:38.800925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:28:25.603590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":864256,"data_stored":698343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":107483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":107519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":107393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":107553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":872448,"data_stored":698399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:50:55.682 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:50:55.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.812+0000 7f4d99a33640 1 Processor -- start 2024-04-26T20:50:55.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.812+0000 7f4d99a33640 1 -- start start 2024-04-26T20:50:55.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.813+0000 7f4d99a33640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4d940d06f0 0x7f4d940d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:55.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.813+0000 7f4d99a33640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f4d940b2850 0x7f4d940b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:55.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.813+0000 7f4d99a33640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d940d0ba0 con 0x7f4d9415f3a0 2024-04-26T20:50:55.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.813+0000 7f4d99a33640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f4d940d0e20 con 0x7f4d940d06f0 2024-04-26T20:50:55.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.813+0000 7f4d99a33640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f4d940b1d70 con 0x7f4d940b2850 2024-04-26T20:50:55.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.813+0000 7f4d8bfff640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4d940d06f0 0x7f4d940d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:55.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.813+0000 7f4d92ffd640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f4d9415f3a0 0x7f4d940d1770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33062/0 (socket says 172.21.15.73:33062) 2024-04-26T20:50:55.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.813+0000 7f4d92ffd640 1 -- 172.21.15.73:0/3760788506 learned_addr learned my addr 172.21.15.73:0/3760788506 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:50:55.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.813+0000 7f4d937fe640 1 --2- 172.21.15.73:0/3760788506 >> v2:172.21.15.120:3300/0 conn(0x7f4d940b2850 0x7f4d940b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:55.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.813+0000 7f4d8bfff640 1 -- 172.21.15.73:0/3760788506 >> v2:172.21.15.120:3300/0 conn(0x7f4d940b2850 msgr2=0x7f4d940b2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:55.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.813+0000 7f4d8bfff640 1 --2- 172.21.15.73:0/3760788506 >> v2:172.21.15.120:3300/0 conn(0x7f4d940b2850 0x7f4d940b2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:55.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.813+0000 7f4d8bfff640 1 -- 172.21.15.73:0/3760788506 >> v1:172.21.15.73:6789/0 conn(0x7f4d9415f3a0 legacy=0x7f4d940d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:55.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.813+0000 7f4d8bfff640 1 -- 172.21.15.73:0/3760788506 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d940b2050 con 0x7f4d940d06f0 2024-04-26T20:50:55.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.813+0000 7f4d927fc640 1 -- 172.21.15.73:0/3760788506 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3176197402 0 0) 0x7f4d84001110 con 0x7f4d9415f3a0 2024-04-26T20:50:55.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.814+0000 7f4d8bfff640 1 --2- 172.21.15.73:0/3760788506 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d940d06f0 0x7f4d940d1e80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4d940c89f0 tx=0x7f4d7c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce9518d202d08a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:55.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.814+0000 7f4d927fc640 1 -- 172.21.15.73:0/3760788506 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c037070 con 0x7f4d940d06f0 2024-04-26T20:50:55.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.814+0000 7f4d927fc640 1 -- 172.21.15.73:0/3760788506 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d7c032440 con 0x7f4d940d06f0 2024-04-26T20:50:55.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.814+0000 7f4d927fc640 1 -- 172.21.15.73:0/3760788506 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c032810 con 0x7f4d940d06f0 2024-04-26T20:50:55.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.815+0000 7f4d99a33640 1 -- 172.21.15.73:0/3760788506 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d940d06f0 msgr2=0x7f4d940d1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:55.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.815+0000 7f4d99a33640 1 --2- 172.21.15.73:0/3760788506 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d940d06f0 0x7f4d940d1e80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4d940c89f0 tx=0x7f4d7c02a1a0 comp rx=0 tx=0).stop 2024-04-26T20:50:55.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.815+0000 7f4d99a33640 1 -- 172.21.15.73:0/3760788506 shutdown_connections 2024-04-26T20:50:55.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.815+0000 7f4d99a33640 1 --2- 172.21.15.73:0/3760788506 >> v2:172.21.15.120:3300/0 conn(0x7f4d940b2850 0x7f4d940b2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:55.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.815+0000 7f4d99a33640 1 --2- 172.21.15.73:0/3760788506 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d940d06f0 0x7f4d940d1e80 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:55.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.815+0000 7f4d99a33640 1 -- 172.21.15.73:0/3760788506 >> 172.21.15.73:0/3760788506 conn(0x7f4d940d4f90 msgr2=0x7f4d940d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:55.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.815+0000 7f4d99a33640 1 -- 172.21.15.73:0/3760788506 shutdown_connections 2024-04-26T20:50:55.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.815+0000 7f4d99a33640 1 -- 172.21.15.73:0/3760788506 wait complete. 2024-04-26T20:50:55.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.816+0000 7f4d99a33640 1 Processor -- start 2024-04-26T20:50:55.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.816+0000 7f4d99a33640 1 -- start start 2024-04-26T20:50:55.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.816+0000 7f4d99a33640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d940b2850 0x7f4d94159950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:55.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.816+0000 7f4d99a33640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d940d06f0 0x7f4d9415da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:55.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.816+0000 7f4d99a33640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4d9415f3a0 0x7f4d9415df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:55.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.816+0000 7f4d99a33640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4d9415f900 con 0x7f4d940d06f0 2024-04-26T20:50:55.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.816+0000 7f4d99a33640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4d9415f780 con 0x7f4d940b2850 2024-04-26T20:50:55.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.816+0000 7f4d99a33640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f4d940d23c0 con 0x7f4d9415f3a0 2024-04-26T20:50:55.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.817+0000 7f4d8bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d940d06f0 0x7f4d9415da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:55.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.817+0000 7f4d8bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d940d06f0 0x7f4d9415da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49400/0 (socket says 172.21.15.73:49400) 2024-04-26T20:50:55.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.817+0000 7f4d8bfff640 1 -- 172.21.15.73:0/1109586945 learned_addr learned my addr 172.21.15.73:0/1109586945 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:55.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.817+0000 7f4d8bfff640 1 -- 172.21.15.73:0/1109586945 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4d9415f3a0 msgr2=0x7f4d9415df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:55.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.817+0000 7f4d8bfff640 1 --2- 172.21.15.73:0/1109586945 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4d9415f3a0 0x7f4d9415df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:55.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.817+0000 7f4d8bfff640 1 -- 172.21.15.73:0/1109586945 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d940b2850 msgr2=0x7f4d94159950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:55.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.817+0000 7f4d8bfff640 1 --2- 172.21.15.73:0/1109586945 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d940b2850 0x7f4d94159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:55.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.817+0000 7f4d8bfff640 1 -- 172.21.15.73:0/1109586945 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d940b7c20 con 0x7f4d940d06f0 2024-04-26T20:50:55.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.817+0000 7f4d8bfff640 1 --2- 172.21.15.73:0/1109586945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d940d06f0 0x7f4d9415da00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c000c00 tx=0x7f4d7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:55.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.817+0000 7f4d90ff9640 1 -- 172.21.15.73:0/1109586945 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c037040 con 0x7f4d940d06f0 2024-04-26T20:50:55.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.817+0000 7f4d90ff9640 1 -- 172.21.15.73:0/1109586945 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d7c042070 con 0x7f4d940d06f0 2024-04-26T20:50:55.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.817+0000 7f4d90ff9640 1 -- 172.21.15.73:0/1109586945 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c030bd0 con 0x7f4d940d06f0 2024-04-26T20:50:55.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.817+0000 7f4d99a33640 1 -- 172.21.15.73:0/1109586945 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d940b7f00 con 0x7f4d940d06f0 2024-04-26T20:50:55.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.817+0000 7f4d99a33640 1 -- 172.21.15.73:0/1109586945 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d940b84c0 con 0x7f4d940d06f0 2024-04-26T20:50:55.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.818+0000 7f4d99a33640 1 -- 172.21.15.73:0/1109586945 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d940d3ec0 con 0x7f4d940d06f0 2024-04-26T20:50:55.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.819+0000 7f4d90ff9640 1 -- 172.21.15.73:0/1109586945 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4d7c047460 con 0x7f4d940d06f0 2024-04-26T20:50:55.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.819+0000 7f4d90ff9640 1 --2- 172.21.15.73:0/1109586945 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f4d7406c920 0x7f4d7406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:55.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.819+0000 7f4d90ff9640 1 -- 172.21.15.73:0/1109586945 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f4d7c0aca30 con 0x7f4d940d06f0 2024-04-26T20:50:55.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.819+0000 7f4d92ffd640 1 --2- 172.21.15.73:0/1109586945 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f4d7406c920 0x7f4d7406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:55.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.819+0000 7f4d92ffd640 1 --2- 172.21.15.73:0/1109586945 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f4d7406c920 0x7f4d7406ede0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4d84007fd0 tx=0x7f4d84007960 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:55.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.823+0000 7f4d90ff9640 1 -- 172.21.15.73:0/1109586945 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4d940d3ec0 con 0x7f4d940d06f0 2024-04-26T20:50:55.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:55.995+0000 7f4d99a33640 1 -- 172.21.15.73:0/1109586945 --> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4d940b8700 con 0x7f4d7406c920 2024-04-26T20:50:56.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.003+0000 7f4d90ff9640 1 -- 172.21.15.73:0/1109586945 <== mgr.4107 v2:172.21.15.73:6832/666905549 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7f4d840070d0 con 0x7f4d7406c920 2024-04-26T20:50:56.007 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:50:56.009 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-26T20:50:56.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.007+0000 7f4d99a33640 1 -- 172.21.15.73:0/1109586945 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f4d7406c920 msgr2=0x7f4d7406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:56.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.007+0000 7f4d99a33640 1 --2- 172.21.15.73:0/1109586945 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f4d7406c920 0x7f4d7406ede0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4d84007fd0 tx=0x7f4d84007960 comp rx=0 tx=0).stop 2024-04-26T20:50:56.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.007+0000 7f4d99a33640 1 -- 172.21.15.73:0/1109586945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d940d06f0 msgr2=0x7f4d9415da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:56.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.007+0000 7f4d99a33640 1 --2- 172.21.15.73:0/1109586945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d940d06f0 0x7f4d9415da00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c000c00 tx=0x7f4d7c0046c0 comp rx=0 tx=0).stop 2024-04-26T20:50:56.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.007+0000 7f4d99a33640 1 -- 172.21.15.73:0/1109586945 shutdown_connections 2024-04-26T20:50:56.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.007+0000 7f4d99a33640 1 --2- 172.21.15.73:0/1109586945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d940d06f0 0x7f4d9415da00 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:56.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.007+0000 7f4d99a33640 1 --2- 172.21.15.73:0/1109586945 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f4d7406c920 0x7f4d7406ede0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:56.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.007+0000 7f4d99a33640 1 --2- 172.21.15.73:0/1109586945 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4d9415f3a0 0x7f4d9415df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:56.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.007+0000 7f4d99a33640 1 --2- 172.21.15.73:0/1109586945 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d940b2850 0x7f4d94159950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:56.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.007+0000 7f4d99a33640 1 -- 172.21.15.73:0/1109586945 >> 172.21.15.73:0/1109586945 conn(0x7f4d940d4f90 msgr2=0x7f4d940d42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:56.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.007+0000 7f4d99a33640 1 -- 172.21.15.73:0/1109586945 shutdown_connections 2024-04-26T20:50:56.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.007+0000 7f4d99a33640 1 -- 172.21.15.73:0/1109586945 wait complete. 2024-04-26T20:50:56.019 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-04-26T20:50:55.983916+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219468,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755252,"statfs":{"total":773094113280,"available":772869378048,"internally_reserved":0,"allocated":3383296,"data_stored":2049160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002579"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803145+0000","last_change":"2024-04-26T20:50:31.630770+0000","last_active":"2024-04-26T20:50:38.803145+0000","last_peered":"2024-04-26T20:50:38.803145+0000","last_clean":"2024-04-26T20:50:38.803145+0000","last_became_active":"2024-04-26T20:50:31.630518+0000","last_became_peered":"2024-04-26T20:50:31.630518+0000","last_unstale":"2024-04-26T20:50:38.803145+0000","last_undegraded":"2024-04-26T20:50:38.803145+0000","last_fullsized":"2024-04-26T20:50:38.803145+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:20:49.853735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803210+0000","last_change":"2024-04-26T20:50:32.649145+0000","last_active":"2024-04-26T20:50:38.803210+0000","last_peered":"2024-04-26T20:50:38.803210+0000","last_clean":"2024-04-26T20:50:38.803210+0000","last_became_active":"2024-04-26T20:50:32.648873+0000","last_became_peered":"2024-04-26T20:50:32.648873+0000","last_unstale":"2024-04-26T20:50:38.803210+0000","last_undegraded":"2024-04-26T20:50:38.803210+0000","last_fullsized":"2024-04-26T20:50:38.803210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:16:25.308786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802643+0000","last_change":"2024-04-26T20:50:37.296160+0000","last_active":"2024-04-26T20:50:38.802643+0000","last_peered":"2024-04-26T20:50:38.802643+0000","last_clean":"2024-04-26T20:50:38.802643+0000","last_became_active":"2024-04-26T20:50:37.295441+0000","last_became_peered":"2024-04-26T20:50:37.295441+0000","last_unstale":"2024-04-26T20:50:38.802643+0000","last_undegraded":"2024-04-26T20:50:38.802643+0000","last_fullsized":"2024-04-26T20:50:38.802643+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:20:48.160586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803389+0000","last_change":"2024-04-26T20:50:31.634354+0000","last_active":"2024-04-26T20:50:38.803389+0000","last_peered":"2024-04-26T20:50:38.803389+0000","last_clean":"2024-04-26T20:50:38.803389+0000","last_became_active":"2024-04-26T20:50:31.634084+0000","last_became_peered":"2024-04-26T20:50:31.634084+0000","last_unstale":"2024-04-26T20:50:38.803389+0000","last_undegraded":"2024-04-26T20:50:38.803389+0000","last_fullsized":"2024-04-26T20:50:38.803389+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:49:28.099373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802875+0000","last_change":"2024-04-26T20:50:32.645829+0000","last_active":"2024-04-26T20:50:38.802875+0000","last_peered":"2024-04-26T20:50:38.802875+0000","last_clean":"2024-04-26T20:50:38.802875+0000","last_became_active":"2024-04-26T20:50:32.645589+0000","last_became_peered":"2024-04-26T20:50:32.645589+0000","last_unstale":"2024-04-26T20:50:38.802875+0000","last_undegraded":"2024-04-26T20:50:38.802875+0000","last_fullsized":"2024-04-26T20:50:38.802875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:36.651960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661856+0000","last_change":"2024-04-26T20:50:36.675389+0000","last_active":"2024-04-26T20:50:38.661856+0000","last_peered":"2024-04-26T20:50:38.661856+0000","last_clean":"2024-04-26T20:50:38.661856+0000","last_became_active":"2024-04-26T20:50:36.675027+0000","last_became_peered":"2024-04-26T20:50:36.675027+0000","last_unstale":"2024-04-26T20:50:38.661856+0000","last_undegraded":"2024-04-26T20:50:38.661856+0000","last_fullsized":"2024-04-26T20:50:38.661856+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:13:14.645783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800942+0000","last_change":"2024-04-26T20:50:31.631388+0000","last_active":"2024-04-26T20:50:38.800942+0000","last_peered":"2024-04-26T20:50:38.800942+0000","last_clean":"2024-04-26T20:50:38.800942+0000","last_became_active":"2024-04-26T20:50:31.631121+0000","last_became_peered":"2024-04-26T20:50:31.631121+0000","last_unstale":"2024-04-26T20:50:38.800942+0000","last_undegraded":"2024-04-26T20:50:38.800942+0000","last_fullsized":"2024-04-26T20:50:38.800942+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:46:19.858595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800306+0000","last_change":"2024-04-26T20:50:32.644708+0000","last_active":"2024-04-26T20:50:38.800306+0000","last_peered":"2024-04-26T20:50:38.800306+0000","last_clean":"2024-04-26T20:50:38.800306+0000","last_became_active":"2024-04-26T20:50:32.644440+0000","last_became_peered":"2024-04-26T20:50:32.644440+0000","last_unstale":"2024-04-26T20:50:38.800306+0000","last_undegraded":"2024-04-26T20:50:38.800306+0000","last_fullsized":"2024-04-26T20:50:38.800306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:40:19.711175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.662138+0000","last_change":"2024-04-26T20:50:36.676949+0000","last_active":"2024-04-26T20:50:38.662138+0000","last_peered":"2024-04-26T20:50:38.662138+0000","last_clean":"2024-04-26T20:50:38.662138+0000","last_became_active":"2024-04-26T20:50:36.676533+0000","last_became_peered":"2024-04-26T20:50:36.676533+0000","last_unstale":"2024-04-26T20:50:38.662138+0000","last_undegraded":"2024-04-26T20:50:38.662138+0000","last_fullsized":"2024-04-26T20:50:38.662138+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:59:08.998553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800828+0000","last_change":"2024-04-26T20:50:31.632017+0000","last_active":"2024-04-26T20:50:38.800828+0000","last_peered":"2024-04-26T20:50:38.800828+0000","last_clean":"2024-04-26T20:50:38.800828+0000","last_became_active":"2024-04-26T20:50:31.631717+0000","last_became_peered":"2024-04-26T20:50:31.631717+0000","last_unstale":"2024-04-26T20:50:38.800828+0000","last_undegraded":"2024-04-26T20:50:38.800828+0000","last_fullsized":"2024-04-26T20:50:38.800828+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:32:17.604576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800366+0000","last_change":"2024-04-26T20:50:32.644081+0000","last_active":"2024-04-26T20:50:38.800366+0000","last_peered":"2024-04-26T20:50:38.800366+0000","last_clean":"2024-04-26T20:50:38.800366+0000","last_became_active":"2024-04-26T20:50:32.643809+0000","last_became_peered":"2024-04-26T20:50:32.643809+0000","last_unstale":"2024-04-26T20:50:38.800366+0000","last_undegraded":"2024-04-26T20:50:38.800366+0000","last_fullsized":"2024-04-26T20:50:38.800366+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:16:25.308786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.662248+0000","last_change":"2024-04-26T20:50:36.679187+0000","last_active":"2024-04-26T20:50:38.662248+0000","last_peered":"2024-04-26T20:50:38.662248+0000","last_clean":"2024-04-26T20:50:38.662248+0000","last_became_active":"2024-04-26T20:50:36.678782+0000","last_became_peered":"2024-04-26T20:50:36.678782+0000","last_unstale":"2024-04-26T20:50:38.662248+0000","last_undegraded":"2024-04-26T20:50:38.662248+0000","last_fullsized":"2024-04-26T20:50:38.662248+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:12.140650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.717057+0000","last_change":"2024-04-26T20:50:32.646429+0000","last_active":"2024-04-26T20:50:38.717057+0000","last_peered":"2024-04-26T20:50:38.717057+0000","last_clean":"2024-04-26T20:50:38.717057+0000","last_became_active":"2024-04-26T20:50:32.646134+0000","last_became_peered":"2024-04-26T20:50:32.646134+0000","last_unstale":"2024-04-26T20:50:38.717057+0000","last_undegraded":"2024-04-26T20:50:38.717057+0000","last_fullsized":"2024-04-26T20:50:38.717057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:09:36.737379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800514+0000","last_change":"2024-04-26T20:50:31.631250+0000","last_active":"2024-04-26T20:50:38.800514+0000","last_peered":"2024-04-26T20:50:38.800514+0000","last_clean":"2024-04-26T20:50:38.800514+0000","last_became_active":"2024-04-26T20:50:31.630277+0000","last_became_peered":"2024-04-26T20:50:31.630277+0000","last_unstale":"2024-04-26T20:50:38.800514+0000","last_undegraded":"2024-04-26T20:50:38.800514+0000","last_fullsized":"2024-04-26T20:50:38.800514+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:06:39.630339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.662616+0000","last_change":"2024-04-26T20:50:36.676056+0000","last_active":"2024-04-26T20:50:38.662616+0000","last_peered":"2024-04-26T20:50:38.662616+0000","last_clean":"2024-04-26T20:50:38.662616+0000","last_became_active":"2024-04-26T20:50:36.675706+0000","last_became_peered":"2024-04-26T20:50:36.675706+0000","last_unstale":"2024-04-26T20:50:38.662616+0000","last_undegraded":"2024-04-26T20:50:38.662616+0000","last_fullsized":"2024-04-26T20:50:38.662616+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:13:08.850173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802373+0000","last_change":"2024-04-26T20:50:32.650473+0000","last_active":"2024-04-26T20:50:38.802373+0000","last_peered":"2024-04-26T20:50:38.802373+0000","last_clean":"2024-04-26T20:50:38.802373+0000","last_became_active":"2024-04-26T20:50:32.649711+0000","last_became_peered":"2024-04-26T20:50:32.649711+0000","last_unstale":"2024-04-26T20:50:38.802373+0000","last_undegraded":"2024-04-26T20:50:38.802373+0000","last_fullsized":"2024-04-26T20:50:38.802373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:36.651960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661208+0000","last_change":"2024-04-26T20:50:31.629572+0000","last_active":"2024-04-26T20:50:38.661208+0000","last_peered":"2024-04-26T20:50:38.661208+0000","last_clean":"2024-04-26T20:50:38.661208+0000","last_became_active":"2024-04-26T20:50:31.629287+0000","last_became_peered":"2024-04-26T20:50:31.629287+0000","last_unstale":"2024-04-26T20:50:38.661208+0000","last_undegraded":"2024-04-26T20:50:38.661208+0000","last_fullsized":"2024-04-26T20:50:38.661208+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:36.499651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.662538+0000","last_change":"2024-04-26T20:50:36.681910+0000","last_active":"2024-04-26T20:50:38.662538+0000","last_peered":"2024-04-26T20:50:38.662538+0000","last_clean":"2024-04-26T20:50:38.662538+0000","last_became_active":"2024-04-26T20:50:36.681432+0000","last_became_peered":"2024-04-26T20:50:36.681432+0000","last_unstale":"2024-04-26T20:50:38.662538+0000","last_undegraded":"2024-04-26T20:50:38.662538+0000","last_fullsized":"2024-04-26T20:50:38.662538+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:27:18.035795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802910+0000","last_change":"2024-04-26T20:50:32.647018+0000","last_active":"2024-04-26T20:50:38.802910+0000","last_peered":"2024-04-26T20:50:38.802910+0000","last_clean":"2024-04-26T20:50:38.802910+0000","last_became_active":"2024-04-26T20:50:32.646748+0000","last_became_peered":"2024-04-26T20:50:32.646748+0000","last_unstale":"2024-04-26T20:50:38.802910+0000","last_undegraded":"2024-04-26T20:50:38.802910+0000","last_fullsized":"2024-04-26T20:50:38.802910+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:56:51.926886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.660717+0000","last_change":"2024-04-26T20:50:31.631760+0000","last_active":"2024-04-26T20:50:38.660717+0000","last_peered":"2024-04-26T20:50:38.660717+0000","last_clean":"2024-04-26T20:50:38.660717+0000","last_became_active":"2024-04-26T20:50:31.629437+0000","last_became_peered":"2024-04-26T20:50:31.629437+0000","last_unstale":"2024-04-26T20:50:38.660717+0000","last_undegraded":"2024-04-26T20:50:38.660717+0000","last_fullsized":"2024-04-26T20:50:38.660717+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:23:29.866794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661242+0000","last_change":"2024-04-26T20:50:37.295810+0000","last_active":"2024-04-26T20:50:38.661242+0000","last_peered":"2024-04-26T20:50:38.661242+0000","last_clean":"2024-04-26T20:50:38.661242+0000","last_became_active":"2024-04-26T20:50:37.295425+0000","last_became_peered":"2024-04-26T20:50:37.295425+0000","last_unstale":"2024-04-26T20:50:38.661242+0000","last_undegraded":"2024-04-26T20:50:38.661242+0000","last_fullsized":"2024-04-26T20:50:38.661242+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:31:21.850102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.662187+0000","last_change":"2024-04-26T20:50:32.638584+0000","last_active":"2024-04-26T20:50:38.662187+0000","last_peered":"2024-04-26T20:50:38.662187+0000","last_clean":"2024-04-26T20:50:38.662187+0000","last_became_active":"2024-04-26T20:50:32.637734+0000","last_became_peered":"2024-04-26T20:50:32.637734+0000","last_unstale":"2024-04-26T20:50:38.662187+0000","last_undegraded":"2024-04-26T20:50:38.662187+0000","last_fullsized":"2024-04-26T20:50:38.662187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:37.504098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800397+0000","last_change":"2024-04-26T20:50:31.634926+0000","last_active":"2024-04-26T20:50:38.800397+0000","last_peered":"2024-04-26T20:50:38.800397+0000","last_clean":"2024-04-26T20:50:38.800397+0000","last_became_active":"2024-04-26T20:50:31.634278+0000","last_became_peered":"2024-04-26T20:50:31.634278+0000","last_unstale":"2024-04-26T20:50:38.800397+0000","last_undegraded":"2024-04-26T20:50:38.800397+0000","last_fullsized":"2024-04-26T20:50:38.800397+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:36.499651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.662287+0000","last_change":"2024-04-26T20:50:36.678377+0000","last_active":"2024-04-26T20:50:38.662287+0000","last_peered":"2024-04-26T20:50:38.662287+0000","last_clean":"2024-04-26T20:50:38.662287+0000","last_became_active":"2024-04-26T20:50:36.677946+0000","last_became_peered":"2024-04-26T20:50:36.677946+0000","last_unstale":"2024-04-26T20:50:38.662287+0000","last_undegraded":"2024-04-26T20:50:38.662287+0000","last_fullsized":"2024-04-26T20:50:38.662287+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:31:21.850102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803961+0000","last_change":"2024-04-26T20:50:32.637345+0000","last_active":"2024-04-26T20:50:38.803961+0000","last_peered":"2024-04-26T20:50:38.803961+0000","last_clean":"2024-04-26T20:50:38.803961+0000","last_became_active":"2024-04-26T20:50:32.637071+0000","last_became_peered":"2024-04-26T20:50:32.637071+0000","last_unstale":"2024-04-26T20:50:38.803961+0000","last_undegraded":"2024-04-26T20:50:38.803961+0000","last_fullsized":"2024-04-26T20:50:38.803961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:47:14.715739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.717764+0000","last_change":"2024-04-26T20:50:31.631119+0000","last_active":"2024-04-26T20:50:38.717764+0000","last_peered":"2024-04-26T20:50:38.717764+0000","last_clean":"2024-04-26T20:50:38.717764+0000","last_became_active":"2024-04-26T20:50:31.630825+0000","last_became_peered":"2024-04-26T20:50:31.630825+0000","last_unstale":"2024-04-26T20:50:38.717764+0000","last_undegraded":"2024-04-26T20:50:38.717764+0000","last_fullsized":"2024-04-26T20:50:38.717764+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:26.601021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661961+0000","last_change":"2024-04-26T20:50:37.295783+0000","last_active":"2024-04-26T20:50:38.661961+0000","last_peered":"2024-04-26T20:50:38.661961+0000","last_clean":"2024-04-26T20:50:38.661961+0000","last_became_active":"2024-04-26T20:50:37.295382+0000","last_became_peered":"2024-04-26T20:50:37.295382+0000","last_unstale":"2024-04-26T20:50:38.661961+0000","last_undegraded":"2024-04-26T20:50:38.661961+0000","last_fullsized":"2024-04-26T20:50:38.661961+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:58:22.587813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803938+0000","last_change":"2024-04-26T20:50:32.649792+0000","last_active":"2024-04-26T20:50:38.803938+0000","last_peered":"2024-04-26T20:50:38.803938+0000","last_clean":"2024-04-26T20:50:38.803938+0000","last_became_active":"2024-04-26T20:50:32.649510+0000","last_became_peered":"2024-04-26T20:50:32.649510+0000","last_unstale":"2024-04-26T20:50:38.803938+0000","last_undegraded":"2024-04-26T20:50:38.803938+0000","last_fullsized":"2024-04-26T20:50:38.803938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:35.639518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661244+0000","last_change":"2024-04-26T20:50:31.631623+0000","last_active":"2024-04-26T20:50:38.661244+0000","last_peered":"2024-04-26T20:50:38.661244+0000","last_clean":"2024-04-26T20:50:38.661244+0000","last_became_active":"2024-04-26T20:50:31.631312+0000","last_became_peered":"2024-04-26T20:50:31.631312+0000","last_unstale":"2024-04-26T20:50:38.661244+0000","last_undegraded":"2024-04-26T20:50:38.661244+0000","last_fullsized":"2024-04-26T20:50:38.661244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:46:19.858595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802802+0000","last_change":"2024-04-26T20:50:37.295508+0000","last_active":"2024-04-26T20:50:38.802802+0000","last_peered":"2024-04-26T20:50:38.802802+0000","last_clean":"2024-04-26T20:50:38.802802+0000","last_became_active":"2024-04-26T20:50:37.294858+0000","last_became_peered":"2024-04-26T20:50:37.294858+0000","last_unstale":"2024-04-26T20:50:38.802802+0000","last_undegraded":"2024-04-26T20:50:38.802802+0000","last_fullsized":"2024-04-26T20:50:38.802802+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:42:14.026063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661292+0000","last_change":"2024-04-26T20:50:32.640053+0000","last_active":"2024-04-26T20:50:38.661292+0000","last_peered":"2024-04-26T20:50:38.661292+0000","last_clean":"2024-04-26T20:50:38.661292+0000","last_became_active":"2024-04-26T20:50:32.639778+0000","last_became_peered":"2024-04-26T20:50:32.639778+0000","last_unstale":"2024-04-26T20:50:38.661292+0000","last_undegraded":"2024-04-26T20:50:38.661292+0000","last_fullsized":"2024-04-26T20:50:38.661292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:09:36.737379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.801056+0000","last_change":"2024-04-26T20:50:31.634409+0000","last_active":"2024-04-26T20:50:38.801056+0000","last_peered":"2024-04-26T20:50:38.801056+0000","last_clean":"2024-04-26T20:50:38.801056+0000","last_became_active":"2024-04-26T20:50:31.634103+0000","last_became_peered":"2024-04-26T20:50:31.634103+0000","last_unstale":"2024-04-26T20:50:38.801056+0000","last_undegraded":"2024-04-26T20:50:38.801056+0000","last_fullsized":"2024-04-26T20:50:38.801056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:40:15.520084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803602+0000","last_change":"2024-04-26T20:50:37.292899+0000","last_active":"2024-04-26T20:50:38.803602+0000","last_peered":"2024-04-26T20:50:38.803602+0000","last_clean":"2024-04-26T20:50:38.803602+0000","last_became_active":"2024-04-26T20:50:37.292379+0000","last_became_peered":"2024-04-26T20:50:37.292379+0000","last_unstale":"2024-04-26T20:50:38.803602+0000","last_undegraded":"2024-04-26T20:50:38.803602+0000","last_fullsized":"2024-04-26T20:50:38.803602+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:28:04.995069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800475+0000","last_change":"2024-04-26T20:50:32.649145+0000","last_active":"2024-04-26T20:50:38.800475+0000","last_peered":"2024-04-26T20:50:38.800475+0000","last_clean":"2024-04-26T20:50:38.800475+0000","last_became_active":"2024-04-26T20:50:32.648860+0000","last_became_peered":"2024-04-26T20:50:32.648860+0000","last_unstale":"2024-04-26T20:50:38.800475+0000","last_undegraded":"2024-04-26T20:50:38.800475+0000","last_fullsized":"2024-04-26T20:50:38.800475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:45:44.950496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661412+0000","last_change":"2024-04-26T20:50:31.627675+0000","last_active":"2024-04-26T20:50:38.661412+0000","last_peered":"2024-04-26T20:50:38.661412+0000","last_clean":"2024-04-26T20:50:38.661412+0000","last_became_active":"2024-04-26T20:50:31.627373+0000","last_became_peered":"2024-04-26T20:50:31.627373+0000","last_unstale":"2024-04-26T20:50:38.661412+0000","last_undegraded":"2024-04-26T20:50:38.661412+0000","last_fullsized":"2024-04-26T20:50:38.661412+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:26.601021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.660956+0000","last_change":"2024-04-26T20:50:36.679728+0000","last_active":"2024-04-26T20:50:38.660956+0000","last_peered":"2024-04-26T20:50:38.660956+0000","last_clean":"2024-04-26T20:50:38.660956+0000","last_became_active":"2024-04-26T20:50:36.678881+0000","last_became_peered":"2024-04-26T20:50:36.678881+0000","last_unstale":"2024-04-26T20:50:38.660956+0000","last_undegraded":"2024-04-26T20:50:38.660956+0000","last_fullsized":"2024-04-26T20:50:38.660956+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:59:08.998553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.662645+0000","last_change":"2024-04-26T20:50:32.633552+0000","last_active":"2024-04-26T20:50:38.662645+0000","last_peered":"2024-04-26T20:50:38.662645+0000","last_clean":"2024-04-26T20:50:38.662645+0000","last_became_active":"2024-04-26T20:50:32.632649+0000","last_became_peered":"2024-04-26T20:50:32.632649+0000","last_unstale":"2024-04-26T20:50:38.662645+0000","last_undegraded":"2024-04-26T20:50:38.662645+0000","last_fullsized":"2024-04-26T20:50:38.662645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:32:18.609023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800518+0000","last_change":"2024-04-26T20:50:31.622898+0000","last_active":"2024-04-26T20:50:38.800518+0000","last_peered":"2024-04-26T20:50:38.800518+0000","last_clean":"2024-04-26T20:50:38.800518+0000","last_became_active":"2024-04-26T20:50:31.622583+0000","last_became_peered":"2024-04-26T20:50:31.622583+0000","last_unstale":"2024-04-26T20:50:38.800518+0000","last_undegraded":"2024-04-26T20:50:38.800518+0000","last_fullsized":"2024-04-26T20:50:38.800518+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:53:10.645804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661250+0000","last_change":"2024-04-26T20:50:36.681863+0000","last_active":"2024-04-26T20:50:38.661250+0000","last_peered":"2024-04-26T20:50:38.661250+0000","last_clean":"2024-04-26T20:50:38.661250+0000","last_became_active":"2024-04-26T20:50:36.681433+0000","last_became_peered":"2024-04-26T20:50:36.681433+0000","last_unstale":"2024-04-26T20:50:38.661250+0000","last_undegraded":"2024-04-26T20:50:38.661250+0000","last_fullsized":"2024-04-26T20:50:38.661250+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:28:04.995069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661447+0000","last_change":"2024-04-26T20:50:32.640962+0000","last_active":"2024-04-26T20:50:38.661447+0000","last_peered":"2024-04-26T20:50:38.661447+0000","last_clean":"2024-04-26T20:50:38.661447+0000","last_became_active":"2024-04-26T20:50:32.640670+0000","last_became_peered":"2024-04-26T20:50:32.640670+0000","last_unstale":"2024-04-26T20:50:38.661447+0000","last_undegraded":"2024-04-26T20:50:38.661447+0000","last_fullsized":"2024-04-26T20:50:38.661447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:54.967490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802635+0000","last_change":"2024-04-26T20:50:31.631694+0000","last_active":"2024-04-26T20:50:38.802635+0000","last_peered":"2024-04-26T20:50:38.802635+0000","last_clean":"2024-04-26T20:50:38.802635+0000","last_became_active":"2024-04-26T20:50:31.631330+0000","last_became_peered":"2024-04-26T20:50:31.631330+0000","last_unstale":"2024-04-26T20:50:38.802635+0000","last_undegraded":"2024-04-26T20:50:38.802635+0000","last_fullsized":"2024-04-26T20:50:38.802635+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:49:28.099373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661389+0000","last_change":"2024-04-26T20:50:36.682799+0000","last_active":"2024-04-26T20:50:38.661389+0000","last_peered":"2024-04-26T20:50:38.661389+0000","last_clean":"2024-04-26T20:50:38.661389+0000","last_became_active":"2024-04-26T20:50:36.682271+0000","last_became_peered":"2024-04-26T20:50:36.682271+0000","last_unstale":"2024-04-26T20:50:38.661389+0000","last_undegraded":"2024-04-26T20:50:38.661389+0000","last_fullsized":"2024-04-26T20:50:38.661389+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:27:18.035795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802689+0000","last_change":"2024-04-26T20:50:32.647241+0000","last_active":"2024-04-26T20:50:38.802689+0000","last_peered":"2024-04-26T20:50:38.802689+0000","last_clean":"2024-04-26T20:50:38.802689+0000","last_became_active":"2024-04-26T20:50:32.646907+0000","last_became_peered":"2024-04-26T20:50:32.646907+0000","last_unstale":"2024-04-26T20:50:38.802689+0000","last_undegraded":"2024-04-26T20:50:38.802689+0000","last_fullsized":"2024-04-26T20:50:38.802689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:57:40.604842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800494+0000","last_change":"2024-04-26T20:50:31.633583+0000","last_active":"2024-04-26T20:50:38.800494+0000","last_peered":"2024-04-26T20:50:38.800494+0000","last_clean":"2024-04-26T20:50:38.800494+0000","last_became_active":"2024-04-26T20:50:31.633272+0000","last_became_peered":"2024-04-26T20:50:31.633272+0000","last_unstale":"2024-04-26T20:50:38.800494+0000","last_undegraded":"2024-04-26T20:50:38.800494+0000","last_fullsized":"2024-04-26T20:50:38.800494+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:53:10.645804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803808+0000","last_change":"2024-04-26T20:50:37.291410+0000","last_active":"2024-04-26T20:50:38.803808+0000","last_peered":"2024-04-26T20:50:38.803808+0000","last_clean":"2024-04-26T20:50:38.803808+0000","last_became_active":"2024-04-26T20:50:37.291016+0000","last_became_peered":"2024-04-26T20:50:37.291016+0000","last_unstale":"2024-04-26T20:50:38.803808+0000","last_undegraded":"2024-04-26T20:50:38.803808+0000","last_fullsized":"2024-04-26T20:50:38.803808+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:13:08.850173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802348+0000","last_change":"2024-04-26T20:50:32.653314+0000","last_active":"2024-04-26T20:50:38.802348+0000","last_peered":"2024-04-26T20:50:38.802348+0000","last_clean":"2024-04-26T20:50:38.802348+0000","last_became_active":"2024-04-26T20:50:32.652980+0000","last_became_peered":"2024-04-26T20:50:32.652980+0000","last_unstale":"2024-04-26T20:50:38.802348+0000","last_undegraded":"2024-04-26T20:50:38.802348+0000","last_fullsized":"2024-04-26T20:50:38.802348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:15:55.563522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800028+0000","last_change":"2024-04-26T20:50:31.629049+0000","last_active":"2024-04-26T20:50:38.800028+0000","last_peered":"2024-04-26T20:50:38.800028+0000","last_clean":"2024-04-26T20:50:38.800028+0000","last_became_active":"2024-04-26T20:50:31.628788+0000","last_became_peered":"2024-04-26T20:50:31.628788+0000","last_unstale":"2024-04-26T20:50:38.800028+0000","last_undegraded":"2024-04-26T20:50:38.800028+0000","last_fullsized":"2024-04-26T20:50:38.800028+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:26.601021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":179,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800588+0000","last_change":"2024-04-26T20:50:21.150132+0000","last_active":"2024-04-26T20:50:38.800588+0000","last_peered":"2024-04-26T20:50:38.800588+0000","last_clean":"2024-04-26T20:50:38.800588+0000","last_became_active":"2024-04-26T20:50:21.147439+0000","last_became_peered":"2024-04-26T20:50:21.147439+0000","last_unstale":"2024-04-26T20:50:38.800588+0000","last_undegraded":"2024-04-26T20:50:38.800588+0000","last_fullsized":"2024-04-26T20:50:38.800588+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:20.084499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:20.084499+0000","last_clean_scrub_stamp":"2024-04-26T20:50:20.084499+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:19:29.395477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.717442+0000","last_change":"2024-04-26T20:50:37.295209+0000","last_active":"2024-04-26T20:50:38.717442+0000","last_peered":"2024-04-26T20:50:38.717442+0000","last_clean":"2024-04-26T20:50:38.717442+0000","last_became_active":"2024-04-26T20:50:37.294715+0000","last_became_peered":"2024-04-26T20:50:37.294715+0000","last_unstale":"2024-04-26T20:50:38.717442+0000","last_undegraded":"2024-04-26T20:50:38.717442+0000","last_fullsized":"2024-04-26T20:50:38.717442+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:13:14.645783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800810+0000","last_change":"2024-04-26T20:50:32.648081+0000","last_active":"2024-04-26T20:50:38.800810+0000","last_peered":"2024-04-26T20:50:38.800810+0000","last_clean":"2024-04-26T20:50:38.800810+0000","last_became_active":"2024-04-26T20:50:32.644812+0000","last_became_peered":"2024-04-26T20:50:32.644812+0000","last_unstale":"2024-04-26T20:50:38.800810+0000","last_undegraded":"2024-04-26T20:50:38.800810+0000","last_fullsized":"2024-04-26T20:50:38.800810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:45:44.950496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803401+0000","last_change":"2024-04-26T20:50:31.630301+0000","last_active":"2024-04-26T20:50:38.803401+0000","last_peered":"2024-04-26T20:50:38.803401+0000","last_clean":"2024-04-26T20:50:38.803401+0000","last_became_active":"2024-04-26T20:50:31.630039+0000","last_became_peered":"2024-04-26T20:50:31.630039+0000","last_unstale":"2024-04-26T20:50:38.803401+0000","last_undegraded":"2024-04-26T20:50:38.803401+0000","last_fullsized":"2024-04-26T20:50:38.803401+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:21:09.963731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.717176+0000","last_change":"2024-04-26T20:50:37.291889+0000","last_active":"2024-04-26T20:50:38.717176+0000","last_peered":"2024-04-26T20:50:38.717176+0000","last_clean":"2024-04-26T20:50:38.717176+0000","last_became_active":"2024-04-26T20:50:37.291401+0000","last_became_peered":"2024-04-26T20:50:37.291401+0000","last_unstale":"2024-04-26T20:50:38.717176+0000","last_undegraded":"2024-04-26T20:50:38.717176+0000","last_fullsized":"2024-04-26T20:50:38.717176+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:28:29.639328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800412+0000","last_change":"2024-04-26T20:50:32.655082+0000","last_active":"2024-04-26T20:50:38.800412+0000","last_peered":"2024-04-26T20:50:38.800412+0000","last_clean":"2024-04-26T20:50:38.800412+0000","last_became_active":"2024-04-26T20:50:32.654751+0000","last_became_peered":"2024-04-26T20:50:32.654751+0000","last_unstale":"2024-04-26T20:50:38.800412+0000","last_undegraded":"2024-04-26T20:50:38.800412+0000","last_fullsized":"2024-04-26T20:50:38.800412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:27:59.302826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661274+0000","last_change":"2024-04-26T20:50:31.628865+0000","last_active":"2024-04-26T20:50:38.661274+0000","last_peered":"2024-04-26T20:50:38.661274+0000","last_clean":"2024-04-26T20:50:38.661274+0000","last_became_active":"2024-04-26T20:50:31.627789+0000","last_became_peered":"2024-04-26T20:50:31.627789+0000","last_unstale":"2024-04-26T20:50:38.661274+0000","last_undegraded":"2024-04-26T20:50:38.661274+0000","last_fullsized":"2024-04-26T20:50:38.661274+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:53:10.645804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803847+0000","last_change":"2024-04-26T20:50:36.679869+0000","last_active":"2024-04-26T20:50:38.803847+0000","last_peered":"2024-04-26T20:50:38.803847+0000","last_clean":"2024-04-26T20:50:38.803847+0000","last_became_active":"2024-04-26T20:50:36.678678+0000","last_became_peered":"2024-04-26T20:50:36.678678+0000","last_unstale":"2024-04-26T20:50:38.803847+0000","last_undegraded":"2024-04-26T20:50:38.803847+0000","last_fullsized":"2024-04-26T20:50:38.803847+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:54.249138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.801078+0000","last_change":"2024-04-26T20:50:32.655912+0000","last_active":"2024-04-26T20:50:38.801078+0000","last_peered":"2024-04-26T20:50:38.801078+0000","last_clean":"2024-04-26T20:50:38.801078+0000","last_became_active":"2024-04-26T20:50:32.655649+0000","last_became_peered":"2024-04-26T20:50:32.655649+0000","last_unstale":"2024-04-26T20:50:38.801078+0000","last_undegraded":"2024-04-26T20:50:38.801078+0000","last_fullsized":"2024-04-26T20:50:38.801078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:59:04.962815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803582+0000","last_change":"2024-04-26T20:50:31.631670+0000","last_active":"2024-04-26T20:50:38.803582+0000","last_peered":"2024-04-26T20:50:38.803582+0000","last_clean":"2024-04-26T20:50:38.803582+0000","last_became_active":"2024-04-26T20:50:31.631380+0000","last_became_peered":"2024-04-26T20:50:31.631380+0000","last_unstale":"2024-04-26T20:50:38.803582+0000","last_undegraded":"2024-04-26T20:50:38.803582+0000","last_fullsized":"2024-04-26T20:50:38.803582+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:23:29.866794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.799715+0000","last_change":"2024-04-26T20:50:37.291812+0000","last_active":"2024-04-26T20:50:38.799715+0000","last_peered":"2024-04-26T20:50:38.799715+0000","last_clean":"2024-04-26T20:50:38.799715+0000","last_became_active":"2024-04-26T20:50:37.291408+0000","last_became_peered":"2024-04-26T20:50:37.291408+0000","last_unstale":"2024-04-26T20:50:38.799715+0000","last_undegraded":"2024-04-26T20:50:38.799715+0000","last_fullsized":"2024-04-26T20:50:38.799715+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:29:58.077073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800564+0000","last_change":"2024-04-26T20:50:32.649031+0000","last_active":"2024-04-26T20:50:38.800564+0000","last_peered":"2024-04-26T20:50:38.800564+0000","last_clean":"2024-04-26T20:50:38.800564+0000","last_became_active":"2024-04-26T20:50:32.648769+0000","last_became_peered":"2024-04-26T20:50:32.648769+0000","last_unstale":"2024-04-26T20:50:38.800564+0000","last_undegraded":"2024-04-26T20:50:38.800564+0000","last_fullsized":"2024-04-26T20:50:38.800564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:16:25.308786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.662578+0000","last_change":"2024-04-26T20:50:31.627514+0000","last_active":"2024-04-26T20:50:38.662578+0000","last_peered":"2024-04-26T20:50:38.662578+0000","last_clean":"2024-04-26T20:50:38.662578+0000","last_became_active":"2024-04-26T20:50:31.627177+0000","last_became_peered":"2024-04-26T20:50:31.627177+0000","last_unstale":"2024-04-26T20:50:38.662578+0000","last_undegraded":"2024-04-26T20:50:38.662578+0000","last_fullsized":"2024-04-26T20:50:38.662578+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:23:21.503395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661857+0000","last_change":"2024-04-26T20:50:36.678980+0000","last_active":"2024-04-26T20:50:38.661857+0000","last_peered":"2024-04-26T20:50:38.661857+0000","last_clean":"2024-04-26T20:50:38.661857+0000","last_became_active":"2024-04-26T20:50:36.678581+0000","last_became_peered":"2024-04-26T20:50:36.678581+0000","last_unstale":"2024-04-26T20:50:38.661857+0000","last_undegraded":"2024-04-26T20:50:38.661857+0000","last_fullsized":"2024-04-26T20:50:38.661857+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:28:29.639328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800927+0000","last_change":"2024-04-26T20:50:32.647060+0000","last_active":"2024-04-26T20:50:38.800927+0000","last_peered":"2024-04-26T20:50:38.800927+0000","last_clean":"2024-04-26T20:50:38.800927+0000","last_became_active":"2024-04-26T20:50:32.646761+0000","last_became_peered":"2024-04-26T20:50:32.646761+0000","last_unstale":"2024-04-26T20:50:38.800927+0000","last_undegraded":"2024-04-26T20:50:38.800927+0000","last_fullsized":"2024-04-26T20:50:38.800927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:35.639518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802576+0000","last_change":"2024-04-26T20:50:31.627228+0000","last_active":"2024-04-26T20:50:38.802576+0000","last_peered":"2024-04-26T20:50:38.802576+0000","last_clean":"2024-04-26T20:50:38.802576+0000","last_became_active":"2024-04-26T20:50:31.626931+0000","last_became_peered":"2024-04-26T20:50:31.626931+0000","last_unstale":"2024-04-26T20:50:38.802576+0000","last_undegraded":"2024-04-26T20:50:38.802576+0000","last_fullsized":"2024-04-26T20:50:38.802576+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:26.601021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803663+0000","last_change":"2024-04-26T20:50:36.684013+0000","last_active":"2024-04-26T20:50:38.803663+0000","last_peered":"2024-04-26T20:50:38.803663+0000","last_clean":"2024-04-26T20:50:38.803663+0000","last_became_active":"2024-04-26T20:50:36.683653+0000","last_became_peered":"2024-04-26T20:50:36.683653+0000","last_unstale":"2024-04-26T20:50:38.803663+0000","last_undegraded":"2024-04-26T20:50:38.803663+0000","last_fullsized":"2024-04-26T20:50:38.803663+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:27:18.035795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.717061+0000","last_change":"2024-04-26T20:50:32.645444+0000","last_active":"2024-04-26T20:50:38.717061+0000","last_peered":"2024-04-26T20:50:38.717061+0000","last_clean":"2024-04-26T20:50:38.717061+0000","last_became_active":"2024-04-26T20:50:32.645155+0000","last_became_peered":"2024-04-26T20:50:32.645155+0000","last_unstale":"2024-04-26T20:50:38.717061+0000","last_undegraded":"2024-04-26T20:50:38.717061+0000","last_fullsized":"2024-04-26T20:50:38.717061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:49:57.427154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.717091+0000","last_change":"2024-04-26T20:50:31.629590+0000","last_active":"2024-04-26T20:50:38.717091+0000","last_peered":"2024-04-26T20:50:38.717091+0000","last_clean":"2024-04-26T20:50:38.717091+0000","last_became_active":"2024-04-26T20:50:31.629324+0000","last_became_peered":"2024-04-26T20:50:31.629324+0000","last_unstale":"2024-04-26T20:50:38.717091+0000","last_undegraded":"2024-04-26T20:50:38.717091+0000","last_fullsized":"2024-04-26T20:50:38.717091+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:49:28.099373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803803+0000","last_change":"2024-04-26T20:50:37.291611+0000","last_active":"2024-04-26T20:50:38.803803+0000","last_peered":"2024-04-26T20:50:38.803803+0000","last_clean":"2024-04-26T20:50:38.803803+0000","last_became_active":"2024-04-26T20:50:37.291068+0000","last_became_peered":"2024-04-26T20:50:37.291068+0000","last_unstale":"2024-04-26T20:50:38.803803+0000","last_undegraded":"2024-04-26T20:50:38.803803+0000","last_fullsized":"2024-04-26T20:50:38.803803+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:31:09.384259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800427+0000","last_change":"2024-04-26T20:50:32.649304+0000","last_active":"2024-04-26T20:50:38.800427+0000","last_peered":"2024-04-26T20:50:38.800427+0000","last_clean":"2024-04-26T20:50:38.800427+0000","last_became_active":"2024-04-26T20:50:32.647714+0000","last_became_peered":"2024-04-26T20:50:32.647714+0000","last_unstale":"2024-04-26T20:50:38.800427+0000","last_undegraded":"2024-04-26T20:50:38.800427+0000","last_fullsized":"2024-04-26T20:50:38.800427+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:27:59.302826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800065+0000","last_change":"2024-04-26T20:50:31.626907+0000","last_active":"2024-04-26T20:50:38.800065+0000","last_peered":"2024-04-26T20:50:38.800065+0000","last_clean":"2024-04-26T20:50:38.800065+0000","last_became_active":"2024-04-26T20:50:31.625955+0000","last_became_peered":"2024-04-26T20:50:31.625955+0000","last_unstale":"2024-04-26T20:50:38.800065+0000","last_undegraded":"2024-04-26T20:50:38.800065+0000","last_fullsized":"2024-04-26T20:50:38.800065+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:46:19.858595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803391+0000","last_change":"2024-04-26T20:50:37.292652+0000","last_active":"2024-04-26T20:50:38.803391+0000","last_peered":"2024-04-26T20:50:38.803391+0000","last_clean":"2024-04-26T20:50:38.803391+0000","last_became_active":"2024-04-26T20:50:37.291717+0000","last_became_peered":"2024-04-26T20:50:37.291717+0000","last_unstale":"2024-04-26T20:50:38.803391+0000","last_undegraded":"2024-04-26T20:50:38.803391+0000","last_fullsized":"2024-04-26T20:50:38.803391+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:31:19.069050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803321+0000","last_change":"2024-04-26T20:50:32.643484+0000","last_active":"2024-04-26T20:50:38.803321+0000","last_peered":"2024-04-26T20:50:38.803321+0000","last_clean":"2024-04-26T20:50:38.803321+0000","last_became_active":"2024-04-26T20:50:32.643225+0000","last_became_peered":"2024-04-26T20:50:32.643225+0000","last_unstale":"2024-04-26T20:50:38.803321+0000","last_undegraded":"2024-04-26T20:50:38.803321+0000","last_fullsized":"2024-04-26T20:50:38.803321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:49:57.427154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803958+0000","last_change":"2024-04-26T20:50:31.627130+0000","last_active":"2024-04-26T20:50:38.803958+0000","last_peered":"2024-04-26T20:50:38.803958+0000","last_clean":"2024-04-26T20:50:38.803958+0000","last_became_active":"2024-04-26T20:50:31.626697+0000","last_became_peered":"2024-04-26T20:50:31.626697+0000","last_unstale":"2024-04-26T20:50:38.803958+0000","last_undegraded":"2024-04-26T20:50:38.803958+0000","last_fullsized":"2024-04-26T20:50:38.803958+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:21:09.963731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800487+0000","last_change":"2024-04-26T20:50:37.294808+0000","last_active":"2024-04-26T20:50:38.800487+0000","last_peered":"2024-04-26T20:50:38.800487+0000","last_clean":"2024-04-26T20:50:38.800487+0000","last_became_active":"2024-04-26T20:50:37.294404+0000","last_became_peered":"2024-04-26T20:50:37.294404+0000","last_unstale":"2024-04-26T20:50:38.800487+0000","last_undegraded":"2024-04-26T20:50:38.800487+0000","last_fullsized":"2024-04-26T20:50:38.800487+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:29:17.876006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800773+0000","last_change":"2024-04-26T20:50:32.649505+0000","last_active":"2024-04-26T20:50:38.800773+0000","last_peered":"2024-04-26T20:50:38.800773+0000","last_clean":"2024-04-26T20:50:38.800773+0000","last_became_active":"2024-04-26T20:50:32.648538+0000","last_became_peered":"2024-04-26T20:50:32.648538+0000","last_unstale":"2024-04-26T20:50:38.800773+0000","last_undegraded":"2024-04-26T20:50:38.800773+0000","last_fullsized":"2024-04-26T20:50:38.800773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:33:28.969815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:41.974976+0000","last_change":"2024-04-26T20:50:31.628758+0000","last_active":"2024-04-26T20:50:41.974976+0000","last_peered":"2024-04-26T20:50:41.974976+0000","last_clean":"2024-04-26T20:50:41.974976+0000","last_became_active":"2024-04-26T20:50:31.628431+0000","last_became_peered":"2024-04-26T20:50:31.628431+0000","last_unstale":"2024-04-26T20:50:41.974976+0000","last_undegraded":"2024-04-26T20:50:41.974976+0000","last_fullsized":"2024-04-26T20:50:41.974976+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:36.499651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661093+0000","last_change":"2024-04-26T20:50:36.679403+0000","last_active":"2024-04-26T20:50:38.661093+0000","last_peered":"2024-04-26T20:50:38.661093+0000","last_clean":"2024-04-26T20:50:38.661093+0000","last_became_active":"2024-04-26T20:50:36.678945+0000","last_became_peered":"2024-04-26T20:50:36.678945+0000","last_unstale":"2024-04-26T20:50:38.661093+0000","last_undegraded":"2024-04-26T20:50:38.661093+0000","last_fullsized":"2024-04-26T20:50:38.661093+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:12.140650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.662571+0000","last_change":"2024-04-26T20:50:32.636718+0000","last_active":"2024-04-26T20:50:38.662571+0000","last_peered":"2024-04-26T20:50:38.662571+0000","last_clean":"2024-04-26T20:50:38.662571+0000","last_became_active":"2024-04-26T20:50:32.636419+0000","last_became_peered":"2024-04-26T20:50:32.636419+0000","last_unstale":"2024-04-26T20:50:38.662571+0000","last_undegraded":"2024-04-26T20:50:38.662571+0000","last_fullsized":"2024-04-26T20:50:38.662571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:06:40.634786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661155+0000","last_change":"2024-04-26T20:50:31.630575+0000","last_active":"2024-04-26T20:50:38.661155+0000","last_peered":"2024-04-26T20:50:38.661155+0000","last_clean":"2024-04-26T20:50:38.661155+0000","last_became_active":"2024-04-26T20:50:31.630304+0000","last_became_peered":"2024-04-26T20:50:31.630304+0000","last_unstale":"2024-04-26T20:50:38.661155+0000","last_undegraded":"2024-04-26T20:50:38.661155+0000","last_fullsized":"2024-04-26T20:50:38.661155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:32:17.604576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.662550+0000","last_change":"2024-04-26T20:50:36.672365+0000","last_active":"2024-04-26T20:50:38.662550+0000","last_peered":"2024-04-26T20:50:38.662550+0000","last_clean":"2024-04-26T20:50:38.662550+0000","last_became_active":"2024-04-26T20:50:36.671917+0000","last_became_peered":"2024-04-26T20:50:36.671917+0000","last_unstale":"2024-04-26T20:50:38.662550+0000","last_undegraded":"2024-04-26T20:50:38.662550+0000","last_fullsized":"2024-04-26T20:50:38.662550+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:28:04.995069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803056+0000","last_change":"2024-04-26T20:50:32.645616+0000","last_active":"2024-04-26T20:50:38.803056+0000","last_peered":"2024-04-26T20:50:38.803056+0000","last_clean":"2024-04-26T20:50:38.803056+0000","last_became_active":"2024-04-26T20:50:32.645372+0000","last_became_peered":"2024-04-26T20:50:32.645372+0000","last_unstale":"2024-04-26T20:50:38.803056+0000","last_undegraded":"2024-04-26T20:50:38.803056+0000","last_fullsized":"2024-04-26T20:50:38.803056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:56:51.926886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800505+0000","last_change":"2024-04-26T20:50:31.632113+0000","last_active":"2024-04-26T20:50:38.800505+0000","last_peered":"2024-04-26T20:50:38.800505+0000","last_clean":"2024-04-26T20:50:38.800505+0000","last_became_active":"2024-04-26T20:50:31.631815+0000","last_became_peered":"2024-04-26T20:50:31.631815+0000","last_unstale":"2024-04-26T20:50:38.800505+0000","last_undegraded":"2024-04-26T20:50:38.800505+0000","last_fullsized":"2024-04-26T20:50:38.800505+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:36.499651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802860+0000","last_change":"2024-04-26T20:50:36.676688+0000","last_active":"2024-04-26T20:50:38.802860+0000","last_peered":"2024-04-26T20:50:38.802860+0000","last_clean":"2024-04-26T20:50:38.802860+0000","last_became_active":"2024-04-26T20:50:36.675819+0000","last_became_peered":"2024-04-26T20:50:36.675819+0000","last_unstale":"2024-04-26T20:50:38.802860+0000","last_undegraded":"2024-04-26T20:50:38.802860+0000","last_fullsized":"2024-04-26T20:50:38.802860+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:12:04.255812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800398+0000","last_change":"2024-04-26T20:50:32.642860+0000","last_active":"2024-04-26T20:50:38.800398+0000","last_peered":"2024-04-26T20:50:38.800398+0000","last_clean":"2024-04-26T20:50:38.800398+0000","last_became_active":"2024-04-26T20:50:32.642567+0000","last_became_peered":"2024-04-26T20:50:32.642567+0000","last_unstale":"2024-04-26T20:50:38.800398+0000","last_undegraded":"2024-04-26T20:50:38.800398+0000","last_fullsized":"2024-04-26T20:50:38.800398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:00:12.394204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.717653+0000","last_change":"2024-04-26T20:50:31.632961+0000","last_active":"2024-04-26T20:50:38.717653+0000","last_peered":"2024-04-26T20:50:38.717653+0000","last_clean":"2024-04-26T20:50:38.717653+0000","last_became_active":"2024-04-26T20:50:31.632044+0000","last_became_peered":"2024-04-26T20:50:31.632044+0000","last_unstale":"2024-04-26T20:50:38.717653+0000","last_undegraded":"2024-04-26T20:50:38.717653+0000","last_fullsized":"2024-04-26T20:50:38.717653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:23:29.866794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802614+0000","last_change":"2024-04-26T20:50:37.293243+0000","last_active":"2024-04-26T20:50:38.802614+0000","last_peered":"2024-04-26T20:50:38.802614+0000","last_clean":"2024-04-26T20:50:38.802614+0000","last_became_active":"2024-04-26T20:50:37.292840+0000","last_became_peered":"2024-04-26T20:50:37.292840+0000","last_unstale":"2024-04-26T20:50:38.802614+0000","last_undegraded":"2024-04-26T20:50:38.802614+0000","last_fullsized":"2024-04-26T20:50:38.802614+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:05:40.897275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803514+0000","last_change":"2024-04-26T20:50:37.294767+0000","last_active":"2024-04-26T20:50:38.803514+0000","last_peered":"2024-04-26T20:50:38.803514+0000","last_clean":"2024-04-26T20:50:38.803514+0000","last_became_active":"2024-04-26T20:50:37.294351+0000","last_became_peered":"2024-04-26T20:50:37.294351+0000","last_unstale":"2024-04-26T20:50:38.803514+0000","last_undegraded":"2024-04-26T20:50:38.803514+0000","last_fullsized":"2024-04-26T20:50:38.803514+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:31:21.850102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.660600+0000","last_change":"2024-04-26T20:50:32.648105+0000","last_active":"2024-04-26T20:50:38.660600+0000","last_peered":"2024-04-26T20:50:38.660600+0000","last_clean":"2024-04-26T20:50:38.660600+0000","last_became_active":"2024-04-26T20:50:32.647833+0000","last_became_peered":"2024-04-26T20:50:32.647833+0000","last_unstale":"2024-04-26T20:50:38.660600+0000","last_undegraded":"2024-04-26T20:50:38.660600+0000","last_fullsized":"2024-04-26T20:50:38.660600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:55.634765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.799847+0000","last_change":"2024-04-26T20:50:31.622923+0000","last_active":"2024-04-26T20:50:38.799847+0000","last_peered":"2024-04-26T20:50:38.799847+0000","last_clean":"2024-04-26T20:50:38.799847+0000","last_became_active":"2024-04-26T20:50:31.622533+0000","last_became_peered":"2024-04-26T20:50:31.622533+0000","last_unstale":"2024-04-26T20:50:38.799847+0000","last_undegraded":"2024-04-26T20:50:38.799847+0000","last_fullsized":"2024-04-26T20:50:38.799847+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:32:17.604576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.717573+0000","last_change":"2024-04-26T20:50:36.680804+0000","last_active":"2024-04-26T20:50:38.717573+0000","last_peered":"2024-04-26T20:50:38.717573+0000","last_clean":"2024-04-26T20:50:38.717573+0000","last_became_active":"2024-04-26T20:50:36.680438+0000","last_became_peered":"2024-04-26T20:50:36.680438+0000","last_unstale":"2024-04-26T20:50:38.717573+0000","last_undegraded":"2024-04-26T20:50:38.717573+0000","last_fullsized":"2024-04-26T20:50:38.717573+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:58:22.587813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802852+0000","last_change":"2024-04-26T20:50:32.649026+0000","last_active":"2024-04-26T20:50:38.802852+0000","last_peered":"2024-04-26T20:50:38.802852+0000","last_clean":"2024-04-26T20:50:38.802852+0000","last_became_active":"2024-04-26T20:50:32.648737+0000","last_became_peered":"2024-04-26T20:50:32.648737+0000","last_unstale":"2024-04-26T20:50:38.802852+0000","last_undegraded":"2024-04-26T20:50:38.802852+0000","last_fullsized":"2024-04-26T20:50:38.802852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:47:14.715739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802756+0000","last_change":"2024-04-26T20:50:31.631362+0000","last_active":"2024-04-26T20:50:38.802756+0000","last_peered":"2024-04-26T20:50:38.802756+0000","last_clean":"2024-04-26T20:50:38.802756+0000","last_became_active":"2024-04-26T20:50:31.631099+0000","last_became_peered":"2024-04-26T20:50:31.631099+0000","last_unstale":"2024-04-26T20:50:38.802756+0000","last_undegraded":"2024-04-26T20:50:38.802756+0000","last_fullsized":"2024-04-26T20:50:38.802756+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:20:49.853735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.717285+0000","last_change":"2024-04-26T20:50:31.629712+0000","last_active":"2024-04-26T20:50:38.717285+0000","last_peered":"2024-04-26T20:50:38.717285+0000","last_clean":"2024-04-26T20:50:38.717285+0000","last_became_active":"2024-04-26T20:50:31.629412+0000","last_became_peered":"2024-04-26T20:50:31.629412+0000","last_unstale":"2024-04-26T20:50:38.717285+0000","last_undegraded":"2024-04-26T20:50:38.717285+0000","last_fullsized":"2024-04-26T20:50:38.717285+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:32:17.604576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803849+0000","last_change":"2024-04-26T20:50:32.638185+0000","last_active":"2024-04-26T20:50:38.803849+0000","last_peered":"2024-04-26T20:50:38.803849+0000","last_clean":"2024-04-26T20:50:38.803849+0000","last_became_active":"2024-04-26T20:50:32.637917+0000","last_became_peered":"2024-04-26T20:50:32.637917+0000","last_unstale":"2024-04-26T20:50:38.803849+0000","last_undegraded":"2024-04-26T20:50:38.803849+0000","last_fullsized":"2024-04-26T20:50:38.803849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:00:12.394204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803278+0000","last_change":"2024-04-26T20:50:36.682087+0000","last_active":"2024-04-26T20:50:38.803278+0000","last_peered":"2024-04-26T20:50:38.803278+0000","last_clean":"2024-04-26T20:50:38.803278+0000","last_became_active":"2024-04-26T20:50:36.681543+0000","last_became_peered":"2024-04-26T20:50:36.681543+0000","last_unstale":"2024-04-26T20:50:38.803278+0000","last_undegraded":"2024-04-26T20:50:38.803278+0000","last_fullsized":"2024-04-26T20:50:38.803278+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:57:44.640580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802464+0000","last_change":"2024-04-26T20:50:31.632761+0000","last_active":"2024-04-26T20:50:38.802464+0000","last_peered":"2024-04-26T20:50:38.802464+0000","last_clean":"2024-04-26T20:50:38.802464+0000","last_became_active":"2024-04-26T20:50:31.632381+0000","last_became_peered":"2024-04-26T20:50:31.632381+0000","last_unstale":"2024-04-26T20:50:38.802464+0000","last_undegraded":"2024-04-26T20:50:38.802464+0000","last_fullsized":"2024-04-26T20:50:38.802464+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:21:09.963731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.662120+0000","last_change":"2024-04-26T20:50:32.637920+0000","last_active":"2024-04-26T20:50:38.662120+0000","last_peered":"2024-04-26T20:50:38.662120+0000","last_clean":"2024-04-26T20:50:38.662120+0000","last_became_active":"2024-04-26T20:50:32.637618+0000","last_became_peered":"2024-04-26T20:50:32.637618+0000","last_unstale":"2024-04-26T20:50:38.662120+0000","last_undegraded":"2024-04-26T20:50:38.662120+0000","last_fullsized":"2024-04-26T20:50:38.662120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:46:20.863042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803215+0000","last_change":"2024-04-26T20:50:36.678219+0000","last_active":"2024-04-26T20:50:38.803215+0000","last_peered":"2024-04-26T20:50:38.803215+0000","last_clean":"2024-04-26T20:50:38.803215+0000","last_became_active":"2024-04-26T20:50:36.677732+0000","last_became_peered":"2024-04-26T20:50:36.677732+0000","last_unstale":"2024-04-26T20:50:38.803215+0000","last_undegraded":"2024-04-26T20:50:38.803215+0000","last_fullsized":"2024-04-26T20:50:38.803215+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:03:12.186398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.716924+0000","last_change":"2024-04-26T20:50:32.638997+0000","last_active":"2024-04-26T20:50:38.716924+0000","last_peered":"2024-04-26T20:50:38.716924+0000","last_clean":"2024-04-26T20:50:38.716924+0000","last_became_active":"2024-04-26T20:50:32.638708+0000","last_became_peered":"2024-04-26T20:50:32.638708+0000","last_unstale":"2024-04-26T20:50:38.716924+0000","last_undegraded":"2024-04-26T20:50:38.716924+0000","last_fullsized":"2024-04-26T20:50:38.716924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:36.651960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802480+0000","last_change":"2024-04-26T20:50:32.648305+0000","last_active":"2024-04-26T20:50:38.802480+0000","last_peered":"2024-04-26T20:50:38.802480+0000","last_clean":"2024-04-26T20:50:38.802480+0000","last_became_active":"2024-04-26T20:50:32.648021+0000","last_became_peered":"2024-04-26T20:50:32.648021+0000","last_unstale":"2024-04-26T20:50:38.802480+0000","last_undegraded":"2024-04-26T20:50:38.802480+0000","last_fullsized":"2024-04-26T20:50:38.802480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:46:20.863042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800030+0000","last_change":"2024-04-26T20:50:32.649133+0000","last_active":"2024-04-26T20:50:38.800030+0000","last_peered":"2024-04-26T20:50:38.800030+0000","last_clean":"2024-04-26T20:50:38.800030+0000","last_became_active":"2024-04-26T20:50:32.648855+0000","last_became_peered":"2024-04-26T20:50:32.648855+0000","last_unstale":"2024-04-26T20:50:38.800030+0000","last_undegraded":"2024-04-26T20:50:38.800030+0000","last_fullsized":"2024-04-26T20:50:38.800030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:58:18.552075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803763+0000","last_change":"2024-04-26T20:50:32.650322+0000","last_active":"2024-04-26T20:50:38.803763+0000","last_peered":"2024-04-26T20:50:38.803763+0000","last_clean":"2024-04-26T20:50:38.803763+0000","last_became_active":"2024-04-26T20:50:32.650011+0000","last_became_peered":"2024-04-26T20:50:32.650011+0000","last_unstale":"2024-04-26T20:50:38.803763+0000","last_undegraded":"2024-04-26T20:50:38.803763+0000","last_fullsized":"2024-04-26T20:50:38.803763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:15:55.563522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803550+0000","last_change":"2024-04-26T20:50:32.649914+0000","last_active":"2024-04-26T20:50:38.803550+0000","last_peered":"2024-04-26T20:50:38.803550+0000","last_clean":"2024-04-26T20:50:38.803550+0000","last_became_active":"2024-04-26T20:50:32.649439+0000","last_became_peered":"2024-04-26T20:50:32.649439+0000","last_unstale":"2024-04-26T20:50:38.803550+0000","last_undegraded":"2024-04-26T20:50:38.803550+0000","last_fullsized":"2024-04-26T20:50:38.803550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:33:28.969815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661177+0000","last_change":"2024-04-26T20:50:32.648570+0000","last_active":"2024-04-26T20:50:38.661177+0000","last_peered":"2024-04-26T20:50:38.661177+0000","last_clean":"2024-04-26T20:50:38.661177+0000","last_became_active":"2024-04-26T20:50:32.646951+0000","last_became_peered":"2024-04-26T20:50:32.646951+0000","last_unstale":"2024-04-26T20:50:38.661177+0000","last_undegraded":"2024-04-26T20:50:38.661177+0000","last_fullsized":"2024-04-26T20:50:38.661177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:15:55.563522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.717644+0000","last_change":"2024-04-26T20:50:32.640208+0000","last_active":"2024-04-26T20:50:38.717644+0000","last_peered":"2024-04-26T20:50:38.717644+0000","last_clean":"2024-04-26T20:50:38.717644+0000","last_became_active":"2024-04-26T20:50:32.639869+0000","last_became_peered":"2024-04-26T20:50:32.639869+0000","last_unstale":"2024-04-26T20:50:38.717644+0000","last_undegraded":"2024-04-26T20:50:38.717644+0000","last_fullsized":"2024-04-26T20:50:38.717644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:15:55.563522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803850+0000","last_change":"2024-04-26T20:50:32.650173+0000","last_active":"2024-04-26T20:50:38.803850+0000","last_peered":"2024-04-26T20:50:38.803850+0000","last_clean":"2024-04-26T20:50:38.803850+0000","last_became_active":"2024-04-26T20:50:32.649876+0000","last_became_peered":"2024-04-26T20:50:32.649876+0000","last_unstale":"2024-04-26T20:50:38.803850+0000","last_undegraded":"2024-04-26T20:50:38.803850+0000","last_fullsized":"2024-04-26T20:50:38.803850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:00:12.394204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.799788+0000","last_change":"2024-04-26T20:50:32.651129+0000","last_active":"2024-04-26T20:50:38.799788+0000","last_peered":"2024-04-26T20:50:38.799788+0000","last_clean":"2024-04-26T20:50:38.799788+0000","last_became_active":"2024-04-26T20:50:32.650853+0000","last_became_peered":"2024-04-26T20:50:32.650853+0000","last_unstale":"2024-04-26T20:50:38.799788+0000","last_undegraded":"2024-04-26T20:50:38.799788+0000","last_fullsized":"2024-04-26T20:50:38.799788+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:35.639518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802726+0000","last_change":"2024-04-26T20:50:32.650192+0000","last_active":"2024-04-26T20:50:38.802726+0000","last_peered":"2024-04-26T20:50:38.802726+0000","last_clean":"2024-04-26T20:50:38.802726+0000","last_became_active":"2024-04-26T20:50:32.649911+0000","last_became_peered":"2024-04-26T20:50:32.649911+0000","last_unstale":"2024-04-26T20:50:38.802726+0000","last_undegraded":"2024-04-26T20:50:38.802726+0000","last_fullsized":"2024-04-26T20:50:38.802726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:09:36.737379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.799890+0000","last_change":"2024-04-26T20:50:32.641183+0000","last_active":"2024-04-26T20:50:38.799890+0000","last_peered":"2024-04-26T20:50:38.799890+0000","last_clean":"2024-04-26T20:50:38.799890+0000","last_became_active":"2024-04-26T20:50:32.640584+0000","last_became_peered":"2024-04-26T20:50:32.640584+0000","last_unstale":"2024-04-26T20:50:38.799890+0000","last_undegraded":"2024-04-26T20:50:38.799890+0000","last_fullsized":"2024-04-26T20:50:38.799890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:13:10.610044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803653+0000","last_change":"2024-04-26T20:50:32.651661+0000","last_active":"2024-04-26T20:50:38.803653+0000","last_peered":"2024-04-26T20:50:38.803653+0000","last_clean":"2024-04-26T20:50:38.803653+0000","last_became_active":"2024-04-26T20:50:32.651316+0000","last_became_peered":"2024-04-26T20:50:32.651316+0000","last_unstale":"2024-04-26T20:50:38.803653+0000","last_undegraded":"2024-04-26T20:50:38.803653+0000","last_fullsized":"2024-04-26T20:50:38.803653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:09:36.737379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.717426+0000","last_change":"2024-04-26T20:50:32.645619+0000","last_active":"2024-04-26T20:50:38.717426+0000","last_peered":"2024-04-26T20:50:38.717426+0000","last_clean":"2024-04-26T20:50:38.717426+0000","last_became_active":"2024-04-26T20:50:32.644645+0000","last_became_peered":"2024-04-26T20:50:32.644645+0000","last_unstale":"2024-04-26T20:50:38.717426+0000","last_undegraded":"2024-04-26T20:50:38.717426+0000","last_fullsized":"2024-04-26T20:50:38.717426+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:54.967490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.801055+0000","last_change":"2024-04-26T20:50:32.646646+0000","last_active":"2024-04-26T20:50:38.801055+0000","last_peered":"2024-04-26T20:50:38.801055+0000","last_clean":"2024-04-26T20:50:38.801055+0000","last_became_active":"2024-04-26T20:50:32.645404+0000","last_became_peered":"2024-04-26T20:50:32.645404+0000","last_unstale":"2024-04-26T20:50:38.801055+0000","last_undegraded":"2024-04-26T20:50:38.801055+0000","last_fullsized":"2024-04-26T20:50:38.801055+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:58:18.552075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803547+0000","last_change":"2024-04-26T20:50:32.646750+0000","last_active":"2024-04-26T20:50:38.803547+0000","last_peered":"2024-04-26T20:50:38.803547+0000","last_clean":"2024-04-26T20:50:38.803547+0000","last_became_active":"2024-04-26T20:50:32.646447+0000","last_became_peered":"2024-04-26T20:50:32.646447+0000","last_unstale":"2024-04-26T20:50:38.803547+0000","last_undegraded":"2024-04-26T20:50:38.803547+0000","last_fullsized":"2024-04-26T20:50:38.803547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:53:11.650251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800765+0000","last_change":"2024-04-26T20:50:32.642132+0000","last_active":"2024-04-26T20:50:38.800765+0000","last_peered":"2024-04-26T20:50:38.800765+0000","last_clean":"2024-04-26T20:50:38.800765+0000","last_became_active":"2024-04-26T20:50:32.641690+0000","last_became_peered":"2024-04-26T20:50:32.641690+0000","last_unstale":"2024-04-26T20:50:38.800765+0000","last_undegraded":"2024-04-26T20:50:38.800765+0000","last_fullsized":"2024-04-26T20:50:38.800765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:59:04.962815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.799661+0000","last_change":"2024-04-26T20:50:32.644332+0000","last_active":"2024-04-26T20:50:38.799661+0000","last_peered":"2024-04-26T20:50:38.799661+0000","last_clean":"2024-04-26T20:50:38.799661+0000","last_became_active":"2024-04-26T20:50:32.643783+0000","last_became_peered":"2024-04-26T20:50:32.643783+0000","last_unstale":"2024-04-26T20:50:38.799661+0000","last_undegraded":"2024-04-26T20:50:38.799661+0000","last_fullsized":"2024-04-26T20:50:38.799661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:56:51.926886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.660696+0000","last_change":"2024-04-26T20:50:32.638177+0000","last_active":"2024-04-26T20:50:38.660696+0000","last_peered":"2024-04-26T20:50:38.660696+0000","last_clean":"2024-04-26T20:50:38.660696+0000","last_became_active":"2024-04-26T20:50:32.637905+0000","last_became_peered":"2024-04-26T20:50:32.637905+0000","last_unstale":"2024-04-26T20:50:38.660696+0000","last_undegraded":"2024-04-26T20:50:38.660696+0000","last_fullsized":"2024-04-26T20:50:38.660696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:36.651960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803236+0000","last_change":"2024-04-26T20:50:32.637456+0000","last_active":"2024-04-26T20:50:38.803236+0000","last_peered":"2024-04-26T20:50:38.803236+0000","last_clean":"2024-04-26T20:50:38.803236+0000","last_became_active":"2024-04-26T20:50:32.637188+0000","last_became_peered":"2024-04-26T20:50:32.637188+0000","last_unstale":"2024-04-26T20:50:38.803236+0000","last_undegraded":"2024-04-26T20:50:38.803236+0000","last_fullsized":"2024-04-26T20:50:38.803236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:54.967490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802268+0000","last_change":"2024-04-26T20:50:32.649659+0000","last_active":"2024-04-26T20:50:38.802268+0000","last_peered":"2024-04-26T20:50:38.802268+0000","last_clean":"2024-04-26T20:50:38.802268+0000","last_became_active":"2024-04-26T20:50:32.645837+0000","last_became_peered":"2024-04-26T20:50:32.645837+0000","last_unstale":"2024-04-26T20:50:38.802268+0000","last_undegraded":"2024-04-26T20:50:38.802268+0000","last_fullsized":"2024-04-26T20:50:38.802268+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:55.634765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.662458+0000","last_change":"2024-04-26T20:50:32.643463+0000","last_active":"2024-04-26T20:50:38.662458+0000","last_peered":"2024-04-26T20:50:38.662458+0000","last_clean":"2024-04-26T20:50:38.662458+0000","last_became_active":"2024-04-26T20:50:32.642783+0000","last_became_peered":"2024-04-26T20:50:32.642783+0000","last_unstale":"2024-04-26T20:50:38.662458+0000","last_undegraded":"2024-04-26T20:50:38.662458+0000","last_fullsized":"2024-04-26T20:50:38.662458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:53:11.650251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.717503+0000","last_change":"2024-04-26T20:50:32.643690+0000","last_active":"2024-04-26T20:50:38.717503+0000","last_peered":"2024-04-26T20:50:38.717503+0000","last_clean":"2024-04-26T20:50:38.717503+0000","last_became_active":"2024-04-26T20:50:32.643412+0000","last_became_peered":"2024-04-26T20:50:32.643412+0000","last_unstale":"2024-04-26T20:50:38.717503+0000","last_undegraded":"2024-04-26T20:50:38.717503+0000","last_fullsized":"2024-04-26T20:50:38.717503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:55.634765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.717608+0000","last_change":"2024-04-26T20:50:32.643116+0000","last_active":"2024-04-26T20:50:38.717608+0000","last_peered":"2024-04-26T20:50:38.717608+0000","last_clean":"2024-04-26T20:50:38.717608+0000","last_became_active":"2024-04-26T20:50:32.642598+0000","last_became_peered":"2024-04-26T20:50:32.642598+0000","last_unstale":"2024-04-26T20:50:38.717608+0000","last_undegraded":"2024-04-26T20:50:38.717608+0000","last_fullsized":"2024-04-26T20:50:38.717608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:47:14.715739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.801040+0000","last_change":"2024-04-26T20:50:32.654019+0000","last_active":"2024-04-26T20:50:38.801040+0000","last_peered":"2024-04-26T20:50:38.801040+0000","last_clean":"2024-04-26T20:50:38.801040+0000","last_became_active":"2024-04-26T20:50:32.653569+0000","last_became_peered":"2024-04-26T20:50:32.653569+0000","last_unstale":"2024-04-26T20:50:38.801040+0000","last_undegraded":"2024-04-26T20:50:38.801040+0000","last_fullsized":"2024-04-26T20:50:38.801040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:13:10.610044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802660+0000","last_change":"2024-04-26T20:50:32.646837+0000","last_active":"2024-04-26T20:50:38.802660+0000","last_peered":"2024-04-26T20:50:38.802660+0000","last_clean":"2024-04-26T20:50:38.802660+0000","last_became_active":"2024-04-26T20:50:32.646528+0000","last_became_peered":"2024-04-26T20:50:32.646528+0000","last_unstale":"2024-04-26T20:50:38.802660+0000","last_undegraded":"2024-04-26T20:50:38.802660+0000","last_fullsized":"2024-04-26T20:50:38.802660+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:54.967490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800781+0000","last_change":"2024-04-26T20:50:32.654006+0000","last_active":"2024-04-26T20:50:38.800781+0000","last_peered":"2024-04-26T20:50:38.800781+0000","last_clean":"2024-04-26T20:50:38.800781+0000","last_became_active":"2024-04-26T20:50:32.653553+0000","last_became_peered":"2024-04-26T20:50:32.653553+0000","last_unstale":"2024-04-26T20:50:38.800781+0000","last_undegraded":"2024-04-26T20:50:38.800781+0000","last_fullsized":"2024-04-26T20:50:38.800781+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:00:12.394204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.799949+0000","last_change":"2024-04-26T20:50:32.641407+0000","last_active":"2024-04-26T20:50:38.799949+0000","last_peered":"2024-04-26T20:50:38.799949+0000","last_clean":"2024-04-26T20:50:38.799949+0000","last_became_active":"2024-04-26T20:50:32.640889+0000","last_became_peered":"2024-04-26T20:50:32.640889+0000","last_unstale":"2024-04-26T20:50:38.799949+0000","last_undegraded":"2024-04-26T20:50:38.799949+0000","last_fullsized":"2024-04-26T20:50:38.799949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:28:25.603590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800212+0000","last_change":"2024-04-26T20:50:32.645181+0000","last_active":"2024-04-26T20:50:38.800212+0000","last_peered":"2024-04-26T20:50:38.800212+0000","last_clean":"2024-04-26T20:50:38.800212+0000","last_became_active":"2024-04-26T20:50:32.644919+0000","last_became_peered":"2024-04-26T20:50:32.644919+0000","last_unstale":"2024-04-26T20:50:38.800212+0000","last_undegraded":"2024-04-26T20:50:38.800212+0000","last_fullsized":"2024-04-26T20:50:38.800212+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:40:19.711175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800200+0000","last_change":"2024-04-26T20:50:32.655095+0000","last_active":"2024-04-26T20:50:38.800200+0000","last_peered":"2024-04-26T20:50:38.800200+0000","last_clean":"2024-04-26T20:50:38.800200+0000","last_became_active":"2024-04-26T20:50:32.654821+0000","last_became_peered":"2024-04-26T20:50:32.654821+0000","last_unstale":"2024-04-26T20:50:38.800200+0000","last_undegraded":"2024-04-26T20:50:38.800200+0000","last_fullsized":"2024-04-26T20:50:38.800200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:33:28.969815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802460+0000","last_change":"2024-04-26T20:50:32.642769+0000","last_active":"2024-04-26T20:50:38.802460+0000","last_peered":"2024-04-26T20:50:38.802460+0000","last_clean":"2024-04-26T20:50:38.802460+0000","last_became_active":"2024-04-26T20:50:32.642506+0000","last_became_peered":"2024-04-26T20:50:32.642506+0000","last_unstale":"2024-04-26T20:50:38.802460+0000","last_undegraded":"2024-04-26T20:50:38.802460+0000","last_fullsized":"2024-04-26T20:50:38.802460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:55.634765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661146+0000","last_change":"2024-04-26T20:50:32.641048+0000","last_active":"2024-04-26T20:50:38.661146+0000","last_peered":"2024-04-26T20:50:38.661146+0000","last_clean":"2024-04-26T20:50:38.661146+0000","last_became_active":"2024-04-26T20:50:32.640572+0000","last_became_peered":"2024-04-26T20:50:32.640572+0000","last_unstale":"2024-04-26T20:50:38.661146+0000","last_undegraded":"2024-04-26T20:50:38.661146+0000","last_fullsized":"2024-04-26T20:50:38.661146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:47:14.715739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800925+0000","last_change":"2024-04-26T20:50:32.650971+0000","last_active":"2024-04-26T20:50:38.800925+0000","last_peered":"2024-04-26T20:50:38.800925+0000","last_clean":"2024-04-26T20:50:38.800925+0000","last_became_active":"2024-04-26T20:50:32.650713+0000","last_became_peered":"2024-04-26T20:50:32.650713+0000","last_unstale":"2024-04-26T20:50:38.800925+0000","last_undegraded":"2024-04-26T20:50:38.800925+0000","last_fullsized":"2024-04-26T20:50:38.800925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:28:25.603590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":864256,"data_stored":698343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":107483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":107519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":107393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":107553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":872448,"data_stored":698399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:50:56.024 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T20:50:56.024 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-26T20:50:56.024 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-26T20:50:56.024 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-26T20:50:56.024 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-26T20:50:56.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.154+0000 7f66c42a0640 1 Processor -- start 2024-04-26T20:50:56.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.154+0000 7f66c42a0640 1 -- start start 2024-04-26T20:50:56.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.155+0000 7f66c42a0640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f66bc0d3bc0 0x7f66bc0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:56.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.155+0000 7f66c42a0640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66bc0b7e50 con 0x7f66bc162880 2024-04-26T20:50:56.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.155+0000 7f66c42a0640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66bc0d1800 con 0x7f66bc0b5f20 2024-04-26T20:50:56.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.155+0000 7f66c42a0640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f66bc0d0580 con 0x7f66bc0d3bc0 2024-04-26T20:50:56.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.155+0000 7f66c183b640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f66bc0d3bc0 0x7f66bc0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:56.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.155+0000 7f66c203c640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f66bc162880 0x7f66bc0d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33068/0 (socket says 172.21.15.73:33068) 2024-04-26T20:50:56.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.155+0000 7f66c203c640 1 -- 172.21.15.73:0/3656813527 learned_addr learned my addr 172.21.15.73:0/3656813527 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:50:56.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.155+0000 7f66c183b640 1 -- 172.21.15.73:0/3656813527 >> v1:172.21.15.120:6789/0 conn(0x7f66bc0b5f20 legacy=0x7f66bc0b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:56.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.155+0000 7f66c103a640 1 -- 172.21.15.73:0/3656813527 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3075153805 0 0) 0x7f66bc0b7e50 con 0x7f66bc162880 2024-04-26T20:50:56.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.155+0000 7f66c183b640 1 -- 172.21.15.73:0/3656813527 >> v1:172.21.15.73:6789/0 conn(0x7f66bc162880 legacy=0x7f66bc0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:56.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.155+0000 7f66c183b640 1 -- 172.21.15.73:0/3656813527 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66bc0d07d0 con 0x7f66bc0d3bc0 2024-04-26T20:50:56.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.156+0000 7f66c183b640 1 --2- 172.21.15.73:0/3656813527 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66bc0d3bc0 0x7f66bc0d2cb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f66ac0097a0 tx=0x7f66ac02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a274ce30bb4aeaf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:56.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.156+0000 7f66c103a640 1 -- 172.21.15.73:0/3656813527 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ac037070 con 0x7f66bc0d3bc0 2024-04-26T20:50:56.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.156+0000 7f66c103a640 1 -- 172.21.15.73:0/3656813527 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66ac032440 con 0x7f66bc0d3bc0 2024-04-26T20:50:56.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.156+0000 7f66c103a640 1 -- 172.21.15.73:0/3656813527 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ac032810 con 0x7f66bc0d3bc0 2024-04-26T20:50:56.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.157+0000 7f66c42a0640 1 -- 172.21.15.73:0/3656813527 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66bc0d3bc0 msgr2=0x7f66bc0d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:56.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.157+0000 7f66c42a0640 1 --2- 172.21.15.73:0/3656813527 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66bc0d3bc0 0x7f66bc0d2cb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f66ac0097a0 tx=0x7f66ac02a1e0 comp rx=0 tx=0).stop 2024-04-26T20:50:56.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.157+0000 7f66c42a0640 1 -- 172.21.15.73:0/3656813527 shutdown_connections 2024-04-26T20:50:56.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.157+0000 7f66c42a0640 1 --2- 172.21.15.73:0/3656813527 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66bc0d3bc0 0x7f66bc0d2cb0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:56.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.157+0000 7f66c42a0640 1 -- 172.21.15.73:0/3656813527 >> 172.21.15.73:0/3656813527 conn(0x7f66bc0d4f60 msgr2=0x7f66bc0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:56.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.157+0000 7f66c42a0640 1 -- 172.21.15.73:0/3656813527 shutdown_connections 2024-04-26T20:50:56.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.157+0000 7f66c42a0640 1 -- 172.21.15.73:0/3656813527 wait complete. 2024-04-26T20:50:56.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.158+0000 7f66c42a0640 1 Processor -- start 2024-04-26T20:50:56.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.158+0000 7f66c42a0640 1 -- start start 2024-04-26T20:50:56.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.159+0000 7f66c42a0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f66bc0b5f20 0x7f66bc1f8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:56.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.159+0000 7f66c42a0640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f66bc0d3bc0 0x7f66bc0b4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:56.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.159+0000 7f66c42a0640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66bc162880 0x7f66bc0a26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:56.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.159+0000 7f66c42a0640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f66bc162c60 con 0x7f66bc162880 2024-04-26T20:50:56.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.159+0000 7f66c42a0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f66bc0d0d90 con 0x7f66bc0b5f20 2024-04-26T20:50:56.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.159+0000 7f66c42a0640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f66bc162de0 con 0x7f66bc0d3bc0 2024-04-26T20:50:56.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.159+0000 7f66c283d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66bc162880 0x7f66bc0a26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:56.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.159+0000 7f66c283d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66bc162880 0x7f66bc0a26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49418/0 (socket says 172.21.15.73:49418) 2024-04-26T20:50:56.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.159+0000 7f66c283d640 1 -- 172.21.15.73:0/2692020109 learned_addr learned my addr 172.21.15.73:0/2692020109 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:56.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.159+0000 7f66c203c640 1 --2- 172.21.15.73:0/2692020109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f66bc0b5f20 0x7f66bc1f8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:56.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.159+0000 7f66c183b640 1 --2- 172.21.15.73:0/2692020109 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f66bc0d3bc0 0x7f66bc0b4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:56.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.159+0000 7f66c283d640 1 -- 172.21.15.73:0/2692020109 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f66bc0d3bc0 msgr2=0x7f66bc0b4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:56.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.159+0000 7f66c283d640 1 --2- 172.21.15.73:0/2692020109 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f66bc0d3bc0 0x7f66bc0b4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:56.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.159+0000 7f66c283d640 1 -- 172.21.15.73:0/2692020109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f66bc0b5f20 msgr2=0x7f66bc1f8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:56.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.159+0000 7f66c283d640 1 --2- 172.21.15.73:0/2692020109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f66bc0b5f20 0x7f66bc1f8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:56.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.159+0000 7f66c283d640 1 -- 172.21.15.73:0/2692020109 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66bc0b4a60 con 0x7f66bc162880 2024-04-26T20:50:56.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.159+0000 7f66c283d640 1 --2- 172.21.15.73:0/2692020109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66bc162880 0x7f66bc0a26a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f66b400c780 tx=0x7f66b400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:56.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.160+0000 7f66b2ffd640 1 -- 172.21.15.73:0/2692020109 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b4004b60 con 0x7f66bc162880 2024-04-26T20:50:56.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.160+0000 7f66b2ffd640 1 -- 172.21.15.73:0/2692020109 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66b4004d00 con 0x7f66bc162880 2024-04-26T20:50:56.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.160+0000 7f66b2ffd640 1 -- 172.21.15.73:0/2692020109 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b400aa90 con 0x7f66bc162880 2024-04-26T20:50:56.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.160+0000 7f66c42a0640 1 -- 172.21.15.73:0/2692020109 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66bc1f8690 con 0x7f66bc162880 2024-04-26T20:50:56.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.160+0000 7f66c42a0640 1 -- 172.21.15.73:0/2692020109 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66bc1f8c50 con 0x7f66bc162880 2024-04-26T20:50:56.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.161+0000 7f66c42a0640 1 -- 172.21.15.73:0/2692020109 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66bc0694f0 con 0x7f66bc162880 2024-04-26T20:50:56.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.161+0000 7f66b2ffd640 1 -- 172.21.15.73:0/2692020109 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f66b400ac30 con 0x7f66bc162880 2024-04-26T20:50:56.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.161+0000 7f66b2ffd640 1 --2- 172.21.15.73:0/2692020109 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f66a006c8f0 0x7f66a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:56.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.161+0000 7f66b2ffd640 1 -- 172.21.15.73:0/2692020109 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f66b408bfe0 con 0x7f66bc162880 2024-04-26T20:50:56.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.161+0000 7f66c203c640 1 --2- 172.21.15.73:0/2692020109 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f66a006c8f0 0x7f66a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:56.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.162+0000 7f66c203c640 1 --2- 172.21.15.73:0/2692020109 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f66a006c8f0 0x7f66a006edb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f66b8007fd0 tx=0x7f66b8007de0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:56.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.166+0000 7f66b2ffd640 1 -- 172.21.15.73:0/2692020109 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f66b4020070 con 0x7f66bc162880 2024-04-26T20:50:56.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.387+0000 7f66c42a0640 1 -- 172.21.15.73:0/2692020109 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f66bc074590 con 0x7f66bc162880 2024-04-26T20:50:56.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.390+0000 7f66b2ffd640 1 -- 172.21.15.73:0/2692020109 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+174132 (secure 0 0 0) 0x7f66b4059a80 con 0x7f66bc162880 2024-04-26T20:50:56.393 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:50:56.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.393+0000 7f66c42a0640 1 -- 172.21.15.73:0/2692020109 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f66a006c8f0 msgr2=0x7f66a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:56.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.393+0000 7f66c42a0640 1 --2- 172.21.15.73:0/2692020109 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f66a006c8f0 0x7f66a006edb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f66b8007fd0 tx=0x7f66b8007de0 comp rx=0 tx=0).stop 2024-04-26T20:50:56.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.393+0000 7f66c42a0640 1 -- 172.21.15.73:0/2692020109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66bc162880 msgr2=0x7f66bc0a26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:56.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.393+0000 7f66c42a0640 1 --2- 172.21.15.73:0/2692020109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66bc162880 0x7f66bc0a26a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f66b400c780 tx=0x7f66b400cc40 comp rx=0 tx=0).stop 2024-04-26T20:50:56.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.393+0000 7f66c42a0640 1 -- 172.21.15.73:0/2692020109 shutdown_connections 2024-04-26T20:50:56.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.393+0000 7f66c42a0640 1 --2- 172.21.15.73:0/2692020109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66bc162880 0x7f66bc0a26a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:56.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.393+0000 7f66c42a0640 1 --2- 172.21.15.73:0/2692020109 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f66a006c8f0 0x7f66a006edb0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:56.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.393+0000 7f66c42a0640 1 --2- 172.21.15.73:0/2692020109 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f66bc0d3bc0 0x7f66bc0b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:56.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.393+0000 7f66c42a0640 1 --2- 172.21.15.73:0/2692020109 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f66bc0b5f20 0x7f66bc1f8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:56.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.393+0000 7f66c42a0640 1 -- 172.21.15.73:0/2692020109 >> 172.21.15.73:0/2692020109 conn(0x7f66bc0d4f60 msgr2=0x7f66bc0d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:56.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.393+0000 7f66c42a0640 1 -- 172.21.15.73:0/2692020109 shutdown_connections 2024-04-26T20:50:56.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.393+0000 7f66c42a0640 1 -- 172.21.15.73:0/2692020109 wait complete. 2024-04-26T20:50:56.404 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":5,"flags":0,"active_gid":4107,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":666905549},{"type":"v1","addr":"172.21.15.73:6833","nonce":666905549}]},"active_addr":"172.21.15.73:6833/666905549","active_change":"2024-04-26T20:50:11.970452+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4102,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.73:0","nonce":3717382563}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.73:0","nonce":734581961}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.73:0","nonce":2753634831}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.73:0","nonce":999054320}]}]} 2024-04-26T20:50:56.409 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-26T20:50:56.409 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-26T20:50:56.409 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:50:56.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.505+0000 7f2daa683640 1 Processor -- start 2024-04-26T20:50:56.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.505+0000 7f2daa683640 1 -- start start 2024-04-26T20:50:56.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.506+0000 7f2daa683640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2da406c950 0x7f2da4144990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:56.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.506+0000 7f2daa683640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2da41455e0 con 0x7f2da4160450 2024-04-26T20:50:56.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.506+0000 7f2daa683640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2da415f970 con 0x7f2da40b4ae0 2024-04-26T20:50:56.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.506+0000 7f2daa683640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f2da415fc50 con 0x7f2da406c950 2024-04-26T20:50:56.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.506+0000 7f2da37fe640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2da406c950 0x7f2da4144990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:56.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.506+0000 7f2da8c20640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f2da4160450 0x7f2da4142900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33072/0 (socket says 172.21.15.73:33072) 2024-04-26T20:50:56.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.506+0000 7f2da37fe640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2da406c950 0x7f2da4144990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49428/0 (socket says 172.21.15.73:49428) 2024-04-26T20:50:56.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.506+0000 7f2da8c20640 1 -- 172.21.15.73:0/2862619083 learned_addr learned my addr 172.21.15.73:0/2862619083 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:50:56.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.507+0000 7f2da37fe640 1 -- 172.21.15.73:0/2862619083 >> v1:172.21.15.120:6789/0 conn(0x7f2da40b4ae0 legacy=0x7f2da406c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:56.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.507+0000 7f2da37fe640 1 -- 172.21.15.73:0/2862619083 >> v1:172.21.15.73:6789/0 conn(0x7f2da4160450 legacy=0x7f2da4142900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:56.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.507+0000 7f2da37fe640 1 -- 172.21.15.73:0/2862619083 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2da415ff30 con 0x7f2da406c950 2024-04-26T20:50:56.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.507+0000 7f2da37fe640 1 --2- 172.21.15.73:0/2862619083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2da406c950 0x7f2da4144990 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2d980097a0 tx=0x7f2d9802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b877b7f89608a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:56.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.507+0000 7f2da2ffd640 1 -- 172.21.15.73:0/2862619083 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d98037070 con 0x7f2da406c950 2024-04-26T20:50:56.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.508+0000 7f2da2ffd640 1 -- 172.21.15.73:0/2862619083 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d98032440 con 0x7f2da406c950 2024-04-26T20:50:56.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.508+0000 7f2da2ffd640 1 -- 172.21.15.73:0/2862619083 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d98032810 con 0x7f2da406c950 2024-04-26T20:50:56.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.508+0000 7f2daa683640 1 -- 172.21.15.73:0/2862619083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2da406c950 msgr2=0x7f2da4144990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:56.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.508+0000 7f2daa683640 1 --2- 172.21.15.73:0/2862619083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2da406c950 0x7f2da4144990 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2d980097a0 tx=0x7f2d9802a230 comp rx=0 tx=0).stop 2024-04-26T20:50:56.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.508+0000 7f2daa683640 1 -- 172.21.15.73:0/2862619083 shutdown_connections 2024-04-26T20:50:56.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.508+0000 7f2daa683640 1 --2- 172.21.15.73:0/2862619083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2da406c950 0x7f2da4144990 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:56.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.508+0000 7f2daa683640 1 -- 172.21.15.73:0/2862619083 >> 172.21.15.73:0/2862619083 conn(0x7f2da40b3110 msgr2=0x7f2da40b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:56.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.508+0000 7f2daa683640 1 -- 172.21.15.73:0/2862619083 shutdown_connections 2024-04-26T20:50:56.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.508+0000 7f2daa683640 1 -- 172.21.15.73:0/2862619083 wait complete. 2024-04-26T20:50:56.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.509+0000 7f2daa683640 1 Processor -- start 2024-04-26T20:50:56.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.509+0000 7f2daa683640 1 -- start start 2024-04-26T20:50:56.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.510+0000 7f2daa683640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2da406c950 0x7f2da41567b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:56.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.510+0000 7f2daa683640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2da40b4ae0 0x7f2da413ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:56.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.510+0000 7f2daa683640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2da4160450 0x7f2da41f5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:56.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.510+0000 7f2daa683640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2da4145050 con 0x7f2da4160450 2024-04-26T20:50:56.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.510+0000 7f2daa683640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2da4144ed0 con 0x7f2da40b4ae0 2024-04-26T20:50:56.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.510+0000 7f2daa683640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f2da41451d0 con 0x7f2da406c950 2024-04-26T20:50:56.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.510+0000 7f2da8c20640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2da4160450 0x7f2da41f5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:56.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.510+0000 7f2da8c20640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2da4160450 0x7f2da41f5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49440/0 (socket says 172.21.15.73:49440) 2024-04-26T20:50:56.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.510+0000 7f2da8c20640 1 -- 172.21.15.73:0/4194875359 learned_addr learned my addr 172.21.15.73:0/4194875359 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:56.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.511+0000 7f2da37fe640 1 --2- 172.21.15.73:0/4194875359 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2da40b4ae0 0x7f2da413ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:56.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.511+0000 7f2da3fff640 1 --2- 172.21.15.73:0/4194875359 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2da406c950 0x7f2da41567b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:56.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.511+0000 7f2da8c20640 1 -- 172.21.15.73:0/4194875359 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2da406c950 msgr2=0x7f2da41567b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:56.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.511+0000 7f2da8c20640 1 --2- 172.21.15.73:0/4194875359 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2da406c950 0x7f2da41567b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:56.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.511+0000 7f2da8c20640 1 -- 172.21.15.73:0/4194875359 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2da40b4ae0 msgr2=0x7f2da413ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:56.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.511+0000 7f2da8c20640 1 --2- 172.21.15.73:0/4194875359 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2da40b4ae0 0x7f2da413ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:56.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.511+0000 7f2da8c20640 1 -- 172.21.15.73:0/4194875359 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2da4156f20 con 0x7f2da4160450 2024-04-26T20:50:56.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.511+0000 7f2da8c20640 1 --2- 172.21.15.73:0/4194875359 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2da4160450 0x7f2da41f5780 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c00cbc0 tx=0x7f2d9c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:56.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.511+0000 7f2da0ff9640 1 -- 172.21.15.73:0/4194875359 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c017070 con 0x7f2da4160450 2024-04-26T20:50:56.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.511+0000 7f2da0ff9640 1 -- 172.21.15.73:0/4194875359 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d9c00ad40 con 0x7f2da4160450 2024-04-26T20:50:56.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.511+0000 7f2daa683640 1 -- 172.21.15.73:0/4194875359 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2da4157200 con 0x7f2da4160450 2024-04-26T20:50:56.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.511+0000 7f2da0ff9640 1 -- 172.21.15.73:0/4194875359 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c012600 con 0x7f2da4160450 2024-04-26T20:50:56.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.511+0000 7f2daa683640 1 -- 172.21.15.73:0/4194875359 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2da41577c0 con 0x7f2da4160450 2024-04-26T20:50:56.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.512+0000 7f2daa683640 1 -- 172.21.15.73:0/4194875359 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2da4068fc0 con 0x7f2da4160450 2024-04-26T20:50:56.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.512+0000 7f2da0ff9640 1 -- 172.21.15.73:0/4194875359 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2d9c01a070 con 0x7f2da4160450 2024-04-26T20:50:56.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.513+0000 7f2da0ff9640 1 --2- 172.21.15.73:0/4194875359 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f2d9006c920 0x7f2d9006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:56.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.513+0000 7f2da0ff9640 1 -- 172.21.15.73:0/4194875359 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f2d9c08bb30 con 0x7f2da4160450 2024-04-26T20:50:56.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.513+0000 7f2da3fff640 1 --2- 172.21.15.73:0/4194875359 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f2d9006c920 0x7f2d9006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:56.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.513+0000 7f2da3fff640 1 --2- 172.21.15.73:0/4194875359 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f2d9006c920 0x7f2d9006ede0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2d940015a0 tx=0x7f2d94009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:56.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.517+0000 7f2da0ff9640 1 -- 172.21.15.73:0/4194875359 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2d9c0595d0 con 0x7f2da4160450 2024-04-26T20:50:56.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.697+0000 7f2daa683640 1 -- 172.21.15.73:0/4194875359 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2da40694f0 con 0x7f2da4160450 2024-04-26T20:50:56.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.698+0000 7f2da0ff9640 1 -- 172.21.15.73:0/4194875359 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18514 (secure 0 0 0) 0x7f2d9c05d8d0 con 0x7f2da4160450 2024-04-26T20:50:56.699 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:50:56.700 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":25,"fsid":"fbd3f548-b547-4713-856b-09f98f5d1764","created":"2024-04-26T20:50:07.876786+0000","modified":"2024-04-26T20:50:38.657510+0000","last_up_change":"2024-04-26T20:50:19.082741+0000","last_in_change":"2024-04-26T20:50:10.018400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:50:19.993731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:50:29.964420+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:50:30.920082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:50:34.650207+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"72d6f4f3-214a-4392-a02f-c33cbcab3c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6809","nonce":3461705124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6811","nonce":3461705124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6815","nonce":3461705124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6813","nonce":3461705124}]},"public_addr":"172.21.15.73:6809/3461705124","cluster_addr":"172.21.15.73:6811/3461705124","heartbeat_back_addr":"172.21.15.73:6815/3461705124","heartbeat_front_addr":"172.21.15.73:6813/3461705124","state":["exists","up"]},{"osd":1,"uuid":"786bf8ca-6ddc-4b3c-be6c-9152398bd5fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6817","nonce":2827619403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6819","nonce":2827619403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6825","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6827","nonce":2827619403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6821","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6823","nonce":2827619403}]},"public_addr":"172.21.15.73:6817/2827619403","cluster_addr":"172.21.15.73:6819/2827619403","heartbeat_back_addr":"172.21.15.73:6827/2827619403","heartbeat_front_addr":"172.21.15.73:6823/2827619403","state":["exists","up"]},{"osd":2,"uuid":"90963b1a-5437-4ef8-ad8b-6b74c534eeb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6801","nonce":2542681064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6803","nonce":2542681064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6808","nonce":2542681064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6805","nonce":2542681064}]},"public_addr":"172.21.15.73:6801/2542681064","cluster_addr":"172.21.15.73:6803/2542681064","heartbeat_back_addr":"172.21.15.73:6808/2542681064","heartbeat_front_addr":"172.21.15.73:6805/2542681064","state":["exists","up"]},{"osd":3,"uuid":"bd3d2148-6e38-41b4-a453-fe6e3b8b4ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6822","nonce":1604042718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6826","nonce":1604042718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6831","nonce":1604042718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6829","nonce":1604042718}]},"public_addr":"172.21.15.73:6822/1604042718","cluster_addr":"172.21.15.73:6826/1604042718","heartbeat_back_addr":"172.21.15.73:6831/1604042718","heartbeat_front_addr":"172.21.15.73:6829/1604042718","state":["exists","up"]},{"osd":4,"uuid":"e8668dc8-a23f-46d2-8311-cb882da08e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6817","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6819","nonce":3591055561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6824","nonce":3591055561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6831","nonce":3591055561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6828","nonce":3591055561}]},"public_addr":"172.21.15.120:6819/3591055561","cluster_addr":"172.21.15.120:6824/3591055561","heartbeat_back_addr":"172.21.15.120:6831/3591055561","heartbeat_front_addr":"172.21.15.120:6828/3591055561","state":["exists","up"]},{"osd":5,"uuid":"cbac0bdb-96bd-4852-bcce-e564495c3fcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6809","nonce":819450433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6811","nonce":819450433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6815","nonce":819450433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6813","nonce":819450433}]},"public_addr":"172.21.15.120:6809/819450433","cluster_addr":"172.21.15.120:6811/819450433","heartbeat_back_addr":"172.21.15.120:6815/819450433","heartbeat_front_addr":"172.21.15.120:6813/819450433","state":["exists","up"]},{"osd":6,"uuid":"1884b2a8-569f-4272-96a9-f25a09364dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6801","nonce":3946210727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6803","nonce":3946210727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6807","nonce":3946210727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6805","nonce":3946210727}]},"public_addr":"172.21.15.120:6801/3946210727","cluster_addr":"172.21.15.120:6803/3946210727","heartbeat_back_addr":"172.21.15.120:6807/3946210727","heartbeat_front_addr":"172.21.15.120:6805/3946210727","state":["exists","up"]},{"osd":7,"uuid":"c0c3bf6d-a4c5-4f1e-aa2d-330951830666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6818","nonce":2915574674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6821","nonce":2915574674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6827","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6829","nonce":2915574674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6823","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6825","nonce":2915574674}]},"public_addr":"172.21.15.120:6818/2915574674","cluster_addr":"172.21.15.120:6821/2915574674","heartbeat_back_addr":"172.21.15.120:6829/2915574674","heartbeat_front_addr":"172.21.15.120:6825/2915574674","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.218216+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.208710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.112705+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.182789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.164324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.174212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.225396+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.261559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:50:56.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.700+0000 7f2daa683640 1 -- 172.21.15.73:0/4194875359 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f2d9006c920 msgr2=0x7f2d9006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:56.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.700+0000 7f2daa683640 1 --2- 172.21.15.73:0/4194875359 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f2d9006c920 0x7f2d9006ede0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2d940015a0 tx=0x7f2d94009040 comp rx=0 tx=0).stop 2024-04-26T20:50:56.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.700+0000 7f2daa683640 1 -- 172.21.15.73:0/4194875359 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2da4160450 msgr2=0x7f2da41f5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:56.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.700+0000 7f2daa683640 1 --2- 172.21.15.73:0/4194875359 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2da4160450 0x7f2da41f5780 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c00cbc0 tx=0x7f2d9c00a430 comp rx=0 tx=0).stop 2024-04-26T20:50:56.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.700+0000 7f2daa683640 1 -- 172.21.15.73:0/4194875359 shutdown_connections 2024-04-26T20:50:56.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.700+0000 7f2daa683640 1 --2- 172.21.15.73:0/4194875359 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2da4160450 0x7f2da41f5780 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:56.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.700+0000 7f2daa683640 1 --2- 172.21.15.73:0/4194875359 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f2d9006c920 0x7f2d9006ede0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:56.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.700+0000 7f2daa683640 1 --2- 172.21.15.73:0/4194875359 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2da40b4ae0 0x7f2da413ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:56.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.700+0000 7f2daa683640 1 --2- 172.21.15.73:0/4194875359 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2da406c950 0x7f2da41567b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:56.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.700+0000 7f2daa683640 1 -- 172.21.15.73:0/4194875359 >> 172.21.15.73:0/4194875359 conn(0x7f2da40b3110 msgr2=0x7f2da406f230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:56.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.701+0000 7f2daa683640 1 -- 172.21.15.73:0/4194875359 shutdown_connections 2024-04-26T20:50:56.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.701+0000 7f2daa683640 1 -- 172.21.15.73:0/4194875359 wait complete. 2024-04-26T20:50:56.712 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-26T20:50:56.713 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:50:56.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.808+0000 7f569a759640 1 Processor -- start 2024-04-26T20:50:56.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.808+0000 7f569a759640 1 -- start start 2024-04-26T20:50:56.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.809+0000 7f569a759640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f56940b0380 0x7f56940ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:56.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.809+0000 7f569a759640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56940cfd80 con 0x7f56940b10b0 2024-04-26T20:50:56.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.809+0000 7f569a759640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56940d0020 con 0x7f569406c950 2024-04-26T20:50:56.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.809+0000 7f569a759640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f56940af8e0 con 0x7f56940b0380 2024-04-26T20:50:56.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.809+0000 7f56937fe640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f56940b10b0 0x7f569406c240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33076/0 (socket says 172.21.15.73:33076) 2024-04-26T20:50:56.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.809+0000 7f56937fe640 1 -- 172.21.15.73:0/2568405511 learned_addr learned my addr 172.21.15.73:0/2568405511 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:50:56.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.809+0000 7f5692ffd640 1 -- 172.21.15.73:0/2568405511 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2996164018 0 0) 0x7f56940cfd80 con 0x7f56940b10b0 2024-04-26T20:50:56.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.809+0000 7f5692ffd640 1 -- 172.21.15.73:0/2568405511 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f567c003520 con 0x7f56940b10b0 2024-04-26T20:50:56.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.809+0000 7f5693fff640 1 --2- 172.21.15.73:0/2568405511 >> v2:172.21.15.120:3300/0 conn(0x7f56940b0380 0x7f56940ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:56.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.809+0000 7f5692ffd640 1 -- 172.21.15.73:0/2568405511 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1837778646 0 0) 0x7f567c003520 con 0x7f56940b10b0 2024-04-26T20:50:56.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.809+0000 7f5692ffd640 1 -- 172.21.15.73:0/2568405511 >> v2:172.21.15.120:3300/0 conn(0x7f56940b0380 msgr2=0x7f56940ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:56.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.809+0000 7f5692ffd640 1 --2- 172.21.15.73:0/2568405511 >> v2:172.21.15.120:3300/0 conn(0x7f56940b0380 0x7f56940ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:56.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.809+0000 7f5692ffd640 1 -- 172.21.15.73:0/2568405511 >> v1:172.21.15.120:6789/0 conn(0x7f569406c950 legacy=0x7f569415e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:56.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.809+0000 7f5692ffd640 1 -- 172.21.15.73:0/2568405511 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56940d0300 con 0x7f56940b10b0 2024-04-26T20:50:56.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.809+0000 7f5692ffd640 1 -- 172.21.15.73:0/2568405511 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7f5688001690 con 0x7f56940b10b0 2024-04-26T20:50:56.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.809+0000 7f5692ffd640 1 -- 172.21.15.73:0/2568405511 >> v1:172.21.15.73:6789/0 conn(0x7f56940b10b0 legacy=0x7f569406c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:56.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.809+0000 7f5692ffd640 1 --2- 172.21.15.73:0/2568405511 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f567c0039b0 0x7f567c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:56.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.809+0000 7f5692ffd640 1 --2- 172.21.15.73:0/2568405511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f567c004800 0x7f567c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:56.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.809+0000 7f5692ffd640 1 --2- 172.21.15.73:0/2568405511 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f567c0051c0 0x7f567c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:56.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.809+0000 7f5692ffd640 1 -- 172.21.15.73:0/2568405511 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f56940af8e0 con 0x7f567c004800 2024-04-26T20:50:56.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.809+0000 7f5692ffd640 1 -- 172.21.15.73:0/2568405511 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f567c00c160 con 0x7f567c0051c0 2024-04-26T20:50:56.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.809+0000 7f5692ffd640 1 -- 172.21.15.73:0/2568405511 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f567c00c360 con 0x7f567c0039b0 2024-04-26T20:50:56.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.810+0000 7f56937fe640 1 --2- 172.21.15.73:0/2568405511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f567c004800 0x7f567c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:56.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.810+0000 7f5693fff640 1 --2- 172.21.15.73:0/2568405511 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f567c0039b0 0x7f567c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:56.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.810+0000 7f5698cf6640 1 --2- 172.21.15.73:0/2568405511 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f567c0051c0 0x7f567c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:56.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.810+0000 7f56937fe640 1 -- 172.21.15.73:0/2568405511 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f567c0039b0 msgr2=0x7f567c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:56.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.810+0000 7f56937fe640 1 --2- 172.21.15.73:0/2568405511 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f567c0039b0 0x7f567c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:56.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.810+0000 7f56937fe640 1 -- 172.21.15.73:0/2568405511 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f567c0051c0 msgr2=0x7f567c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:56.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.810+0000 7f56937fe640 1 --2- 172.21.15.73:0/2568405511 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f567c0051c0 0x7f567c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:56.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.810+0000 7f5698cf6640 1 -- 172.21.15.73:0/2568405511 reap_dead start 2024-04-26T20:50:56.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.810+0000 7f56937fe640 1 -- 172.21.15.73:0/2568405511 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f567c003520 con 0x7f567c004800 2024-04-26T20:50:56.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.810+0000 7f56937fe640 1 --2- 172.21.15.73:0/2568405511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f567c004800 0x7f567c004c80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f568800c210 tx=0x7f567c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37f7f95492450ca2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:56.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.811+0000 7f5692ffd640 1 -- 172.21.15.73:0/2568405511 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5688007d30 con 0x7f567c004800 2024-04-26T20:50:56.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.811+0000 7f5692ffd640 1 -- 172.21.15.73:0/2568405511 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56880058d0 con 0x7f567c004800 2024-04-26T20:50:56.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.811+0000 7f5692ffd640 1 -- 172.21.15.73:0/2568405511 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5688005ce0 con 0x7f567c004800 2024-04-26T20:50:56.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.811+0000 7f569a759640 1 -- 172.21.15.73:0/2568405511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f567c004800 msgr2=0x7f567c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:56.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.811+0000 7f569a759640 1 --2- 172.21.15.73:0/2568405511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f567c004800 0x7f567c004c80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f568800c210 tx=0x7f567c005830 comp rx=0 tx=0).stop 2024-04-26T20:50:56.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.811+0000 7f569a759640 1 -- 172.21.15.73:0/2568405511 shutdown_connections 2024-04-26T20:50:56.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.811+0000 7f569a759640 1 --2- 172.21.15.73:0/2568405511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f567c004800 0x7f567c004c80 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:56.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.811+0000 7f569a759640 1 -- 172.21.15.73:0/2568405511 >> 172.21.15.73:0/2568405511 conn(0x7f56940d1cf0 msgr2=0x7f56940d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:56.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.811+0000 7f569a759640 1 -- 172.21.15.73:0/2568405511 shutdown_connections 2024-04-26T20:50:56.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.811+0000 7f569a759640 1 -- 172.21.15.73:0/2568405511 wait complete. 2024-04-26T20:50:56.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.812+0000 7f569a759640 1 Processor -- start 2024-04-26T20:50:56.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.812+0000 7f569a759640 1 -- start start 2024-04-26T20:50:56.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.813+0000 7f569a759640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f567c004800 0x7f569414e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:56.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.813+0000 7f569a759640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f56940b6740 0x7f56940b6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:56.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.813+0000 7f569a759640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f569406fdf0 0x7f5694070270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:56.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.813+0000 7f569a759640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f56940b0760 con 0x7f569406fdf0 2024-04-26T20:50:56.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.813+0000 7f569a759640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f56940d1980 con 0x7f567c004800 2024-04-26T20:50:56.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.813+0000 7f569a759640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f56940d15c0 con 0x7f56940b6740 2024-04-26T20:50:56.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.813+0000 7f5698cf6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f569406fdf0 0x7f5694070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:56.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.813+0000 7f5698cf6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f569406fdf0 0x7f5694070270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49456/0 (socket says 172.21.15.73:49456) 2024-04-26T20:50:56.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.813+0000 7f5698cf6640 1 -- 172.21.15.73:0/2336847574 learned_addr learned my addr 172.21.15.73:0/2336847574 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:56.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.813+0000 7f56937fe640 1 --2- 172.21.15.73:0/2336847574 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f56940b6740 0x7f56940b6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:56.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.813+0000 7f5693fff640 1 --2- 172.21.15.73:0/2336847574 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f567c004800 0x7f569414e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:56.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.813+0000 7f5698cf6640 1 -- 172.21.15.73:0/2336847574 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f56940b6740 msgr2=0x7f56940b6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:56.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.813+0000 7f5698cf6640 1 --2- 172.21.15.73:0/2336847574 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f56940b6740 0x7f56940b6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:56.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.813+0000 7f5698cf6640 1 -- 172.21.15.73:0/2336847574 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f567c004800 msgr2=0x7f569414e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:56.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.813+0000 7f5698cf6640 1 --2- 172.21.15.73:0/2336847574 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f567c004800 0x7f569414e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:56.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.814+0000 7f5698cf6640 1 -- 172.21.15.73:0/2336847574 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f569415a8c0 con 0x7f569406fdf0 2024-04-26T20:50:56.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.814+0000 7f5698cf6640 1 --2- 172.21.15.73:0/2336847574 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f569406fdf0 0x7f5694070270 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5694074650 tx=0x7f568c00ea00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:56.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.814+0000 7f5690ff9640 1 -- 172.21.15.73:0/2336847574 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568c019070 con 0x7f569406fdf0 2024-04-26T20:50:56.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.814+0000 7f5690ff9640 1 -- 172.21.15.73:0/2336847574 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f568c006d60 con 0x7f569406fdf0 2024-04-26T20:50:56.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.814+0000 7f569a759640 1 -- 172.21.15.73:0/2336847574 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56940b72b0 con 0x7f569406fdf0 2024-04-26T20:50:56.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.814+0000 7f5690ff9640 1 -- 172.21.15.73:0/2336847574 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568c003b90 con 0x7f569406fdf0 2024-04-26T20:50:56.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.814+0000 7f569a759640 1 -- 172.21.15.73:0/2336847574 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56940b7870 con 0x7f569406fdf0 2024-04-26T20:50:56.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.815+0000 7f569a759640 1 -- 172.21.15.73:0/2336847574 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56940694f0 con 0x7f569406fdf0 2024-04-26T20:50:56.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.815+0000 7f5690ff9640 1 -- 172.21.15.73:0/2336847574 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f568c01c070 con 0x7f569406fdf0 2024-04-26T20:50:56.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.816+0000 7f5690ff9640 1 --2- 172.21.15.73:0/2336847574 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f568006c8f0 0x7f568006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:56.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.816+0000 7f5690ff9640 1 -- 172.21.15.73:0/2336847574 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f568c08b440 con 0x7f569406fdf0 2024-04-26T20:50:56.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.816+0000 7f5693fff640 1 --2- 172.21.15.73:0/2336847574 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f568006c8f0 0x7f568006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:56.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.816+0000 7f5693fff640 1 --2- 172.21.15.73:0/2336847574 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f568006c8f0 0x7f568006edb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5694052070 tx=0x7f5684008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:56.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.820+0000 7f5690ff9640 1 -- 172.21.15.73:0/2336847574 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f568c059530 con 0x7f569406fdf0 2024-04-26T20:50:56.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.997+0000 7f569a759640 1 -- 172.21.15.73:0/2336847574 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f56940cd3a0 con 0x7f569406fdf0 2024-04-26T20:50:56.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:56.998+0000 7f5690ff9640 1 -- 172.21.15.73:0/2336847574 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18514 (secure 0 0 0) 0x7f568c058e30 con 0x7f569406fdf0 2024-04-26T20:50:57.000 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:50:57.000 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":25,"fsid":"fbd3f548-b547-4713-856b-09f98f5d1764","created":"2024-04-26T20:50:07.876786+0000","modified":"2024-04-26T20:50:38.657510+0000","last_up_change":"2024-04-26T20:50:19.082741+0000","last_in_change":"2024-04-26T20:50:10.018400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:50:19.993731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:50:29.964420+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:50:30.920082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:50:34.650207+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"72d6f4f3-214a-4392-a02f-c33cbcab3c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6809","nonce":3461705124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6811","nonce":3461705124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6815","nonce":3461705124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6813","nonce":3461705124}]},"public_addr":"172.21.15.73:6809/3461705124","cluster_addr":"172.21.15.73:6811/3461705124","heartbeat_back_addr":"172.21.15.73:6815/3461705124","heartbeat_front_addr":"172.21.15.73:6813/3461705124","state":["exists","up"]},{"osd":1,"uuid":"786bf8ca-6ddc-4b3c-be6c-9152398bd5fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6817","nonce":2827619403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6819","nonce":2827619403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6825","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6827","nonce":2827619403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6821","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6823","nonce":2827619403}]},"public_addr":"172.21.15.73:6817/2827619403","cluster_addr":"172.21.15.73:6819/2827619403","heartbeat_back_addr":"172.21.15.73:6827/2827619403","heartbeat_front_addr":"172.21.15.73:6823/2827619403","state":["exists","up"]},{"osd":2,"uuid":"90963b1a-5437-4ef8-ad8b-6b74c534eeb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6801","nonce":2542681064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6803","nonce":2542681064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6808","nonce":2542681064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6805","nonce":2542681064}]},"public_addr":"172.21.15.73:6801/2542681064","cluster_addr":"172.21.15.73:6803/2542681064","heartbeat_back_addr":"172.21.15.73:6808/2542681064","heartbeat_front_addr":"172.21.15.73:6805/2542681064","state":["exists","up"]},{"osd":3,"uuid":"bd3d2148-6e38-41b4-a453-fe6e3b8b4ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6822","nonce":1604042718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6826","nonce":1604042718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6831","nonce":1604042718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6829","nonce":1604042718}]},"public_addr":"172.21.15.73:6822/1604042718","cluster_addr":"172.21.15.73:6826/1604042718","heartbeat_back_addr":"172.21.15.73:6831/1604042718","heartbeat_front_addr":"172.21.15.73:6829/1604042718","state":["exists","up"]},{"osd":4,"uuid":"e8668dc8-a23f-46d2-8311-cb882da08e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6817","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6819","nonce":3591055561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6824","nonce":3591055561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6831","nonce":3591055561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6828","nonce":3591055561}]},"public_addr":"172.21.15.120:6819/3591055561","cluster_addr":"172.21.15.120:6824/3591055561","heartbeat_back_addr":"172.21.15.120:6831/3591055561","heartbeat_front_addr":"172.21.15.120:6828/3591055561","state":["exists","up"]},{"osd":5,"uuid":"cbac0bdb-96bd-4852-bcce-e564495c3fcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6809","nonce":819450433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6811","nonce":819450433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6815","nonce":819450433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6813","nonce":819450433}]},"public_addr":"172.21.15.120:6809/819450433","cluster_addr":"172.21.15.120:6811/819450433","heartbeat_back_addr":"172.21.15.120:6815/819450433","heartbeat_front_addr":"172.21.15.120:6813/819450433","state":["exists","up"]},{"osd":6,"uuid":"1884b2a8-569f-4272-96a9-f25a09364dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6801","nonce":3946210727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6803","nonce":3946210727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6807","nonce":3946210727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6805","nonce":3946210727}]},"public_addr":"172.21.15.120:6801/3946210727","cluster_addr":"172.21.15.120:6803/3946210727","heartbeat_back_addr":"172.21.15.120:6807/3946210727","heartbeat_front_addr":"172.21.15.120:6805/3946210727","state":["exists","up"]},{"osd":7,"uuid":"c0c3bf6d-a4c5-4f1e-aa2d-330951830666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6818","nonce":2915574674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6821","nonce":2915574674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6827","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6829","nonce":2915574674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6823","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6825","nonce":2915574674}]},"public_addr":"172.21.15.120:6818/2915574674","cluster_addr":"172.21.15.120:6821/2915574674","heartbeat_back_addr":"172.21.15.120:6829/2915574674","heartbeat_front_addr":"172.21.15.120:6825/2915574674","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.218216+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.208710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.112705+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.182789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.164324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.174212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.225396+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.261559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:50:57.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.001+0000 7f569a759640 1 -- 172.21.15.73:0/2336847574 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f568006c8f0 msgr2=0x7f568006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.001+0000 7f569a759640 1 --2- 172.21.15.73:0/2336847574 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f568006c8f0 0x7f568006edb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5694052070 tx=0x7f5684008040 comp rx=0 tx=0).stop 2024-04-26T20:50:57.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.001+0000 7f569a759640 1 -- 172.21.15.73:0/2336847574 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f569406fdf0 msgr2=0x7f5694070270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.001+0000 7f569a759640 1 --2- 172.21.15.73:0/2336847574 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f569406fdf0 0x7f5694070270 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5694074650 tx=0x7f568c00ea00 comp rx=0 tx=0).stop 2024-04-26T20:50:57.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.001+0000 7f569a759640 1 -- 172.21.15.73:0/2336847574 shutdown_connections 2024-04-26T20:50:57.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.001+0000 7f569a759640 1 --2- 172.21.15.73:0/2336847574 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f569406fdf0 0x7f5694070270 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.001+0000 7f569a759640 1 --2- 172.21.15.73:0/2336847574 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f568006c8f0 0x7f568006edb0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.001+0000 7f569a759640 1 --2- 172.21.15.73:0/2336847574 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f56940b6740 0x7f56940b6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.001+0000 7f569a759640 1 --2- 172.21.15.73:0/2336847574 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f567c004800 0x7f569414e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.001+0000 7f569a759640 1 -- 172.21.15.73:0/2336847574 >> 172.21.15.73:0/2336847574 conn(0x7f56940d1cf0 msgr2=0x7f56940ce6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:57.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.001+0000 7f569a759640 1 -- 172.21.15.73:0/2336847574 shutdown_connections 2024-04-26T20:50:57.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.001+0000 7f569a759640 1 -- 172.21.15.73:0/2336847574 wait complete. 2024-04-26T20:50:57.013 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-26T20:50:57.014 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-26T20:50:57.014 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-26T20:50:57.015 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-26T20:50:57.015 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-26T20:50:57.016 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-26T20:50:57.016 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-26T20:50:57.017 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-26T20:50:57.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.179+0000 7f01f5f05640 1 Processor -- start 2024-04-26T20:50:57.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.179+0000 7f01f5f05640 1 -- start start 2024-04-26T20:50:57.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.180+0000 7f01f5f05640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f01f014cf00 0x7f01f0073a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.180+0000 7f01f5f05640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f01f0167a40 0x7f01f0166e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.181+0000 7f3d93d0a640 1 Processor -- start 2024-04-26T20:50:57.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.181+0000 7f3d93d0a640 1 -- start start 2024-04-26T20:50:57.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.180+0000 7f01f5f05640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f01f01673a0 0x7f01f0163820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.180+0000 7f01f5f05640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f01f00b4e30 con 0x7f01f014cf00 2024-04-26T20:50:57.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.180+0000 7f01f5f05640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f01f0073fb0 con 0x7f01f0167a40 2024-04-26T20:50:57.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.180+0000 7f01f5f05640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f01f00b4220 con 0x7f01f01673a0 2024-04-26T20:50:57.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.181+0000 7f9cb6652640 1 Processor -- start 2024-04-26T20:50:57.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.181+0000 7f9cb6652640 1 -- start start 2024-04-26T20:50:57.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.181+0000 7f01f5704640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f01f01673a0 0x7f01f0163820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.181+0000 7f01f4f03640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f01f014cf00 0x7f01f0073a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.181+0000 7f01f5704640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f01f01673a0 0x7f01f0163820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.73:37736/0 (socket says 172.21.15.73:37736) 2024-04-26T20:50:57.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.181+0000 7f01f5704640 1 -- 172.21.15.73:0/442188170 learned_addr learned my addr 172.21.15.73:0/442188170 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:57.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.181+0000 7f01effff640 1 --2- 172.21.15.73:0/442188170 >> v2:172.21.15.120:3300/0 conn(0x7f01f0167a40 0x7f01f0166e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.182+0000 7f0fc3fff640 1 Processor -- start 2024-04-26T20:50:57.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.182+0000 7f0fc3fff640 1 -- start start 2024-04-26T20:50:57.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.182+0000 7f9cb6652640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f9cb00c41d0 0x7f9cb00cfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.182+0000 7f9cb6652640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9cb01678f0 0x7f9cb00cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.182+0000 7f9cb6652640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9cb0163a50 con 0x7f9cb0073f30 2024-04-26T20:50:57.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.182+0000 7f9cb6652640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f9cb00cf7a0 con 0x7f9cb00c41d0 2024-04-26T20:50:57.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.182+0000 7f9cb6652640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f9cb00ceb90 con 0x7f9cb01678f0 2024-04-26T20:50:57.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.182+0000 7f3d93d0a640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f3d8c0d1700 0x7f3d8c0722f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.182+0000 7f3d93d0a640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3d8c06f600 0x7f3d8c06e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.182+0000 7f3d93d0a640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d8c0d06a0 con 0x7f3d8c0aff60 2024-04-26T20:50:57.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.182+0000 7f3d93d0a640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f3d8c071e60 con 0x7f3d8c06f600 2024-04-26T20:50:57.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.182+0000 7f3d93d0a640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f3d8c073f80 con 0x7f3d8c0d1700 2024-04-26T20:50:57.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.183+0000 7f3d922a7640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3d8c06f600 0x7f3d8c06e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.183+0000 7f9caffff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f9cb00c41d0 0x7f9cb00cfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.183+0000 7f33692ea640 1 Processor -- start 2024-04-26T20:50:57.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.183+0000 7f89b6a4e640 1 Processor -- start 2024-04-26T20:50:57.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.183+0000 7f3d922a7640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3d8c06f600 0x7f3d8c06e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49478/0 (socket says 172.21.15.73:49478) 2024-04-26T20:50:57.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.183+0000 7f3d922a7640 1 -- 172.21.15.73:0/3831402724 learned_addr learned my addr 172.21.15.73:0/3831402724 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:57.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.183+0000 7f3d91aa6640 1 --2- 172.21.15.73:0/3831402724 >> v2:172.21.15.120:3300/0 conn(0x7f3d8c0d1700 0x7f3d8c0722f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.181+0000 7f01f5704640 1 -- 172.21.15.73:0/442188170 >> v2:172.21.15.120:3300/0 conn(0x7f01f0167a40 msgr2=0x7f01f0166e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:57.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.181+0000 7f01f5704640 1 --2- 172.21.15.73:0/442188170 >> v2:172.21.15.120:3300/0 conn(0x7f01f0167a40 0x7f01f0166e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.181+0000 7f01f5704640 1 -- 172.21.15.73:0/442188170 >> v2:172.21.15.73:3300/0 conn(0x7f01f014cf00 msgr2=0x7f01f0073a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:57.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.181+0000 7f01f5704640 1 --2- 172.21.15.73:0/442188170 >> v2:172.21.15.73:3300/0 conn(0x7f01f014cf00 0x7f01f0073a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.181+0000 7f01f5704640 1 -- 172.21.15.73:0/442188170 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01f00b4500 con 0x7f01f01673a0 2024-04-26T20:50:57.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.182+0000 7f01f4f03640 1 --2- 172.21.15.73:0/442188170 >> v2:172.21.15.73:3300/0 conn(0x7f01f014cf00 0x7f01f0073a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:50:57.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.183+0000 7f33692ea640 1 -- start start 2024-04-26T20:50:57.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.183+0000 7f89b6a4e640 1 -- start start 2024-04-26T20:50:57.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.184+0000 7f01f5704640 1 --2- 172.21.15.73:0/442188170 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f01f01673a0 0x7f01f0163820 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f01f00d0a40 tx=0x7f01e803b6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25dd4a03aed7d9a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:57.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.183+0000 7f0fc3fff640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0fbc005e80 0x7f0fbc0062c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.183+0000 7f0fc3fff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f0fbc006800 0x7f0fbc006c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.183+0000 7f0fc3fff640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fbc053710 con 0x7f0fbc0eacf0 2024-04-26T20:50:57.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.183+0000 7f0fc3fff640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f0fbc054320 con 0x7f0fbc006800 2024-04-26T20:50:57.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.183+0000 7f0fc3fff640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f0fbc0539f0 con 0x7f0fbc005e80 2024-04-26T20:50:57.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.184+0000 7f0fc2ffd640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f0fbc0eacf0 0x7f0fbc04fa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.73:37782/0 (socket says 172.21.15.73:37782) 2024-04-26T20:50:57.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.184+0000 7f0fc2ffd640 1 -- 172.21.15.73:0/886343660 learned_addr learned my addr 172.21.15.73:0/886343660 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:50:57.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.184+0000 7f0fc27fc640 1 --2- 172.21.15.73:0/886343660 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0fbc005e80 0x7f0fbc0062c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.184+0000 7f0fc37fe640 1 --2- 172.21.15.73:0/886343660 >> v2:172.21.15.120:3300/0 conn(0x7f0fbc006800 0x7f0fbc006c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.184+0000 7f33692ea640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f336415f5a0 0x7f33640d5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.184+0000 7f33692ea640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f3364163610 0x7f33640cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.184+0000 7f33692ea640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33640d1a70 con 0x7f33640d2830 2024-04-26T20:50:57.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.184+0000 7f89b6a4e640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f89b00b4180 0x7f89b006dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.184+0000 7f89b6a4e640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f89b0164650 0x7f89b006e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.184+0000 7f89b6a4e640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89b01638f0 con 0x7f89b00d11e0 2024-04-26T20:50:57.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.184+0000 7f89b6a4e640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f89b006e920 con 0x7f89b0164650 2024-04-26T20:50:57.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.183+0000 7f3d922a7640 1 -- 172.21.15.73:0/3831402724 >> v2:172.21.15.120:3300/0 conn(0x7f3d8c0d1700 msgr2=0x7f3d8c0722f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:57.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.183+0000 7f3d922a7640 1 --2- 172.21.15.73:0/3831402724 >> v2:172.21.15.120:3300/0 conn(0x7f3d8c0d1700 0x7f3d8c0722f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.183+0000 7f3d922a7640 1 -- 172.21.15.73:0/3831402724 >> v1:172.21.15.73:6789/0 conn(0x7f3d8c0aff60 legacy=0x7f3d8c06ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.183+0000 7f3d922a7640 1 -- 172.21.15.73:0/3831402724 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d8c06fa10 con 0x7f3d8c06f600 2024-04-26T20:50:57.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.184+0000 7f89b6a4e640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f89b00d15c0 con 0x7f89b00b4180 2024-04-26T20:50:57.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.183+0000 7f9cb4bef640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9cb01678f0 0x7f9cb00cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.183+0000 7f9cb4bef640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9cb01678f0 0x7f9cb00cc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.73:37744/0 (socket says 172.21.15.73:37744) 2024-04-26T20:50:57.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.183+0000 7f9cb4bef640 1 -- 172.21.15.73:0/139567912 learned_addr learned my addr 172.21.15.73:0/139567912 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:57.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.183+0000 7f9cb4bef640 1 -- 172.21.15.73:0/139567912 >> v2:172.21.15.120:3300/0 conn(0x7f9cb00c41d0 msgr2=0x7f9cb00cfc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:57.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.183+0000 7f9cb4bef640 1 --2- 172.21.15.73:0/139567912 >> v2:172.21.15.120:3300/0 conn(0x7f9cb00c41d0 0x7f9cb00cfc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.183+0000 7f9cb4bef640 1 -- 172.21.15.73:0/139567912 >> v1:172.21.15.120:6789/0 conn(0x7f9cb0073f30 legacy=0x7f9cb01671e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:57.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.183+0000 7f9cb4bef640 1 -- 172.21.15.73:0/139567912 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cb00cee70 con 0x7f9cb01678f0 2024-04-26T20:50:57.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.185+0000 7f0fc1ffb640 1 -- 172.21.15.73:0/886343660 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3364645607 0 0) 0x7f0fbc053710 con 0x7f0fbc0eacf0 2024-04-26T20:50:57.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.185+0000 7f0fc1ffb640 1 -- 172.21.15.73:0/886343660 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0fac003520 con 0x7f0fbc0eacf0 2024-04-26T20:50:57.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.185+0000 7f0fc37fe640 1 -- 172.21.15.73:0/886343660 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0fbc005e80 msgr2=0x7f0fbc0062c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:57.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.185+0000 7f9cb4bef640 1 --2- 172.21.15.73:0/139567912 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9cb01678f0 0x7f9cb00cc240 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4009870 tx=0x7f9ca402a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=907f6463f53490ab server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:57.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.185+0000 7f0fc37fe640 1 --2- 172.21.15.73:0/886343660 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0fbc005e80 0x7f0fbc0062c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.185+0000 7f0fc37fe640 1 -- 172.21.15.73:0/886343660 >> v1:172.21.15.120:6789/0 conn(0x7f0fbc0eacf0 legacy=0x7f0fbc04fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.185+0000 7f0fc37fe640 1 -- 172.21.15.73:0/886343660 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fbc053cd0 con 0x7f0fbc006800 2024-04-26T20:50:57.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.184+0000 7f33692ea640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f3364163d30 con 0x7f336415f5a0 2024-04-26T20:50:57.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.184+0000 7f33692ea640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f33640d1d50 con 0x7f3364163610 2024-04-26T20:50:57.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.185+0000 7f89b624d640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f89b0164650 0x7f89b006e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.185+0000 7f89b524b640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f89b00b4180 0x7f89b006dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.185+0000 7f89b624d640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f89b0164650 0x7f89b006e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49488/0 (socket says 172.21.15.73:49488) 2024-04-26T20:50:57.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.185+0000 7f89b624d640 1 -- 172.21.15.73:0/3809523663 learned_addr learned my addr 172.21.15.73:0/3809523663 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:57.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.185+0000 7f3d922a7640 1 --2- 172.21.15.73:0/3831402724 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3d8c06f600 0x7f3d8c06e500 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c06c4e0 tx=0x7f3d8802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c09905a9fa3c0b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:57.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.185+0000 7f3362ffd640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f336415f5a0 0x7f33640d5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.185+0000 7f3362ffd640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f336415f5a0 0x7f33640d5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49490/0 (socket says 172.21.15.73:49490) 2024-04-26T20:50:57.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.184+0000 7f01ef7fe640 1 -- 172.21.15.73:0/442188170 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e8047070 con 0x7f01f01673a0 2024-04-26T20:50:57.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.185+0000 7f3362ffd640 1 -- 172.21.15.73:0/1528087399 learned_addr learned my addr 172.21.15.73:0/1528087399 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:57.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.184+0000 7f01ef7fe640 1 -- 172.21.15.73:0/442188170 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01e8042600 con 0x7f01f01673a0 2024-04-26T20:50:57.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.184+0000 7f01ef7fe640 1 -- 172.21.15.73:0/442188170 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e80429d0 con 0x7f01f01673a0 2024-04-26T20:50:57.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.185+0000 7f01f5f05640 1 -- 172.21.15.73:0/442188170 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f01f01673a0 msgr2=0x7f01f0163820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.185+0000 7f01f5f05640 1 --2- 172.21.15.73:0/442188170 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f01f01673a0 0x7f01f0163820 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f01f00d0a40 tx=0x7f01e803b6b0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.185+0000 7f01f5f05640 1 -- 172.21.15.73:0/442188170 shutdown_connections 2024-04-26T20:50:57.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.185+0000 7f01f5f05640 1 --2- 172.21.15.73:0/442188170 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f01f01673a0 0x7f01f0163820 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.185+0000 7f01f5f05640 1 --2- 172.21.15.73:0/442188170 >> v2:172.21.15.120:3300/0 conn(0x7f01f0167a40 0x7f01f0166e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.185+0000 7f01f5f05640 1 --2- 172.21.15.73:0/442188170 >> v2:172.21.15.73:3300/0 conn(0x7f01f014cf00 0x7f01f0073a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.185+0000 7f01f5f05640 1 -- 172.21.15.73:0/442188170 >> 172.21.15.73:0/442188170 conn(0x7f01f00b2720 msgr2=0x7f01f00b2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:57.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.185+0000 7f9caeffd640 1 -- 172.21.15.73:0/139567912 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4037070 con 0x7f9cb01678f0 2024-04-26T20:50:57.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.185+0000 7f3d90aa4640 1 -- 172.21.15.73:0/3831402724 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d88037070 con 0x7f3d8c06f600 2024-04-26T20:50:57.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.185+0000 7f3d90aa4640 1 -- 172.21.15.73:0/3831402724 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d88032440 con 0x7f3d8c06f600 2024-04-26T20:50:57.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.186+0000 7f3d90aa4640 1 -- 172.21.15.73:0/3831402724 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d88032850 con 0x7f3d8c06f600 2024-04-26T20:50:57.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.185+0000 7f9caeffd640 1 -- 172.21.15.73:0/139567912 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ca4032440 con 0x7f9cb01678f0 2024-04-26T20:50:57.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.185+0000 7f9caeffd640 1 -- 172.21.15.73:0/139567912 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca40327f0 con 0x7f9cb01678f0 2024-04-26T20:50:57.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.185+0000 7f89b624d640 1 -- 172.21.15.73:0/3809523663 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f89b00b4180 msgr2=0x7f89b006dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:57.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.185+0000 7f89b624d640 1 --2- 172.21.15.73:0/3809523663 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f89b00b4180 0x7f89b006dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.185+0000 7f89b624d640 1 -- 172.21.15.73:0/3809523663 >> v1:172.21.15.73:6789/0 conn(0x7f89b00d11e0 legacy=0x7f89b00b3a70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T20:50:57.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.185+0000 7f89b624d640 1 -- 172.21.15.73:0/3809523663 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89b006ebd0 con 0x7f89b0164650 2024-04-26T20:50:57.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.186+0000 7f3362ffd640 1 -- 172.21.15.73:0/1528087399 >> v2:172.21.15.120:3300/0 conn(0x7f3364163610 msgr2=0x7f33640cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:57.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.186+0000 7f33637fe640 1 --2- 172.21.15.73:0/1528087399 >> v2:172.21.15.120:3300/0 conn(0x7f3364163610 0x7f33640cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.186+0000 7f3362ffd640 1 --2- 172.21.15.73:0/1528087399 >> v2:172.21.15.120:3300/0 conn(0x7f3364163610 0x7f33640cfda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.186+0000 7f3362ffd640 1 -- 172.21.15.73:0/1528087399 >> v1:172.21.15.120:6789/0 conn(0x7f33640d2830 legacy=0x7f3364162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.186+0000 7f01f5f05640 1 -- 172.21.15.73:0/442188170 shutdown_connections 2024-04-26T20:50:57.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.186+0000 7f3362ffd640 1 -- 172.21.15.73:0/1528087399 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33640d2030 con 0x7f336415f5a0 2024-04-26T20:50:57.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.186+0000 7f33637fe640 1 --2- 172.21.15.73:0/1528087399 >> v2:172.21.15.120:3300/0 conn(0x7f3364163610 0x7f33640cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:50:57.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.186+0000 7f0fc37fe640 1 --2- 172.21.15.73:0/886343660 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0fbc006800 0x7f0fbc006c90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0fb80097a0 tx=0x7f0fb802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc9c6c2a90951aaf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:57.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.186+0000 7f3d93d0a640 1 -- 172.21.15.73:0/3831402724 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3d8c06f600 msgr2=0x7f3d8c06e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.186+0000 7f3d93d0a640 1 --2- 172.21.15.73:0/3831402724 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3d8c06f600 0x7f3d8c06e500 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c06c4e0 tx=0x7f3d8802a2a0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.186+0000 7f01f5f05640 1 -- 172.21.15.73:0/442188170 wait complete. 2024-04-26T20:50:57.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.186+0000 7f9cb6652640 1 -- 172.21.15.73:0/139567912 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9cb01678f0 msgr2=0x7f9cb00cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.186+0000 7f9cb6652640 1 --2- 172.21.15.73:0/139567912 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9cb01678f0 0x7f9cb00cc240 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4009870 tx=0x7f9ca402a230 comp rx=0 tx=0).stop 2024-04-26T20:50:57.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.186+0000 7f9cb6652640 1 -- 172.21.15.73:0/139567912 shutdown_connections 2024-04-26T20:50:57.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.186+0000 7f9cb6652640 1 --2- 172.21.15.73:0/139567912 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9cb01678f0 0x7f9cb00cc240 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.186+0000 7f9cb6652640 1 --2- 172.21.15.73:0/139567912 >> v2:172.21.15.120:3300/0 conn(0x7f9cb00c41d0 0x7f9cb00cfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.186+0000 7f9cb6652640 1 -- 172.21.15.73:0/139567912 >> 172.21.15.73:0/139567912 conn(0x7f9cb00d0be0 msgr2=0x7f9cb00d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:57.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.186+0000 7f0fc1ffb640 1 -- 172.21.15.73:0/886343660 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb8037070 con 0x7f0fbc006800 2024-04-26T20:50:57.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.186+0000 7f0fc1ffb640 1 -- 172.21.15.73:0/886343660 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fb8032440 con 0x7f0fbc006800 2024-04-26T20:50:57.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.186+0000 7f0fc1ffb640 1 -- 172.21.15.73:0/886343660 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb8037070 con 0x7f0fbc006800 2024-04-26T20:50:57.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.186+0000 7fc99710f640 1 Processor -- start 2024-04-26T20:50:57.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.187+0000 7f3d93d0a640 1 -- 172.21.15.73:0/3831402724 shutdown_connections 2024-04-26T20:50:57.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.187+0000 7f3d93d0a640 1 --2- 172.21.15.73:0/3831402724 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3d8c06f600 0x7f3d8c06e500 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.187+0000 7f3d93d0a640 1 --2- 172.21.15.73:0/3831402724 >> v2:172.21.15.120:3300/0 conn(0x7f3d8c0d1700 0x7f3d8c0722f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.187+0000 7f3d93d0a640 1 -- 172.21.15.73:0/3831402724 >> 172.21.15.73:0/3831402724 conn(0x7f3d8c0d0be0 msgr2=0x7f3d8c0d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:57.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.187+0000 7f3d93d0a640 1 -- 172.21.15.73:0/3831402724 shutdown_connections 2024-04-26T20:50:57.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.187+0000 7f9cb6652640 1 -- 172.21.15.73:0/139567912 shutdown_connections 2024-04-26T20:50:57.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.187+0000 7f0fc3fff640 1 -- 172.21.15.73:0/886343660 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0fbc006800 msgr2=0x7f0fbc006c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.187+0000 7f0fc3fff640 1 --2- 172.21.15.73:0/886343660 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0fbc006800 0x7f0fbc006c90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0fb80097a0 tx=0x7f0fb802a180 comp rx=0 tx=0).stop 2024-04-26T20:50:57.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.187+0000 7f3362ffd640 1 --2- 172.21.15.73:0/1528087399 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f336415f5a0 0x7f33640d5cb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f336408e3e0 tx=0x7f335c02a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2e30db485a8ac12 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:57.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.187+0000 7f3361ffb640 1 -- 172.21.15.73:0/1528087399 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335c036070 con 0x7f336415f5a0 2024-04-26T20:50:57.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.187+0000 7f3361ffb640 1 -- 172.21.15.73:0/1528087399 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f335c031600 con 0x7f336415f5a0 2024-04-26T20:50:57.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.187+0000 7fc90bfff640 1 Processor -- start 2024-04-26T20:50:57.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.187+0000 7fc90bfff640 1 -- start start 2024-04-26T20:50:57.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.187+0000 7f3361ffb640 1 -- 172.21.15.73:0/1528087399 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335c0319d0 con 0x7f336415f5a0 2024-04-26T20:50:57.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.188+0000 7f0fc3fff640 1 -- 172.21.15.73:0/886343660 shutdown_connections 2024-04-26T20:50:57.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.188+0000 7f0fc3fff640 1 --2- 172.21.15.73:0/886343660 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0fbc006800 0x7f0fbc006c90 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.188+0000 7f0fc3fff640 1 --2- 172.21.15.73:0/886343660 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0fbc005e80 0x7f0fbc0062c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.188+0000 7f0fc3fff640 1 -- 172.21.15.73:0/886343660 >> 172.21.15.73:0/886343660 conn(0x7f0fbc009fa0 msgr2=0x7f0fbc00a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:57.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.187+0000 7f3d93d0a640 1 -- 172.21.15.73:0/3831402724 wait complete. 2024-04-26T20:50:57.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.187+0000 7fc99710f640 1 -- start start 2024-04-26T20:50:57.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.188+0000 7fc99710f640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc9900c8350 0x7fc99016e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.188+0000 7fc99710f640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9900d3c80 con 0x7fc9900d1d60 2024-04-26T20:50:57.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.188+0000 7f33692ea640 1 -- 172.21.15.73:0/1528087399 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f336415f5a0 msgr2=0x7f33640d5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.188+0000 7f33692ea640 1 --2- 172.21.15.73:0/1528087399 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f336415f5a0 0x7f33640d5cb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f336408e3e0 tx=0x7f335c02a5b0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.188+0000 7fc99710f640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9900d5a90 con 0x7fc9900d1980 2024-04-26T20:50:57.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.188+0000 7fc99710f640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fc9900d5d70 con 0x7fc9900c8350 2024-04-26T20:50:57.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.188+0000 7fc99590c640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fc9900d1d60 0x7fc9900d3570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33096/0 (socket says 172.21.15.73:33096) 2024-04-26T20:50:57.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.188+0000 7fc99590c640 1 -- 172.21.15.73:0/1348629237 learned_addr learned my addr 172.21.15.73:0/1348629237 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:50:57.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.188+0000 7fc99690e640 1 --2- 172.21.15.73:0/1348629237 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc9900c8350 0x7fc99016e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.188+0000 7fc90bfff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fc90c0730a0 0x7fc90c0cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.187+0000 7f9cb6652640 1 -- 172.21.15.73:0/139567912 wait complete. 2024-04-26T20:50:57.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.188+0000 7f9cb6652640 1 Processor -- start 2024-04-26T20:50:57.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.188+0000 7fc90bfff640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc90c071ef0 0x7fc90c0b4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.188+0000 7fc90bfff640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc90c06dde0 con 0x7fc90c15bc70 2024-04-26T20:50:57.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.188+0000 7fc90bfff640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7fc90c06e3a0 con 0x7fc90c071ef0 2024-04-26T20:50:57.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.188+0000 7fc90bfff640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fc90c0b4840 con 0x7fc90c0730a0 2024-04-26T20:50:57.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.189+0000 7fc90b7fe640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc90c071ef0 0x7fc90c0b4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.189+0000 7f9cb6652640 1 -- start start 2024-04-26T20:50:57.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.189+0000 7fc90b7fe640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc90c071ef0 0x7fc90c0b4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49502/0 (socket says 172.21.15.73:49502) 2024-04-26T20:50:57.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.189+0000 7fc90b7fe640 1 -- 172.21.15.73:0/519520648 learned_addr learned my addr 172.21.15.73:0/519520648 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:57.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.189+0000 7fc99690e640 1 -- 172.21.15.73:0/1348629237 >> v1:172.21.15.120:6789/0 conn(0x7fc9900d1980 legacy=0x7fc9900d2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.189+0000 7fc99690e640 1 -- 172.21.15.73:0/1348629237 >> v1:172.21.15.73:6789/0 conn(0x7fc9900d1d60 legacy=0x7fc9900d3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.189+0000 7fc99690e640 1 -- 172.21.15.73:0/1348629237 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9900d0690 con 0x7fc9900c8350 2024-04-26T20:50:57.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.189+0000 7f3d93d0a640 1 Processor -- start 2024-04-26T20:50:57.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.188+0000 7f01f5f05640 1 Processor -- start 2024-04-26T20:50:57.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.188+0000 7f01f5f05640 1 -- start start 2024-04-26T20:50:57.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.189+0000 7f01f5f05640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f01f014cf00 0x7f01f0162eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.189+0000 7f3d93d0a640 1 -- start start 2024-04-26T20:50:57.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.189+0000 7fc90a7fc640 1 --2- 172.21.15.73:0/519520648 >> v2:172.21.15.120:3300/0 conn(0x7fc90c0730a0 0x7fc90c0cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.189+0000 7f01f5f05640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01f01673a0 0x7f01f013bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.187+0000 7f89b624d640 1 --2- 172.21.15.73:0/3809523663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89b0164650 0x7f89b006e3e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f89ac009870 tx=0x7f89ac02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a176b3866a333d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:57.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.187+0000 7f89b4a4a640 1 -- 172.21.15.73:0/3809523663 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ac037070 con 0x7f89b0164650 2024-04-26T20:50:57.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.187+0000 7f89b4a4a640 1 -- 172.21.15.73:0/3809523663 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89ac032440 con 0x7f89b0164650 2024-04-26T20:50:57.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.187+0000 7f89b4a4a640 1 -- 172.21.15.73:0/3809523663 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ac037070 con 0x7f89b0164650 2024-04-26T20:50:57.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.188+0000 7f89b6a4e640 1 -- 172.21.15.73:0/3809523663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89b0164650 msgr2=0x7f89b006e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.188+0000 7f89b6a4e640 1 --2- 172.21.15.73:0/3809523663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89b0164650 0x7f89b006e3e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f89ac009870 tx=0x7f89ac02a210 comp rx=0 tx=0).stop 2024-04-26T20:50:57.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.188+0000 7f89b6a4e640 1 -- 172.21.15.73:0/3809523663 shutdown_connections 2024-04-26T20:50:57.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.188+0000 7f89b6a4e640 1 --2- 172.21.15.73:0/3809523663 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f89b00b4180 0x7f89b006dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.188+0000 7f89b6a4e640 1 --2- 172.21.15.73:0/3809523663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89b0164650 0x7f89b006e3e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.188+0000 7f89b6a4e640 1 -- 172.21.15.73:0/3809523663 >> 172.21.15.73:0/3809523663 conn(0x7f89b0075eb0 msgr2=0x7f89b0068090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:57.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.188+0000 7f89b6a4e640 1 -- 172.21.15.73:0/3809523663 shutdown_connections 2024-04-26T20:50:57.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.188+0000 7f89b6a4e640 1 -- 172.21.15.73:0/3809523663 wait complete. 2024-04-26T20:50:57.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.189+0000 7fc90b7fe640 1 -- 172.21.15.73:0/519520648 >> v2:172.21.15.120:3300/0 conn(0x7fc90c0730a0 msgr2=0x7fc90c0cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:57.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.189+0000 7fc90b7fe640 1 --2- 172.21.15.73:0/519520648 >> v2:172.21.15.120:3300/0 conn(0x7fc90c0730a0 0x7fc90c0cdf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.189+0000 7fc90b7fe640 1 -- 172.21.15.73:0/519520648 >> v1:172.21.15.120:6789/0 conn(0x7fc90c15bc70 legacy=0x7fc90c072990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.189+0000 7fc90b7fe640 1 -- 172.21.15.73:0/519520648 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc90c06e0c0 con 0x7fc90c071ef0 2024-04-26T20:50:57.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.189+0000 7fc90a7fc640 1 --2- 172.21.15.73:0/519520648 >> v2:172.21.15.120:3300/0 conn(0x7fc90c0730a0 0x7fc90c0cdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:50:57.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.189+0000 7f01f4f03640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f01f014cf00 0x7f01f0162eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.189+0000 7f33692ea640 1 -- 172.21.15.73:0/1528087399 shutdown_connections 2024-04-26T20:50:57.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.189+0000 7f33692ea640 1 --2- 172.21.15.73:0/1528087399 >> v2:172.21.15.120:3300/0 conn(0x7f3364163610 0x7f33640cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.189+0000 7f33692ea640 1 --2- 172.21.15.73:0/1528087399 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f336415f5a0 0x7f33640d5cb0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.189+0000 7f0fc3fff640 1 -- 172.21.15.73:0/886343660 shutdown_connections 2024-04-26T20:50:57.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.189+0000 7f33692ea640 1 -- 172.21.15.73:0/1528087399 >> 172.21.15.73:0/1528087399 conn(0x7f33640d5190 msgr2=0x7f33640d55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:57.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.190+0000 7f33692ea640 1 -- 172.21.15.73:0/1528087399 shutdown_connections 2024-04-26T20:50:57.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.189+0000 7f01f4f03640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f01f014cf00 0x7f01f0162eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49514/0 (socket says 172.21.15.73:49514) 2024-04-26T20:50:57.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.190+0000 7f01f4f03640 1 -- 172.21.15.73:0/3048503941 learned_addr learned my addr 172.21.15.73:0/3048503941 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:57.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.190+0000 7f33692ea640 1 -- 172.21.15.73:0/1528087399 wait complete. 2024-04-26T20:50:57.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.190+0000 7fc99690e640 1 --2- 172.21.15.73:0/1348629237 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc9900c8350 0x7fc99016e8e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc9880097d0 tx=0x7fc98802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9091734e384436f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:57.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.190+0000 7fc99510b640 1 -- 172.21.15.73:0/1348629237 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc988037070 con 0x7fc9900c8350 2024-04-26T20:50:57.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.190+0000 7fc99510b640 1 -- 172.21.15.73:0/1348629237 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc988032440 con 0x7fc9900c8350 2024-04-26T20:50:57.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.190+0000 7f3d93d0a640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3d8c06f600 0x7f3d8c0c0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.190+0000 7fc99510b640 1 -- 172.21.15.73:0/1348629237 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc988032810 con 0x7fc9900c8350 2024-04-26T20:50:57.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.190+0000 7f3d93d0a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3d8c0aff60 0x7f3d8c0ca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.190+0000 7f89b6a4e640 1 Processor -- start 2024-04-26T20:50:57.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.190+0000 7f89b6a4e640 1 -- start start 2024-04-26T20:50:57.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.190+0000 7f3d93d0a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3d8c0d1700 0x7f3d8c150950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.191+0000 7f3d93d0a640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8c06ea40 con 0x7f3d8c0aff60 2024-04-26T20:50:57.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.191+0000 7f3d93d0a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8c0d0980 con 0x7f3d8c0d1700 2024-04-26T20:50:57.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.191+0000 7f3d93d0a640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f3d8c06ebc0 con 0x7f3d8c06f600 2024-04-26T20:50:57.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.191+0000 7f3d912a5640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3d8c0aff60 0x7f3d8c0ca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.191+0000 7f3d91aa6640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3d8c06f600 0x7f3d8c0c0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.191+0000 7f3d912a5640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3d8c0aff60 0x7f3d8c0ca5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49516/0 (socket says 172.21.15.73:49516) 2024-04-26T20:50:57.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.190+0000 7f01f5f05640 1 --2- 172.21.15.73:0/3048503941 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f01f0167a40 0x7f01f00b17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.190+0000 7f01f5f05640 1 -- 172.21.15.73:0/3048503941 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f01f00b52c0 con 0x7f01f014cf00 2024-04-26T20:50:57.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.190+0000 7f01f5f05640 1 -- 172.21.15.73:0/3048503941 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f01f00b4ac0 con 0x7f01f01673a0 2024-04-26T20:50:57.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.190+0000 7f01f5f05640 1 -- 172.21.15.73:0/3048503941 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f01f00b3ed0 con 0x7f01f0167a40 2024-04-26T20:50:57.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.190+0000 7f01effff640 1 --2- 172.21.15.73:0/3048503941 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01f01673a0 0x7f01f013bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.190+0000 7f01f4f03640 1 -- 172.21.15.73:0/3048503941 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f01f0167a40 msgr2=0x7f01f00b17c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T20:50:57.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.190+0000 7f01f4f03640 1 --2- 172.21.15.73:0/3048503941 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f01f0167a40 0x7f01f00b17c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.190+0000 7f01f4f03640 1 -- 172.21.15.73:0/3048503941 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01f01673a0 msgr2=0x7f01f013bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.190+0000 7f01f4f03640 1 --2- 172.21.15.73:0/3048503941 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f01f01673a0 0x7f01f013bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.190+0000 7f01f4f03640 1 -- 172.21.15.73:0/3048503941 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01f013eab0 con 0x7f01f014cf00 2024-04-26T20:50:57.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.191+0000 7f01f4f03640 1 --2- 172.21.15.73:0/3048503941 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f01f014cf00 0x7f01f0162eb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f01e400ec10 tx=0x7f01e400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:57.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.191+0000 7f9cb6652640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9cb0073f30 0x7f9cb01512f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.191+0000 7f3d912a5640 1 -- 172.21.15.73:0/3324027480 learned_addr learned my addr 172.21.15.73:0/3324027480 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:57.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.191+0000 7f9cb6652640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9cb00c41d0 0x7f9cb0165110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.191+0000 7f3d922a7640 1 --2- 172.21.15.73:0/3324027480 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3d8c0d1700 0x7f3d8c150950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.191+0000 7f01ed7fa640 1 -- 172.21.15.73:0/3048503941 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e4017070 con 0x7f01f014cf00 2024-04-26T20:50:57.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.191+0000 7f01f5f05640 1 -- 172.21.15.73:0/3048503941 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01f013ed50 con 0x7f01f014cf00 2024-04-26T20:50:57.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.191+0000 7f9cb6652640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9cb01678f0 0x7f9cb00c5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.191+0000 7fc99710f640 1 -- 172.21.15.73:0/1348629237 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc9900c8350 msgr2=0x7fc99016e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.191+0000 7fc99710f640 1 --2- 172.21.15.73:0/1348629237 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc9900c8350 0x7fc99016e8e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc9880097d0 tx=0x7fc98802a210 comp rx=0 tx=0).stop 2024-04-26T20:50:57.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.191+0000 7fc90b7fe640 1 --2- 172.21.15.73:0/519520648 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc90c071ef0 0x7fc90c0b4cd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc90c071670 tx=0x7fc90002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea936b383e7e68f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:57.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.191+0000 7f3d91aa6640 1 -- 172.21.15.73:0/3324027480 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3d8c0d1700 msgr2=0x7f3d8c150950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.191+0000 7f3d91aa6640 1 --2- 172.21.15.73:0/3324027480 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3d8c0d1700 0x7f3d8c150950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.191+0000 7f3d91aa6640 1 -- 172.21.15.73:0/3324027480 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3d8c0aff60 msgr2=0x7f3d8c0ca5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.191+0000 7f3d91aa6640 1 --2- 172.21.15.73:0/3324027480 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3d8c0aff60 0x7f3d8c0ca5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.191+0000 7f3d91aa6640 1 -- 172.21.15.73:0/3324027480 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d8c0c3700 con 0x7f3d8c06f600 2024-04-26T20:50:57.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.192+0000 7f3d91aa6640 1 --2- 172.21.15.73:0/3324027480 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3d8c06f600 0x7f3d8c0c0be0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c00d550 tx=0x7f3d7c00da10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:57.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.192+0000 7f3d827fc640 1 -- 172.21.15.73:0/3324027480 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c0125f0 con 0x7f3d8c06f600 2024-04-26T20:50:57.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.193+0000 7f3d93d0a640 1 -- 172.21.15.73:0/3324027480 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d8c0c39e0 con 0x7f3d8c06f600 2024-04-26T20:50:57.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.193+0000 7f3d93d0a640 1 -- 172.21.15.73:0/3324027480 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d8c0c8d00 con 0x7f3d8c06f600 2024-04-26T20:50:57.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.193+0000 7f3d827fc640 1 -- 172.21.15.73:0/3324027480 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d7c019070 con 0x7f3d8c06f600 2024-04-26T20:50:57.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.193+0000 7f3d827fc640 1 -- 172.21.15.73:0/3324027480 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c0129b0 con 0x7f3d8c06f600 2024-04-26T20:50:57.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.194+0000 7f01f5f05640 1 -- 172.21.15.73:0/3048503941 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01f013f310 con 0x7f01f014cf00 2024-04-26T20:50:57.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.195+0000 7f01ed7fa640 1 -- 172.21.15.73:0/3048503941 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01e400cc30 con 0x7f01f014cf00 2024-04-26T20:50:57.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.191+0000 7f9caffff640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9cb0073f30 0x7f9cb01512f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.191+0000 7f9caffff640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9cb0073f30 0x7f9cb01512f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.73:37814/0 (socket says 172.21.15.73:37814) 2024-04-26T20:50:57.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.191+0000 7f9caf7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9cb00c41d0 0x7f9cb0165110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.191+0000 7f9caf7fe640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9cb00c41d0 0x7f9cb0165110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.73:33948/0 (socket says 172.21.15.73:33948) 2024-04-26T20:50:57.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.193+0000 7f9cb6652640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb00d0170 con 0x7f9cb01678f0 2024-04-26T20:50:57.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.193+0000 7f9cb6652640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb00cf430 con 0x7f9cb00c41d0 2024-04-26T20:50:57.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.193+0000 7f9cb6652640 1 -- 172.21.15.73:0/2870802174 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f9cb00ce7b0 con 0x7f9cb0073f30 2024-04-26T20:50:57.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.193+0000 7f9caffff640 1 -- 172.21.15.73:0/2870802174 learned_addr learned my addr 172.21.15.73:0/2870802174 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:57.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.194+0000 7f9cb4bef640 1 --2- 172.21.15.73:0/2870802174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9cb01678f0 0x7f9cb00c5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.194+0000 7f9caf7fe640 1 -- 172.21.15.73:0/2870802174 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9cb0073f30 msgr2=0x7f9cb01512f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.194+0000 7f9caf7fe640 1 --2- 172.21.15.73:0/2870802174 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9cb0073f30 0x7f9cb01512f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.194+0000 7f9caf7fe640 1 -- 172.21.15.73:0/2870802174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9cb01678f0 msgr2=0x7f9cb00c5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.194+0000 7f9caf7fe640 1 --2- 172.21.15.73:0/2870802174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9cb01678f0 0x7f9cb00c5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.194+0000 7f9caf7fe640 1 -- 172.21.15.73:0/2870802174 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cb00b38e0 con 0x7f9cb00c41d0 2024-04-26T20:50:57.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.194+0000 7f9cb4bef640 1 --2- 172.21.15.73:0/2870802174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9cb01678f0 0x7f9cb00c5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:50:57.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.194+0000 7f9caf7fe640 1 --2- 172.21.15.73:0/2870802174 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9cb00c41d0 0x7f9cb0165110 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9ca000c360 tx=0x7f9ca000c820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:57.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.194+0000 7f9cacff9640 1 -- 172.21.15.73:0/2870802174 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca0017070 con 0x7f9cb00c41d0 2024-04-26T20:50:57.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.194+0000 7f9cb6652640 1 -- 172.21.15.73:0/2870802174 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cb00b3b80 con 0x7f9cb00c41d0 2024-04-26T20:50:57.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.195+0000 7f9cb6652640 1 -- 172.21.15.73:0/2870802174 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cb00b4140 con 0x7f9cb00c41d0 2024-04-26T20:50:57.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.191+0000 7f0fc3fff640 1 -- 172.21.15.73:0/886343660 wait complete. 2024-04-26T20:50:57.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.193+0000 7f0fc3fff640 1 Processor -- start 2024-04-26T20:50:57.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.191+0000 7f89b6a4e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89b00b4180 0x7f89b015d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.191+0000 7f89b6a4e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89b00d11e0 0x7f89b01f5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.192+0000 7f89b6a4e640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f89b0164650 0x7f89b01f5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.192+0000 7f89b6a4e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f89b0164130 con 0x7f89b00d11e0 2024-04-26T20:50:57.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.192+0000 7f89b6a4e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f89b0163e50 con 0x7f89b00b4180 2024-04-26T20:50:57.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.192+0000 7f89b6a4e640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f89b0166ee0 con 0x7f89b0164650 2024-04-26T20:50:57.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.192+0000 7f89b5a4c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89b00b4180 0x7f89b015d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.192+0000 7f89b5a4c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89b00b4180 0x7f89b015d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.73:33958/0 (socket says 172.21.15.73:33958) 2024-04-26T20:50:57.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.192+0000 7f89b5a4c640 1 -- 172.21.15.73:0/3764893031 learned_addr learned my addr 172.21.15.73:0/3764893031 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:57.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.192+0000 7f89b624d640 1 --2- 172.21.15.73:0/3764893031 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f89b0164650 0x7f89b01f5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.192+0000 7f89b524b640 1 --2- 172.21.15.73:0/3764893031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89b00d11e0 0x7f89b01f5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.192+0000 7f89b5a4c640 1 -- 172.21.15.73:0/3764893031 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f89b0164650 msgr2=0x7f89b01f5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.192+0000 7f89b5a4c640 1 --2- 172.21.15.73:0/3764893031 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f89b0164650 0x7f89b01f5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.192+0000 7f89b5a4c640 1 -- 172.21.15.73:0/3764893031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89b00d11e0 msgr2=0x7f89b01f5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.192+0000 7f89b5a4c640 1 --2- 172.21.15.73:0/3764893031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89b00d11e0 0x7f89b01f5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.192+0000 7f89b5a4c640 1 -- 172.21.15.73:0/3764893031 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89b01457b0 con 0x7f89b00b4180 2024-04-26T20:50:57.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.193+0000 7f89b5a4c640 1 --2- 172.21.15.73:0/3764893031 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89b00b4180 0x7f89b015d120 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f89a000b450 tx=0x7f89a000b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:57.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.193+0000 7f89a67fc640 1 -- 172.21.15.73:0/3764893031 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a0015020 con 0x7f89b00b4180 2024-04-26T20:50:57.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.193+0000 7f89a67fc640 1 -- 172.21.15.73:0/3764893031 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89a00118b0 con 0x7f89b00b4180 2024-04-26T20:50:57.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.193+0000 7f89b6a4e640 1 -- 172.21.15.73:0/3764893031 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89b0145a90 con 0x7f89b00b4180 2024-04-26T20:50:57.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.193+0000 7f89a67fc640 1 -- 172.21.15.73:0/3764893031 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a0011ca0 con 0x7f89b00b4180 2024-04-26T20:50:57.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.193+0000 7f89b6a4e640 1 -- 172.21.15.73:0/3764893031 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89b0146050 con 0x7f89b00b4180 2024-04-26T20:50:57.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.195+0000 7f8993fff640 1 -- 172.21.15.73:0/3764893031 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f89b0201c60 con 0x7f89b00b4180 2024-04-26T20:50:57.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.192+0000 7f33692ea640 1 Processor -- start 2024-04-26T20:50:57.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.192+0000 7f33692ea640 1 -- start start 2024-04-26T20:50:57.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.193+0000 7f33692ea640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f33640d2830 0x7f33640b8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.193+0000 7f33692ea640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3364163610 0x7f33641f8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.193+0000 7f33692ea640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33640cdff0 0x7f33640ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.193+0000 7f33692ea640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f33640d3c20 con 0x7f33640cdff0 2024-04-26T20:50:57.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.193+0000 7f33692ea640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f33640d4190 con 0x7f3364163610 2024-04-26T20:50:57.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.193+0000 7f33637fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33640cdff0 0x7f33640ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.193+0000 7f33637fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33640cdff0 0x7f33640ce4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49542/0 (socket says 172.21.15.73:49542) 2024-04-26T20:50:57.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.193+0000 7f33637fe640 1 -- 172.21.15.73:0/3774634518 learned_addr learned my addr 172.21.15.73:0/3774634518 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:57.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.193+0000 7f33692ea640 1 -- 172.21.15.73:0/3774634518 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f33640d3da0 con 0x7f33640d2830 2024-04-26T20:50:57.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.193+0000 7f33627fc640 1 --2- 172.21.15.73:0/3774634518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3364163610 0x7f33641f8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.193+0000 7f3362ffd640 1 --2- 172.21.15.73:0/3774634518 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f33640d2830 0x7f33640b8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.194+0000 7f33627fc640 1 -- 172.21.15.73:0/3774634518 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f33640d2830 msgr2=0x7f33640b8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.194+0000 7f33627fc640 1 --2- 172.21.15.73:0/3774634518 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f33640d2830 0x7f33640b8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.194+0000 7f33627fc640 1 -- 172.21.15.73:0/3774634518 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33640cdff0 msgr2=0x7f33640ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.194+0000 7f33627fc640 1 --2- 172.21.15.73:0/3774634518 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33640cdff0 0x7f33640ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.194+0000 7f33627fc640 1 -- 172.21.15.73:0/3774634518 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33640cbb20 con 0x7f3364163610 2024-04-26T20:50:57.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.191+0000 7fc99710f640 1 -- 172.21.15.73:0/1348629237 shutdown_connections 2024-04-26T20:50:57.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.191+0000 7fc99710f640 1 --2- 172.21.15.73:0/1348629237 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc9900c8350 0x7fc99016e8e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.191+0000 7fc99710f640 1 -- 172.21.15.73:0/1348629237 >> 172.21.15.73:0/1348629237 conn(0x7fc9900d4dd0 msgr2=0x7fc9900d51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:57.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.192+0000 7fc99710f640 1 -- 172.21.15.73:0/1348629237 shutdown_connections 2024-04-26T20:50:57.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.192+0000 7fc99710f640 1 -- 172.21.15.73:0/1348629237 wait complete. 2024-04-26T20:50:57.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.193+0000 7fc99710f640 1 Processor -- start 2024-04-26T20:50:57.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.193+0000 7fc99710f640 1 -- start start 2024-04-26T20:50:57.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.195+0000 7fc99710f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc9900c8350 0x7fc9900ba320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.191+0000 7fc909ffb640 1 -- 172.21.15.73:0/519520648 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc900037070 con 0x7fc90c071ef0 2024-04-26T20:50:57.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.191+0000 7fc909ffb640 1 -- 172.21.15.73:0/519520648 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc900032440 con 0x7fc90c071ef0 2024-04-26T20:50:57.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.191+0000 7fc909ffb640 1 -- 172.21.15.73:0/519520648 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc900032810 con 0x7fc90c071ef0 2024-04-26T20:50:57.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.193+0000 7fc90bfff640 1 -- 172.21.15.73:0/519520648 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc90c071ef0 msgr2=0x7fc90c0b4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.193+0000 7fc90bfff640 1 --2- 172.21.15.73:0/519520648 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc90c071ef0 0x7fc90c0b4cd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc90c071670 tx=0x7fc90002a270 comp rx=0 tx=0).stop 2024-04-26T20:50:57.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.193+0000 7fc90bfff640 1 -- 172.21.15.73:0/519520648 shutdown_connections 2024-04-26T20:50:57.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.193+0000 7fc90bfff640 1 --2- 172.21.15.73:0/519520648 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc90c071ef0 0x7fc90c0b4cd0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.193+0000 7fc90bfff640 1 --2- 172.21.15.73:0/519520648 >> v2:172.21.15.120:3300/0 conn(0x7fc90c0730a0 0x7fc90c0cdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.193+0000 7fc90bfff640 1 -- 172.21.15.73:0/519520648 >> 172.21.15.73:0/519520648 conn(0x7fc90c0b2720 msgr2=0x7fc90c0b2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:57.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.195+0000 7f9cacff9640 1 -- 172.21.15.73:0/2870802174 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ca00035f0 con 0x7f9cb00c41d0 2024-04-26T20:50:57.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.195+0000 7f9cacff9640 1 -- 172.21.15.73:0/2870802174 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca00039c0 con 0x7f9cb00c41d0 2024-04-26T20:50:57.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.195+0000 7fc99710f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9900d1980 0x7fc9900ca950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.195+0000 7fc99710f640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc9900d1d60 0x7fc9900b8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.195+0000 7fc99710f640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc9900b5f10 con 0x7fc9900d1980 2024-04-26T20:50:57.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.195+0000 7fc99710f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc9900b5d90 con 0x7fc9900c8350 2024-04-26T20:50:57.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.195+0000 7fc99710f640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fc9900d0e70 con 0x7fc9900d1d60 2024-04-26T20:50:57.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.195+0000 7fc99610d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc9900c8350 0x7fc9900ba320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.195+0000 7fc99610d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc9900c8350 0x7fc9900ba320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.73:33974/0 (socket says 172.21.15.73:33974) 2024-04-26T20:50:57.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.195+0000 7fc99610d640 1 -- 172.21.15.73:0/2983759973 learned_addr learned my addr 172.21.15.73:0/2983759973 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:57.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.195+0000 7fc99690e640 1 --2- 172.21.15.73:0/2983759973 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc9900d1d60 0x7fc9900b8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.195+0000 7fc99590c640 1 --2- 172.21.15.73:0/2983759973 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9900d1980 0x7fc9900ca950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.196+0000 7f33627fc640 1 --2- 172.21.15.73:0/3774634518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3364163610 0x7f33641f8290 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f335400c780 tx=0x7f335400cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:57.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.196+0000 7f01ed7fa640 1 -- 172.21.15.73:0/3048503941 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e4013470 con 0x7f01f014cf00 2024-04-26T20:50:57.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.196+0000 7f01ed7fa640 1 -- 172.21.15.73:0/3048503941 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f01e401a070 con 0x7f01f014cf00 2024-04-26T20:50:57.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.196+0000 7fc99590c640 1 -- 172.21.15.73:0/2983759973 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc9900d1d60 msgr2=0x7fc9900b8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.196+0000 7fc99590c640 1 --2- 172.21.15.73:0/2983759973 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc9900d1d60 0x7fc9900b8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.196+0000 7fc99590c640 1 -- 172.21.15.73:0/2983759973 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc9900c8350 msgr2=0x7fc9900ba320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.196+0000 7fc99590c640 1 --2- 172.21.15.73:0/2983759973 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc9900c8350 0x7fc9900ba320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.196+0000 7fc99590c640 1 -- 172.21.15.73:0/2983759973 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9900cc950 con 0x7fc9900d1980 2024-04-26T20:50:57.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.196+0000 7fc90bfff640 1 -- 172.21.15.73:0/519520648 shutdown_connections 2024-04-26T20:50:57.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.196+0000 7f89a67fc640 1 -- 172.21.15.73:0/3764893031 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f89a0018020 con 0x7f89b00b4180 2024-04-26T20:50:57.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.196+0000 7f89a67fc640 1 --2- 172.21.15.73:0/3764893031 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f899406c8f0 0x7f899406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.196+0000 7f334b7fe640 1 -- 172.21.15.73:0/3774634518 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3354017070 con 0x7f3364163610 2024-04-26T20:50:57.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.196+0000 7f334b7fe640 1 -- 172.21.15.73:0/3774634518 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3354005290 con 0x7f3364163610 2024-04-26T20:50:57.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.196+0000 7f89a67fc640 1 -- 172.21.15.73:0/3764893031 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f89a0057b80 con 0x7f89b00b4180 2024-04-26T20:50:57.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.196+0000 7f9cacff9640 1 -- 172.21.15.73:0/2870802174 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9ca001a070 con 0x7f9cb00c41d0 2024-04-26T20:50:57.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.196+0000 7f01ed7fa640 1 --2- 172.21.15.73:0/3048503941 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f01dc06ca20 0x7f01dc06eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.196+0000 7f89a67fc640 1 --2- 172.21.15.73:0/3764893031 >> [v2:172.21.15.120:6817/3591055561,v1:172.21.15.120:6819/3591055561] conn(0x7f8994076c40 0x7f89940790e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.197+0000 7f89a67fc640 1 -- 172.21.15.73:0/3764893031 --> [v2:172.21.15.120:6817/3591055561,v1:172.21.15.120:6819/3591055561] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f89940797b0 con 0x7f8994076c40 2024-04-26T20:50:57.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.195+0000 7f0fc3fff640 1 -- start start 2024-04-26T20:50:57.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.197+0000 7f0fc3fff640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0fbc005e80 0x7f0fbc188850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.197+0000 7f334b7fe640 1 -- 172.21.15.73:0/3774634518 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335400a880 con 0x7f3364163610 2024-04-26T20:50:57.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.197+0000 7f33692ea640 1 -- 172.21.15.73:0/3774634518 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33641f87d0 con 0x7f3364163610 2024-04-26T20:50:57.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.197+0000 7f33692ea640 1 -- 172.21.15.73:0/3774634518 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33640b9810 con 0x7f3364163610 2024-04-26T20:50:57.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.197+0000 7f89b524b640 1 --2- 172.21.15.73:0/3764893031 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f899406c8f0 0x7f899406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.197+0000 7f0fc3fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0fbc006800 0x7f0fbc0364b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.197+0000 7f0fc3fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0fbc0eacf0 0x7f0fbc0369f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.197+0000 7f0fc3fff640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0fbc0547b0 con 0x7f0fbc006800 2024-04-26T20:50:57.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.197+0000 7f0fc3fff640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0fbc050190 con 0x7f0fbc0eacf0 2024-04-26T20:50:57.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.197+0000 7f0fc3fff640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f0fbc054930 con 0x7f0fbc005e80 2024-04-26T20:50:57.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.197+0000 7f9cb6652640 1 -- 172.21.15.73:0/2870802174 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9c90001030 con 0x7f9cb00c41d0 2024-04-26T20:50:57.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.195+0000 7f3d827fc640 1 -- 172.21.15.73:0/3324027480 <== mon.2 v2:172.21.15.120:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3d7c012b50 con 0x7f3d8c06f600 2024-04-26T20:50:57.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.195+0000 7f3d93d0a640 1 -- 172.21.15.73:0/3324027480 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3d74001030 con 0x7f3d8c06f600 2024-04-26T20:50:57.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.197+0000 7fc99590c640 1 --2- 172.21.15.73:0/2983759973 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9900d1980 0x7fc9900ca950 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc98400caa0 tx=0x7fc98400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:57.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.197+0000 7f9cacff9640 1 --2- 172.21.15.73:0/2870802174 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f9c9c06c9c0 0x7f9c9c06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.197+0000 7fc90bfff640 1 -- 172.21.15.73:0/519520648 wait complete. 2024-04-26T20:50:57.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.195+0000 7f3d827fc640 1 --2- 172.21.15.73:0/3324027480 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f3d7006c9c0 0x7f3d7006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.196+0000 7f3d912a5640 1 --2- 172.21.15.73:0/3324027480 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f3d7006c9c0 0x7f3d7006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.196+0000 7f3d827fc640 1 -- 172.21.15.73:0/3324027480 <== mon.2 v2:172.21.15.120:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f3d7c08c040 con 0x7f3d8c06f600 2024-04-26T20:50:57.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.196+0000 7f3d827fc640 1 --2- 172.21.15.73:0/3324027480 >> [v2:172.21.15.73:6807/3461705124,v1:172.21.15.73:6809/3461705124] conn(0x7f3d70076130 0x7f3d70078570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.196+0000 7f3d827fc640 1 -- 172.21.15.73:0/3324027480 --> [v2:172.21.15.73:6807/3461705124,v1:172.21.15.73:6809/3461705124] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3d7c001d30 con 0x7f3d70076130 2024-04-26T20:50:57.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.196+0000 7f3d912a5640 1 --2- 172.21.15.73:0/3324027480 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f3d7006c9c0 0x7f3d7006ee80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3d84001fe0 tx=0x7f3d84009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:57.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.197+0000 7fc982ffd640 1 -- 172.21.15.73:0/2983759973 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc984017070 con 0x7fc9900d1980 2024-04-26T20:50:57.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.196+0000 7f3d827fc640 1 -- 172.21.15.73:0/3324027480 <== mon.2 v2:172.21.15.120:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3d7c08f5d0 con 0x7f3d8c06f600 2024-04-26T20:50:57.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.197+0000 7f3d922a7640 1 --2- 172.21.15.73:0/3324027480 >> [v2:172.21.15.73:6807/3461705124,v1:172.21.15.73:6809/3461705124] conn(0x7f3d70076130 0x7f3d70078570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.197+0000 7fc982ffd640 1 -- 172.21.15.73:0/2983759973 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc98400ab40 con 0x7fc9900d1980 2024-04-26T20:50:57.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.197+0000 7fc982ffd640 1 -- 172.21.15.73:0/2983759973 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc984012400 con 0x7fc9900d1980 2024-04-26T20:50:57.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.197+0000 7f3d922a7640 1 --2- 172.21.15.73:0/3324027480 >> [v2:172.21.15.73:6807/3461705124,v1:172.21.15.73:6809/3461705124] conn(0x7f3d70076130 0x7f3d70078570 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:57.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.197+0000 7f9cacff9640 1 -- 172.21.15.73:0/2870802174 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f9ca008b010 con 0x7f9cb00c41d0 2024-04-26T20:50:57.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.197+0000 7f9cacff9640 1 --2- 172.21.15.73:0/2870802174 >> [v2:172.21.15.73:6800/2542681064,v1:172.21.15.73:6801/2542681064] conn(0x7f9c9c0760c0 0x7f9c9c078500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.197+0000 7f9cacff9640 1 -- 172.21.15.73:0/2870802174 --> [v2:172.21.15.73:6800/2542681064,v1:172.21.15.73:6801/2542681064] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9ca001eec0 con 0x7f9c9c0760c0 2024-04-26T20:50:57.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.197+0000 7f9cacff9640 1 -- 172.21.15.73:0/2870802174 <== mon.1 v2:172.21.15.120:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9ca0054e00 con 0x7f9cb00c41d0 2024-04-26T20:50:57.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.197+0000 7f01d6ffd640 1 -- 172.21.15.73:0/3048503941 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f01f0069370 con 0x7f01f014cf00 2024-04-26T20:50:57.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.197+0000 7f01effff640 1 --2- 172.21.15.73:0/3048503941 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f01dc06ca20 0x7f01dc06eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.198+0000 7f01ed7fa640 1 -- 172.21.15.73:0/3048503941 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f01e4090060 con 0x7f01f014cf00 2024-04-26T20:50:57.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.198+0000 7f9cb4bef640 1 --2- 172.21.15.73:0/2870802174 >> [v2:172.21.15.73:6800/2542681064,v1:172.21.15.73:6801/2542681064] conn(0x7f9c9c0760c0 0x7f9c9c078500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.198+0000 7f01ed7fa640 1 --2- 172.21.15.73:0/3048503941 >> [v2:172.21.15.73:6816/2827619403,v1:172.21.15.73:6817/2827619403] conn(0x7f01dc076120 0x7f01dc078560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.198+0000 7f0fc2ffd640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0fbc005e80 0x7f0fbc188850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.198+0000 7f33692ea640 1 -- 172.21.15.73:0/3774634518 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f33640694f0 con 0x7f3364163610 2024-04-26T20:50:57.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.198+0000 7f0fc2ffd640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0fbc005e80 0x7f0fbc188850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.73:37834/0 (socket says 172.21.15.73:37834) 2024-04-26T20:50:57.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.198+0000 7f0fc2ffd640 1 -- 172.21.15.73:0/73486179 learned_addr learned my addr 172.21.15.73:0/73486179 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:57.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.198+0000 7f0fc37fe640 1 --2- 172.21.15.73:0/73486179 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0fbc0eacf0 0x7f0fbc0369f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.197+0000 7f89a67fc640 1 -- 172.21.15.73:0/3764893031 <== mon.1 v2:172.21.15.120:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f89a0056510 con 0x7f89b00b4180 2024-04-26T20:50:57.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.197+0000 7f89b624d640 1 --2- 172.21.15.73:0/3764893031 >> [v2:172.21.15.120:6817/3591055561,v1:172.21.15.120:6819/3591055561] conn(0x7f8994076c40 0x7f89940790e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.198+0000 7f0fc2ffd640 1 -- 172.21.15.73:0/73486179 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0fbc0eacf0 msgr2=0x7f0fbc0369f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.198+0000 7f0fc2ffd640 1 --2- 172.21.15.73:0/73486179 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0fbc0eacf0 0x7f0fbc0369f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.198+0000 7f0fc2ffd640 1 -- 172.21.15.73:0/73486179 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0fbc006800 msgr2=0x7f0fbc0364b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.198+0000 7f0fc2ffd640 1 --2- 172.21.15.73:0/73486179 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0fbc006800 0x7f0fbc0364b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.198+0000 7f0fc2ffd640 1 -- 172.21.15.73:0/73486179 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fbc0e3770 con 0x7f0fbc005e80 2024-04-26T20:50:57.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.198+0000 7fc99710f640 1 -- 172.21.15.73:0/2983759973 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9900ccc30 con 0x7fc9900d1980 2024-04-26T20:50:57.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.198+0000 7fc99710f640 1 -- 172.21.15.73:0/2983759973 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9900cd1f0 con 0x7fc9900d1980 2024-04-26T20:50:57.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.198+0000 7f01f5704640 1 --2- 172.21.15.73:0/3048503941 >> [v2:172.21.15.73:6816/2827619403,v1:172.21.15.73:6817/2827619403] conn(0x7f01dc076120 0x7f01dc078560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.199+0000 7f01ed7fa640 1 -- 172.21.15.73:0/3048503941 --> [v2:172.21.15.73:6816/2827619403,v1:172.21.15.73:6817/2827619403] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f01e400cc30 con 0x7f01dc076120 2024-04-26T20:50:57.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.199+0000 7f3d827fc640 1 -- 172.21.15.73:0/3324027480 <== osd.0 v2:172.21.15.73:6807/3461705124 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3d7c001d30 con 0x7f3d70076130 2024-04-26T20:50:57.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.199+0000 7f01ed7fa640 1 -- 172.21.15.73:0/3048503941 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f01e4055490 con 0x7f01f014cf00 2024-04-26T20:50:57.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.199+0000 7f01effff640 1 --2- 172.21.15.73:0/3048503941 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f01dc06ca20 0x7f01dc06eee0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f01f01361e0 tx=0x7f01e0009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:57.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.199+0000 7f0fc2ffd640 1 --2- 172.21.15.73:0/73486179 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0fbc005e80 0x7f0fbc188850 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0fc402e060 tx=0x7f0fc4075270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:57.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.199+0000 7f89b624d640 1 --2- 172.21.15.73:0/3764893031 >> [v2:172.21.15.120:6817/3591055561,v1:172.21.15.120:6819/3591055561] conn(0x7f8994076c40 0x7f89940790e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:57.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.199+0000 7fc99710f640 1 -- 172.21.15.73:0/2983759973 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc978001030 con 0x7fc9900d1980 2024-04-26T20:50:57.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.199+0000 7f89b524b640 1 --2- 172.21.15.73:0/3764893031 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f899406c8f0 0x7f899406edb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f89b015e9b0 tx=0x7f89a8009830 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:57.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.199+0000 7f01f5704640 1 --2- 172.21.15.73:0/3048503941 >> [v2:172.21.15.73:6816/2827619403,v1:172.21.15.73:6817/2827619403] conn(0x7f01dc076120 0x7f01dc078560 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:57.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.199+0000 7f0fab7fe640 1 -- 172.21.15.73:0/73486179 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc4072460 con 0x7f0fbc005e80 2024-04-26T20:50:57.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.199+0000 7f9caffff640 1 --2- 172.21.15.73:0/2870802174 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f9c9c06c9c0 0x7f9c9c06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.199+0000 7f0fc3fff640 1 -- 172.21.15.73:0/73486179 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fbc0e3a10 con 0x7f0fbc005e80 2024-04-26T20:50:57.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.199+0000 7f0fc3fff640 1 -- 172.21.15.73:0/73486179 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fbc0e3fd0 con 0x7f0fbc005e80 2024-04-26T20:50:57.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.199+0000 7f89a67fc640 1 -- 172.21.15.73:0/3764893031 <== osd.4 v2:172.21.15.120:6817/3591055561 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f89940797b0 con 0x7f8994076c40 2024-04-26T20:50:57.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.200+0000 7f0fab7fe640 1 -- 172.21.15.73:0/73486179 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fc4072600 con 0x7f0fbc005e80 2024-04-26T20:50:57.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.200+0000 7fc90bfff640 1 Processor -- start 2024-04-26T20:50:57.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.200+0000 7fc90bfff640 1 -- start start 2024-04-26T20:50:57.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.200+0000 7f9caffff640 1 --2- 172.21.15.73:0/2870802174 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f9c9c06c9c0 0x7f9c9c06ee80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9ca800f5f0 tx=0x7f9ca8007870 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:57.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.200+0000 7f334b7fe640 1 -- 172.21.15.73:0/3774634518 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f335401a070 con 0x7f3364163610 2024-04-26T20:50:57.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.200+0000 7f01ed7fa640 1 -- 172.21.15.73:0/3048503941 <== osd.1 v2:172.21.15.73:6816/2827619403 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f01e400cc30 con 0x7f01dc076120 2024-04-26T20:50:57.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.200+0000 7f9cb4bef640 1 --2- 172.21.15.73:0/2870802174 >> [v2:172.21.15.73:6800/2542681064,v1:172.21.15.73:6801/2542681064] conn(0x7f9c9c0760c0 0x7f9c9c078500 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:57.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.200+0000 7f0fab7fe640 1 -- 172.21.15.73:0/73486179 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc4072a10 con 0x7f0fbc005e80 2024-04-26T20:50:57.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.200+0000 7fc982ffd640 1 -- 172.21.15.73:0/2983759973 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc98401a070 con 0x7fc9900d1980 2024-04-26T20:50:57.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.201+0000 7f334b7fe640 1 --2- 172.21.15.73:0/3774634518 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f335006c9f0 0x7f335006eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.201+0000 7fc982ffd640 1 --2- 172.21.15.73:0/2983759973 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc97406c9f0 0x7fc97406eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.202+0000 7f9cacff9640 1 -- 172.21.15.73:0/2870802174 <== osd.2 v2:172.21.15.73:6800/2542681064 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9ca001eec0 con 0x7f9c9c0760c0 2024-04-26T20:50:57.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.202+0000 7f0fab7fe640 1 -- 172.21.15.73:0/73486179 <== mon.2 v2:172.21.15.120:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0fc406f030 con 0x7f0fbc005e80 2024-04-26T20:50:57.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.201+0000 7f3362ffd640 1 --2- 172.21.15.73:0/3774634518 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f335006c9f0 0x7f335006eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.202+0000 7f3362ffd640 1 --2- 172.21.15.73:0/3774634518 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f335006c9f0 0x7f335006eeb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f335c033000 tx=0x7f335c009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:57.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.202+0000 7f334b7fe640 1 -- 172.21.15.73:0/3774634518 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f335408b3b0 con 0x7f3364163610 2024-04-26T20:50:57.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.201+0000 7fc982ffd640 1 -- 172.21.15.73:0/2983759973 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc98408c260 con 0x7fc9900d1980 2024-04-26T20:50:57.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.201+0000 7fc982ffd640 1 --2- 172.21.15.73:0/2983759973 >> [v2:172.21.15.120:6800/3946210727,v1:172.21.15.120:6801/3946210727] conn(0x7fc974076160 0x7fc9740785a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.201+0000 7fc982ffd640 1 -- 172.21.15.73:0/2983759973 --> [v2:172.21.15.120:6800/3946210727,v1:172.21.15.120:6801/3946210727] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc984016ec0 con 0x7fc974076160 2024-04-26T20:50:57.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.201+0000 7fc982ffd640 1 -- 172.21.15.73:0/2983759973 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc98408c650 con 0x7fc9900d1980 2024-04-26T20:50:57.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.202+0000 7fc99690e640 1 --2- 172.21.15.73:0/2983759973 >> [v2:172.21.15.120:6800/3946210727,v1:172.21.15.120:6801/3946210727] conn(0x7fc974076160 0x7fc9740785a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.201+0000 7fc90bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc90c071ef0 0x7fc90c15d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.201+0000 7fc90bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc90c0730a0 0x7fc90c144f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.201+0000 7fc90bfff640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc90c15bc70 0x7fc90c0cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.201+0000 7fc90bfff640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc90c15c050 con 0x7fc90c0730a0 2024-04-26T20:50:57.256 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.201+0000 7fc90bfff640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc90c06fad0 con 0x7fc90c071ef0 2024-04-26T20:50:57.256 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.201+0000 7fc90bfff640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fc90c15c1d0 con 0x7fc90c15bc70 2024-04-26T20:50:57.256 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.201+0000 7fc90b7fe640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc90c15bc70 0x7fc90c0cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.256 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.201+0000 7fc90b7fe640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc90c15bc70 0x7fc90c0cf2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.73:37840/0 (socket says 172.21.15.73:37840) 2024-04-26T20:50:57.256 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.201+0000 7fc90b7fe640 1 -- 172.21.15.73:0/660037484 learned_addr learned my addr 172.21.15.73:0/660037484 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:57.256 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.202+0000 7fc90a7fc640 1 --2- 172.21.15.73:0/660037484 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc90c0730a0 0x7fc90c144f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.256 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.202+0000 7f0fab7fe640 1 --2- 172.21.15.73:0/73486179 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f0fb006c9c0 0x7f0fb006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.256 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.202+0000 7f0fab7fe640 1 -- 172.21.15.73:0/73486179 <== mon.2 v2:172.21.15.120:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f0fc41087c0 con 0x7f0fbc005e80 2024-04-26T20:50:57.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.202+0000 7fc90a7fc640 1 -- 172.21.15.73:0/660037484 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc90c15bc70 msgr2=0x7fc90c0cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.203+0000 7fc90a7fc640 1 --2- 172.21.15.73:0/660037484 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc90c15bc70 0x7fc90c0cf2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.203+0000 7fc90a7fc640 1 -- 172.21.15.73:0/660037484 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc90c071ef0 msgr2=0x7fc90c15d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.203+0000 7fc90a7fc640 1 --2- 172.21.15.73:0/660037484 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc90c071ef0 0x7fc90c15d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.203+0000 7fc90a7fc640 1 -- 172.21.15.73:0/660037484 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc90c0b5ce0 con 0x7fc90c0730a0 2024-04-26T20:50:57.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.203+0000 7f334b7fe640 1 --2- 172.21.15.73:0/3774634518 >> [v2:172.21.15.120:6808/819450433,v1:172.21.15.120:6809/819450433] conn(0x7f33500760e0 0x7f3350078520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.203+0000 7f334b7fe640 1 -- 172.21.15.73:0/3774634518 --> [v2:172.21.15.120:6808/819450433,v1:172.21.15.120:6809/819450433] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3354016ec0 con 0x7f33500760e0 2024-04-26T20:50:57.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.203+0000 7f334b7fe640 1 -- 172.21.15.73:0/3774634518 <== mon.1 v2:172.21.15.120:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f335408b7a0 con 0x7f3364163610 2024-04-26T20:50:57.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.203+0000 7f0fc3fff640 1 --2- 172.21.15.73:0/73486179 >> [v2:172.21.15.73:6820/1604042718,v1:172.21.15.73:6822/1604042718] conn(0x7f0fa4001650 0x7f0fa4003b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.203+0000 7f0fc3fff640 1 -- 172.21.15.73:0/73486179 --> [v2:172.21.15.73:6820/1604042718,v1:172.21.15.73:6822/1604042718] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0fa4006bb0 con 0x7f0fa4001650 2024-04-26T20:50:57.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.203+0000 7f0fc27fc640 1 --2- 172.21.15.73:0/73486179 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f0fb006c9c0 0x7f0fb006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.203+0000 7f0fc37fe640 1 --2- 172.21.15.73:0/73486179 >> [v2:172.21.15.73:6820/1604042718,v1:172.21.15.73:6822/1604042718] conn(0x7f0fa4001650 0x7f0fa4003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.203+0000 7f33637fe640 1 --2- 172.21.15.73:0/3774634518 >> [v2:172.21.15.120:6808/819450433,v1:172.21.15.120:6809/819450433] conn(0x7f33500760e0 0x7f3350078520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.202+0000 7fc99610d640 1 --2- 172.21.15.73:0/2983759973 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc97406c9f0 0x7fc97406eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.203+0000 7fc99610d640 1 --2- 172.21.15.73:0/2983759973 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc97406c9f0 0x7fc97406eeb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc98c001700 tx=0x7fc98c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:57.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.203+0000 7f0fc27fc640 1 --2- 172.21.15.73:0/73486179 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f0fb006c9c0 0x7f0fb006ee80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4006a90 tx=0x7f0fb4006a20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:57.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.203+0000 7f0fc37fe640 1 --2- 172.21.15.73:0/73486179 >> [v2:172.21.15.73:6820/1604042718,v1:172.21.15.73:6822/1604042718] conn(0x7f0fa4001650 0x7f0fa4003b10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:57.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.204+0000 7fc90a7fc640 1 --2- 172.21.15.73:0/660037484 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc90c0730a0 0x7fc90c144f30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc904007e10 tx=0x7fc90400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:57.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.204+0000 7fc8f37fe640 1 -- 172.21.15.73:0/660037484 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc904014620 con 0x7fc90c0730a0 2024-04-26T20:50:57.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.204+0000 7fc8f37fe640 1 -- 172.21.15.73:0/660037484 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc904019070 con 0x7fc90c0730a0 2024-04-26T20:50:57.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.204+0000 7fc90bfff640 1 -- 172.21.15.73:0/660037484 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc90c144980 con 0x7fc90c0730a0 2024-04-26T20:50:57.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.204+0000 7fc99690e640 1 --2- 172.21.15.73:0/2983759973 >> [v2:172.21.15.120:6800/3946210727,v1:172.21.15.120:6801/3946210727] conn(0x7fc974076160 0x7fc9740785a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:57.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.204+0000 7f33637fe640 1 --2- 172.21.15.73:0/3774634518 >> [v2:172.21.15.120:6808/819450433,v1:172.21.15.120:6809/819450433] conn(0x7f33500760e0 0x7f3350078520 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:57.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.204+0000 7fc8f37fe640 1 -- 172.21.15.73:0/660037484 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9040149e0 con 0x7fc90c0730a0 2024-04-26T20:50:57.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.204+0000 7fc90bfff640 1 -- 172.21.15.73:0/660037484 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc90c144c20 con 0x7fc90c0730a0 2024-04-26T20:50:57.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.204+0000 7f0fab7fe640 1 -- 172.21.15.73:0/73486179 <== osd.3 v2:172.21.15.73:6820/1604042718 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0fa4006bb0 con 0x7f0fa4001650 2024-04-26T20:50:57.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.205+0000 7fc90bfff640 1 -- 172.21.15.73:0/660037484 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc8ec001030 con 0x7fc90c0730a0 2024-04-26T20:50:57.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.206+0000 7f334b7fe640 1 -- 172.21.15.73:0/3774634518 <== osd.5 v2:172.21.15.120:6808/819450433 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3354016ec0 con 0x7f33500760e0 2024-04-26T20:50:57.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.205+0000 7fc982ffd640 1 -- 172.21.15.73:0/2983759973 <== osd.6 v2:172.21.15.120:6800/3946210727 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc984016ec0 con 0x7fc974076160 2024-04-26T20:50:57.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.206+0000 7fc8f37fe640 1 -- 172.21.15.73:0/660037484 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc904014b80 con 0x7fc90c0730a0 2024-04-26T20:50:57.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.207+0000 7fc8f37fe640 1 --2- 172.21.15.73:0/660037484 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc8f806c8f0 0x7fc8f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.207+0000 7fc8f37fe640 1 -- 172.21.15.73:0/660037484 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc904058b90 con 0x7fc90c0730a0 2024-04-26T20:50:57.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.207+0000 7fc8f37fe640 1 --2- 172.21.15.73:0/660037484 >> [v2:172.21.15.120:6816/2915574674,v1:172.21.15.120:6818/2915574674] conn(0x7fc8f8076bc0 0x7fc8f8079060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.207+0000 7fc8f37fe640 1 -- 172.21.15.73:0/660037484 --> [v2:172.21.15.120:6816/2915574674,v1:172.21.15.120:6818/2915574674] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc8f8079730 con 0x7fc8f8076bc0 2024-04-26T20:50:57.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.207+0000 7fc8f37fe640 1 -- 172.21.15.73:0/660037484 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc904057520 con 0x7fc90c0730a0 2024-04-26T20:50:57.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.208+0000 7fc90affd640 1 --2- 172.21.15.73:0/660037484 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc8f806c8f0 0x7fc8f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.208+0000 7fc90affd640 1 --2- 172.21.15.73:0/660037484 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc8f806c8f0 0x7fc8f806edb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc000f60 tx=0x7fc8fc0079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:57.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.209+0000 7fc90b7fe640 1 --2- 172.21.15.73:0/660037484 >> [v2:172.21.15.120:6816/2915574674,v1:172.21.15.120:6818/2915574674] conn(0x7fc8f8076bc0 0x7fc8f8079060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.209+0000 7fc90b7fe640 1 --2- 172.21.15.73:0/660037484 >> [v2:172.21.15.120:6816/2915574674,v1:172.21.15.120:6818/2915574674] conn(0x7fc8f8076bc0 0x7fc8f8079060 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:57.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.210+0000 7fc8f37fe640 1 -- 172.21.15.73:0/660037484 <== osd.7 v2:172.21.15.120:6816/2915574674 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc8f8079730 con 0x7fc8f8076bc0 2024-04-26T20:50:57.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.228+0000 7f3d93d0a640 1 -- 172.21.15.73:0/3324027480 --> [v2:172.21.15.73:6807/3461705124,v1:172.21.15.73:6809/3461705124] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3d74002dd0 con 0x7f3d70076130 2024-04-26T20:50:57.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.229+0000 7f3d827fc640 1 -- 172.21.15.73:0/3324027480 <== osd.0 v2:172.21.15.73:6807/3461705124 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3d74002dd0 con 0x7f3d70076130 2024-04-26T20:50:57.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.230+0000 7f3d6ffff640 1 -- 172.21.15.73:0/3324027480 >> [v2:172.21.15.73:6807/3461705124,v1:172.21.15.73:6809/3461705124] conn(0x7f3d70076130 msgr2=0x7f3d70078570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.230+0000 7f3d6ffff640 1 --2- 172.21.15.73:0/3324027480 >> [v2:172.21.15.73:6807/3461705124,v1:172.21.15.73:6809/3461705124] conn(0x7f3d70076130 0x7f3d70078570 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.230+0000 7f3d6ffff640 1 -- 172.21.15.73:0/3324027480 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f3d7006c9c0 msgr2=0x7f3d7006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.231+0000 7f3d6ffff640 1 --2- 172.21.15.73:0/3324027480 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f3d7006c9c0 0x7f3d7006ee80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3d84001fe0 tx=0x7f3d84009040 comp rx=0 tx=0).stop 2024-04-26T20:50:57.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.231+0000 7f3d6ffff640 1 -- 172.21.15.73:0/3324027480 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3d8c06f600 msgr2=0x7f3d8c0c0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.231+0000 7f3d6ffff640 1 --2- 172.21.15.73:0/3324027480 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3d8c06f600 0x7f3d8c0c0be0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c00d550 tx=0x7f3d7c00da10 comp rx=0 tx=0).stop 2024-04-26T20:50:57.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.231+0000 7f3d922a7640 1 -- 172.21.15.73:0/3324027480 reap_dead start 2024-04-26T20:50:57.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.231+0000 7f3d6ffff640 1 -- 172.21.15.73:0/3324027480 shutdown_connections 2024-04-26T20:50:57.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.231+0000 7f3d6ffff640 1 -- 172.21.15.73:0/3324027480 >> 172.21.15.73:0/3324027480 conn(0x7f3d8c0d0be0 msgr2=0x7f3d8c0b1920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:57.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.231+0000 7f3d6ffff640 1 -- 172.21.15.73:0/3324027480 shutdown_connections 2024-04-26T20:50:57.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.232+0000 7f3d6ffff640 1 -- 172.21.15.73:0/3324027480 wait complete. 2024-04-26T20:50:57.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.234+0000 7f8993fff640 1 -- 172.21.15.73:0/3764893031 --> [v2:172.21.15.120:6817/3591055561,v1:172.21.15.120:6819/3591055561] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f89b0069370 con 0x7f8994076c40 2024-04-26T20:50:57.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.234+0000 7f01d6ffd640 1 -- 172.21.15.73:0/3048503941 --> [v2:172.21.15.73:6816/2827619403,v1:172.21.15.73:6817/2827619403] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f01f0071f90 con 0x7f01dc076120 2024-04-26T20:50:57.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.234+0000 7f89a67fc640 1 -- 172.21.15.73:0/3764893031 <== osd.4 v2:172.21.15.120:6817/3591055561 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f89b0069370 con 0x7f8994076c40 2024-04-26T20:50:57.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.235+0000 7f01ed7fa640 1 -- 172.21.15.73:0/3048503941 <== osd.1 v2:172.21.15.73:6816/2827619403 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f01f0071f90 con 0x7f01dc076120 2024-04-26T20:50:57.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.235+0000 7f89b6a4e640 1 -- 172.21.15.73:0/3764893031 >> [v2:172.21.15.120:6817/3591055561,v1:172.21.15.120:6819/3591055561] conn(0x7f8994076c40 msgr2=0x7f89940790e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.235+0000 7f0fc3fff640 1 -- 172.21.15.73:0/73486179 --> [v2:172.21.15.73:6820/1604042718,v1:172.21.15.73:6822/1604042718] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0fa4005c30 con 0x7f0fa4001650 2024-04-26T20:50:57.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.235+0000 7f89b6a4e640 1 --2- 172.21.15.73:0/3764893031 >> [v2:172.21.15.120:6817/3591055561,v1:172.21.15.120:6819/3591055561] conn(0x7f8994076c40 0x7f89940790e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.235+0000 7f9cb6652640 1 -- 172.21.15.73:0/2870802174 --> [v2:172.21.15.73:6800/2542681064,v1:172.21.15.73:6801/2542681064] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9c90002dd0 con 0x7f9c9c0760c0 2024-04-26T20:50:57.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.235+0000 7f01f5f05640 1 -- 172.21.15.73:0/3048503941 >> [v2:172.21.15.73:6816/2827619403,v1:172.21.15.73:6817/2827619403] conn(0x7f01dc076120 msgr2=0x7f01dc078560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.235+0000 7f01f5f05640 1 --2- 172.21.15.73:0/3048503941 >> [v2:172.21.15.73:6816/2827619403,v1:172.21.15.73:6817/2827619403] conn(0x7f01dc076120 0x7f01dc078560 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.236+0000 7f0fab7fe640 1 -- 172.21.15.73:0/73486179 <== osd.3 v2:172.21.15.73:6820/1604042718 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0fa4005c30 con 0x7f0fa4001650 2024-04-26T20:50:57.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.236+0000 7f01f5f05640 1 -- 172.21.15.73:0/3048503941 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f01dc06ca20 msgr2=0x7f01dc06eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.236+0000 7f01f5f05640 1 --2- 172.21.15.73:0/3048503941 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f01dc06ca20 0x7f01dc06eee0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f01f01361e0 tx=0x7f01e0009190 comp rx=0 tx=0).stop 2024-04-26T20:50:57.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.236+0000 7f01f5f05640 1 -- 172.21.15.73:0/3048503941 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f01f014cf00 msgr2=0x7f01f0162eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.236+0000 7f01f5f05640 1 --2- 172.21.15.73:0/3048503941 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f01f014cf00 0x7f01f0162eb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f01e400ec10 tx=0x7f01e400c520 comp rx=0 tx=0).stop 2024-04-26T20:50:57.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.236+0000 7f01f5704640 1 -- 172.21.15.73:0/3048503941 reap_dead start 2024-04-26T20:50:57.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.236+0000 7f01f5f05640 1 -- 172.21.15.73:0/3048503941 shutdown_connections 2024-04-26T20:50:57.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.236+0000 7f01f5f05640 1 -- 172.21.15.73:0/3048503941 >> 172.21.15.73:0/3048503941 conn(0x7f01f00b2720 msgr2=0x7f01f006ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:57.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.236+0000 7f89b6a4e640 1 -- 172.21.15.73:0/3764893031 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f899406c8f0 msgr2=0x7f899406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.236+0000 7f89b6a4e640 1 --2- 172.21.15.73:0/3764893031 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f899406c8f0 0x7f899406edb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f89b015e9b0 tx=0x7f89a8009830 comp rx=0 tx=0).stop 2024-04-26T20:50:57.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.236+0000 7f89b6a4e640 1 -- 172.21.15.73:0/3764893031 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89b00b4180 msgr2=0x7f89b015d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.236+0000 7f89b6a4e640 1 --2- 172.21.15.73:0/3764893031 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89b00b4180 0x7f89b015d120 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f89a000b450 tx=0x7f89a000b910 comp rx=0 tx=0).stop 2024-04-26T20:50:57.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.236+0000 7f89b624d640 1 -- 172.21.15.73:0/3764893031 reap_dead start 2024-04-26T20:50:57.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.236+0000 7f89b6a4e640 1 -- 172.21.15.73:0/3764893031 shutdown_connections 2024-04-26T20:50:57.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.236+0000 7f89b6a4e640 1 -- 172.21.15.73:0/3764893031 >> 172.21.15.73:0/3764893031 conn(0x7f89b0075eb0 msgr2=0x7f89b0161f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:57.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.236+0000 7fc99710f640 1 -- 172.21.15.73:0/2983759973 --> [v2:172.21.15.120:6800/3946210727,v1:172.21.15.120:6801/3946210727] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc978002d50 con 0x7fc974076160 2024-04-26T20:50:57.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.236+0000 7f9cacff9640 1 -- 172.21.15.73:0/2870802174 <== osd.2 v2:172.21.15.73:6800/2542681064 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9c90002dd0 con 0x7f9c9c0760c0 2024-04-26T20:50:57.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.236+0000 7f89b6a4e640 1 -- 172.21.15.73:0/3764893031 shutdown_connections 2024-04-26T20:50:57.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.236+0000 7fc982ffd640 1 -- 172.21.15.73:0/2983759973 <== osd.6 v2:172.21.15.120:6800/3946210727 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc978002d50 con 0x7fc974076160 2024-04-26T20:50:57.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.236+0000 7f89b6a4e640 1 -- 172.21.15.73:0/3764893031 wait complete. 2024-04-26T20:50:57.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.237+0000 7f0fa97fa640 1 -- 172.21.15.73:0/73486179 >> [v2:172.21.15.73:6820/1604042718,v1:172.21.15.73:6822/1604042718] conn(0x7f0fa4001650 msgr2=0x7f0fa4003b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.237+0000 7f0fa97fa640 1 --2- 172.21.15.73:0/73486179 >> [v2:172.21.15.73:6820/1604042718,v1:172.21.15.73:6822/1604042718] conn(0x7f0fa4001650 0x7f0fa4003b10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.237+0000 7f01f5f05640 1 -- 172.21.15.73:0/3048503941 shutdown_connections 2024-04-26T20:50:57.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.237+0000 7f01f5f05640 1 -- 172.21.15.73:0/3048503941 wait complete. 2024-04-26T20:50:57.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.237+0000 7f9cb6652640 1 -- 172.21.15.73:0/2870802174 >> [v2:172.21.15.73:6800/2542681064,v1:172.21.15.73:6801/2542681064] conn(0x7f9c9c0760c0 msgr2=0x7f9c9c078500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.237+0000 7f9cb6652640 1 --2- 172.21.15.73:0/2870802174 >> [v2:172.21.15.73:6800/2542681064,v1:172.21.15.73:6801/2542681064] conn(0x7f9c9c0760c0 0x7f9c9c078500 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.237+0000 7f0fa97fa640 1 -- 172.21.15.73:0/73486179 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f0fb006c9c0 msgr2=0x7f0fb006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.237+0000 7f0fa97fa640 1 --2- 172.21.15.73:0/73486179 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f0fb006c9c0 0x7f0fb006ee80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0fb4006a90 tx=0x7f0fb4006a20 comp rx=0 tx=0).stop 2024-04-26T20:50:57.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.237+0000 7f0fa97fa640 1 -- 172.21.15.73:0/73486179 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0fbc005e80 msgr2=0x7f0fbc188850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.237+0000 7f0fa97fa640 1 --2- 172.21.15.73:0/73486179 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0fbc005e80 0x7f0fbc188850 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0fc402e060 tx=0x7f0fc4075270 comp rx=0 tx=0).stop 2024-04-26T20:50:57.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.237+0000 7f9cb6652640 1 -- 172.21.15.73:0/2870802174 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f9c9c06c9c0 msgr2=0x7f9c9c06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.237+0000 7f9cb6652640 1 --2- 172.21.15.73:0/2870802174 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f9c9c06c9c0 0x7f9c9c06ee80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9ca800f5f0 tx=0x7f9ca8007870 comp rx=0 tx=0).stop 2024-04-26T20:50:57.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.237+0000 7fc99710f640 1 -- 172.21.15.73:0/2983759973 >> [v2:172.21.15.120:6800/3946210727,v1:172.21.15.120:6801/3946210727] conn(0x7fc974076160 msgr2=0x7fc9740785a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.237+0000 7fc99710f640 1 --2- 172.21.15.73:0/2983759973 >> [v2:172.21.15.120:6800/3946210727,v1:172.21.15.120:6801/3946210727] conn(0x7fc974076160 0x7fc9740785a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.237+0000 7f9cb6652640 1 -- 172.21.15.73:0/2870802174 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9cb00c41d0 msgr2=0x7f9cb0165110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.237+0000 7f9cb6652640 1 --2- 172.21.15.73:0/2870802174 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9cb00c41d0 0x7f9cb0165110 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9ca000c360 tx=0x7f9ca000c820 comp rx=0 tx=0).stop 2024-04-26T20:50:57.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.237+0000 7f9cb4bef640 1 -- 172.21.15.73:0/2870802174 reap_dead start 2024-04-26T20:50:57.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.237+0000 7f0fc37fe640 1 -- 172.21.15.73:0/73486179 reap_dead start 2024-04-26T20:50:57.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.238+0000 7f9cb6652640 1 -- 172.21.15.73:0/2870802174 shutdown_connections 2024-04-26T20:50:57.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.238+0000 7f9cb6652640 1 -- 172.21.15.73:0/2870802174 >> 172.21.15.73:0/2870802174 conn(0x7f9cb00d0be0 msgr2=0x7f9cb0162340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:57.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.238+0000 7fc99710f640 1 -- 172.21.15.73:0/2983759973 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc97406c9f0 msgr2=0x7fc97406eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.238+0000 7fc99710f640 1 --2- 172.21.15.73:0/2983759973 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc97406c9f0 0x7fc97406eeb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc98c001700 tx=0x7fc98c009040 comp rx=0 tx=0).stop 2024-04-26T20:50:57.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.238+0000 7fc99710f640 1 -- 172.21.15.73:0/2983759973 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9900d1980 msgr2=0x7fc9900ca950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.238+0000 7fc99710f640 1 --2- 172.21.15.73:0/2983759973 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9900d1980 0x7fc9900ca950 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc98400caa0 tx=0x7fc98400cf60 comp rx=0 tx=0).stop 2024-04-26T20:50:57.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.238+0000 7fc99690e640 1 -- 172.21.15.73:0/2983759973 reap_dead start 2024-04-26T20:50:57.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.238+0000 7f9cb6652640 1 -- 172.21.15.73:0/2870802174 shutdown_connections 2024-04-26T20:50:57.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.238+0000 7f9cb6652640 1 -- 172.21.15.73:0/2870802174 wait complete. 2024-04-26T20:50:57.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.238+0000 7fc99710f640 1 -- 172.21.15.73:0/2983759973 shutdown_connections 2024-04-26T20:50:57.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.238+0000 7fc99710f640 1 -- 172.21.15.73:0/2983759973 >> 172.21.15.73:0/2983759973 conn(0x7fc9900d4dd0 msgr2=0x7fc9900d2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:57.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.238+0000 7f0fa97fa640 1 -- 172.21.15.73:0/73486179 shutdown_connections 2024-04-26T20:50:57.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.238+0000 7fc99710f640 1 -- 172.21.15.73:0/2983759973 shutdown_connections 2024-04-26T20:50:57.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.238+0000 7fc99710f640 1 -- 172.21.15.73:0/2983759973 wait complete. 2024-04-26T20:50:57.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.238+0000 7f0fa97fa640 1 -- 172.21.15.73:0/73486179 >> 172.21.15.73:0/73486179 conn(0x7f0fbc009fa0 msgr2=0x7f0fbc049ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:57.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.238+0000 7f33692ea640 1 -- 172.21.15.73:0/3774634518 --> [v2:172.21.15.120:6808/819450433,v1:172.21.15.120:6809/819450433] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3364068d30 con 0x7f33500760e0 2024-04-26T20:50:57.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.239+0000 7f0fa97fa640 1 -- 172.21.15.73:0/73486179 shutdown_connections 2024-04-26T20:50:57.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.239+0000 7f0fa97fa640 1 -- 172.21.15.73:0/73486179 wait complete. 2024-04-26T20:50:57.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.239+0000 7f334b7fe640 1 -- 172.21.15.73:0/3774634518 <== osd.5 v2:172.21.15.120:6808/819450433 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3364068d30 con 0x7f33500760e0 2024-04-26T20:50:57.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.239+0000 7f33692ea640 1 -- 172.21.15.73:0/3774634518 >> [v2:172.21.15.120:6808/819450433,v1:172.21.15.120:6809/819450433] conn(0x7f33500760e0 msgr2=0x7f3350078520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.240+0000 7f33692ea640 1 --2- 172.21.15.73:0/3774634518 >> [v2:172.21.15.120:6808/819450433,v1:172.21.15.120:6809/819450433] conn(0x7f33500760e0 0x7f3350078520 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.240+0000 7f33692ea640 1 -- 172.21.15.73:0/3774634518 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f335006c9f0 msgr2=0x7f335006eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.240+0000 7f33692ea640 1 --2- 172.21.15.73:0/3774634518 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f335006c9f0 0x7f335006eeb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f335c033000 tx=0x7f335c009000 comp rx=0 tx=0).stop 2024-04-26T20:50:57.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.240+0000 7f33692ea640 1 -- 172.21.15.73:0/3774634518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3364163610 msgr2=0x7f33641f8290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.240+0000 7f33692ea640 1 --2- 172.21.15.73:0/3774634518 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3364163610 0x7f33641f8290 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f335400c780 tx=0x7f335400cc40 comp rx=0 tx=0).stop 2024-04-26T20:50:57.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.240+0000 7f33637fe640 1 -- 172.21.15.73:0/3774634518 reap_dead start 2024-04-26T20:50:57.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.240+0000 7f33692ea640 1 -- 172.21.15.73:0/3774634518 shutdown_connections 2024-04-26T20:50:57.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.240+0000 7f33692ea640 1 -- 172.21.15.73:0/3774634518 >> 172.21.15.73:0/3774634518 conn(0x7f33640d5190 msgr2=0x7f336406d970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:57.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.241+0000 7f33692ea640 1 -- 172.21.15.73:0/3774634518 shutdown_connections 2024-04-26T20:50:57.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.241+0000 7f33692ea640 1 -- 172.21.15.73:0/3774634518 wait complete. 2024-04-26T20:50:57.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.241+0000 7fc90bfff640 1 -- 172.21.15.73:0/660037484 --> [v2:172.21.15.120:6816/2915574674,v1:172.21.15.120:6818/2915574674] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc8ec002dd0 con 0x7fc8f8076bc0 2024-04-26T20:50:57.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.242+0000 7fc8f37fe640 1 -- 172.21.15.73:0/660037484 <== osd.7 v2:172.21.15.120:6816/2915574674 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc8ec002dd0 con 0x7fc8f8076bc0 2024-04-26T20:50:57.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.243+0000 7fc8f17fa640 1 -- 172.21.15.73:0/660037484 >> [v2:172.21.15.120:6816/2915574674,v1:172.21.15.120:6818/2915574674] conn(0x7fc8f8076bc0 msgr2=0x7fc8f8079060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.244+0000 7fc8f17fa640 1 --2- 172.21.15.73:0/660037484 >> [v2:172.21.15.120:6816/2915574674,v1:172.21.15.120:6818/2915574674] conn(0x7fc8f8076bc0 0x7fc8f8079060 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.244+0000 7fc8f17fa640 1 -- 172.21.15.73:0/660037484 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc8f806c8f0 msgr2=0x7fc8f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.244+0000 7fc8f17fa640 1 --2- 172.21.15.73:0/660037484 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc8f806c8f0 0x7fc8f806edb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc000f60 tx=0x7fc8fc0079b0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.244+0000 7fc8f17fa640 1 -- 172.21.15.73:0/660037484 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc90c0730a0 msgr2=0x7fc90c144f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.244+0000 7fc8f17fa640 1 --2- 172.21.15.73:0/660037484 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc90c0730a0 0x7fc90c144f30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc904007e10 tx=0x7fc90400e4b0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.244+0000 7fc90b7fe640 1 -- 172.21.15.73:0/660037484 reap_dead start 2024-04-26T20:50:57.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.244+0000 7fc8f17fa640 1 -- 172.21.15.73:0/660037484 shutdown_connections 2024-04-26T20:50:57.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.245+0000 7fc8f17fa640 1 -- 172.21.15.73:0/660037484 >> 172.21.15.73:0/660037484 conn(0x7fc90c0b2720 msgr2=0x7fc90c0b11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:57.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.245+0000 7fc8f17fa640 1 -- 172.21.15.73:0/660037484 shutdown_connections 2024-04-26T20:50:57.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.245+0000 7fc8f17fa640 1 -- 172.21.15.73:0/660037484 wait complete. 2024-04-26T20:50:57.286 INFO:teuthology.orchestra.run.smithi073.stdout:55834574858 2024-04-26T20:50:57.286 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-26T20:50:57.287 INFO:teuthology.orchestra.run.smithi073.stdout:55834574859 2024-04-26T20:50:57.287 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-26T20:50:57.288 INFO:teuthology.orchestra.run.smithi073.stdout:55834574858 2024-04-26T20:50:57.288 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-26T20:50:57.289 INFO:teuthology.orchestra.run.smithi073.stdout:55834574859 2024-04-26T20:50:57.289 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-26T20:50:57.290 INFO:teuthology.orchestra.run.smithi073.stdout:55834574858 2024-04-26T20:50:57.290 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-26T20:50:57.291 INFO:teuthology.orchestra.run.smithi073.stdout:55834574859 2024-04-26T20:50:57.291 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-26T20:50:57.292 INFO:teuthology.orchestra.run.smithi073.stdout:55834574859 2024-04-26T20:50:57.292 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-26T20:50:57.293 INFO:teuthology.orchestra.run.smithi073.stdout:55834574858 2024-04-26T20:50:57.294 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-26T20:50:57.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.492+0000 7f3b30cfd640 1 Processor -- start 2024-04-26T20:50:57.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.492+0000 7f3b30cfd640 1 -- start start 2024-04-26T20:50:57.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.493+0000 7fbefbfff640 1 Processor -- start 2024-04-26T20:50:57.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.493+0000 7fbefbfff640 1 -- start start 2024-04-26T20:50:57.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.493+0000 7f463c686640 1 Processor -- start 2024-04-26T20:50:57.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.493+0000 7f3b30cfd640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3b240e12b0 0x7f3b240534f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.494+0000 7f3b30cfd640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b24054fe0 con 0x7f3b240e4d30 2024-04-26T20:50:57.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.494+0000 7f0e04a06640 1 Processor -- start 2024-04-26T20:50:57.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.494+0000 7f3b2b7fe640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3b240e12b0 0x7f3b240534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.494+0000 7f8f719b3640 1 Processor -- start 2024-04-26T20:50:57.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.494+0000 7f3b2b7fe640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3b240e12b0 0x7f3b240534f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49598/0 (socket says 172.21.15.73:49598) 2024-04-26T20:50:57.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.494+0000 7f3b2b7fe640 1 -- 172.21.15.73:0/1557022561 learned_addr learned my addr 172.21.15.73:0/1557022561 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:57.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.494+0000 7f516ceef640 1 Processor -- start 2024-04-26T20:50:57.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.494+0000 7f516ceef640 1 -- start start 2024-04-26T20:50:57.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.494+0000 7fbefbfff640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbef40e77c0 0x7fbef4006a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.494+0000 7f3b30cfd640 1 -- 172.21.15.73:0/1557022561 --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b240552c0 con 0x7f3b24054b50 2024-04-26T20:50:57.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.494+0000 7f3b30cfd640 1 -- 172.21.15.73:0/1557022561 --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f3b240555a0 con 0x7f3b240e12b0 2024-04-26T20:50:57.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.494+0000 7fbefbfff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fbef40093b0 0x7fbef4006f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.494+0000 7fbefbfff640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbef4048ec0 0x7fbef40eb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.494+0000 7fbefbfff640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7fbef40495d0 con 0x7fbef40e77c0 2024-04-26T20:50:57.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.494+0000 7fbefbfff640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fbef4051350 con 0x7fbef40093b0 2024-04-26T20:50:57.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.494+0000 7fbefbfff640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fbef40517e0 con 0x7fbef4048ec0 2024-04-26T20:50:57.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.494+0000 7f8f719b3640 1 -- start start 2024-04-26T20:50:57.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.494+0000 7f1adbcc8640 1 Processor -- start 2024-04-26T20:50:57.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.494+0000 7f463c686640 1 -- start start 2024-04-26T20:50:57.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.494+0000 7f0e04a06640 1 -- start start 2024-04-26T20:50:57.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.495+0000 7f0e04a06640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f0df80ead00 0x7f0df804fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.495+0000 7f3b2a7fc640 1 -- 172.21.15.73:0/1557022561 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 858894949 0 0) 0x7f3b24054fe0 con 0x7f3b240e4d30 2024-04-26T20:50:57.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.495+0000 7f3b2a7fc640 1 -- 172.21.15.73:0/1557022561 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b1c003520 con 0x7f3b240e4d30 2024-04-26T20:50:57.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.495+0000 7f3b2a7fc640 1 -- 172.21.15.73:0/1557022561 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1133220584 0 0) 0x7f3b240552c0 con 0x7f3b24054b50 2024-04-26T20:50:57.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.495+0000 7f463c686640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46340c41d0 0x7f46340cfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.495+0000 7fbefaffd640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbef40e77c0 0x7fbef4006a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.495+0000 7fbefaffd640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbef40e77c0 0x7fbef4006a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49602/0 (socket says 172.21.15.73:49602) 2024-04-26T20:50:57.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.495+0000 7fbefaffd640 1 -- 172.21.15.73:0/2229244754 learned_addr learned my addr 172.21.15.73:0/2229244754 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:57.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.495+0000 7f0e04a06640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0df804ffd0 0x7f0df8037bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.495+0000 7f0e04a06640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0df8038110 0x7f0df80385a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.495+0000 7f0e04a06640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f0df8054330 con 0x7f0df8038110 2024-04-26T20:50:57.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.495+0000 7f0e04a06640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f0df8053720 con 0x7f0df80ead00 2024-04-26T20:50:57.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.495+0000 7f0e04a06640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f0df8053a00 con 0x7f0df804ffd0 2024-04-26T20:50:57.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.495+0000 7f1adbcc8640 1 -- start start 2024-04-26T20:50:57.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.495+0000 7f516ceef640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f51600e00a0 0x7f5160034fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.495+0000 7f516ceef640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f5160050430 0x7f5160038de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.495+0000 7f516ceef640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5160034d10 con 0x7f5160039320 2024-04-26T20:50:57.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.495+0000 7f516ceef640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f5160007b00 con 0x7f51600e00a0 2024-04-26T20:50:57.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.495+0000 7f516ceef640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f5160050930 con 0x7f5160050430 2024-04-26T20:50:57.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.495+0000 7f8f719b3640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8f640e7820 0x7f8f64038280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.495+0000 7f8f719b3640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8f640eb300 0x7f8f640eb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.495+0000 7f463c686640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f46341678f0 0x7f46340cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.495+0000 7f51677fe640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f51600e00a0 0x7f5160034fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.495+0000 7f463a422640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46340c41d0 0x7f46340cfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.495+0000 7f51677fe640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f51600e00a0 0x7f5160034fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49614/0 (socket says 172.21.15.73:49614) 2024-04-26T20:50:57.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.495+0000 7f51677fe640 1 -- 172.21.15.73:0/3966305613 learned_addr learned my addr 172.21.15.73:0/3966305613 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:57.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.496+0000 7f0dfed9d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f0df80ead00 0x7f0df804fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.496+0000 7f0dfed9d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f0df80ead00 0x7f0df804fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.73:34008/0 (socket says 172.21.15.73:34008) 2024-04-26T20:50:57.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.496+0000 7f0dfed9d640 1 -- 172.21.15.73:0/133388959 learned_addr learned my addr 172.21.15.73:0/133388959 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:57.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.496+0000 7f4639c21640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f4634073f30 0x7f46341671e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.73:37852/0 (socket says 172.21.15.73:37852) 2024-04-26T20:50:57.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.496+0000 7f4639c21640 1 -- 172.21.15.73:0/2239702916 learned_addr learned my addr 172.21.15.73:0/2239702916 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:50:57.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.496+0000 7f0dff59e640 1 --2- 172.21.15.73:0/133388959 >> v2:172.21.15.73:3300/0 conn(0x7f0df8038110 0x7f0df80385a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.496+0000 7f463ac23640 1 --2- 172.21.15.73:0/2239702916 >> v2:172.21.15.120:3300/0 conn(0x7f46341678f0 0x7f46340cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.496+0000 7f463c686640 1 -- 172.21.15.73:0/2239702916 --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4634163a50 con 0x7f4634073f30 2024-04-26T20:50:57.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.496+0000 7f0dfe59c640 1 --2- 172.21.15.73:0/133388959 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0df804ffd0 0x7f0df8037bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.496+0000 7f5166ffd640 1 --2- 172.21.15.73:0/3966305613 >> v2:172.21.15.120:3300/0 conn(0x7f5160050430 0x7f5160038de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.496+0000 7f51677fe640 1 -- 172.21.15.73:0/3966305613 >> v2:172.21.15.120:3300/0 conn(0x7f5160050430 msgr2=0x7f5160038de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:57.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.496+0000 7f51677fe640 1 --2- 172.21.15.73:0/3966305613 >> v2:172.21.15.120:3300/0 conn(0x7f5160050430 0x7f5160038de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.496+0000 7f51677fe640 1 -- 172.21.15.73:0/3966305613 >> v1:172.21.15.120:6789/0 conn(0x7f5160039320 legacy=0x7f5160008af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.496+0000 7f51677fe640 1 -- 172.21.15.73:0/3966305613 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5160007d30 con 0x7f51600e00a0 2024-04-26T20:50:57.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.496+0000 7f463c686640 1 -- 172.21.15.73:0/2239702916 --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f46340cf7a0 con 0x7f46341678f0 2024-04-26T20:50:57.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.496+0000 7f463c686640 1 -- 172.21.15.73:0/2239702916 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f46340ceb90 con 0x7f46340c41d0 2024-04-26T20:50:57.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.495+0000 7f3b2a7fc640 1 -- 172.21.15.73:0/1557022561 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b24054fe0 con 0x7f3b24054b50 2024-04-26T20:50:57.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.495+0000 7f3b2b7fe640 1 -- 172.21.15.73:0/1557022561 >> v1:172.21.15.120:6789/0 conn(0x7f3b24054b50 legacy=0x7f3b240e1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.495+0000 7f3b2b7fe640 1 -- 172.21.15.73:0/1557022561 >> v1:172.21.15.73:6789/0 conn(0x7f3b240e4d30 legacy=0x7f3b240e1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.495+0000 7f3b2b7fe640 1 -- 172.21.15.73:0/1557022561 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b24055880 con 0x7f3b240e12b0 2024-04-26T20:50:57.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.496+0000 7f5166ffd640 1 --2- 172.21.15.73:0/3966305613 >> v2:172.21.15.120:3300/0 conn(0x7f5160050430 0x7f5160038de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:50:57.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.496+0000 7fbefa7fc640 1 --2- 172.21.15.73:0/2229244754 >> v2:172.21.15.120:3300/0 conn(0x7fbef40093b0 0x7fbef4006f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.496+0000 7fbefb7fe640 1 --2- 172.21.15.73:0/2229244754 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbef4048ec0 0x7fbef40eb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.496+0000 7f0dfed9d640 1 -- 172.21.15.73:0/133388959 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0df804ffd0 msgr2=0x7f0df8037bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:57.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.496+0000 7f1adbcc8640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1ad4071110 0x7f1ad4071570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.496+0000 7f1adbcc8640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ad40cfbf0 con 0x7f1ad415bcc0 2024-04-26T20:50:57.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.496+0000 7f1adbcc8640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ad40cfed0 con 0x7f1ad4072f50 2024-04-26T20:50:57.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.496+0000 7f1adbcc8640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f1ad40d0800 con 0x7f1ad4071110 2024-04-26T20:50:57.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.496+0000 7fbefa7fc640 1 -- 172.21.15.73:0/2229244754 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbef4048ec0 msgr2=0x7fbef40eb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:57.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.496+0000 7fbefa7fc640 1 --2- 172.21.15.73:0/2229244754 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbef4048ec0 0x7fbef40eb3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.496+0000 7fbefa7fc640 1 -- 172.21.15.73:0/2229244754 >> v2:172.21.15.73:3300/0 conn(0x7fbef40e77c0 msgr2=0x7fbef4006a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:57.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.496+0000 7fbefa7fc640 1 --2- 172.21.15.73:0/2229244754 >> v2:172.21.15.73:3300/0 conn(0x7fbef40e77c0 0x7fbef4006a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.496+0000 7fbefa7fc640 1 -- 172.21.15.73:0/2229244754 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbef4050740 con 0x7fbef40093b0 2024-04-26T20:50:57.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.496+0000 7f0dfed9d640 1 --2- 172.21.15.73:0/133388959 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0df804ffd0 0x7f0df8037bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.496+0000 7f0dfed9d640 1 -- 172.21.15.73:0/133388959 >> v2:172.21.15.73:3300/0 conn(0x7f0df8038110 msgr2=0x7f0df80385a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:57.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.496+0000 7f0dfed9d640 1 --2- 172.21.15.73:0/133388959 >> v2:172.21.15.73:3300/0 conn(0x7f0df8038110 0x7f0df80385a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.496+0000 7f0dfed9d640 1 -- 172.21.15.73:0/133388959 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0df8053ce0 con 0x7f0df80ead00 2024-04-26T20:50:57.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.497+0000 7f4639420640 1 -- 172.21.15.73:0/2239702916 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1412203265 0 0) 0x7f4634163a50 con 0x7f4634073f30 2024-04-26T20:50:57.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.497+0000 7f4639420640 1 -- 172.21.15.73:0/2239702916 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4620003520 con 0x7f4634073f30 2024-04-26T20:50:57.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.497+0000 7f4639420640 1 -- 172.21.15.73:0/2239702916 <== mon.1 v1:172.21.15.120:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2700630050 0 0) 0x7f4620003520 con 0x7f4634073f30 2024-04-26T20:50:57.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.497+0000 7f4639420640 1 -- 172.21.15.73:0/2239702916 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46340c41d0 msgr2=0x7f46340cfc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:57.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.497+0000 7f4639420640 1 --2- 172.21.15.73:0/2239702916 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46340c41d0 0x7f46340cfc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.497+0000 7f1ad9263640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1ad4071110 0x7f1ad4071570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.497+0000 7f1ad9263640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1ad4071110 0x7f1ad4071570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.73:37894/0 (socket says 172.21.15.73:37894) 2024-04-26T20:50:57.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.497+0000 7f1ad9263640 1 -- 172.21.15.73:0/995864065 learned_addr learned my addr 172.21.15.73:0/995864065 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:57.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.497+0000 7f4639420640 1 -- 172.21.15.73:0/2239702916 >> v2:172.21.15.120:3300/0 conn(0x7f46341678f0 msgr2=0x7f46340cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:57.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.497+0000 7f4639420640 1 --2- 172.21.15.73:0/2239702916 >> v2:172.21.15.120:3300/0 conn(0x7f46341678f0 0x7f46340cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.497+0000 7f4639420640 1 -- 172.21.15.73:0/2239702916 --> v1:172.21.15.120:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46340cee70 con 0x7f4634073f30 2024-04-26T20:50:57.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.497+0000 7f463ac23640 1 --2- 172.21.15.73:0/2239702916 >> v2:172.21.15.120:3300/0 conn(0x7f46341678f0 0x7f46340cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:50:57.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.497+0000 7f4639420640 1 -- 172.21.15.73:0/2239702916 <== mon.1 v1:172.21.15.120:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7f4624001670 con 0x7f4634073f30 2024-04-26T20:50:57.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.496+0000 7f8f719b3640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f640e7c00 con 0x7f8f640ebcc0 2024-04-26T20:50:57.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.496+0000 7f8f719b3640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f8f64057f20 con 0x7f8f640e7820 2024-04-26T20:50:57.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.496+0000 7f8f709b1640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8f640e7820 0x7f8f64038280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.497+0000 7f4639420640 1 -- 172.21.15.73:0/2239702916 >> v1:172.21.15.120:6789/0 conn(0x7f4634073f30 legacy=0x7f46341671e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.496+0000 7f8f719b3640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f8f64058200 con 0x7f8f640eb300 2024-04-26T20:50:57.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.496+0000 7f8f6bfff640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8f640eb300 0x7f8f640eb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.496+0000 7f8f6bfff640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8f640eb300 0x7f8f640eb780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.73:37880/0 (socket says 172.21.15.73:37880) 2024-04-26T20:50:57.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.496+0000 7f8f6bfff640 1 -- 172.21.15.73:0/285252014 learned_addr learned my addr 172.21.15.73:0/285252014 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:57.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.496+0000 7f8f709b1640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8f640e7820 0x7f8f64038280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49612/0 (socket says 172.21.15.73:49612) 2024-04-26T20:50:57.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.497+0000 7f8f6bfff640 1 -- 172.21.15.73:0/285252014 >> v2:172.21.15.73:3300/0 conn(0x7f8f640e7820 msgr2=0x7f8f64038280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:57.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.497+0000 7f8f6bfff640 1 --2- 172.21.15.73:0/285252014 >> v2:172.21.15.73:3300/0 conn(0x7f8f640e7820 0x7f8f64038280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.497+0000 7f8f6bfff640 1 -- 172.21.15.73:0/285252014 >> v1:172.21.15.120:6789/0 conn(0x7f8f640ebcc0 legacy=0x7f8f64053520 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T20:50:57.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.497+0000 7f8f6bfff640 1 -- 172.21.15.73:0/285252014 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f640584e0 con 0x7f8f640eb300 2024-04-26T20:50:57.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.497+0000 7f4639420640 1 --2- 172.21.15.73:0/2239702916 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46340c41d0 0x7f46200039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.497+0000 7f4639420640 1 --2- 172.21.15.73:0/2239702916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4620003f80 0x7f4620004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.497+0000 7fddbbfff640 1 Processor -- start 2024-04-26T20:50:57.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.497+0000 7f4639420640 1 --2- 172.21.15.73:0/2239702916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4620004d40 0x7f46200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.497+0000 7f4639420640 1 -- 172.21.15.73:0/2239702916 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f46340cf7a0 con 0x7f4620004d40 2024-04-26T20:50:57.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.497+0000 7f4639420640 1 -- 172.21.15.73:0/2239702916 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f46340ceb90 con 0x7f4620003f80 2024-04-26T20:50:57.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.497+0000 7f4639420640 1 -- 172.21.15.73:0/2239702916 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f462000b9c0 con 0x7f46340c41d0 2024-04-26T20:50:57.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.497+0000 7f4639c21640 1 --2- 172.21.15.73:0/2239702916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4620003f80 0x7f4620004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.497+0000 7f463a422640 1 --2- 172.21.15.73:0/2239702916 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46340c41d0 0x7f46200039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.497+0000 7fddbbfff640 1 -- start start 2024-04-26T20:50:57.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.497+0000 7f0dfed9d640 1 --2- 172.21.15.73:0/133388959 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0df80ead00 0x7f0df804fa90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0df8007a60 tx=0x7f0df402a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d5ae4911f3dfd54 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:57.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.497+0000 7f463ac23640 1 --2- 172.21.15.73:0/2239702916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4620004d40 0x7f46200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.498+0000 7f1ad8a62640 1 -- 172.21.15.73:0/995864065 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 979352076 0 0) 0x7f1ad40cfed0 con 0x7f1ad4072f50 2024-04-26T20:50:57.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.498+0000 7f1ad8a62640 1 -- 172.21.15.73:0/995864065 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ac0003520 con 0x7f1ad4072f50 2024-04-26T20:50:57.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.498+0000 7f0dfdd9b640 1 -- 172.21.15.73:0/133388959 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df4037070 con 0x7f0df80ead00 2024-04-26T20:50:57.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.498+0000 7f0dfdd9b640 1 -- 172.21.15.73:0/133388959 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0df4032440 con 0x7f0df80ead00 2024-04-26T20:50:57.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.498+0000 7f0dfdd9b640 1 -- 172.21.15.73:0/133388959 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df4032810 con 0x7f0df80ead00 2024-04-26T20:50:57.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.498+0000 7f51677fe640 1 --2- 172.21.15.73:0/3966305613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51600e00a0 0x7f5160034fc0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5160053ad0 tx=0x7f515c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdadfeb15da44a28 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:57.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.498+0000 7f3b2b7fe640 1 --2- 172.21.15.73:0/1557022561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b240e12b0 0x7f3b240534f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3b18009870 tx=0x7f3b1802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db661f762da22d4b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:57.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.498+0000 7fbefa7fc640 1 --2- 172.21.15.73:0/2229244754 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbef40093b0 0x7fbef4006f70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbefc051fc0 tx=0x7fbefc0a6ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42c6a198f21d0aac server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:57.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.498+0000 7f1ad9263640 1 -- 172.21.15.73:0/995864065 >> v1:172.21.15.120:6789/0 conn(0x7f1ad4072f50 legacy=0x7f1ad40d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.498+0000 7f463ac23640 1 -- 172.21.15.73:0/2239702916 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46340c41d0 msgr2=0x7f46200039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:57.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.498+0000 7f463ac23640 1 --2- 172.21.15.73:0/2239702916 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46340c41d0 0x7f46200039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.498+0000 7f463ac23640 1 -- 172.21.15.73:0/2239702916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4620003f80 msgr2=0x7f4620004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:57.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.498+0000 7f463ac23640 1 --2- 172.21.15.73:0/2239702916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4620003f80 0x7f4620004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.498+0000 7f463ac23640 1 -- 172.21.15.73:0/2239702916 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4620003520 con 0x7f4620004d40 2024-04-26T20:50:57.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.498+0000 7f4639c21640 1 --2- 172.21.15.73:0/2239702916 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4620003f80 0x7f4620004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:50:57.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.498+0000 7f51667fc640 1 -- 172.21.15.73:0/3966305613 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f515c037070 con 0x7f51600e00a0 2024-04-26T20:50:57.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.498+0000 7f3b2a7fc640 1 -- 172.21.15.73:0/1557022561 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b18037070 con 0x7f3b240e12b0 2024-04-26T20:50:57.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.498+0000 7f3b2a7fc640 1 -- 172.21.15.73:0/1557022561 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b1802ae50 con 0x7f3b240e12b0 2024-04-26T20:50:57.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.498+0000 7f3b2a7fc640 1 -- 172.21.15.73:0/1557022561 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b18032860 con 0x7f3b240e12b0 2024-04-26T20:50:57.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.498+0000 7f51667fc640 1 -- 172.21.15.73:0/3966305613 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f515c032440 con 0x7f51600e00a0 2024-04-26T20:50:57.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.498+0000 7f8f6bfff640 1 --2- 172.21.15.73:0/285252014 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8f640eb300 0x7f8f640eb780 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8f60009870 tx=0x7f8f6003b220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0172f4723e8b785 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:57.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.498+0000 7f1ad8a62640 1 -- 172.21.15.73:0/995864065 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 8523689 0 0) 0x7f1ad40cfbf0 con 0x7f1ad415bcc0 2024-04-26T20:50:57.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.498+0000 7f1ad9263640 1 -- 172.21.15.73:0/995864065 >> v1:172.21.15.73:6789/0 conn(0x7f1ad415bcc0 legacy=0x7f1ad40cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.498+0000 7f1ad9263640 1 -- 172.21.15.73:0/995864065 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ad4161960 con 0x7f1ad4071110 2024-04-26T20:50:57.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.498+0000 7f51667fc640 1 -- 172.21.15.73:0/3966305613 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f515c032810 con 0x7f51600e00a0 2024-04-26T20:50:57.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.498+0000 7fbef9ffb640 1 -- 172.21.15.73:0/2229244754 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbefc0b4070 con 0x7fbef40093b0 2024-04-26T20:50:57.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.498+0000 7f8f6b7fe640 1 -- 172.21.15.73:0/285252014 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f60048070 con 0x7f8f640eb300 2024-04-26T20:50:57.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.498+0000 7f8f6b7fe640 1 -- 172.21.15.73:0/285252014 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f60043440 con 0x7f8f640eb300 2024-04-26T20:50:57.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.498+0000 7f8f6b7fe640 1 -- 172.21.15.73:0/285252014 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f60043810 con 0x7f8f640eb300 2024-04-26T20:50:57.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.498+0000 7fbef9ffb640 1 -- 172.21.15.73:0/2229244754 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbefc0a7af0 con 0x7fbef40093b0 2024-04-26T20:50:57.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.498+0000 7fddbbfff640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fddbc0d17d0 0x7fddbc0b6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.499+0000 7fddbbfff640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fddbc0d2400 0x7fddbc0d1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.499+0000 7fbef9ffb640 1 -- 172.21.15.73:0/2229244754 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbefc0af440 con 0x7fbef40093b0 2024-04-26T20:50:57.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.499+0000 7fddbbfff640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddbc0b66c0 con 0x7fddbc0d0c80 2024-04-26T20:50:57.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.499+0000 7f0e04a06640 1 -- 172.21.15.73:0/133388959 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0df80ead00 msgr2=0x7f0df804fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.499+0000 7f0e04a06640 1 --2- 172.21.15.73:0/133388959 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0df80ead00 0x7f0df804fa90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0df8007a60 tx=0x7f0df402a1e0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.499+0000 7f0e04a06640 1 -- 172.21.15.73:0/133388959 shutdown_connections 2024-04-26T20:50:57.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.499+0000 7f0e04a06640 1 --2- 172.21.15.73:0/133388959 >> v2:172.21.15.73:3300/0 conn(0x7f0df8038110 0x7f0df80385a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.499+0000 7f0e04a06640 1 --2- 172.21.15.73:0/133388959 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0df804ffd0 0x7f0df8037bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.499+0000 7f0e04a06640 1 --2- 172.21.15.73:0/133388959 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0df80ead00 0x7f0df804fa90 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.499+0000 7f0e04a06640 1 -- 172.21.15.73:0/133388959 >> 172.21.15.73:0/133388959 conn(0x7f0df8009fb0 msgr2=0x7f0df800a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:57.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.499+0000 7fddbb7fe640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7fddbc0d0c80 0x7fddbc0b5210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.73:37882/0 (socket says 172.21.15.73:37882) 2024-04-26T20:50:57.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.499+0000 7fddbb7fe640 1 -- 172.21.15.73:0/1306333635 learned_addr learned my addr 172.21.15.73:0/1306333635 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:50:57.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.499+0000 7fddbbfff640 1 -- 172.21.15.73:0/1306333635 --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7fddbc0d37b0 con 0x7fddbc0d17d0 2024-04-26T20:50:57.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.499+0000 7fddbbfff640 1 -- 172.21.15.73:0/1306333635 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fddbc0d3d70 con 0x7fddbc0d2400 2024-04-26T20:50:57.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.499+0000 7fddbaffd640 1 --2- 172.21.15.73:0/1306333635 >> v2:172.21.15.73:3300/0 conn(0x7fddbc0d17d0 0x7fddbc0b6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.499+0000 7f0e04a06640 1 -- 172.21.15.73:0/133388959 shutdown_connections 2024-04-26T20:50:57.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.499+0000 7f1ad9263640 1 --2- 172.21.15.73:0/995864065 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1ad4071110 0x7f1ad4071570 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1acc009e80 tx=0x7f1acc02a750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a4d2225b2c6c49f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:57.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.499+0000 7f3b30cfd640 1 -- 172.21.15.73:0/1557022561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b240e12b0 msgr2=0x7f3b240534f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.499+0000 7f3b30cfd640 1 --2- 172.21.15.73:0/1557022561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b240e12b0 0x7f3b240534f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3b18009870 tx=0x7f3b1802a300 comp rx=0 tx=0).stop 2024-04-26T20:50:57.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.499+0000 7f0e04a06640 1 -- 172.21.15.73:0/133388959 wait complete. 2024-04-26T20:50:57.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.499+0000 7f1ad8a62640 1 -- 172.21.15.73:0/995864065 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad00015f0 con 0x7f1ad4071110 2024-04-26T20:50:57.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.499+0000 7fddb9ffb640 1 -- 172.21.15.73:0/1306333635 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 671073474 0 0) 0x7fddb4001490 con 0x7fddbc0d0c80 2024-04-26T20:50:57.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.499+0000 7fddb9ffb640 1 -- 172.21.15.73:0/1306333635 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdda4003520 con 0x7fddbc0d0c80 2024-04-26T20:50:57.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.499+0000 7fddba7fc640 1 --2- 172.21.15.73:0/1306333635 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fddbc0d2400 0x7fddbc0d1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.499+0000 7f8f719b3640 1 -- 172.21.15.73:0/285252014 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8f640eb300 msgr2=0x7f8f640eb780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.499+0000 7f8f719b3640 1 --2- 172.21.15.73:0/285252014 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8f640eb300 0x7f8f640eb780 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8f60009870 tx=0x7f8f6003b220 comp rx=0 tx=0).stop 2024-04-26T20:50:57.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.499+0000 7f1ad8a62640 1 -- 172.21.15.73:0/995864065 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1acc031400 con 0x7f1ad4071110 2024-04-26T20:50:57.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.500+0000 7fbefbfff640 1 -- 172.21.15.73:0/2229244754 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbef40093b0 msgr2=0x7fbef4006f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.500+0000 7fbefbfff640 1 --2- 172.21.15.73:0/2229244754 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbef40093b0 0x7fbef4006f70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbefc051fc0 tx=0x7fbefc0a6ce0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.500+0000 7f1ad8a62640 1 -- 172.21.15.73:0/995864065 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad00015f0 con 0x7f1ad4071110 2024-04-26T20:50:57.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.499+0000 7f516ceef640 1 -- 172.21.15.73:0/3966305613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51600e00a0 msgr2=0x7f5160034fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.499+0000 7f516ceef640 1 --2- 172.21.15.73:0/3966305613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51600e00a0 0x7f5160034fc0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5160053ad0 tx=0x7f515c02a270 comp rx=0 tx=0).stop 2024-04-26T20:50:57.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.500+0000 7f516ceef640 1 -- 172.21.15.73:0/3966305613 shutdown_connections 2024-04-26T20:50:57.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.500+0000 7f516ceef640 1 --2- 172.21.15.73:0/3966305613 >> v2:172.21.15.120:3300/0 conn(0x7f5160050430 0x7f5160038de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.500+0000 7f516ceef640 1 --2- 172.21.15.73:0/3966305613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51600e00a0 0x7f5160034fc0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.500+0000 7f516ceef640 1 -- 172.21.15.73:0/3966305613 >> 172.21.15.73:0/3966305613 conn(0x7f5160055bd0 msgr2=0x7f5160055fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:57.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.499+0000 7f8f719b3640 1 -- 172.21.15.73:0/285252014 shutdown_connections 2024-04-26T20:50:57.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.500+0000 7f8f719b3640 1 --2- 172.21.15.73:0/285252014 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8f640eb300 0x7f8f640eb780 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.500+0000 7f8f719b3640 1 --2- 172.21.15.73:0/285252014 >> v2:172.21.15.73:3300/0 conn(0x7f8f640e7820 0x7f8f64038280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.500+0000 7f8f719b3640 1 -- 172.21.15.73:0/285252014 >> 172.21.15.73:0/285252014 conn(0x7f8f640559e0 msgr2=0x7f8f64055df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:57.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.500+0000 7fbefbfff640 1 -- 172.21.15.73:0/2229244754 shutdown_connections 2024-04-26T20:50:57.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.500+0000 7fbefbfff640 1 --2- 172.21.15.73:0/2229244754 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbef4048ec0 0x7fbef40eb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.500+0000 7f516ceef640 1 -- 172.21.15.73:0/3966305613 shutdown_connections 2024-04-26T20:50:57.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.500+0000 7fbefbfff640 1 --2- 172.21.15.73:0/2229244754 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbef40093b0 0x7fbef4006f70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.500+0000 7f516ceef640 1 -- 172.21.15.73:0/3966305613 wait complete. 2024-04-26T20:50:57.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.500+0000 7fbefbfff640 1 --2- 172.21.15.73:0/2229244754 >> v2:172.21.15.73:3300/0 conn(0x7fbef40e77c0 0x7fbef4006a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.500+0000 7f3b30cfd640 1 -- 172.21.15.73:0/1557022561 shutdown_connections 2024-04-26T20:50:57.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.500+0000 7f3b30cfd640 1 --2- 172.21.15.73:0/1557022561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b240e12b0 0x7f3b240534f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.500+0000 7f3b30cfd640 1 -- 172.21.15.73:0/1557022561 >> 172.21.15.73:0/1557022561 conn(0x7f3b24020040 msgr2=0x7f3b24056d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:57.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.500+0000 7fbefbfff640 1 -- 172.21.15.73:0/2229244754 >> 172.21.15.73:0/2229244754 conn(0x7fbef4055c50 msgr2=0x7fbef4056060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:57.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.499+0000 7fddb9ffb640 1 -- 172.21.15.73:0/1306333635 <== mon.1 v1:172.21.15.120:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3855463857 0 0) 0x7fdda4003520 con 0x7fddbc0d0c80 2024-04-26T20:50:57.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.499+0000 7fddb9ffb640 1 -- 172.21.15.73:0/1306333635 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fddbc0d2400 msgr2=0x7fddbc0d1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:57.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.499+0000 7fddb9ffb640 1 --2- 172.21.15.73:0/1306333635 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fddbc0d2400 0x7fddbc0d1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.499+0000 7fddb9ffb640 1 -- 172.21.15.73:0/1306333635 >> v2:172.21.15.73:3300/0 conn(0x7fddbc0d17d0 msgr2=0x7fddbc0b6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:57.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.499+0000 7fddb9ffb640 1 --2- 172.21.15.73:0/1306333635 >> v2:172.21.15.73:3300/0 conn(0x7fddbc0d17d0 0x7fddbc0b6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.500+0000 7fddb9ffb640 1 -- 172.21.15.73:0/1306333635 --> v1:172.21.15.120:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddbc0d20f0 con 0x7fddbc0d0c80 2024-04-26T20:50:57.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.500+0000 7fddb9ffb640 1 -- 172.21.15.73:0/1306333635 <== mon.1 v1:172.21.15.120:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7fddb4001bc0 con 0x7fddbc0d0c80 2024-04-26T20:50:57.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.500+0000 7fddb9ffb640 1 -- 172.21.15.73:0/1306333635 >> v1:172.21.15.120:6789/0 conn(0x7fddbc0d0c80 legacy=0x7fddbc0b5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.500+0000 7fddb9ffb640 1 --2- 172.21.15.73:0/1306333635 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdda40039b0 0x7fdda4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.500+0000 7fddb9ffb640 1 --2- 172.21.15.73:0/1306333635 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdda4004800 0x7fdda4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.500+0000 7fddb9ffb640 1 --2- 172.21.15.73:0/1306333635 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fddbc0d2400 0x7fdda40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.500+0000 7fddb9ffb640 1 -- 172.21.15.73:0/1306333635 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fddbc0d37b0 con 0x7fdda40039b0 2024-04-26T20:50:57.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.500+0000 7fddb9ffb640 1 -- 172.21.15.73:0/1306333635 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fddbc0d3d70 con 0x7fdda4004800 2024-04-26T20:50:57.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.500+0000 7fddb9ffb640 1 -- 172.21.15.73:0/1306333635 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fdda400baf0 con 0x7fddbc0d2400 2024-04-26T20:50:57.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.500+0000 7fddbb7fe640 1 --2- 172.21.15.73:0/1306333635 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fddbc0d2400 0x7fdda40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.500+0000 7fddba7fc640 1 --2- 172.21.15.73:0/1306333635 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdda4004800 0x7fdda4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.500+0000 7f8f719b3640 1 -- 172.21.15.73:0/285252014 shutdown_connections 2024-04-26T20:50:57.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.500+0000 7fbefbfff640 1 -- 172.21.15.73:0/2229244754 shutdown_connections 2024-04-26T20:50:57.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.500+0000 7f3b30cfd640 1 -- 172.21.15.73:0/1557022561 shutdown_connections 2024-04-26T20:50:57.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.500+0000 7f3b30cfd640 1 -- 172.21.15.73:0/1557022561 wait complete. 2024-04-26T20:50:57.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.500+0000 7fbefbfff640 1 -- 172.21.15.73:0/2229244754 wait complete. 2024-04-26T20:50:57.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.500+0000 7fddbb7fe640 1 -- 172.21.15.73:0/1306333635 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdda4004800 msgr2=0x7fdda4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:57.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.500+0000 7fddbb7fe640 1 --2- 172.21.15.73:0/1306333635 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdda4004800 0x7fdda4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.500+0000 7fddbb7fe640 1 -- 172.21.15.73:0/1306333635 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdda40039b0 msgr2=0x7fdda4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:57.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.500+0000 7fddbb7fe640 1 --2- 172.21.15.73:0/1306333635 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdda40039b0 0x7fdda4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.500+0000 7fddbb7fe640 1 -- 172.21.15.73:0/1306333635 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdda4003520 con 0x7fddbc0d2400 2024-04-26T20:50:57.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.500+0000 7f1adbcc8640 1 -- 172.21.15.73:0/995864065 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1ad4071110 msgr2=0x7f1ad4071570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.500+0000 7f1adbcc8640 1 --2- 172.21.15.73:0/995864065 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1ad4071110 0x7f1ad4071570 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1acc009e80 tx=0x7f1acc02a750 comp rx=0 tx=0).stop 2024-04-26T20:50:57.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.500+0000 7f8f719b3640 1 -- 172.21.15.73:0/285252014 wait complete. 2024-04-26T20:50:57.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.501+0000 7f1adbcc8640 1 -- 172.21.15.73:0/995864065 shutdown_connections 2024-04-26T20:50:57.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.501+0000 7f1adbcc8640 1 --2- 172.21.15.73:0/995864065 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1ad4071110 0x7f1ad4071570 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.501+0000 7f1adbcc8640 1 -- 172.21.15.73:0/995864065 >> 172.21.15.73:0/995864065 conn(0x7f1ad40d1cb0 msgr2=0x7f1ad40d2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:57.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.501+0000 7f1adbcc8640 1 -- 172.21.15.73:0/995864065 shutdown_connections 2024-04-26T20:50:57.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.501+0000 7f463ac23640 1 --2- 172.21.15.73:0/2239702916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4620004d40 0x7f46200051c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f462c00e320 tx=0x7f4620005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1656034c788efc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:57.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.501+0000 7f4639420640 1 -- 172.21.15.73:0/2239702916 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462c009de0 con 0x7f4620004d40 2024-04-26T20:50:57.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.502+0000 7f4639420640 1 -- 172.21.15.73:0/2239702916 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f462c02e610 con 0x7f4620004d40 2024-04-26T20:50:57.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.502+0000 7f4639420640 1 -- 172.21.15.73:0/2239702916 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462c02e9c0 con 0x7f4620004d40 2024-04-26T20:50:57.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.502+0000 7f516ceef640 1 Processor -- start 2024-04-26T20:50:57.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.502+0000 7f0e04a06640 1 Processor -- start 2024-04-26T20:50:57.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.502+0000 7fddbb7fe640 1 --2- 172.21.15.73:0/1306333635 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fddbc0d2400 0x7fdda40051c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fddb400a9b0 tx=0x7fdda4005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e603fb7806d8d6c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:57.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.502+0000 7f3b30cfd640 1 Processor -- start 2024-04-26T20:50:57.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.503+0000 7f463c686640 1 -- 172.21.15.73:0/2239702916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4620004d40 msgr2=0x7f46200051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.503+0000 7f463c686640 1 --2- 172.21.15.73:0/2239702916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4620004d40 0x7f46200051c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f462c00e320 tx=0x7f4620005830 comp rx=0 tx=0).stop 2024-04-26T20:50:57.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.503+0000 7f463ac23640 1 -- 172.21.15.73:0/2239702916 reap_dead start 2024-04-26T20:50:57.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.503+0000 7f463c686640 1 -- 172.21.15.73:0/2239702916 shutdown_connections 2024-04-26T20:50:57.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.503+0000 7f463c686640 1 -- 172.21.15.73:0/2239702916 >> 172.21.15.73:0/2239702916 conn(0x7f46340d0be0 msgr2=0x7f46340d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:57.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.502+0000 7fbefbfff640 1 Processor -- start 2024-04-26T20:50:57.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.502+0000 7fbefbfff640 1 -- start start 2024-04-26T20:50:57.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.502+0000 7f0e04a06640 1 -- start start 2024-04-26T20:50:57.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.503+0000 7f0e04a06640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0df8038110 0x7f0df80def50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.503+0000 7f0e04a06640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0df804ffd0 0x7f0df8049a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.503+0000 7f0e04a06640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0df80ead00 0x7f0df8049f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.502+0000 7fddb9ffb640 1 -- 172.21.15.73:0/1306333635 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb400a5d0 con 0x7fddbc0d2400 2024-04-26T20:50:57.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.502+0000 7fddb9ffb640 1 -- 172.21.15.73:0/1306333635 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddb402f070 con 0x7fddbc0d2400 2024-04-26T20:50:57.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.502+0000 7fddb9ffb640 1 -- 172.21.15.73:0/1306333635 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb402bdc0 con 0x7fddbc0d2400 2024-04-26T20:50:57.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.503+0000 7fddbbfff640 1 -- 172.21.15.73:0/1306333635 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fddbc0d2400 msgr2=0x7fdda40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.503+0000 7fddbbfff640 1 --2- 172.21.15.73:0/1306333635 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fddbc0d2400 0x7fdda40051c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fddb400a9b0 tx=0x7fdda4005830 comp rx=0 tx=0).stop 2024-04-26T20:50:57.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.502+0000 7f516ceef640 1 -- start start 2024-04-26T20:50:57.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.503+0000 7f516ceef640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5160039320 0x7f51600d7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.503+0000 7f516ceef640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5160050430 0x7f51600e0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.503+0000 7f516ceef640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51600e00a0 0x7f5160037ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.503+0000 7f516ceef640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f51600085d0 con 0x7f51600e00a0 2024-04-26T20:50:57.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.503+0000 7f516ceef640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f51600082f0 con 0x7f5160050430 2024-04-26T20:50:57.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.503+0000 7f516ceef640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f5160009200 con 0x7f5160039320 2024-04-26T20:50:57.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.502+0000 7f8f719b3640 1 Processor -- start 2024-04-26T20:50:57.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.502+0000 7f8f719b3640 1 -- start start 2024-04-26T20:50:57.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.502+0000 7f3b30cfd640 1 -- start start 2024-04-26T20:50:57.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.503+0000 7f3b30cfd640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3b240e12b0 0x7f3b240ce310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.503+0000 7f3b30cfd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b240e4d30 0x7f3b2404cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.503+0000 7f0e04a06640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0df8054940 con 0x7f0df804ffd0 2024-04-26T20:50:57.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.503+0000 7f0e04a06640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0df80547c0 con 0x7f0df8038110 2024-04-26T20:50:57.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.503+0000 7f0e04a06640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f0df804f550 con 0x7f0df80ead00 2024-04-26T20:50:57.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.503+0000 7f0dfe59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0df804ffd0 0x7f0df8049a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.503+0000 7fddbb7fe640 1 -- 172.21.15.73:0/1306333635 reap_dead start 2024-04-26T20:50:57.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.503+0000 7fddbbfff640 1 -- 172.21.15.73:0/1306333635 shutdown_connections 2024-04-26T20:50:57.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.503+0000 7fddbbfff640 1 -- 172.21.15.73:0/1306333635 >> 172.21.15.73:0/1306333635 conn(0x7fddbc09e210 msgr2=0x7fddbc0d4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:57.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.504+0000 7f463c686640 1 -- 172.21.15.73:0/2239702916 shutdown_connections 2024-04-26T20:50:57.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.504+0000 7fbefbfff640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbef40093b0 0x7fbef40e54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.504+0000 7f3b2b7fe640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3b240e12b0 0x7f3b240ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.504+0000 7f3b2b7fe640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3b240e12b0 0x7f3b240ce310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.73:37926/0 (socket says 172.21.15.73:37926) 2024-04-26T20:50:57.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.504+0000 7fbefbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbef4048ec0 0x7fbef40e59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.504+0000 7fbefbfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbef40e77c0 0x7fbef40e4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.504+0000 7fbefaffd640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbef40093b0 0x7fbef40e54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.504+0000 7fddbbfff640 1 -- 172.21.15.73:0/1306333635 shutdown_connections 2024-04-26T20:50:57.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.504+0000 7f8f719b3640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8f640e7820 0x7f8f6404ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.504+0000 7f8f719b3640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8f640eb300 0x7f8f64009e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.504+0000 7f8f719b3640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f640ebcc0 0x7f8f640e8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.504+0000 7f8f719b3640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8f64058ce0 con 0x7f8f640ebcc0 2024-04-26T20:50:57.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.504+0000 7fddbbfff640 1 -- 172.21.15.73:0/1306333635 wait complete. 2024-04-26T20:50:57.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.504+0000 7f8f719b3640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8f640ec1a0 con 0x7f8f640eb300 2024-04-26T20:50:57.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.504+0000 7f8f719b3640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f8f640546f0 con 0x7f8f640e7820 2024-04-26T20:50:57.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.503+0000 7f0dfe59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0df804ffd0 0x7f0df8049a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49676/0 (socket says 172.21.15.73:49676) 2024-04-26T20:50:57.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.503+0000 7f0dfe59c640 1 -- 172.21.15.73:0/3407733903 learned_addr learned my addr 172.21.15.73:0/3407733903 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:57.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.503+0000 7f0dfed9d640 1 --2- 172.21.15.73:0/3407733903 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0df8038110 0x7f0df80def50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.504+0000 7f0dff59e640 1 --2- 172.21.15.73:0/3407733903 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0df80ead00 0x7f0df8049f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.504+0000 7f0dfed9d640 1 -- 172.21.15.73:0/3407733903 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0df80ead00 msgr2=0x7f0df8049f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.504+0000 7f0dfed9d640 1 --2- 172.21.15.73:0/3407733903 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0df80ead00 0x7f0df8049f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.504+0000 7f0dfed9d640 1 -- 172.21.15.73:0/3407733903 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0df804ffd0 msgr2=0x7f0df8049a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.504+0000 7f0dfed9d640 1 --2- 172.21.15.73:0/3407733903 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0df804ffd0 0x7f0df8049a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.504+0000 7f8f6bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8f640eb300 0x7f8f64009e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.504+0000 7f8f6bfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8f640eb300 0x7f8f64009e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.73:34098/0 (socket says 172.21.15.73:34098) 2024-04-26T20:50:57.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.504+0000 7f8f6bfff640 1 -- 172.21.15.73:0/835798291 learned_addr learned my addr 172.21.15.73:0/835798291 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:57.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.504+0000 7f0dfed9d640 1 -- 172.21.15.73:0/3407733903 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0df817d3f0 con 0x7f0df8038110 2024-04-26T20:50:57.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.504+0000 7f463c686640 1 -- 172.21.15.73:0/2239702916 wait complete. 2024-04-26T20:50:57.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.504+0000 7f3b2affd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b240e4d30 0x7f3b2404cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.504+0000 7f3b2affd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b240e4d30 0x7f3b2404cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49678/0 (socket says 172.21.15.73:49678) 2024-04-26T20:50:57.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.504+0000 7f3b30cfd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b2404a880 0x7f3b24007bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.504+0000 7f3b2affd640 1 -- 172.21.15.73:0/624630551 learned_addr learned my addr 172.21.15.73:0/624630551 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:57.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.504+0000 7f3b30cfd640 1 -- 172.21.15.73:0/624630551 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3b24052c80 con 0x7f3b240e4d30 2024-04-26T20:50:57.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.504+0000 7f3b30cfd640 1 -- 172.21.15.73:0/624630551 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3b24052b00 con 0x7f3b2404a880 2024-04-26T20:50:57.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.504+0000 7f3b30cfd640 1 -- 172.21.15.73:0/624630551 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f3b240576a0 con 0x7f3b240e12b0 2024-04-26T20:50:57.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.504+0000 7fbefa7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbef4048ec0 0x7fbef40e59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.504+0000 7fbefa7fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbef4048ec0 0x7fbef40e59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.73:34082/0 (socket says 172.21.15.73:34082) 2024-04-26T20:50:57.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.504+0000 7f8f709b1640 1 --2- 172.21.15.73:0/835798291 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8f640e7820 0x7f8f6404ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.504+0000 7f3b2bfff640 1 --2- 172.21.15.73:0/624630551 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b2404a880 0x7f3b24007bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.501+0000 7f1adbcc8640 1 -- 172.21.15.73:0/995864065 wait complete. 2024-04-26T20:50:57.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.505+0000 7f3b2affd640 1 -- 172.21.15.73:0/624630551 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3b240e12b0 msgr2=0x7f3b240ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.505+0000 7f3b2affd640 1 --2- 172.21.15.73:0/624630551 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3b240e12b0 0x7f3b240ce310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.505+0000 7f3b2affd640 1 -- 172.21.15.73:0/624630551 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b2404a880 msgr2=0x7f3b24007bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.505+0000 7f3b2affd640 1 --2- 172.21.15.73:0/624630551 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b2404a880 0x7f3b24007bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.505+0000 7f3b2affd640 1 -- 172.21.15.73:0/624630551 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b2404bc50 con 0x7f3b240e4d30 2024-04-26T20:50:57.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.505+0000 7f3b2bfff640 1 --2- 172.21.15.73:0/624630551 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b2404a880 0x7f3b24007bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:50:57.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.504+0000 7f8f6bfff640 1 -- 172.21.15.73:0/835798291 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8f640e7820 msgr2=0x7f8f6404ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.504+0000 7f8f6bfff640 1 --2- 172.21.15.73:0/835798291 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8f640e7820 0x7f8f6404ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.504+0000 7f8f6bfff640 1 -- 172.21.15.73:0/835798291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f640ebcc0 msgr2=0x7f8f640e8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.505+0000 7f8f6bfff640 1 --2- 172.21.15.73:0/835798291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f640ebcc0 0x7f8f640e8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.505+0000 7f8f6bfff640 1 -- 172.21.15.73:0/835798291 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f6400caf0 con 0x7f8f640eb300 2024-04-26T20:50:57.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.505+0000 7f0dfed9d640 1 --2- 172.21.15.73:0/3407733903 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0df8038110 0x7f0df80def50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0df80544b0 tx=0x7f0df40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:57.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.504+0000 7f5167fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51600e00a0 0x7f5160037ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.504+0000 7f5166ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5160050430 0x7f51600e0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.504+0000 7f5166ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5160050430 0x7f51600e0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.73:34074/0 (socket says 172.21.15.73:34074) 2024-04-26T20:50:57.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.504+0000 7f5166ffd640 1 -- 172.21.15.73:0/790312225 learned_addr learned my addr 172.21.15.73:0/790312225 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:57.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.504+0000 7f5166ffd640 1 -- 172.21.15.73:0/790312225 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5160039320 msgr2=0x7f51600d7d40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T20:50:57.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.504+0000 7f5166ffd640 1 --2- 172.21.15.73:0/790312225 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5160039320 0x7f51600d7d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.504+0000 7f5166ffd640 1 -- 172.21.15.73:0/790312225 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51600e00a0 msgr2=0x7f5160037ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.504+0000 7f5166ffd640 1 --2- 172.21.15.73:0/790312225 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51600e00a0 0x7f5160037ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.504+0000 7f5166ffd640 1 -- 172.21.15.73:0/790312225 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51600d8730 con 0x7f5160050430 2024-04-26T20:50:57.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.505+0000 7f5166ffd640 1 --2- 172.21.15.73:0/790312225 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5160050430 0x7f51600e0c50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f515400c7f0 tx=0x7f515400ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:57.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.505+0000 7f8f6bfff640 1 --2- 172.21.15.73:0/835798291 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8f640eb300 0x7f8f64009e50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8f60009870 tx=0x7f8f600046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:57.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.505+0000 7f5153fff640 1 -- 172.21.15.73:0/790312225 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5154015070 con 0x7f5160050430 2024-04-26T20:50:57.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.505+0000 7f3b2affd640 1 --2- 172.21.15.73:0/624630551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b240e4d30 0x7f3b2404cde0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3b2000c490 tx=0x7f3b2000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:57.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.505+0000 7f5153fff640 1 -- 172.21.15.73:0/790312225 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5154007a30 con 0x7f5160050430 2024-04-26T20:50:57.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.505+0000 7f5153fff640 1 -- 172.21.15.73:0/790312225 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5154007e00 con 0x7f5160050430 2024-04-26T20:50:57.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.505+0000 7f0de77fe640 1 -- 172.21.15.73:0/3407733903 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df4037040 con 0x7f0df8038110 2024-04-26T20:50:57.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.505+0000 7f0de77fe640 1 -- 172.21.15.73:0/3407733903 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0df4042070 con 0x7f0df8038110 2024-04-26T20:50:57.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.505+0000 7f0de77fe640 1 -- 172.21.15.73:0/3407733903 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df4030bf0 con 0x7f0df8038110 2024-04-26T20:50:57.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.503+0000 7f1adbcc8640 1 Processor -- start 2024-04-26T20:50:57.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.505+0000 7f516ceef640 1 -- 172.21.15.73:0/790312225 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f516004de20 con 0x7f5160050430 2024-04-26T20:50:57.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.503+0000 7f1adbcc8640 1 -- start start 2024-04-26T20:50:57.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.505+0000 7f516ceef640 1 -- 172.21.15.73:0/790312225 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f516004e3e0 con 0x7f5160050430 2024-04-26T20:50:57.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.505+0000 7f463c686640 1 Processor -- start 2024-04-26T20:50:57.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.504+0000 7f1adbcc8640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1ad4071110 0x7f1ad41f22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.504+0000 7f1adbcc8640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ad4072f50 0x7f1ad41599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.504+0000 7f1adbcc8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ad415bcc0 0x7f1ad41f78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.504+0000 7f1adbcc8640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad40d0490 con 0x7f1ad4072f50 2024-04-26T20:50:57.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.504+0000 7f1adbcc8640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad40cca10 con 0x7f1ad415bcc0 2024-04-26T20:50:57.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.504+0000 7f1adbcc8640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f1ad40cf880 con 0x7f1ad4071110 2024-04-26T20:50:57.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.504+0000 7f1ada265640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ad415bcc0 0x7f1ad41f78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.505+0000 7f1ada265640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ad415bcc0 0x7f1ad41f78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.73:34122/0 (socket says 172.21.15.73:34122) 2024-04-26T20:50:57.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.505+0000 7f1ada265640 1 -- 172.21.15.73:0/302209557 learned_addr learned my addr 172.21.15.73:0/302209557 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:57.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.505+0000 7f1ad9263640 1 --2- 172.21.15.73:0/302209557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ad4072f50 0x7f1ad41599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.505+0000 7f1ad9a64640 1 --2- 172.21.15.73:0/302209557 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1ad4071110 0x7f1ad41f22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.505+0000 7f1ad9263640 1 -- 172.21.15.73:0/302209557 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1ad4071110 msgr2=0x7f1ad41f22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.505+0000 7f1ad9263640 1 --2- 172.21.15.73:0/302209557 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1ad4071110 0x7f1ad41f22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.505+0000 7f1ad9263640 1 -- 172.21.15.73:0/302209557 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ad415bcc0 msgr2=0x7f1ad41f78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.505+0000 7f1ad9263640 1 --2- 172.21.15.73:0/302209557 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ad415bcc0 0x7f1ad41f78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.505+0000 7f1ad9263640 1 -- 172.21.15.73:0/302209557 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ad415a420 con 0x7f1ad4072f50 2024-04-26T20:50:57.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.505+0000 7f3b17fff640 1 -- 172.21.15.73:0/624630551 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b200052d0 con 0x7f3b240e4d30 2024-04-26T20:50:57.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.505+0000 7f3b17fff640 1 -- 172.21.15.73:0/624630551 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b20005470 con 0x7f3b240e4d30 2024-04-26T20:50:57.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.506+0000 7f463c686640 1 -- start start 2024-04-26T20:50:57.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.505+0000 7f3b17fff640 1 -- 172.21.15.73:0/624630551 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b20012870 con 0x7f3b240e4d30 2024-04-26T20:50:57.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.505+0000 7f3b30cfd640 1 -- 172.21.15.73:0/624630551 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b2404b0d0 con 0x7f3b240e4d30 2024-04-26T20:50:57.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.506+0000 7f3b30cfd640 1 -- 172.21.15.73:0/624630551 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b2404b690 con 0x7f3b240e4d30 2024-04-26T20:50:57.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.504+0000 7fbefaffd640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbef40093b0 0x7fbef40e54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.73:37954/0 (socket says 172.21.15.73:37954) 2024-04-26T20:50:57.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.505+0000 7fbefbfff640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbef4050fe0 con 0x7fbef40e77c0 2024-04-26T20:50:57.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.505+0000 7fbefbfff640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbef4050d00 con 0x7fbef4048ec0 2024-04-26T20:50:57.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.505+0000 7fbefbfff640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fbef4050430 con 0x7fbef40093b0 2024-04-26T20:50:57.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.506+0000 7fddbbfff640 1 Processor -- start 2024-04-26T20:50:57.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.506+0000 7fbefa7fc640 1 -- 172.21.15.73:0/137708806 learned_addr learned my addr 172.21.15.73:0/137708806 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:57.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.506+0000 7f0e04a06640 1 -- 172.21.15.73:0/3407733903 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0df8035000 con 0x7f0df8038110 2024-04-26T20:50:57.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.506+0000 7f1ad9263640 1 --2- 172.21.15.73:0/302209557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ad4072f50 0x7f1ad41599d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1acc030000 tx=0x7f1acc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:57.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.507+0000 7f463c686640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4620004d40 0x7f46340c8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.507+0000 7f463c686640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4634164eb0 0x7f4634165770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.508+0000 7f463a422640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4620004d40 0x7f46340c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.508+0000 7f463a422640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4620004d40 0x7f46340c8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.73:34132/0 (socket says 172.21.15.73:34132) 2024-04-26T20:50:57.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.508+0000 7f4639c21640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4634164eb0 0x7f4634165770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.508+0000 7f4639c21640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4634164eb0 0x7f4634165770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49718/0 (socket says 172.21.15.73:49718) 2024-04-26T20:50:57.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.508+0000 7f463a422640 1 -- 172.21.15.73:0/1904519345 learned_addr learned my addr 172.21.15.73:0/1904519345 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:57.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.508+0000 7f463c686640 1 --2- 172.21.15.73:0/1904519345 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4634165330 0x7f46340b4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.508+0000 7f463c686640 1 -- 172.21.15.73:0/1904519345 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f46340ce7b0 con 0x7f4634164eb0 2024-04-26T20:50:57.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.508+0000 7f463c686640 1 -- 172.21.15.73:0/1904519345 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f46340cf430 con 0x7f4620004d40 2024-04-26T20:50:57.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.508+0000 7f463c686640 1 -- 172.21.15.73:0/1904519345 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f46340ce930 con 0x7f4634165330 2024-04-26T20:50:57.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.508+0000 7f463ac23640 1 --2- 172.21.15.73:0/1904519345 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4634165330 0x7f46340b4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.509+0000 7f463a422640 1 -- 172.21.15.73:0/1904519345 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4634165330 msgr2=0x7f46340b4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.509+0000 7f463a422640 1 --2- 172.21.15.73:0/1904519345 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4634165330 0x7f46340b4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.509+0000 7f463a422640 1 -- 172.21.15.73:0/1904519345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4634164eb0 msgr2=0x7f4634165770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.509+0000 7f463a422640 1 --2- 172.21.15.73:0/1904519345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4634164eb0 0x7f4634165770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.509+0000 7f463a422640 1 -- 172.21.15.73:0/1904519345 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46340c9c10 con 0x7f4620004d40 2024-04-26T20:50:57.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.506+0000 7f0e04a06640 1 -- 172.21.15.73:0/3407733903 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0df80438d0 con 0x7f0df8038110 2024-04-26T20:50:57.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.506+0000 7f1aca7fc640 1 -- 172.21.15.73:0/302209557 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1acc041020 con 0x7f1ad4072f50 2024-04-26T20:50:57.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.506+0000 7f1aca7fc640 1 -- 172.21.15.73:0/302209557 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1acc0077f0 con 0x7f1ad4072f50 2024-04-26T20:50:57.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.507+0000 7f1aca7fc640 1 -- 172.21.15.73:0/302209557 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad00015f0 con 0x7f1ad4072f50 2024-04-26T20:50:57.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.507+0000 7f1adbcc8640 1 -- 172.21.15.73:0/302209557 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ad415a700 con 0x7f1ad4072f50 2024-04-26T20:50:57.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.507+0000 7f1adbcc8640 1 -- 172.21.15.73:0/302209557 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ad415acc0 con 0x7f1ad4072f50 2024-04-26T20:50:57.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.509+0000 7f1adbcc8640 1 -- 172.21.15.73:0/302209557 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ad40694f0 con 0x7f1ad4072f50 2024-04-26T20:50:57.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.508+0000 7f5153fff640 1 -- 172.21.15.73:0/790312225 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5154018070 con 0x7f5160050430 2024-04-26T20:50:57.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.505+0000 7f8f697fa640 1 -- 172.21.15.73:0/835798291 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f60048040 con 0x7f8f640eb300 2024-04-26T20:50:57.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.505+0000 7f8f697fa640 1 -- 172.21.15.73:0/835798291 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f60053070 con 0x7f8f640eb300 2024-04-26T20:50:57.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.506+0000 7f8f697fa640 1 -- 172.21.15.73:0/835798291 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f60041bd0 con 0x7f8f640eb300 2024-04-26T20:50:57.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.506+0000 7f8f719b3640 1 -- 172.21.15.73:0/835798291 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f6400cdd0 con 0x7f8f640eb300 2024-04-26T20:50:57.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.506+0000 7f8f719b3640 1 -- 172.21.15.73:0/835798291 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f640501d0 con 0x7f8f640eb300 2024-04-26T20:50:57.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.509+0000 7f4639c21640 1 --2- 172.21.15.73:0/1904519345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4634164eb0 0x7f4634165770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:50:57.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.508+0000 7f5153fff640 1 --2- 172.21.15.73:0/790312225 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f514c06c970 0x7f514c06ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.509+0000 7f463a422640 1 --2- 172.21.15.73:0/1904519345 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4620004d40 0x7f46340c8150 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f46300118e0 tx=0x7f4630011da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:57.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.507+0000 7fbefb7fe640 1 --2- 172.21.15.73:0/137708806 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbef40e77c0 0x7fbef40e4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.507+0000 7fbefb7fe640 1 -- 172.21.15.73:0/137708806 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbef40093b0 msgr2=0x7fbef40e54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.507+0000 7fbefb7fe640 1 --2- 172.21.15.73:0/137708806 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbef40093b0 0x7fbef40e54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.507+0000 7fbefb7fe640 1 -- 172.21.15.73:0/137708806 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbef4048ec0 msgr2=0x7fbef40e59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.507+0000 7fbefb7fe640 1 --2- 172.21.15.73:0/137708806 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbef4048ec0 0x7fbef40e59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.507+0000 7fbefb7fe640 1 -- 172.21.15.73:0/137708806 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbef4038910 con 0x7fbef40e77c0 2024-04-26T20:50:57.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.507+0000 7fbefb7fe640 1 --2- 172.21.15.73:0/137708806 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbef40e77c0 0x7fbef40e4010 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbef000d500 tx=0x7fbef000d9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:57.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.508+0000 7fbee37fe640 1 -- 172.21.15.73:0/137708806 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef0018030 con 0x7fbef40e77c0 2024-04-26T20:50:57.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.508+0000 7fbefbfff640 1 -- 172.21.15.73:0/137708806 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbef40e4c90 con 0x7fbef40e77c0 2024-04-26T20:50:57.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.508+0000 7fbefbfff640 1 -- 172.21.15.73:0/137708806 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbef40e4f70 con 0x7fbef40e77c0 2024-04-26T20:50:57.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.509+0000 7f0de77fe640 1 -- 172.21.15.73:0/3407733903 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0df4047460 con 0x7f0df8038110 2024-04-26T20:50:57.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.508+0000 7f5153fff640 1 -- 172.21.15.73:0/790312225 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f515408bc60 con 0x7f5160050430 2024-04-26T20:50:57.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.509+0000 7f462affd640 1 -- 172.21.15.73:0/1904519345 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463000faf0 con 0x7f4620004d40 2024-04-26T20:50:57.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.509+0000 7f462affd640 1 -- 172.21.15.73:0/1904519345 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f463000fc90 con 0x7f4620004d40 2024-04-26T20:50:57.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.509+0000 7f462affd640 1 -- 172.21.15.73:0/1904519345 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4630014440 con 0x7f4620004d40 2024-04-26T20:50:57.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.509+0000 7f463c686640 1 -- 172.21.15.73:0/1904519345 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46340c9eb0 con 0x7f4620004d40 2024-04-26T20:50:57.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.509+0000 7f516ceef640 1 -- 172.21.15.73:0/790312225 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51600057e0 con 0x7f5160050430 2024-04-26T20:50:57.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.510+0000 7f463c686640 1 -- 172.21.15.73:0/1904519345 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46340ca470 con 0x7f4620004d40 2024-04-26T20:50:57.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.508+0000 7fddbbfff640 1 -- start start 2024-04-26T20:50:57.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.509+0000 7fddbbfff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fddbc0d2400 0x7fddbc1f97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.509+0000 7fddbbfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddbc0cadf0 0x7fddbc0b86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.509+0000 7fddbbfff640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fddbc15bc30 0x7fddbc15c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.509+0000 7fddbbfff640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fddbc0d5a10 con 0x7fddbc0cadf0 2024-04-26T20:50:57.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.509+0000 7fddbbfff640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fddbc0d5730 con 0x7fddbc0d2400 2024-04-26T20:50:57.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.509+0000 7fddbbfff640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fddbc0d3a00 con 0x7fddbc15bc30 2024-04-26T20:50:57.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.509+0000 7fddba7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddbc0cadf0 0x7fddbc0b86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.509+0000 7fddba7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddbc0cadf0 0x7fddbc0b86d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49728/0 (socket says 172.21.15.73:49728) 2024-04-26T20:50:57.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.509+0000 7fddba7fc640 1 -- 172.21.15.73:0/1887953119 learned_addr learned my addr 172.21.15.73:0/1887953119 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:57.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.509+0000 7fddbb7fe640 1 --2- 172.21.15.73:0/1887953119 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fddbc15bc30 0x7fddbc15c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.509+0000 7f3b17fff640 1 -- 172.21.15.73:0/624630551 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3b20012a10 con 0x7f3b240e4d30 2024-04-26T20:50:57.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.509+0000 7f3b17fff640 1 --2- 172.21.15.73:0/624630551 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f3b1006c9f0 0x7f3b1006eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.510+0000 7f3b17fff640 1 -- 172.21.15.73:0/624630551 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f3b2008ba40 con 0x7f3b240e4d30 2024-04-26T20:50:57.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.510+0000 7fbefbfff640 1 -- 172.21.15.73:0/137708806 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbef40057e0 con 0x7fbef40e77c0 2024-04-26T20:50:57.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.510+0000 7f0de77fe640 1 --2- 172.21.15.73:0/3407733903 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f0dec06c9c0 0x7f0dec06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.510+0000 7f0de77fe640 1 -- 172.21.15.73:0/3407733903 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f0df40ac880 con 0x7f0df8038110 2024-04-26T20:50:57.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.509+0000 7f0e04a06640 1 -- 172.21.15.73:0/3407733903 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0df8005390 con 0x7f0df8038110 2024-04-26T20:50:57.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.510+0000 7f3b2b7fe640 1 --2- 172.21.15.73:0/624630551 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f3b1006c9f0 0x7f3b1006eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.510+0000 7f0dfe59c640 1 --2- 172.21.15.73:0/3407733903 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f0dec06c9c0 0x7f0dec06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.510+0000 7f3b30cfd640 1 -- 172.21.15.73:0/624630551 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b0c005290 con 0x7f3b240e4d30 2024-04-26T20:50:57.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.510+0000 7fddba7fc640 1 -- 172.21.15.73:0/1887953119 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fddbc15bc30 msgr2=0x7fddbc15c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.510+0000 7fddba7fc640 1 --2- 172.21.15.73:0/1887953119 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fddbc15bc30 0x7fddbc15c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.510+0000 7f51677fe640 1 --2- 172.21.15.73:0/790312225 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f514c06c970 0x7f514c06ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.510+0000 7fddba7fc640 1 -- 172.21.15.73:0/1887953119 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fddbc0d2400 msgr2=0x7fddbc1f97a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T20:50:57.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.510+0000 7fddba7fc640 1 --2- 172.21.15.73:0/1887953119 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fddbc0d2400 0x7fddbc1f97a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.510+0000 7fddba7fc640 1 -- 172.21.15.73:0/1887953119 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddbc1f9ff0 con 0x7fddbc0cadf0 2024-04-26T20:50:57.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.510+0000 7fddba7fc640 1 --2- 172.21.15.73:0/1887953119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddbc0cadf0 0x7fddbc0b86d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fddac00b7f0 tx=0x7fddac00bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:57.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.510+0000 7fddbb7fe640 1 --2- 172.21.15.73:0/1887953119 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fddbc15bc30 0x7fddbc15c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:50:57.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.511+0000 7fbee37fe640 1 -- 172.21.15.73:0/137708806 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbef0004020 con 0x7fbef40e77c0 2024-04-26T20:50:57.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.511+0000 7fbee37fe640 1 -- 172.21.15.73:0/137708806 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef00041d0 con 0x7fbef40e77c0 2024-04-26T20:50:57.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.511+0000 7f0dfe59c640 1 --2- 172.21.15.73:0/3407733903 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f0dec06c9c0 0x7f0dec06ee80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0df8035c70 tx=0x7f0df0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:57.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.511+0000 7fdda3fff640 1 -- 172.21.15.73:0/1887953119 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddac018070 con 0x7fddbc0cadf0 2024-04-26T20:50:57.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.511+0000 7fddbbfff640 1 -- 172.21.15.73:0/1887953119 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddbc1fa2d0 con 0x7fddbc0cadf0 2024-04-26T20:50:57.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.511+0000 7f3b2b7fe640 1 --2- 172.21.15.73:0/624630551 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f3b1006c9f0 0x7f3b1006eeb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3b18034000 tx=0x7f3b1802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:57.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.511+0000 7f51677fe640 1 --2- 172.21.15.73:0/790312225 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f514c06c970 0x7f514c06ee30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f515c034000 tx=0x7f515c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:57.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.511+0000 7f8f697fa640 1 -- 172.21.15.73:0/835798291 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8f60058460 con 0x7f8f640eb300 2024-04-26T20:50:57.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.512+0000 7fddbbfff640 1 -- 172.21.15.73:0/1887953119 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddbc0cd700 con 0x7fddbc0cadf0 2024-04-26T20:50:57.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.512+0000 7f463c686640 1 -- 172.21.15.73:0/1904519345 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f461c005290 con 0x7f4620004d40 2024-04-26T20:50:57.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.511+0000 7f8f697fa640 1 --2- 172.21.15.73:0/835798291 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f8f5006c9c0 0x7f8f5006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.512+0000 7f8f697fa640 1 -- 172.21.15.73:0/835798291 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f8f600bd960 con 0x7f8f640eb300 2024-04-26T20:50:57.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.512+0000 7f8f709b1640 1 --2- 172.21.15.73:0/835798291 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f8f5006c9c0 0x7f8f5006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.512+0000 7f8f719b3640 1 -- 172.21.15.73:0/835798291 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f640057e0 con 0x7f8f640eb300 2024-04-26T20:50:57.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.512+0000 7f8f709b1640 1 --2- 172.21.15.73:0/835798291 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f8f5006c9c0 0x7f8f5006ee80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8f58009aa0 tx=0x7f8f58009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:57.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.512+0000 7f462affd640 1 -- 172.21.15.73:0/1904519345 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4630012040 con 0x7f4620004d40 2024-04-26T20:50:57.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.512+0000 7fbee37fe640 1 -- 172.21.15.73:0/137708806 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbef001b070 con 0x7fbef40e77c0 2024-04-26T20:50:57.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.512+0000 7f462affd640 1 --2- 172.21.15.73:0/1904519345 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f461806c940 0x7f461806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.513+0000 7fbee37fe640 1 --2- 172.21.15.73:0/137708806 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fbee806c920 0x7fbee806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.513+0000 7f462affd640 1 -- 172.21.15.73:0/1904519345 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f463008c450 con 0x7f4620004d40 2024-04-26T20:50:57.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.513+0000 7f4639c21640 1 --2- 172.21.15.73:0/1904519345 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f461806c940 0x7f461806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.513+0000 7fbefaffd640 1 --2- 172.21.15.73:0/137708806 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fbee806c920 0x7fbee806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.513+0000 7fbee37fe640 1 -- 172.21.15.73:0/137708806 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fbef008bbc0 con 0x7fbef40e77c0 2024-04-26T20:50:57.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.512+0000 7f1aca7fc640 1 -- 172.21.15.73:0/302209557 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1acc044020 con 0x7f1ad4072f50 2024-04-26T20:50:57.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.513+0000 7fddbbfff640 1 -- 172.21.15.73:0/1887953119 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddbc0cd4f0 con 0x7fddbc0cadf0 2024-04-26T20:50:57.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.513+0000 7f1aca7fc640 1 --2- 172.21.15.73:0/302209557 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1ab806c9c0 0x7f1ab806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.513+0000 7fbefaffd640 1 --2- 172.21.15.73:0/137708806 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fbee806c920 0x7fbee806ede0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbef4180b30 tx=0x7fbeec008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:57.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.513+0000 7f1aca7fc640 1 -- 172.21.15.73:0/302209557 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f1acc0ac9b0 con 0x7f1ad4072f50 2024-04-26T20:50:57.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.513+0000 7f1ad9a64640 1 --2- 172.21.15.73:0/302209557 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1ab806c9c0 0x7f1ab806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.514+0000 7f4639c21640 1 --2- 172.21.15.73:0/1904519345 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f461806c940 0x7f461806ee00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f46240017e0 tx=0x7f4624001e00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:57.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.514+0000 7f1ad9a64640 1 --2- 172.21.15.73:0/302209557 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1ab806c9c0 0x7f1ab806ee80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4001120 tx=0x7f1ac4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:57.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.515+0000 7f0de77fe640 1 -- 172.21.15.73:0/3407733903 <== mon.1 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0df407a8c0 con 0x7f0df8038110 2024-04-26T20:50:57.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.517+0000 7f3b17fff640 1 -- 172.21.15.73:0/624630551 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3b20059b10 con 0x7f3b240e4d30 2024-04-26T20:50:57.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.520+0000 7f5153fff640 1 -- 172.21.15.73:0/790312225 <== mon.1 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5154059ec0 con 0x7f5160050430 2024-04-26T20:50:57.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.522+0000 7fbee37fe640 1 -- 172.21.15.73:0/137708806 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbef0022070 con 0x7fbef40e77c0 2024-04-26T20:50:57.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.525+0000 7f8f697fa640 1 -- 172.21.15.73:0/835798291 <== mon.1 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8f6008ba30 con 0x7f8f640eb300 2024-04-26T20:50:57.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.526+0000 7f1aca7fc640 1 -- 172.21.15.73:0/302209557 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1acc04b030 con 0x7f1ad4072f50 2024-04-26T20:50:57.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.526+0000 7fdda3fff640 1 -- 172.21.15.73:0/1887953119 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddac0109e0 con 0x7fddbc0cadf0 2024-04-26T20:50:57.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.526+0000 7fdda3fff640 1 -- 172.21.15.73:0/1887953119 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddac022060 con 0x7fddbc0cadf0 2024-04-26T20:50:57.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.528+0000 7fdda3fff640 1 -- 172.21.15.73:0/1887953119 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fddac01b070 con 0x7fddbc0cadf0 2024-04-26T20:50:57.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.528+0000 7fdda3fff640 1 --2- 172.21.15.73:0/1887953119 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fdda806c920 0x7fdda806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:57.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.528+0000 7fdda3fff640 1 -- 172.21.15.73:0/1887953119 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fddac08c8e0 con 0x7fddbc0cadf0 2024-04-26T20:50:57.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.529+0000 7fddbaffd640 1 --2- 172.21.15.73:0/1887953119 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fdda806c920 0x7fdda806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:57.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.529+0000 7f462affd640 1 -- 172.21.15.73:0/1904519345 <== mon.1 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4630019030 con 0x7f4620004d40 2024-04-26T20:50:57.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.530+0000 7fddbaffd640 1 --2- 172.21.15.73:0/1887953119 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fdda806c920 0x7fdda806ede0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fddb00099f0 tx=0x7fddb0004730 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:57.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.535+0000 7fdda3fff640 1 -- 172.21.15.73:0/1887953119 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fddac05a9b0 con 0x7fddbc0cadf0 2024-04-26T20:50:57.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.940+0000 7f3b30cfd640 1 -- 172.21.15.73:0/624630551 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f3b0c005580 con 0x7f3b240e4d30 2024-04-26T20:50:57.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.940+0000 7f3b17fff640 1 -- 172.21.15.73:0/624630551 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3b20059410 con 0x7f3b240e4d30 2024-04-26T20:50:57.941 INFO:teuthology.orchestra.run.smithi073.stdout:55834574857 2024-04-26T20:50:57.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.943+0000 7f0e04a06640 1 -- 172.21.15.73:0/3407733903 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f0df8037b40 con 0x7f0df8038110 2024-04-26T20:50:57.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.943+0000 7f3b30cfd640 1 -- 172.21.15.73:0/624630551 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f3b1006c9f0 msgr2=0x7f3b1006eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.943+0000 7f3b30cfd640 1 --2- 172.21.15.73:0/624630551 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f3b1006c9f0 0x7f3b1006eeb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3b18034000 tx=0x7f3b1802e000 comp rx=0 tx=0).stop 2024-04-26T20:50:57.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.944+0000 7f3b30cfd640 1 -- 172.21.15.73:0/624630551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b240e4d30 msgr2=0x7f3b2404cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.944+0000 7f3b30cfd640 1 --2- 172.21.15.73:0/624630551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b240e4d30 0x7f3b2404cde0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3b2000c490 tx=0x7f3b2000c950 comp rx=0 tx=0).stop 2024-04-26T20:50:57.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.944+0000 7f0de77fe640 1 -- 172.21.15.73:0/3407733903 <== mon.1 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0df407a1c0 con 0x7f0df8038110 2024-04-26T20:50:57.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.944+0000 7f3b30cfd640 1 -- 172.21.15.73:0/624630551 shutdown_connections 2024-04-26T20:50:57.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.944+0000 7f3b30cfd640 1 --2- 172.21.15.73:0/624630551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b240e4d30 0x7f3b2404cde0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.944+0000 7f3b30cfd640 1 --2- 172.21.15.73:0/624630551 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f3b1006c9f0 0x7f3b1006eeb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.944+0000 7f3b30cfd640 1 --2- 172.21.15.73:0/624630551 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b2404a880 0x7f3b24007bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.944+0000 7f3b30cfd640 1 --2- 172.21.15.73:0/624630551 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3b240e12b0 0x7f3b240ce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.944+0000 7f3b30cfd640 1 -- 172.21.15.73:0/624630551 >> 172.21.15.73:0/624630551 conn(0x7f3b24020040 msgr2=0x7f3b24008b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:57.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.944+0000 7f3b30cfd640 1 -- 172.21.15.73:0/624630551 shutdown_connections 2024-04-26T20:50:57.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.944+0000 7f3b30cfd640 1 -- 172.21.15.73:0/624630551 wait complete. 2024-04-26T20:50:57.948 INFO:teuthology.orchestra.run.smithi073.stdout:55834574858 2024-04-26T20:50:57.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.947+0000 7f0e04a06640 1 -- 172.21.15.73:0/3407733903 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f0dec06c9c0 msgr2=0x7f0dec06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.947+0000 7f0e04a06640 1 --2- 172.21.15.73:0/3407733903 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f0dec06c9c0 0x7f0dec06ee80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0df8035c70 tx=0x7f0df0008040 comp rx=0 tx=0).stop 2024-04-26T20:50:57.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.947+0000 7f0e04a06640 1 -- 172.21.15.73:0/3407733903 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0df8038110 msgr2=0x7f0df80def50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.947+0000 7f0e04a06640 1 --2- 172.21.15.73:0/3407733903 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0df8038110 0x7f0df80def50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0df80544b0 tx=0x7f0df40046c0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.948+0000 7f0e04a06640 1 -- 172.21.15.73:0/3407733903 shutdown_connections 2024-04-26T20:50:57.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.948+0000 7f0e04a06640 1 --2- 172.21.15.73:0/3407733903 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0df804ffd0 0x7f0df8049a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.948+0000 7f0e04a06640 1 --2- 172.21.15.73:0/3407733903 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f0dec06c9c0 0x7f0dec06ee80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.948+0000 7f0e04a06640 1 --2- 172.21.15.73:0/3407733903 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0df80ead00 0x7f0df8049f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.948+0000 7f0e04a06640 1 --2- 172.21.15.73:0/3407733903 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0df8038110 0x7f0df80def50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.948+0000 7f0e04a06640 1 -- 172.21.15.73:0/3407733903 >> 172.21.15.73:0/3407733903 conn(0x7f0df8009fb0 msgr2=0x7f0df8004bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:57.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.948+0000 7f0e04a06640 1 -- 172.21.15.73:0/3407733903 shutdown_connections 2024-04-26T20:50:57.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.948+0000 7f0e04a06640 1 -- 172.21.15.73:0/3407733903 wait complete. 2024-04-26T20:50:57.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.950+0000 7fbefbfff640 1 -- 172.21.15.73:0/137708806 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fbef4048630 con 0x7fbef40e77c0 2024-04-26T20:50:57.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.950+0000 7fbee37fe640 1 -- 172.21.15.73:0/137708806 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbef0059ca0 con 0x7fbef40e77c0 2024-04-26T20:50:57.951 INFO:teuthology.orchestra.run.smithi073.stdout:55834574857 2024-04-26T20:50:57.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.951+0000 7f516ceef640 1 -- 172.21.15.73:0/790312225 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f51600e48a0 con 0x7f5160050430 2024-04-26T20:50:57.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.951+0000 7f5153fff640 1 -- 172.21.15.73:0/790312225 <== mon.1 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f51540597c0 con 0x7f5160050430 2024-04-26T20:50:57.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.951+0000 7f8f719b3640 1 -- 172.21.15.73:0/835798291 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f8f64055210 con 0x7f8f640eb300 2024-04-26T20:50:57.952 INFO:teuthology.orchestra.run.smithi073.stdout:55834574858 2024-04-26T20:50:57.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.954+0000 7f463c686640 1 -- 172.21.15.73:0/1904519345 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f461c005580 con 0x7f4620004d40 2024-04-26T20:50:57.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.954+0000 7f8f697fa640 1 -- 172.21.15.73:0/835798291 <== mon.1 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8f6008b330 con 0x7f8f640eb300 2024-04-26T20:50:57.955 INFO:teuthology.orchestra.run.smithi073.stdout:55834574857 2024-04-26T20:50:57.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.954+0000 7fbee17fa640 1 -- 172.21.15.73:0/137708806 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fbee806c920 msgr2=0x7fbee806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.954+0000 7fbee17fa640 1 --2- 172.21.15.73:0/137708806 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fbee806c920 0x7fbee806ede0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbef4180b30 tx=0x7fbeec008040 comp rx=0 tx=0).stop 2024-04-26T20:50:57.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.954+0000 7fbee17fa640 1 -- 172.21.15.73:0/137708806 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbef40e77c0 msgr2=0x7fbef40e4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.954+0000 7fbee17fa640 1 --2- 172.21.15.73:0/137708806 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbef40e77c0 0x7fbef40e4010 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbef000d500 tx=0x7fbef000d9c0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.954+0000 7fbee17fa640 1 -- 172.21.15.73:0/137708806 shutdown_connections 2024-04-26T20:50:57.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.954+0000 7fbee17fa640 1 --2- 172.21.15.73:0/137708806 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbef40e77c0 0x7fbef40e4010 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.954+0000 7fbee17fa640 1 --2- 172.21.15.73:0/137708806 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fbee806c920 0x7fbee806ede0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.954+0000 7fbee17fa640 1 --2- 172.21.15.73:0/137708806 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbef4048ec0 0x7fbef40e59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.954+0000 7fbee17fa640 1 --2- 172.21.15.73:0/137708806 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbef40093b0 0x7fbef40e54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.955+0000 7fbee17fa640 1 -- 172.21.15.73:0/137708806 >> 172.21.15.73:0/137708806 conn(0x7fbef4055c50 msgr2=0x7fbef4006cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:57.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.955+0000 7fbee17fa640 1 -- 172.21.15.73:0/137708806 shutdown_connections 2024-04-26T20:50:57.957 INFO:teuthology.orchestra.run.smithi073.stdout:55834574858 2024-04-26T20:50:57.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.955+0000 7fbee17fa640 1 -- 172.21.15.73:0/137708806 wait complete. 2024-04-26T20:50:57.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.956+0000 7fddbbfff640 1 -- 172.21.15.73:0/1887953119 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fddbc069370 con 0x7fddbc0cadf0 2024-04-26T20:50:57.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.955+0000 7f516ceef640 1 -- 172.21.15.73:0/790312225 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f514c06c970 msgr2=0x7f514c06ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.955+0000 7f516ceef640 1 --2- 172.21.15.73:0/790312225 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f514c06c970 0x7f514c06ee30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f515c034000 tx=0x7f515c02e000 comp rx=0 tx=0).stop 2024-04-26T20:50:57.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.955+0000 7f516ceef640 1 -- 172.21.15.73:0/790312225 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5160050430 msgr2=0x7f51600e0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.955+0000 7f516ceef640 1 --2- 172.21.15.73:0/790312225 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5160050430 0x7f51600e0c50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f515400c7f0 tx=0x7f515400ccb0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.956+0000 7f516ceef640 1 -- 172.21.15.73:0/790312225 shutdown_connections 2024-04-26T20:50:57.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.956+0000 7f516ceef640 1 --2- 172.21.15.73:0/790312225 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51600e00a0 0x7f5160037ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.956+0000 7f516ceef640 1 --2- 172.21.15.73:0/790312225 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f514c06c970 0x7f514c06ee30 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.956+0000 7f516ceef640 1 --2- 172.21.15.73:0/790312225 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5160050430 0x7f51600e0c50 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.956+0000 7f516ceef640 1 --2- 172.21.15.73:0/790312225 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5160039320 0x7f51600d7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.956+0000 7f516ceef640 1 -- 172.21.15.73:0/790312225 >> 172.21.15.73:0/790312225 conn(0x7f5160055bd0 msgr2=0x7f5160050ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:57.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.956+0000 7fdda3fff640 1 -- 172.21.15.73:0/1887953119 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fddac05a2b0 con 0x7fddbc0cadf0 2024-04-26T20:50:57.959 INFO:teuthology.orchestra.run.smithi073.stdout:55834574858 2024-04-26T20:50:57.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.956+0000 7f516ceef640 1 -- 172.21.15.73:0/790312225 shutdown_connections 2024-04-26T20:50:57.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.956+0000 7f516ceef640 1 -- 172.21.15.73:0/790312225 wait complete. 2024-04-26T20:50:57.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.955+0000 7f462affd640 1 -- 172.21.15.73:0/1904519345 <== mon.1 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4630059ed0 con 0x7f4620004d40 2024-04-26T20:50:57.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.958+0000 7f463c686640 1 -- 172.21.15.73:0/1904519345 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f461806c940 msgr2=0x7f461806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.958+0000 7f463c686640 1 --2- 172.21.15.73:0/1904519345 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f461806c940 0x7f461806ee00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f46240017e0 tx=0x7f4624001e00 comp rx=0 tx=0).stop 2024-04-26T20:50:57.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.958+0000 7f463c686640 1 -- 172.21.15.73:0/1904519345 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4620004d40 msgr2=0x7f46340c8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.958+0000 7f463c686640 1 --2- 172.21.15.73:0/1904519345 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4620004d40 0x7f46340c8150 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f46300118e0 tx=0x7f4630011da0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.958+0000 7f463c686640 1 -- 172.21.15.73:0/1904519345 shutdown_connections 2024-04-26T20:50:57.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.958+0000 7f463c686640 1 --2- 172.21.15.73:0/1904519345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4634164eb0 0x7f4634165770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.958+0000 7f463c686640 1 --2- 172.21.15.73:0/1904519345 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f461806c940 0x7f461806ee00 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.958+0000 7f463c686640 1 --2- 172.21.15.73:0/1904519345 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4634165330 0x7f46340b4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.958+0000 7f463c686640 1 --2- 172.21.15.73:0/1904519345 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4620004d40 0x7f46340c8150 secure :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f46300118e0 tx=0x7f4630011da0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.958+0000 7f463c686640 1 -- 172.21.15.73:0/1904519345 >> 172.21.15.73:0/1904519345 conn(0x7f46340d0be0 msgr2=0x7f46340d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:57.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.958+0000 7f8f719b3640 1 -- 172.21.15.73:0/835798291 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f8f5006c9c0 msgr2=0x7f8f5006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.958+0000 7f8f719b3640 1 --2- 172.21.15.73:0/835798291 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f8f5006c9c0 0x7f8f5006ee80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8f58009aa0 tx=0x7f8f58009190 comp rx=0 tx=0).stop 2024-04-26T20:50:57.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.958+0000 7f8f719b3640 1 -- 172.21.15.73:0/835798291 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8f640eb300 msgr2=0x7f8f64009e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.958+0000 7f8f719b3640 1 --2- 172.21.15.73:0/835798291 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8f640eb300 0x7f8f64009e50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8f60009870 tx=0x7f8f600046c0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.958+0000 7f463c686640 1 -- 172.21.15.73:0/1904519345 shutdown_connections 2024-04-26T20:50:57.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.959+0000 7f463c686640 1 -- 172.21.15.73:0/1904519345 wait complete. 2024-04-26T20:50:57.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.959+0000 7f8f719b3640 1 -- 172.21.15.73:0/835798291 shutdown_connections 2024-04-26T20:50:57.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.959+0000 7f8f719b3640 1 --2- 172.21.15.73:0/835798291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f640ebcc0 0x7f8f640e8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.959+0000 7f8f719b3640 1 --2- 172.21.15.73:0/835798291 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f8f5006c9c0 0x7f8f5006ee80 secure :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8f58009aa0 tx=0x7f8f58009190 comp rx=0 tx=0).stop 2024-04-26T20:50:57.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.959+0000 7f8f719b3640 1 --2- 172.21.15.73:0/835798291 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8f640eb300 0x7f8f64009e50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.959+0000 7f8f719b3640 1 --2- 172.21.15.73:0/835798291 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8f640e7820 0x7f8f6404ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.959+0000 7f8f719b3640 1 -- 172.21.15.73:0/835798291 >> 172.21.15.73:0/835798291 conn(0x7f8f640559e0 msgr2=0x7f8f64039c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:57.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.959+0000 7f8f719b3640 1 -- 172.21.15.73:0/835798291 shutdown_connections 2024-04-26T20:50:57.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.959+0000 7f8f719b3640 1 -- 172.21.15.73:0/835798291 wait complete. 2024-04-26T20:50:57.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.960+0000 7fddbbfff640 1 -- 172.21.15.73:0/1887953119 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fdda806c920 msgr2=0x7fdda806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.960+0000 7fddbbfff640 1 --2- 172.21.15.73:0/1887953119 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fdda806c920 0x7fdda806ede0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fddb00099f0 tx=0x7fddb0004730 comp rx=0 tx=0).stop 2024-04-26T20:50:57.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.960+0000 7fddbbfff640 1 -- 172.21.15.73:0/1887953119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddbc0cadf0 msgr2=0x7fddbc0b86d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.960+0000 7fddbbfff640 1 --2- 172.21.15.73:0/1887953119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddbc0cadf0 0x7fddbc0b86d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fddac00b7f0 tx=0x7fddac00bcb0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.960+0000 7fddbbfff640 1 -- 172.21.15.73:0/1887953119 shutdown_connections 2024-04-26T20:50:57.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.960+0000 7fddbbfff640 1 --2- 172.21.15.73:0/1887953119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddbc0cadf0 0x7fddbc0b86d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.960+0000 7fddbbfff640 1 --2- 172.21.15.73:0/1887953119 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fdda806c920 0x7fdda806ede0 secure :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fddb00099f0 tx=0x7fddb0004730 comp rx=0 tx=0).stop 2024-04-26T20:50:57.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.960+0000 7fddbbfff640 1 --2- 172.21.15.73:0/1887953119 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fddbc15bc30 0x7fddbc15c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.960+0000 7fddbbfff640 1 --2- 172.21.15.73:0/1887953119 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fddbc0d2400 0x7fddbc1f97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.960+0000 7fddbbfff640 1 -- 172.21.15.73:0/1887953119 >> 172.21.15.73:0/1887953119 conn(0x7fddbc09e210 msgr2=0x7fddbc0b2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:57.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.961+0000 7fddbbfff640 1 -- 172.21.15.73:0/1887953119 shutdown_connections 2024-04-26T20:50:57.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.961+0000 7fddbbfff640 1 -- 172.21.15.73:0/1887953119 wait complete. 2024-04-26T20:50:57.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.962+0000 7f1adbcc8640 1 -- 172.21.15.73:0/302209557 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f1ad4070120 con 0x7f1ad4072f50 2024-04-26T20:50:57.966 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.0 2024-04-26T20:50:57.966 INFO:teuthology.orchestra.run.smithi073.stdout:55834574857 2024-04-26T20:50:57.967 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.963+0000 7f1aca7fc640 1 -- 172.21.15.73:0/302209557 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1acc07aa80 con 0x7f1ad4072f50 2024-04-26T20:50:57.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.967+0000 7f1adbcc8640 1 -- 172.21.15.73:0/302209557 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1ab806c9c0 msgr2=0x7f1ab806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.967+0000 7f1adbcc8640 1 --2- 172.21.15.73:0/302209557 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1ab806c9c0 0x7f1ab806ee80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4001120 tx=0x7f1ac4009040 comp rx=0 tx=0).stop 2024-04-26T20:50:57.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.967+0000 7f1adbcc8640 1 -- 172.21.15.73:0/302209557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ad4072f50 msgr2=0x7f1ad41599d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:57.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.967+0000 7f1adbcc8640 1 --2- 172.21.15.73:0/302209557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ad4072f50 0x7f1ad41599d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1acc030000 tx=0x7f1acc004a40 comp rx=0 tx=0).stop 2024-04-26T20:50:57.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.967+0000 7f1adbcc8640 1 -- 172.21.15.73:0/302209557 shutdown_connections 2024-04-26T20:50:57.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.967+0000 7f1adbcc8640 1 --2- 172.21.15.73:0/302209557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ad4072f50 0x7f1ad41599d0 secure :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1acc030000 tx=0x7f1acc004a40 comp rx=0 tx=0).stop 2024-04-26T20:50:57.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.967+0000 7f1adbcc8640 1 --2- 172.21.15.73:0/302209557 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1ab806c9c0 0x7f1ab806ee80 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.967+0000 7f1adbcc8640 1 --2- 172.21.15.73:0/302209557 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ad415bcc0 0x7f1ad41f78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.967+0000 7f1adbcc8640 1 --2- 172.21.15.73:0/302209557 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1ad4071110 0x7f1ad41f22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:57.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.967+0000 7f1adbcc8640 1 -- 172.21.15.73:0/302209557 >> 172.21.15.73:0/302209557 conn(0x7f1ad40d1cb0 msgr2=0x7f1ad40c4d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:57.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.967+0000 7f1adbcc8640 1 -- 172.21.15.73:0/302209557 shutdown_connections 2024-04-26T20:50:57.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:57.967+0000 7f1adbcc8640 1 -- 172.21.15.73:0/302209557 wait complete. 2024-04-26T20:50:57.972 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.5 2024-04-26T20:50:57.979 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.1 2024-04-26T20:50:57.980 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.4 2024-04-26T20:50:57.981 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.3 2024-04-26T20:50:57.982 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.6 2024-04-26T20:50:57.982 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.7 2024-04-26T20:50:57.985 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.2 2024-04-26T20:50:58.967 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-26T20:50:58.973 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-26T20:50:58.980 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-26T20:50:58.981 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-26T20:50:58.982 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-26T20:50:58.982 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-26T20:50:58.983 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-26T20:50:58.986 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-26T20:50:59.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.121+0000 7f4fed899640 1 Processor -- start 2024-04-26T20:50:59.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.121+0000 7f4fed899640 1 -- start start 2024-04-26T20:50:59.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.122+0000 7f4fed899640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f4fe815afe0 0x7f4fe806ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.122+0000 7f4fed899640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fe80cf8e0 con 0x7f4fe80cbdd0 2024-04-26T20:50:59.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.122+0000 7f4fed899640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fe80cfbc0 con 0x7f4fe80704b0 2024-04-26T20:50:59.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.122+0000 7f4fed899640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f4fe80cfea0 con 0x7f4fe815afe0 2024-04-26T20:50:59.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.123+0000 7f4fe77fe640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f4fe80cbdd0 0x7f4fe80c5dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33118/0 (socket says 172.21.15.73:33118) 2024-04-26T20:50:59.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.123+0000 7f4fe77fe640 1 -- 172.21.15.73:0/1452944277 learned_addr learned my addr 172.21.15.73:0/1452944277 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:50:59.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.124+0000 7f4fe6ffd640 1 --2- 172.21.15.73:0/1452944277 >> v2:172.21.15.120:3300/0 conn(0x7f4fe815afe0 0x7f4fe806ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:59.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.124+0000 7f4fe5ffb640 1 -- 172.21.15.73:0/1452944277 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3695182171 0 0) 0x7f4fe80cf8e0 con 0x7f4fe80cbdd0 2024-04-26T20:50:59.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.124+0000 7f4fe5ffb640 1 -- 172.21.15.73:0/1452944277 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4fd0003520 con 0x7f4fe80cbdd0 2024-04-26T20:50:59.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.124+0000 7f4fe5ffb640 1 -- 172.21.15.73:0/1452944277 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2009300030 0 0) 0x7f4fe80cfbc0 con 0x7f4fe80704b0 2024-04-26T20:50:59.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.124+0000 7f4fe5ffb640 1 -- 172.21.15.73:0/1452944277 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4fe80cf8e0 con 0x7f4fe80704b0 2024-04-26T20:50:59.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.124+0000 7f4fe5ffb640 1 -- 172.21.15.73:0/1452944277 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2378847266 0 0) 0x7f4fd0003520 con 0x7f4fe80cbdd0 2024-04-26T20:50:59.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.124+0000 7f4fe5ffb640 1 -- 172.21.15.73:0/1452944277 >> v2:172.21.15.120:3300/0 conn(0x7f4fe815afe0 msgr2=0x7f4fe806ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:59.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.124+0000 7f4fe5ffb640 1 --2- 172.21.15.73:0/1452944277 >> v2:172.21.15.120:3300/0 conn(0x7f4fe815afe0 0x7f4fe806ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.124+0000 7f4fe5ffb640 1 -- 172.21.15.73:0/1452944277 >> v1:172.21.15.120:6789/0 conn(0x7f4fe80704b0 legacy=0x7f4fe80cb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:59.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.125+0000 7f4fe5ffb640 1 -- 172.21.15.73:0/1452944277 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fe80d0180 con 0x7f4fe80cbdd0 2024-04-26T20:50:59.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.125+0000 7f4fe5ffb640 1 -- 172.21.15.73:0/1452944277 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7f4fdc001670 con 0x7f4fe80cbdd0 2024-04-26T20:50:59.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.125+0000 7f4fe5ffb640 1 -- 172.21.15.73:0/1452944277 >> v1:172.21.15.73:6789/0 conn(0x7f4fe80cbdd0 legacy=0x7f4fe80c5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:59.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.125+0000 7f4fe5ffb640 1 --2- 172.21.15.73:0/1452944277 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4fd0003b70 0x7f4fd0003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.125+0000 7f4fe5ffb640 1 --2- 172.21.15.73:0/1452944277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4fd0004800 0x7f4fd0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.125+0000 7f4fe5ffb640 1 --2- 172.21.15.73:0/1452944277 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4fd00051c0 0x7f4fd000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.125+0000 7f4fe5ffb640 1 -- 172.21.15.73:0/1452944277 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe80cfea0 con 0x7f4fd0003b70 2024-04-26T20:50:59.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.125+0000 7f4fe5ffb640 1 -- 172.21.15.73:0/1452944277 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd000c0c0 con 0x7f4fd0004800 2024-04-26T20:50:59.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.125+0000 7f4fe5ffb640 1 -- 172.21.15.73:0/1452944277 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f4fd000c2c0 con 0x7f4fd00051c0 2024-04-26T20:50:59.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.126+0000 7f4fe6ffd640 1 --2- 172.21.15.73:0/1452944277 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4fd0003b70 0x7f4fd0003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:59.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.126+0000 7f4fe6ffd640 1 -- 172.21.15.73:0/1452944277 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4fd00051c0 msgr2=0x7f4fd000b9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T20:50:59.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.126+0000 7f4fe6ffd640 1 --2- 172.21.15.73:0/1452944277 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4fd00051c0 0x7f4fd000b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.126+0000 7f4fe6ffd640 1 -- 172.21.15.73:0/1452944277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4fd0004800 msgr2=0x7f4fd0004c80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T20:50:59.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.126+0000 7f4fe6ffd640 1 --2- 172.21.15.73:0/1452944277 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4fd0004800 0x7f4fd0004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.126+0000 7f4fe6ffd640 1 -- 172.21.15.73:0/1452944277 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fe80d0180 con 0x7f4fd0003b70 2024-04-26T20:50:59.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.127+0000 7f4fe6ffd640 1 --2- 172.21.15.73:0/1452944277 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4fd0003b70 0x7f4fd0003f50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4fd800f100 tx=0x7f4fd0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4158910b36069e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:59.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.128+0000 7f4fe5ffb640 1 -- 172.21.15.73:0/1452944277 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd803a070 con 0x7f4fd0003b70 2024-04-26T20:50:59.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.128+0000 7f4fe5ffb640 1 -- 172.21.15.73:0/1452944277 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fd8035410 con 0x7f4fd0003b70 2024-04-26T20:50:59.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.129+0000 7f4fed899640 1 -- 172.21.15.73:0/1452944277 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4fd0003b70 msgr2=0x7f4fd0003f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:59.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.129+0000 7f4fed899640 1 --2- 172.21.15.73:0/1452944277 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4fd0003b70 0x7f4fd0003f50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4fd800f100 tx=0x7f4fd0005830 comp rx=0 tx=0).stop 2024-04-26T20:50:59.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.130+0000 7f4fe77fe640 1 -- 172.21.15.73:0/1452944277 reap_dead start 2024-04-26T20:50:59.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.130+0000 7f4fed899640 1 -- 172.21.15.73:0/1452944277 shutdown_connections 2024-04-26T20:50:59.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.130+0000 7f4fed899640 1 -- 172.21.15.73:0/1452944277 >> 172.21.15.73:0/1452944277 conn(0x7f4fe80d09f0 msgr2=0x7f4fe80d0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:59.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.130+0000 7f4fed899640 1 -- 172.21.15.73:0/1452944277 shutdown_connections 2024-04-26T20:50:59.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.130+0000 7f4fed899640 1 -- 172.21.15.73:0/1452944277 wait complete. 2024-04-26T20:50:59.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.132+0000 7f4fed899640 1 Processor -- start 2024-04-26T20:50:59.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.132+0000 7f4fed899640 1 -- start start 2024-04-26T20:50:59.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.134+0000 7f4fed899640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4fe814ad40 0x7f4fe8153140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.134+0000 7f4fed899640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4fe8153680 0x7f4fe80c8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.134+0000 7f4fed899640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4fe80c88d0 0x7f4fe81fd460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.134+0000 7f4fe6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4fe814ad40 0x7f4fe8153140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:59.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.134+0000 7f4fe6ffd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4fe814ad40 0x7f4fe8153140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.73:34182/0 (socket says 172.21.15.73:34182) 2024-04-26T20:50:59.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.134+0000 7f4fe67fc640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4fe8153680 0x7f4fe80c8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:59.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.134+0000 7f4fe67fc640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4fe8153680 0x7f4fe80c8390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.73:37998/0 (socket says 172.21.15.73:37998) 2024-04-26T20:50:59.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.135+0000 7f4fe6ffd640 1 -- 172.21.15.73:0/3457945439 learned_addr learned my addr 172.21.15.73:0/3457945439 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:59.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.135+0000 7f4fed899640 1 -- 172.21.15.73:0/3457945439 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe815b3c0 con 0x7f4fe80c88d0 2024-04-26T20:50:59.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.135+0000 7f4fe77fe640 1 --2- 172.21.15.73:0/3457945439 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4fe80c88d0 0x7f4fe81fd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:59.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.135+0000 7f4fed899640 1 -- 172.21.15.73:0/3457945439 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe80cf570 con 0x7f4fe814ad40 2024-04-26T20:50:59.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.135+0000 7f4fed899640 1 -- 172.21.15.73:0/3457945439 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f4fe815b540 con 0x7f4fe8153680 2024-04-26T20:50:59.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.136+0000 7f4fe6ffd640 1 -- 172.21.15.73:0/3457945439 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4fe8153680 msgr2=0x7f4fe80c8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:59.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.136+0000 7f4fe6ffd640 1 --2- 172.21.15.73:0/3457945439 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4fe8153680 0x7f4fe80c8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.136+0000 7f4fe6ffd640 1 -- 172.21.15.73:0/3457945439 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4fe80c88d0 msgr2=0x7f4fe81fd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:59.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.136+0000 7f4fe6ffd640 1 --2- 172.21.15.73:0/3457945439 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4fe80c88d0 0x7f4fe81fd460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.136+0000 7f4fe6ffd640 1 -- 172.21.15.73:0/3457945439 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fe81fcf80 con 0x7f4fe814ad40 2024-04-26T20:50:59.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.136+0000 7f4fe6ffd640 1 --2- 172.21.15.73:0/3457945439 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4fe814ad40 0x7f4fe8153140 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4fd800f220 tx=0x7f4fd8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:59.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.136+0000 7f4fcffff640 1 -- 172.21.15.73:0/3457945439 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd8008e60 con 0x7f4fe814ad40 2024-04-26T20:50:59.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.137+0000 7f4fcffff640 1 -- 172.21.15.73:0/3457945439 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fd800b0c0 con 0x7f4fe814ad40 2024-04-26T20:50:59.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.137+0000 7f4fcffff640 1 -- 172.21.15.73:0/3457945439 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd803a040 con 0x7f4fe814ad40 2024-04-26T20:50:59.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.137+0000 7f4fed899640 1 -- 172.21.15.73:0/3457945439 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fe815bb70 con 0x7f4fe814ad40 2024-04-26T20:50:59.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.137+0000 7f4fed899640 1 -- 172.21.15.73:0/3457945439 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fe815c130 con 0x7f4fe814ad40 2024-04-26T20:50:59.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.139+0000 7f4fed899640 1 -- 172.21.15.73:0/3457945439 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fc8005290 con 0x7f4fe814ad40 2024-04-26T20:50:59.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.141+0000 7f4fcffff640 1 -- 172.21.15.73:0/3457945439 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4fd800b470 con 0x7f4fe814ad40 2024-04-26T20:50:59.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.141+0000 7f4fcffff640 1 --2- 172.21.15.73:0/3457945439 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f4fd406c9c0 0x7f4fd406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.141+0000 7f4fcffff640 1 -- 172.21.15.73:0/3457945439 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f4fd80ac070 con 0x7f4fe814ad40 2024-04-26T20:50:59.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.142+0000 7f4fe67fc640 1 --2- 172.21.15.73:0/3457945439 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f4fd406c9c0 0x7f4fd406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:59.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.143+0000 7f4fe67fc640 1 --2- 172.21.15.73:0/3457945439 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f4fd406c9c0 0x7f4fd406ee80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4fe0007fd0 tx=0x7f4fe0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:59.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.145+0000 7f4fcffff640 1 -- 172.21.15.73:0/3457945439 <== mon.1 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4fd8079af0 con 0x7f4fe814ad40 2024-04-26T20:50:59.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.177+0000 7f52c759e640 1 Processor -- start 2024-04-26T20:50:59.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.177+0000 7f52c759e640 1 -- start start 2024-04-26T20:50:59.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.178+0000 7f45ecf3b640 1 Processor -- start 2024-04-26T20:50:59.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.179+0000 7f52c759e640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f52c000aaa0 0x7f52c004fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.179+0000 7f52c759e640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f52c00df390 0x7f52c004ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.179+0000 7f52c759e640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52c0006af0 con 0x7f52c00382c0 2024-04-26T20:50:59.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.179+0000 7f52c759e640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f52c00068a0 con 0x7f52c000aaa0 2024-04-26T20:50:59.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.179+0000 7f52c759e640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f52c0006dd0 con 0x7f52c00df390 2024-04-26T20:50:59.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.178+0000 7f45ecf3b640 1 -- start start 2024-04-26T20:50:59.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.179+0000 7f582c322640 1 Processor -- start 2024-04-26T20:50:59.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.180+0000 7f45ecf3b640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f45e80d1d60 0x7f45e80c8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.180+0000 7f45e7fff640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f45e80d3570 0x7f45e80b7c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33140/0 (socket says 172.21.15.73:33140) 2024-04-26T20:50:59.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.180+0000 7f45e7fff640 1 -- 172.21.15.73:0/3881131146 learned_addr learned my addr 172.21.15.73:0/3881131146 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:50:59.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.180+0000 7f45ecf3b640 1 -- 172.21.15.73:0/3881131146 --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45e80d5a90 con 0x7f45e80d3570 2024-04-26T20:50:59.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.181+0000 7f45ecf3b640 1 -- 172.21.15.73:0/3881131146 --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45e80d0690 con 0x7f45e80d1980 2024-04-26T20:50:59.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.181+0000 7f45e67fc640 1 -- 172.21.15.73:0/3881131146 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 139069671 0 0) 0x7f45e80d5a90 con 0x7f45e80d3570 2024-04-26T20:50:59.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.181+0000 7f45ecf3b640 1 -- 172.21.15.73:0/3881131146 --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f45e80d5d70 con 0x7f45e80d1d60 2024-04-26T20:50:59.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.181+0000 7f45e67fc640 1 -- 172.21.15.73:0/3881131146 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f45d0003520 con 0x7f45e80d3570 2024-04-26T20:50:59.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.179+0000 7f52c659c640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f52c000aaa0 0x7f52c004fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:59.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.179+0000 7f52c659c640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f52c000aaa0 0x7f52c004fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.73:34184/0 (socket says 172.21.15.73:34184) 2024-04-26T20:50:59.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.179+0000 7f52c659c640 1 -- 172.21.15.73:0/3996927421 learned_addr learned my addr 172.21.15.73:0/3996927421 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:59.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.179+0000 7f52c5d9b640 1 --2- 172.21.15.73:0/3996927421 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f52c00df390 0x7f52c004ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:59.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.179+0000 7f52c6d9d640 1 --1- 172.21.15.73:0/3996927421 >> v1:172.21.15.73:6789/0 conn(0x7f52c00382c0 0x7f52c0038750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33124/0 (socket says 172.21.15.73:33124) 2024-04-26T20:50:59.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.180+0000 7f52c659c640 1 -- 172.21.15.73:0/3996927421 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f52c00df390 msgr2=0x7f52c004ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:59.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.180+0000 7f52c659c640 1 --2- 172.21.15.73:0/3996927421 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f52c00df390 0x7f52c004ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.180+0000 7f52c659c640 1 -- 172.21.15.73:0/3996927421 >> v1:172.21.15.73:6789/0 conn(0x7f52c00382c0 legacy=0x7f52c0038750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:59.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.180+0000 7f52c5d9b640 1 --2- 172.21.15.73:0/3996927421 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f52c00df390 0x7f52c004ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:50:59.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.180+0000 7f52c659c640 1 -- 172.21.15.73:0/3996927421 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52c00070b0 con 0x7f52c000aaa0 2024-04-26T20:50:59.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.181+0000 7f52c659c640 1 --2- 172.21.15.73:0/3996927421 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f52c000aaa0 0x7f52c004fa90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f52c8051470 tx=0x7f52c80a6d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac18aa905e6bfcfd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:59.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.179+0000 7f582c322640 1 -- start start 2024-04-26T20:50:59.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.181+0000 7f45e67fc640 1 -- 172.21.15.73:0/3881131146 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3985871653 0 0) 0x7f45d0003520 con 0x7f45e80d3570 2024-04-26T20:50:59.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.181+0000 7f45e67fc640 1 -- 172.21.15.73:0/3881131146 >> v2:172.21.15.120:3300/0 conn(0x7f45e80d1d60 msgr2=0x7f45e80c8350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T20:50:59.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.181+0000 7f45e67fc640 1 --2- 172.21.15.73:0/3881131146 >> v2:172.21.15.120:3300/0 conn(0x7f45e80d1d60 0x7f45e80c8350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.181+0000 7f45e67fc640 1 -- 172.21.15.73:0/3881131146 >> v1:172.21.15.120:6789/0 conn(0x7f45e80d1980 legacy=0x7f45e80d2e60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T20:50:59.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.181+0000 7f582c322640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5824166f60 0x7f5824167340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.181+0000 7f582c322640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f58240cd030 0x7f58240cb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.181+0000 7f582c322640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5824163920 0x7f5824163d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.181+0000 7f45e67fc640 1 -- 172.21.15.73:0/3881131146 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45e80d0970 con 0x7f45e80d3570 2024-04-26T20:50:59.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.181+0000 7f45e67fc640 1 -- 172.21.15.73:0/3881131146 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7f45e00019f0 con 0x7f45e80d3570 2024-04-26T20:50:59.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.181+0000 7f45e67fc640 1 -- 172.21.15.73:0/3881131146 >> v1:172.21.15.73:6789/0 conn(0x7f45e80d3570 legacy=0x7f45e80b7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:59.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.181+0000 7f45e67fc640 1 --2- 172.21.15.73:0/3881131146 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45d0003a00 0x7f45d0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.181+0000 7f45e67fc640 1 --2- 172.21.15.73:0/3881131146 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f45d0004800 0x7f45d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.181+0000 7f45e67fc640 1 --2- 172.21.15.73:0/3881131146 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45d00052e0 0x7f45d000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.181+0000 7f45e67fc640 1 -- 172.21.15.73:0/3881131146 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f45d000c200 con 0x7f45d00052e0 2024-04-26T20:50:59.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.181+0000 7f45e67fc640 1 -- 172.21.15.73:0/3881131146 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f45d000c3b0 con 0x7f45d0003a00 2024-04-26T20:50:59.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.181+0000 7f45e67fc640 1 -- 172.21.15.73:0/3881131146 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f45d000c5b0 con 0x7f45d0004800 2024-04-26T20:50:59.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.181+0000 7f582c322640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f5824167880 con 0x7f5824163920 2024-04-26T20:50:59.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.181+0000 7f582c322640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f58240cf1f0 con 0x7f58240cd030 2024-04-26T20:50:59.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.181+0000 7f582c322640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f58240cf4d0 con 0x7f5824166f60 2024-04-26T20:50:59.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.182+0000 7f582a0be640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5824166f60 0x7f5824167340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:59.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.182+0000 7f582a0be640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5824166f60 0x7f5824167340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.73:38022/0 (socket says 172.21.15.73:38022) 2024-04-26T20:50:59.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.182+0000 7f582a0be640 1 -- 172.21.15.73:0/212887726 learned_addr learned my addr 172.21.15.73:0/212887726 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:59.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.182+0000 7f58298bd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f58240cd030 0x7f58240cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:59.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.181+0000 7f52c559a640 1 -- 172.21.15.73:0/3996927421 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c80b4070 con 0x7f52c000aaa0 2024-04-26T20:50:59.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.182+0000 7f52c559a640 1 -- 172.21.15.73:0/3996927421 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52c80a7b80 con 0x7f52c000aaa0 2024-04-26T20:50:59.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.182+0000 7f582a8bf640 1 --2- 172.21.15.73:0/212887726 >> v2:172.21.15.73:3300/0 conn(0x7f5824163920 0x7f5824163d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:59.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.182+0000 7f45e7fff640 1 --2- 172.21.15.73:0/3881131146 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45d00052e0 0x7f45d000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:59.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.182+0000 7f45e77fe640 1 --2- 172.21.15.73:0/3881131146 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45d0003a00 0x7f45d0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:59.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.182+0000 7f45e6ffd640 1 --2- 172.21.15.73:0/3881131146 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f45d0004800 0x7f45d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:59.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.182+0000 7f582a0be640 1 -- 172.21.15.73:0/212887726 >> v2:172.21.15.120:3300/0 conn(0x7f58240cd030 msgr2=0x7f58240cb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:59.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.182+0000 7f582a0be640 1 --2- 172.21.15.73:0/212887726 >> v2:172.21.15.120:3300/0 conn(0x7f58240cd030 0x7f58240cb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.182+0000 7f582a0be640 1 -- 172.21.15.73:0/212887726 >> v2:172.21.15.73:3300/0 conn(0x7f5824163920 msgr2=0x7f5824163d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:59.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.182+0000 7f582a0be640 1 --2- 172.21.15.73:0/212887726 >> v2:172.21.15.73:3300/0 conn(0x7f5824163920 0x7f5824163d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.180+0000 7f8cd89c6640 1 Processor -- start 2024-04-26T20:50:59.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.180+0000 7f8cd89c6640 1 -- start start 2024-04-26T20:50:59.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.182+0000 7f582a0be640 1 -- 172.21.15.73:0/212887726 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58240cf7b0 con 0x7f5824166f60 2024-04-26T20:50:59.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.182+0000 7f45e7fff640 1 -- 172.21.15.73:0/3881131146 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f45d0004800 msgr2=0x7f45d0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:59.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.182+0000 7f45e7fff640 1 --2- 172.21.15.73:0/3881131146 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f45d0004800 0x7f45d0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.182+0000 7f45e7fff640 1 -- 172.21.15.73:0/3881131146 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45d0003a00 msgr2=0x7f45d0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:59.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.182+0000 7f45e7fff640 1 --2- 172.21.15.73:0/3881131146 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45d0003a00 0x7f45d0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.182+0000 7f45e7fff640 1 -- 172.21.15.73:0/3881131146 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45d0003520 con 0x7f45d00052e0 2024-04-26T20:50:59.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.183+0000 7f52c759e640 1 -- 172.21.15.73:0/3996927421 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f52c000aaa0 msgr2=0x7f52c004fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:59.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.183+0000 7f52c759e640 1 --2- 172.21.15.73:0/3996927421 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f52c000aaa0 0x7f52c004fa90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f52c8051470 tx=0x7f52c80a6d70 comp rx=0 tx=0).stop 2024-04-26T20:50:59.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.183+0000 7fd01a685640 1 Processor -- start 2024-04-26T20:50:59.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.183+0000 7f582a0be640 1 --2- 172.21.15.73:0/212887726 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5824166f60 0x7f5824167340 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5824074670 tx=0x7f582002a1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0be2626c480734b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:59.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.184+0000 7f58290bc640 1 -- 172.21.15.73:0/212887726 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5820037070 con 0x7f5824166f60 2024-04-26T20:50:59.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.184+0000 7f58290bc640 1 -- 172.21.15.73:0/212887726 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5820032440 con 0x7f5824166f60 2024-04-26T20:50:59.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.183+0000 7f45e7fff640 1 -- 172.21.15.73:0/3881131146 reap_dead start 2024-04-26T20:50:59.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.184+0000 7f45e7fff640 1 --2- 172.21.15.73:0/3881131146 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45d00052e0 0x7f45d000b9c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f45e000a780 tx=0x7f45d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e0afc352e3e6390 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:59.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.183+0000 7f52c759e640 1 -- 172.21.15.73:0/3996927421 shutdown_connections 2024-04-26T20:50:59.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.183+0000 7f52c759e640 1 --2- 172.21.15.73:0/3996927421 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f52c00df390 0x7f52c004ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.183+0000 7f52c759e640 1 --2- 172.21.15.73:0/3996927421 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f52c000aaa0 0x7f52c004fa90 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.183+0000 7f52c759e640 1 -- 172.21.15.73:0/3996927421 >> 172.21.15.73:0/3996927421 conn(0x7f52c0009fb0 msgr2=0x7f52c000a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:59.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.183+0000 7f52c759e640 1 -- 172.21.15.73:0/3996927421 shutdown_connections 2024-04-26T20:50:59.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.182+0000 7f8cd89c6640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8cd40b02b0 0x7f8cd406fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.182+0000 7f8cd89c6640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8cd40cb230 0x7f8cd40cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.182+0000 7f8cd89c6640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8cd40cd490 con 0x7f8cd4070350 2024-04-26T20:50:59.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.182+0000 7f8cd89c6640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f8cd40cec30 con 0x7f8cd40cb230 2024-04-26T20:50:59.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.182+0000 7f8cd89c6640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f8cd40cd1b0 con 0x7f8cd40b02b0 2024-04-26T20:50:59.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.182+0000 7f8cd259c640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8cd40b02b0 0x7f8cd406fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:59.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.182+0000 7f8cd259c640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8cd40b02b0 0x7f8cd406fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.73:38030/0 (socket says 172.21.15.73:38030) 2024-04-26T20:50:59.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.182+0000 7f8cd259c640 1 -- 172.21.15.73:0/1109712165 learned_addr learned my addr 172.21.15.73:0/1109712165 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:59.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.182+0000 7f8cd259c640 1 -- 172.21.15.73:0/1109712165 >> v2:172.21.15.73:3300/0 conn(0x7f8cd40cb230 msgr2=0x7f8cd40cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:59.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.182+0000 7f8cd2d9d640 1 --2- 172.21.15.73:0/1109712165 >> v2:172.21.15.73:3300/0 conn(0x7f8cd40cb230 0x7f8cd40cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:59.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.183+0000 7f8cd259c640 1 --2- 172.21.15.73:0/1109712165 >> v2:172.21.15.73:3300/0 conn(0x7f8cd40cb230 0x7f8cd40cb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.183+0000 7f8cd259c640 1 -- 172.21.15.73:0/1109712165 >> v1:172.21.15.120:6789/0 conn(0x7f8cd4070350 legacy=0x7f8cd40cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:59.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.183+0000 7f8cd259c640 1 -- 172.21.15.73:0/1109712165 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cd40ce8c0 con 0x7f8cd40b02b0 2024-04-26T20:50:59.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.183+0000 7f8cd2d9d640 1 --2- 172.21.15.73:0/1109712165 >> v2:172.21.15.73:3300/0 conn(0x7f8cd40cb230 0x7f8cd40cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:50:59.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.183+0000 7f52c759e640 1 -- 172.21.15.73:0/3996927421 wait complete. 2024-04-26T20:50:59.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.184+0000 7f45e67fc640 1 -- 172.21.15.73:0/3881131146 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e000a3d0 con 0x7f45d00052e0 2024-04-26T20:50:59.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.184+0000 7f45e67fc640 1 -- 172.21.15.73:0/3881131146 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45e002f070 con 0x7f45d00052e0 2024-04-26T20:50:59.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.184+0000 7f8cd259c640 1 --2- 172.21.15.73:0/1109712165 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8cd40b02b0 0x7f8cd406fe10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8cc400ffd0 tx=0x7f8cc4033190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3783e2e48973ec2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:59.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.183+0000 7fd01a685640 1 -- start start 2024-04-26T20:50:59.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.184+0000 7fd01a685640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd0140b56c0 0x7fd0140afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.184+0000 7fd01a685640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0140b5e30 con 0x7fd014072fb0 2024-04-26T20:50:59.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.184+0000 7fd01a685640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0140b6110 con 0x7fd014070750 2024-04-26T20:50:59.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.184+0000 7fd01a685640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fd01406f2b0 con 0x7fd0140b56c0 2024-04-26T20:50:59.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.184+0000 7fd013fff640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fd014072fb0 0x7fd014073390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33150/0 (socket says 172.21.15.73:33150) 2024-04-26T20:50:59.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.184+0000 7fd013fff640 1 -- 172.21.15.73:0/3715837664 learned_addr learned my addr 172.21.15.73:0/3715837664 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:50:59.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.185+0000 7fd018c22640 1 --2- 172.21.15.73:0/3715837664 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd0140b56c0 0x7fd0140afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:59.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.185+0000 7f582c322640 1 -- 172.21.15.73:0/212887726 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5824166f60 msgr2=0x7f5824167340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:59.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.185+0000 7f582c322640 1 --2- 172.21.15.73:0/212887726 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5824166f60 0x7f5824167340 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5824074670 tx=0x7f582002a1e0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.185+0000 7fd018c22640 1 -- 172.21.15.73:0/3715837664 >> v1:172.21.15.120:6789/0 conn(0x7fd014070750 legacy=0x7fd0140af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:59.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.185+0000 7fd018c22640 1 -- 172.21.15.73:0/3715837664 >> v1:172.21.15.73:6789/0 conn(0x7fd014072fb0 legacy=0x7fd014073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:59.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.185+0000 7fd018c22640 1 -- 172.21.15.73:0/3715837664 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0140b63f0 con 0x7fd0140b56c0 2024-04-26T20:50:59.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.185+0000 7f45ecf3b640 1 -- 172.21.15.73:0/3881131146 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45d00052e0 msgr2=0x7f45d000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:59.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.185+0000 7f45ecf3b640 1 --2- 172.21.15.73:0/3881131146 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45d00052e0 0x7f45d000b9c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f45e000a780 tx=0x7f45d0005830 comp rx=0 tx=0).stop 2024-04-26T20:50:59.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.185+0000 7f45ecf3b640 1 -- 172.21.15.73:0/3881131146 shutdown_connections 2024-04-26T20:50:59.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.185+0000 7f45ecf3b640 1 --2- 172.21.15.73:0/3881131146 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45d00052e0 0x7f45d000b9c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.185+0000 7f45ecf3b640 1 -- 172.21.15.73:0/3881131146 >> 172.21.15.73:0/3881131146 conn(0x7f45e80d4dd0 msgr2=0x7f45e80d51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:59.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.186+0000 7f52c759e640 1 Processor -- start 2024-04-26T20:50:59.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.186+0000 7f52c759e640 1 -- start start 2024-04-26T20:50:59.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.186+0000 7f45ecf3b640 1 -- 172.21.15.73:0/3881131146 shutdown_connections 2024-04-26T20:50:59.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.186+0000 7f45ecf3b640 1 -- 172.21.15.73:0/3881131146 wait complete. 2024-04-26T20:50:59.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.185+0000 7f8cd159a640 1 -- 172.21.15.73:0/1109712165 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc4040070 con 0x7f8cd40b02b0 2024-04-26T20:50:59.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.185+0000 7f8cd159a640 1 -- 172.21.15.73:0/1109712165 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cc4033e20 con 0x7f8cd40b02b0 2024-04-26T20:50:59.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.186+0000 7f8cd89c6640 1 -- 172.21.15.73:0/1109712165 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8cd40b02b0 msgr2=0x7f8cd406fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:59.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.186+0000 7f8cd89c6640 1 --2- 172.21.15.73:0/1109712165 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8cd40b02b0 0x7f8cd406fe10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8cc400ffd0 tx=0x7f8cc4033190 comp rx=0 tx=0).stop 2024-04-26T20:50:59.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.186+0000 7f9b4d4fd640 1 Processor -- start 2024-04-26T20:50:59.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.186+0000 7f9b4d4fd640 1 -- start start 2024-04-26T20:50:59.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.186+0000 7f582c322640 1 -- 172.21.15.73:0/212887726 shutdown_connections 2024-04-26T20:50:59.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.186+0000 7f582c322640 1 --2- 172.21.15.73:0/212887726 >> v2:172.21.15.73:3300/0 conn(0x7f5824163920 0x7f5824163d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.186+0000 7f582c322640 1 --2- 172.21.15.73:0/212887726 >> v2:172.21.15.120:3300/0 conn(0x7f58240cd030 0x7f58240cb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.186+0000 7f582c322640 1 --2- 172.21.15.73:0/212887726 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5824166f60 0x7f5824167340 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.186+0000 7f582c322640 1 -- 172.21.15.73:0/212887726 >> 172.21.15.73:0/212887726 conn(0x7f58240d09f0 msgr2=0x7f58240d0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:59.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.186+0000 7f582c322640 1 -- 172.21.15.73:0/212887726 shutdown_connections 2024-04-26T20:50:59.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.186+0000 7f582c322640 1 -- 172.21.15.73:0/212887726 wait complete. 2024-04-26T20:50:59.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.186+0000 7f8cd89c6640 1 -- 172.21.15.73:0/1109712165 shutdown_connections 2024-04-26T20:50:59.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.186+0000 7f8cd89c6640 1 --2- 172.21.15.73:0/1109712165 >> v2:172.21.15.73:3300/0 conn(0x7f8cd40cb230 0x7f8cd40cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.186+0000 7f8cd89c6640 1 --2- 172.21.15.73:0/1109712165 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8cd40b02b0 0x7f8cd406fe10 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.186+0000 7f8cd89c6640 1 -- 172.21.15.73:0/1109712165 >> 172.21.15.73:0/1109712165 conn(0x7f8cd40d0be0 msgr2=0x7f8cd40d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:59.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.186+0000 7f8cd89c6640 1 -- 172.21.15.73:0/1109712165 shutdown_connections 2024-04-26T20:50:59.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.186+0000 7f8cd89c6640 1 -- 172.21.15.73:0/1109712165 wait complete. 2024-04-26T20:50:59.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.187+0000 7fd018c22640 1 --2- 172.21.15.73:0/3715837664 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd0140b56c0 0x7fd0140afdf0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd00c009dc0 tx=0x7fd00c02a580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9648765485b8352 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:59.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.187+0000 7fd012ffd640 1 -- 172.21.15.73:0/3715837664 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00c007740 con 0x7fd0140b56c0 2024-04-26T20:50:59.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.187+0000 7fd012ffd640 1 -- 172.21.15.73:0/3715837664 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd00c0078e0 con 0x7fd0140b56c0 2024-04-26T20:50:59.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.187+0000 7fd012ffd640 1 -- 172.21.15.73:0/3715837664 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00c007cb0 con 0x7fd0140b56c0 2024-04-26T20:50:59.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.187+0000 7f9b4d4fd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f9b40004b00 0x7f9b4003a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.187+0000 7f9b4d4fd640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b40054920 con 0x7f9b400defb0 2024-04-26T20:50:59.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.187+0000 7f9b4d4fd640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b40054c00 con 0x7f9b400e3490 2024-04-26T20:50:59.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.187+0000 7f9b4d4fd640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f9b400543f0 con 0x7f9b40004b00 2024-04-26T20:50:59.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.188+0000 7f9b47fff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f9b40004b00 0x7f9b4003a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:59.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.187+0000 7f45ecf3b640 1 Processor -- start 2024-04-26T20:50:59.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.188+0000 7f582c322640 1 Processor -- start 2024-04-26T20:50:59.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.188+0000 7f8cd89c6640 1 Processor -- start 2024-04-26T20:50:59.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.188+0000 7f8cd89c6640 1 -- start start 2024-04-26T20:50:59.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.187+0000 7f211b5c8640 1 Processor -- start 2024-04-26T20:50:59.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.187+0000 7f211b5c8640 1 -- start start 2024-04-26T20:50:59.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.188+0000 7fd01a685640 1 -- 172.21.15.73:0/3715837664 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd0140b56c0 msgr2=0x7fd0140afdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:59.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.188+0000 7fd01a685640 1 --2- 172.21.15.73:0/3715837664 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd0140b56c0 0x7fd0140afdf0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd00c009dc0 tx=0x7fd00c02a580 comp rx=0 tx=0).stop 2024-04-26T20:50:59.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.188+0000 7fd01a685640 1 -- 172.21.15.73:0/3715837664 shutdown_connections 2024-04-26T20:50:59.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.188+0000 7fd01a685640 1 --2- 172.21.15.73:0/3715837664 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd0140b56c0 0x7fd0140afdf0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.188+0000 7fd01a685640 1 -- 172.21.15.73:0/3715837664 >> 172.21.15.73:0/3715837664 conn(0x7fd0140b39b0 msgr2=0x7fd0140b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:59.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.189+0000 7fd01a685640 1 -- 172.21.15.73:0/3715837664 shutdown_connections 2024-04-26T20:50:59.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.189+0000 7fd01a685640 1 -- 172.21.15.73:0/3715837664 wait complete. 2024-04-26T20:50:59.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.188+0000 7f52c759e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f52c000aaa0 0x7f52c00e0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.188+0000 7f52c759e640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f52c00382c0 0x7f52c004d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.188+0000 7f52c759e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52c00df390 0x7f52c00d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.188+0000 7f52c759e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f52c0052cc0 con 0x7f52c00df390 2024-04-26T20:50:59.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.188+0000 7f52c759e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f52c0052b40 con 0x7f52c000aaa0 2024-04-26T20:50:59.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.188+0000 7f52c759e640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f52c0054270 con 0x7f52c00382c0 2024-04-26T20:50:59.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.188+0000 7f52c6d9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52c00df390 0x7f52c00d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:59.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.188+0000 7f52c6d9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52c00df390 0x7f52c00d10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49772/0 (socket says 172.21.15.73:49772) 2024-04-26T20:50:59.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.188+0000 7f52c6d9d640 1 -- 172.21.15.73:0/3677451779 learned_addr learned my addr 172.21.15.73:0/3677451779 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:59.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.189+0000 7f52c6d9d640 1 -- 172.21.15.73:0/3677451779 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f52c00382c0 msgr2=0x7f52c004d180 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T20:50:59.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.189+0000 7f52c6d9d640 1 --2- 172.21.15.73:0/3677451779 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f52c00382c0 0x7f52c004d180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.189+0000 7f52c6d9d640 1 -- 172.21.15.73:0/3677451779 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f52c000aaa0 msgr2=0x7f52c00e0fc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T20:50:59.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.189+0000 7f52c6d9d640 1 --2- 172.21.15.73:0/3677451779 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f52c000aaa0 0x7f52c00e0fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.189+0000 7f52c6d9d640 1 -- 172.21.15.73:0/3677451779 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52c004c0e0 con 0x7f52c00df390 2024-04-26T20:50:59.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.188+0000 7f45ecf3b640 1 -- start start 2024-04-26T20:50:59.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.189+0000 7f52c6d9d640 1 --2- 172.21.15.73:0/3677451779 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52c00df390 0x7f52c00d10a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f52bc00c410 tx=0x7f52bc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:59.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.188+0000 7f9b477fe640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f9b400defb0 0x7f9b400e2d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33152/0 (socket says 172.21.15.73:33152) 2024-04-26T20:50:59.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.188+0000 7f9b477fe640 1 -- 172.21.15.73:0/4001956704 learned_addr learned my addr 172.21.15.73:0/4001956704 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:50:59.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.188+0000 7f9b47fff640 1 -- 172.21.15.73:0/4001956704 >> v1:172.21.15.120:6789/0 conn(0x7f9b400e3490 legacy=0x7f9b40006ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:59.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.188+0000 7f9b47fff640 1 -- 172.21.15.73:0/4001956704 >> v1:172.21.15.73:6789/0 conn(0x7f9b400defb0 legacy=0x7f9b400e2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:59.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.189+0000 7f8cd89c6640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8cd4070350 0x7f8cd408c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.189+0000 7f52b6ffd640 1 -- 172.21.15.73:0/3677451779 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52bc017070 con 0x7f52c00df390 2024-04-26T20:50:59.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.188+0000 7f582c322640 1 -- start start 2024-04-26T20:50:59.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.189+0000 7f582c322640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58240cd030 0x7f582414f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.189+0000 7f582c322640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5824163920 0x7f582414e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.189+0000 7f582c322640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5824166f60 0x7f58240b52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.189+0000 7f8cd89c6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8cd40b02b0 0x7f8cd40b5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.190+0000 7f52c759e640 1 -- 172.21.15.73:0/3677451779 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52c004c3c0 con 0x7f52c00df390 2024-04-26T20:50:59.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.190+0000 7f582c322640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f58240cb700 con 0x7f58240cd030 2024-04-26T20:50:59.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.190+0000 7f582c322640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f58240cb580 con 0x7f5824163920 2024-04-26T20:50:59.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.189+0000 7f45ecf3b640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f45d00052e0 0x7f45e81ecc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.190+0000 7f45ecf3b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45e8205770 0x7f45e8204dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.190+0000 7f45ecf3b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45e82008a0 0x7f45e8200d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.190+0000 7f45ecf3b640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f45e80d0e70 con 0x7f45e8205770 2024-04-26T20:50:59.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.190+0000 7f45ecf3b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f45e80b5fa0 con 0x7f45e82008a0 2024-04-26T20:50:59.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.190+0000 7f45ecf3b640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f45e80d0ff0 con 0x7f45d00052e0 2024-04-26T20:50:59.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.190+0000 7f52c759e640 1 -- 172.21.15.73:0/3677451779 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52c004c980 con 0x7f52c00df390 2024-04-26T20:50:59.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.190+0000 7f582a8bf640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5824166f60 0x7f58240b52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:59.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.190+0000 7f582a8bf640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5824166f60 0x7f58240b52d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.73:38056/0 (socket says 172.21.15.73:38056) 2024-04-26T20:50:59.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.190+0000 7f582a8bf640 1 -- 172.21.15.73:0/2185217826 learned_addr learned my addr 172.21.15.73:0/2185217826 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:59.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.190+0000 7f582c322640 1 -- 172.21.15.73:0/2185217826 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f582415a6c0 con 0x7f5824166f60 2024-04-26T20:50:59.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.188+0000 7f211b5c8640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f210c04a3e0 0x7f210c038750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.188+0000 7f211b5c8640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f210c0e7a70 0x7f210c038c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.189+0000 7f2119364640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f210c0e7690 0x7f210c052c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33162/0 (socket says 172.21.15.73:33162) 2024-04-26T20:50:59.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.189+0000 7f2119b65640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f210c0e7a70 0x7f210c038c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:59.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.189+0000 7f2119b65640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f210c0e7a70 0x7f210c038c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.73:34222/0 (socket says 172.21.15.73:34222) 2024-04-26T20:50:59.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.189+0000 7f2118b63640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f210c04a3e0 0x7f210c038750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:59.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.189+0000 7f2119364640 1 -- 172.21.15.73:0/1316384105 learned_addr learned my addr 172.21.15.73:0/1316384105 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:50:59.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.189+0000 7f211b5c8640 1 -- 172.21.15.73:0/1316384105 --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f210c051bc0 con 0x7f210c0e7690 2024-04-26T20:50:59.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.189+0000 7f211b5c8640 1 -- 172.21.15.73:0/1316384105 --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f210c0527d0 con 0x7f210c0e7a70 2024-04-26T20:50:59.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.189+0000 7f211b5c8640 1 -- 172.21.15.73:0/1316384105 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f210c051ea0 con 0x7f210c04a3e0 2024-04-26T20:50:59.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.189+0000 7f210bfff640 1 -- 172.21.15.73:0/1316384105 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1259830918 0 0) 0x7f210c051bc0 con 0x7f210c0e7690 2024-04-26T20:50:59.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.189+0000 7f210bfff640 1 -- 172.21.15.73:0/1316384105 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2100003520 con 0x7f210c0e7690 2024-04-26T20:50:59.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.190+0000 7f210bfff640 1 -- 172.21.15.73:0/1316384105 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3137036399 0 0) 0x7f2100003520 con 0x7f210c0e7690 2024-04-26T20:50:59.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.190+0000 7f210bfff640 1 -- 172.21.15.73:0/1316384105 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f210c04a3e0 msgr2=0x7f210c038750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:59.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.190+0000 7f210bfff640 1 --2- 172.21.15.73:0/1316384105 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f210c04a3e0 0x7f210c038750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.190+0000 7f210bfff640 1 -- 172.21.15.73:0/1316384105 >> v2:172.21.15.120:3300/0 conn(0x7f210c0e7a70 msgr2=0x7f210c038c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:59.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.190+0000 7f210bfff640 1 --2- 172.21.15.73:0/1316384105 >> v2:172.21.15.120:3300/0 conn(0x7f210c0e7a70 0x7f210c038c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.190+0000 7f210bfff640 1 -- 172.21.15.73:0/1316384105 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f210c052180 con 0x7f210c0e7690 2024-04-26T20:50:59.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.190+0000 7f8cd89c6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8cd40cb230 0x7f8cd40b5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.190+0000 7f210bfff640 1 -- 172.21.15.73:0/1316384105 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7f2114050f10 con 0x7f210c0e7690 2024-04-26T20:50:59.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.190+0000 7f210bfff640 1 -- 172.21.15.73:0/1316384105 >> v1:172.21.15.73:6789/0 conn(0x7f210c0e7690 legacy=0x7f210c052c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:59.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.190+0000 7f210bfff640 1 --2- 172.21.15.73:0/1316384105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21000039b0 0x7f2100003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.190+0000 7f210bfff640 1 --2- 172.21.15.73:0/1316384105 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2100004800 0x7f2100004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.190+0000 7f210bfff640 1 --2- 172.21.15.73:0/1316384105 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f21000051c0 0x7f210000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.190+0000 7f210bfff640 1 -- 172.21.15.73:0/1316384105 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f210c0527d0 con 0x7f21000039b0 2024-04-26T20:50:59.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.190+0000 7f210bfff640 1 -- 172.21.15.73:0/1316384105 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f210c051ea0 con 0x7f2100004800 2024-04-26T20:50:59.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.190+0000 7f210bfff640 1 -- 172.21.15.73:0/1316384105 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f210000c0c0 con 0x7f21000051c0 2024-04-26T20:50:59.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.190+0000 7f2119b65640 1 --2- 172.21.15.73:0/1316384105 >> v2:172.21.15.120:3300/0 conn(0x7f210c0e7a70 0x7f210c038c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:50:59.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.190+0000 7f2118b63640 1 --2- 172.21.15.73:0/1316384105 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f210c04a3e0 0x7f210c038750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:50:59.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.190+0000 7fd01a685640 1 Processor -- start 2024-04-26T20:50:59.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.190+0000 7f8cd1d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8cd40b02b0 0x7f8cd40b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:59.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.190+0000 7f8cd1d9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8cd40b02b0 0x7f8cd40b5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.73:34248/0 (socket says 172.21.15.73:34248) 2024-04-26T20:50:59.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.190+0000 7f8cd259c640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8cd4070350 0x7f8cd408c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:59.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.190+0000 7f45e77fe640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f45d00052e0 0x7f45e81ecc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:59.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.190+0000 7f45e77fe640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f45d00052e0 0x7f45e81ecc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.73:38072/0 (socket says 172.21.15.73:38072) 2024-04-26T20:50:59.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.190+0000 7f45e77fe640 1 -- 172.21.15.73:0/4267575452 learned_addr learned my addr 172.21.15.73:0/4267575452 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:59.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.190+0000 7f8cd259c640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8cd4070350 0x7f8cd408c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.73:38086/0 (socket says 172.21.15.73:38086) 2024-04-26T20:50:59.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.190+0000 7f2119364640 1 --2- 172.21.15.73:0/1316384105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21000039b0 0x7f2100003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:59.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.190+0000 7f45e7fff640 1 --2- 172.21.15.73:0/4267575452 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45e82008a0 0x7f45e8200d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:59.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.191+0000 7f8cd1d9b640 1 -- 172.21.15.73:0/478819180 learned_addr learned my addr 172.21.15.73:0/478819180 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:59.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.191+0000 7fd01a685640 1 -- start start 2024-04-26T20:50:59.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.191+0000 7f8cd89c6640 1 -- 172.21.15.73:0/478819180 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd4073750 con 0x7f8cd40cb230 2024-04-26T20:50:59.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.190+0000 7f582a0be640 1 --2- 172.21.15.73:0/2185217826 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58240cd030 0x7f582414f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:59.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.190+0000 7f582a0be640 1 -- 172.21.15.73:0/2185217826 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5824166f60 msgr2=0x7f58240b52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:59.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.190+0000 7f582a0be640 1 --2- 172.21.15.73:0/2185217826 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5824166f60 0x7f58240b52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.190+0000 7f582a0be640 1 -- 172.21.15.73:0/2185217826 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5824163920 msgr2=0x7f582414e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:59.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.190+0000 7f582a0be640 1 --2- 172.21.15.73:0/2185217826 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5824163920 0x7f582414e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.190+0000 7f582a0be640 1 -- 172.21.15.73:0/2185217826 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58240ca9b0 con 0x7f58240cd030 2024-04-26T20:50:59.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.191+0000 7f582a0be640 1 --2- 172.21.15.73:0/2185217826 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58240cd030 0x7f582414f590 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5820031000 tx=0x7f58200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:59.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.191+0000 7f45e77fe640 1 -- 172.21.15.73:0/4267575452 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45e82008a0 msgr2=0x7f45e8200d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:59.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.191+0000 7f8cd89c6640 1 -- 172.21.15.73:0/478819180 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd4072080 con 0x7f8cd40b02b0 2024-04-26T20:50:59.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.191+0000 7f8cd89c6640 1 -- 172.21.15.73:0/478819180 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f8cd40738d0 con 0x7f8cd4070350 2024-04-26T20:50:59.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.191+0000 7f45e77fe640 1 --2- 172.21.15.73:0/4267575452 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45e82008a0 0x7f45e8200d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.191+0000 7f45e77fe640 1 -- 172.21.15.73:0/4267575452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45e8205770 msgr2=0x7f45e8204dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:59.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.191+0000 7f45e77fe640 1 --2- 172.21.15.73:0/4267575452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45e8205770 0x7f45e8204dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.191+0000 7f45e77fe640 1 -- 172.21.15.73:0/4267575452 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45e81edd80 con 0x7f45d00052e0 2024-04-26T20:50:59.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.191+0000 7f45e77fe640 1 --2- 172.21.15.73:0/4267575452 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f45d00052e0 0x7f45e81ecc80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f45d800e870 tx=0x7f45d800ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:59.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.191+0000 7f8cd2d9d640 1 --2- 172.21.15.73:0/478819180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8cd40cb230 0x7f8cd40b5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:59.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.191+0000 7f52b6ffd640 1 -- 172.21.15.73:0/3677451779 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52bc0074e0 con 0x7f52c00df390 2024-04-26T20:50:59.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.191+0000 7f8cd1d9b640 1 -- 172.21.15.73:0/478819180 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8cd4070350 msgr2=0x7f8cd408c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:59.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.191+0000 7f8cd1d9b640 1 --2- 172.21.15.73:0/478819180 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8cd4070350 0x7f8cd408c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.191+0000 7f8cd1d9b640 1 -- 172.21.15.73:0/478819180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8cd40cb230 msgr2=0x7f8cd40b5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:59.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.191+0000 7f45cffff640 1 -- 172.21.15.73:0/4267575452 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d8017070 con 0x7f45d00052e0 2024-04-26T20:50:59.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.191+0000 7f9b47fff640 1 -- 172.21.15.73:0/4001956704 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b400e60a0 con 0x7f9b40004b00 2024-04-26T20:50:59.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.191+0000 7f8cd1d9b640 1 --2- 172.21.15.73:0/478819180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8cd40cb230 0x7f8cd40b5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.192+0000 7f52b6ffd640 1 -- 172.21.15.73:0/3677451779 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52bc0059a0 con 0x7f52c00df390 2024-04-26T20:50:59.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.191+0000 7f2119364640 1 -- 172.21.15.73:0/1316384105 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f21000051c0 msgr2=0x7f210000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:59.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.191+0000 7f2119364640 1 --2- 172.21.15.73:0/1316384105 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f21000051c0 0x7f210000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.191+0000 7f2119364640 1 -- 172.21.15.73:0/1316384105 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2100004800 msgr2=0x7f2100004c80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T20:50:59.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.191+0000 7f2119364640 1 --2- 172.21.15.73:0/1316384105 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2100004800 0x7f2100004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.191+0000 7f2119364640 1 -- 172.21.15.73:0/1316384105 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f210c052180 con 0x7f21000039b0 2024-04-26T20:50:59.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.192+0000 7f52b4ff9640 1 -- 172.21.15.73:0/3677451779 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52c00089d0 con 0x7f52c00df390 2024-04-26T20:50:59.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.192+0000 7fd01a685640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd014070750 0x7fd0141ee310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.192+0000 7f8cd1d9b640 1 -- 172.21.15.73:0/478819180 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cd40bd5b0 con 0x7f8cd40b02b0 2024-04-26T20:50:59.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.192+0000 7fd01a685640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd014072fb0 0x7fd014158140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.192+0000 7f581affd640 1 -- 172.21.15.73:0/2185217826 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5820032e60 con 0x7f58240cd030 2024-04-26T20:50:59.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.192+0000 7fd01a685640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd0140b56c0 0x7fd01415ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.192+0000 7f581affd640 1 -- 172.21.15.73:0/2185217826 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5820042070 con 0x7f58240cd030 2024-04-26T20:50:59.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.192+0000 7fd013fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd014070750 0x7fd0141ee310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:59.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.192+0000 7fd013fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd014070750 0x7fd0141ee310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.73:34270/0 (socket says 172.21.15.73:34270) 2024-04-26T20:50:59.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.192+0000 7fd013fff640 1 -- 172.21.15.73:0/2810494177 learned_addr learned my addr 172.21.15.73:0/2810494177 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:59.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.192+0000 7f8cd2d9d640 1 --2- 172.21.15.73:0/478819180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8cd40cb230 0x7f8cd40b5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:50:59.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.192+0000 7fd018c22640 1 --2- 172.21.15.73:0/2810494177 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd0140b56c0 0x7fd01415ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:59.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.193+0000 7fd01a685640 1 -- 172.21.15.73:0/2810494177 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd01415c2e0 con 0x7fd014072fb0 2024-04-26T20:50:59.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.193+0000 7f52b6ffd640 1 -- 172.21.15.73:0/3677451779 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f52bc01a070 con 0x7f52c00df390 2024-04-26T20:50:59.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.193+0000 7fd01a685640 1 -- 172.21.15.73:0/2810494177 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd01415c160 con 0x7fd014070750 2024-04-26T20:50:59.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.193+0000 7fd01a685640 1 -- 172.21.15.73:0/2810494177 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fd0140b0330 con 0x7fd0140b56c0 2024-04-26T20:50:59.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.193+0000 7f8cd1d9b640 1 --2- 172.21.15.73:0/478819180 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8cd40b02b0 0x7f8cd40b5890 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc00cb20 tx=0x7f8ccc0020d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:59.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.193+0000 7f581affd640 1 -- 172.21.15.73:0/2185217826 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5820037040 con 0x7f58240cd030 2024-04-26T20:50:59.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.192+0000 7f45ecf3b640 1 -- 172.21.15.73:0/4267575452 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45e81ed4e0 con 0x7f45d00052e0 2024-04-26T20:50:59.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.192+0000 7f45ecf3b640 1 -- 172.21.15.73:0/4267575452 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f45e81edaa0 con 0x7f45d00052e0 2024-04-26T20:50:59.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.192+0000 7f45cffff640 1 -- 172.21.15.73:0/4267575452 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45d800ee60 con 0x7f45d00052e0 2024-04-26T20:50:59.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.193+0000 7fd0137fe640 1 --2- 172.21.15.73:0/2810494177 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd014072fb0 0x7fd014158140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:59.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.193+0000 7f45cffff640 1 -- 172.21.15.73:0/4267575452 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d800cb80 con 0x7f45d00052e0 2024-04-26T20:50:59.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.192+0000 7f2119364640 1 --2- 172.21.15.73:0/1316384105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21000039b0 0x7f2100003dd0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f211406b190 tx=0x7f2100005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d45f8a92478ab1e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:59.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.193+0000 7f45ecf3b640 1 -- 172.21.15.73:0/4267575452 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45c8005290 con 0x7f45d00052e0 2024-04-26T20:50:59.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.193+0000 7f210bfff640 1 -- 172.21.15.73:0/1316384105 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2114069730 con 0x7f21000039b0 2024-04-26T20:50:59.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.193+0000 7f9b47fff640 1 --2- 172.21.15.73:0/4001956704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9b40004b00 0x7f9b4003a7f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c0097a0 tx=0x7f9b3c02a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=787cc1262ae84c77 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:59.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.193+0000 7f9b46ffd640 1 -- 172.21.15.73:0/4001956704 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c037070 con 0x7f9b40004b00 2024-04-26T20:50:59.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.193+0000 7f9b46ffd640 1 -- 172.21.15.73:0/4001956704 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b3c032440 con 0x7f9b40004b00 2024-04-26T20:50:59.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.193+0000 7f9b46ffd640 1 -- 172.21.15.73:0/4001956704 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c032850 con 0x7f9b40004b00 2024-04-26T20:50:59.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.193+0000 7fd0137fe640 1 -- 172.21.15.73:0/2810494177 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd0140b56c0 msgr2=0x7fd01415ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:59.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.193+0000 7fd0137fe640 1 --2- 172.21.15.73:0/2810494177 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd0140b56c0 0x7fd01415ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.193+0000 7fd0137fe640 1 -- 172.21.15.73:0/2810494177 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd014070750 msgr2=0x7fd0141ee310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:59.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.193+0000 7fd0137fe640 1 --2- 172.21.15.73:0/2810494177 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd014070750 0x7fd0141ee310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.193+0000 7fd0137fe640 1 -- 172.21.15.73:0/2810494177 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd01409d170 con 0x7fd014072fb0 2024-04-26T20:50:59.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.193+0000 7fd013fff640 1 --2- 172.21.15.73:0/2810494177 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd014070750 0x7fd0141ee310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:50:59.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.193+0000 7f8cc2ffd640 1 -- 172.21.15.73:0/478819180 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc018070 con 0x7f8cd40b02b0 2024-04-26T20:50:59.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.193+0000 7f8cc2ffd640 1 -- 172.21.15.73:0/478819180 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ccc003da0 con 0x7f8cd40b02b0 2024-04-26T20:50:59.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.193+0000 7f8cc2ffd640 1 -- 172.21.15.73:0/478819180 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc006d30 con 0x7f8cd40b02b0 2024-04-26T20:50:59.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.193+0000 7f8cd89c6640 1 -- 172.21.15.73:0/478819180 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cd40c0de0 con 0x7f8cd40b02b0 2024-04-26T20:50:59.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.193+0000 7f8cd89c6640 1 -- 172.21.15.73:0/478819180 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cd40c13a0 con 0x7f8cd40b02b0 2024-04-26T20:50:59.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.194+0000 7fd0137fe640 1 --2- 172.21.15.73:0/2810494177 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd014072fb0 0x7fd014158140 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd00400c440 tx=0x7fd00400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:59.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.194+0000 7f9b4d4fd640 1 -- 172.21.15.73:0/4001956704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9b40004b00 msgr2=0x7f9b4003a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:59.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.194+0000 7f52b6ffd640 1 --2- 172.21.15.73:0/3677451779 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f52a806c9c0 0x7f52a806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.194+0000 7f9b4d4fd640 1 --2- 172.21.15.73:0/4001956704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9b40004b00 0x7f9b4003a7f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c0097a0 tx=0x7f9b3c02a1c0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.195+0000 7f9b4d4fd640 1 -- 172.21.15.73:0/4001956704 shutdown_connections 2024-04-26T20:50:59.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.194+0000 7f52c659c640 1 --2- 172.21.15.73:0/3677451779 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f52a806c9c0 0x7f52a806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:59.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.195+0000 7f8cd89c6640 1 -- 172.21.15.73:0/478819180 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cd40b1f90 con 0x7f8cd40b02b0 2024-04-26T20:50:59.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.195+0000 7f9b4d4fd640 1 --2- 172.21.15.73:0/4001956704 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9b40004b00 0x7f9b4003a7f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.195+0000 7f9b4d4fd640 1 -- 172.21.15.73:0/4001956704 >> 172.21.15.73:0/4001956704 conn(0x7f9b4000a400 msgr2=0x7f9b4000a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:59.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.195+0000 7f45cffff640 1 -- 172.21.15.73:0/4267575452 <== mon.2 v2:172.21.15.120:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f45d801a070 con 0x7f45d00052e0 2024-04-26T20:50:59.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.195+0000 7f9b4d4fd640 1 -- 172.21.15.73:0/4001956704 shutdown_connections 2024-04-26T20:50:59.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.195+0000 7f9b4d4fd640 1 -- 172.21.15.73:0/4001956704 wait complete. 2024-04-26T20:50:59.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.195+0000 7f45cffff640 1 --2- 172.21.15.73:0/4267575452 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f45d406c9f0 0x7f45d406eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.195+0000 7f52b6ffd640 1 -- 172.21.15.73:0/3677451779 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f52bc08b1d0 con 0x7f52c00df390 2024-04-26T20:50:59.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.195+0000 7f52c659c640 1 --2- 172.21.15.73:0/3677451779 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f52a806c9c0 0x7f52a806ee80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f52c0008680 tx=0x7f52c80ab000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:59.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.195+0000 7f45cffff640 1 -- 172.21.15.73:0/4267575452 <== mon.2 v2:172.21.15.120:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f45d808c480 con 0x7f45d00052e0 2024-04-26T20:50:59.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.194+0000 7f582c322640 1 -- 172.21.15.73:0/2185217826 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58240ca6d0 con 0x7f58240cd030 2024-04-26T20:50:59.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.194+0000 7f582c322640 1 -- 172.21.15.73:0/2185217826 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58240b4450 con 0x7f58240cd030 2024-04-26T20:50:59.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.195+0000 7f582c322640 1 -- 172.21.15.73:0/2185217826 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5824069370 con 0x7f58240cd030 2024-04-26T20:50:59.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.195+0000 7f45e6ffd640 1 --2- 172.21.15.73:0/4267575452 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f45d406c9f0 0x7f45d406eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:59.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.196+0000 7f45e6ffd640 1 --2- 172.21.15.73:0/4267575452 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f45d406c9f0 0x7f45d406eeb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f45e8166890 tx=0x7f45dc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:59.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.196+0000 7f8cc2ffd640 1 -- 172.21.15.73:0/478819180 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8ccc00a700 con 0x7f8cd40b02b0 2024-04-26T20:50:59.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.196+0000 7f9b4d4fd640 1 Processor -- start 2024-04-26T20:50:59.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.197+0000 7f9b4d4fd640 1 -- start start 2024-04-26T20:50:59.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.197+0000 7f8cc2ffd640 1 --2- 172.21.15.73:0/478819180 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f8cb406c8f0 0x7f8cb406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.197+0000 7f8cd259c640 1 --2- 172.21.15.73:0/478819180 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f8cb406c8f0 0x7f8cb406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:59.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.197+0000 7f8cc2ffd640 1 -- 172.21.15.73:0/478819180 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f8ccc08ba60 con 0x7f8cd40b02b0 2024-04-26T20:50:59.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.197+0000 7f8cd259c640 1 --2- 172.21.15.73:0/478819180 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f8cb406c8f0 0x7f8cb406edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8cc403d000 tx=0x7f8cc4037000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:59.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.198+0000 7f9b4d4fd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9b40004b00 0x7f9b40037380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.198+0000 7f9b4d4fd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b400defb0 0x7f9b400d6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.198+0000 7f9b4d4fd640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9b400e3490 0x7f9b400df8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.198+0000 7f9b477fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b400defb0 0x7f9b400d6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:59.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.198+0000 7f9b477fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b400defb0 0x7f9b400d6d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49812/0 (socket says 172.21.15.73:49812) 2024-04-26T20:50:59.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.198+0000 7f9b47fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9b40004b00 0x7f9b40037380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:59.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.198+0000 7f9b47fff640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9b40004b00 0x7f9b40037380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.73:34274/0 (socket says 172.21.15.73:34274) 2024-04-26T20:50:59.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.198+0000 7f9b477fe640 1 -- 172.21.15.73:0/3536493397 learned_addr learned my addr 172.21.15.73:0/3536493397 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:59.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.198+0000 7f9b4d4fd640 1 -- 172.21.15.73:0/3536493397 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9b400e3a60 con 0x7f9b400defb0 2024-04-26T20:50:59.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.198+0000 7f9b4d4fd640 1 -- 172.21.15.73:0/3536493397 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9b400065c0 con 0x7f9b40004b00 2024-04-26T20:50:59.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.198+0000 7f9b4d4fd640 1 -- 172.21.15.73:0/3536493397 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f9b400e3be0 con 0x7f9b400e3490 2024-04-26T20:50:59.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.198+0000 7f9b4ccfc640 1 --2- 172.21.15.73:0/3536493397 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9b400e3490 0x7f9b400df8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:59.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.198+0000 7f9b477fe640 1 -- 172.21.15.73:0/3536493397 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9b400e3490 msgr2=0x7f9b400df8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:59.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.198+0000 7f9b477fe640 1 --2- 172.21.15.73:0/3536493397 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9b400e3490 0x7f9b400df8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.198+0000 7f9b477fe640 1 -- 172.21.15.73:0/3536493397 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9b40004b00 msgr2=0x7f9b40037380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:59.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.198+0000 7f9b477fe640 1 --2- 172.21.15.73:0/3536493397 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9b40004b00 0x7f9b40037380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.198+0000 7f9b477fe640 1 -- 172.21.15.73:0/3536493397 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b4004e140 con 0x7f9b400defb0 2024-04-26T20:50:59.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.198+0000 7f9b477fe640 1 --2- 172.21.15.73:0/3536493397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b400defb0 0x7f9b400d6d60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9b3400c4d0 tx=0x7f9b3400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:59.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.199+0000 7f45cffff640 1 -- 172.21.15.73:0/4267575452 <== mon.2 v2:172.21.15.120:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f45d8021070 con 0x7f45d00052e0 2024-04-26T20:50:59.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.200+0000 7f8cc2ffd640 1 -- 172.21.15.73:0/478819180 <== mon.1 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8cd40b1f90 con 0x7f8cd40b02b0 2024-04-26T20:50:59.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.202+0000 7f210bfff640 1 -- 172.21.15.73:0/1316384105 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21140ae400 con 0x7f21000039b0 2024-04-26T20:50:59.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.202+0000 7f210bfff640 1 -- 172.21.15.73:0/1316384105 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21140ae8b0 con 0x7f21000039b0 2024-04-26T20:50:59.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.202+0000 7f52b6ffd640 1 -- 172.21.15.73:0/3677451779 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f52bc0593f0 con 0x7f52c00df390 2024-04-26T20:50:59.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.202+0000 7fd010ff9640 1 -- 172.21.15.73:0/2810494177 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd004011790 con 0x7fd014072fb0 2024-04-26T20:50:59.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.203+0000 7fd01a685640 1 -- 172.21.15.73:0/2810494177 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd01409d450 con 0x7fd014072fb0 2024-04-26T20:50:59.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.203+0000 7f9b44ff9640 1 -- 172.21.15.73:0/3536493397 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b34017070 con 0x7f9b400defb0 2024-04-26T20:50:59.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.203+0000 7fd01a685640 1 -- 172.21.15.73:0/2810494177 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd01409da10 con 0x7fd014072fb0 2024-04-26T20:50:59.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.203+0000 7f9b4d4fd640 1 -- 172.21.15.73:0/3536493397 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b4004e420 con 0x7f9b400defb0 2024-04-26T20:50:59.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.203+0000 7f9b4d4fd640 1 -- 172.21.15.73:0/3536493397 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b4004e9e0 con 0x7f9b400defb0 2024-04-26T20:50:59.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.203+0000 7f211b5c8640 1 -- 172.21.15.73:0/1316384105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21000039b0 msgr2=0x7f2100003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:59.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.203+0000 7f211b5c8640 1 --2- 172.21.15.73:0/1316384105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21000039b0 0x7f2100003dd0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f211406b190 tx=0x7f2100005830 comp rx=0 tx=0).stop 2024-04-26T20:50:59.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.203+0000 7f2119b65640 1 -- 172.21.15.73:0/1316384105 reap_dead start 2024-04-26T20:50:59.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.204+0000 7f9b44ff9640 1 -- 172.21.15.73:0/3536493397 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b34005ce0 con 0x7f9b400defb0 2024-04-26T20:50:59.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.204+0000 7f9b44ff9640 1 -- 172.21.15.73:0/3536493397 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b34007270 con 0x7f9b400defb0 2024-04-26T20:50:59.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.204+0000 7f211b5c8640 1 -- 172.21.15.73:0/1316384105 shutdown_connections 2024-04-26T20:50:59.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.205+0000 7f211b5c8640 1 -- 172.21.15.73:0/1316384105 >> 172.21.15.73:0/1316384105 conn(0x7f210c053750 msgr2=0x7f210c053b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:59.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.205+0000 7f9b4d4fd640 1 -- 172.21.15.73:0/3536493397 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b40008e20 con 0x7f9b400defb0 2024-04-26T20:50:59.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.205+0000 7f211b5c8640 1 -- 172.21.15.73:0/1316384105 shutdown_connections 2024-04-26T20:50:59.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.205+0000 7f211b5c8640 1 -- 172.21.15.73:0/1316384105 wait complete. 2024-04-26T20:50:59.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.205+0000 7f581affd640 1 -- 172.21.15.73:0/2185217826 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5820032440 con 0x7f58240cd030 2024-04-26T20:50:59.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.206+0000 7f581affd640 1 --2- 172.21.15.73:0/2185217826 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f580806c9c0 0x7f580806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.206+0000 7f581affd640 1 -- 172.21.15.73:0/2185217826 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f58200ad240 con 0x7f58240cd030 2024-04-26T20:50:59.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.207+0000 7f58298bd640 1 --2- 172.21.15.73:0/2185217826 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f580806c9c0 0x7f580806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:59.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.207+0000 7f211b5c8640 1 Processor -- start 2024-04-26T20:50:59.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.207+0000 7fd01a685640 1 -- 172.21.15.73:0/2810494177 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0140694f0 con 0x7fd014072fb0 2024-04-26T20:50:59.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.207+0000 7f211b5c8640 1 -- start start 2024-04-26T20:50:59.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.207+0000 7fd010ff9640 1 -- 172.21.15.73:0/2810494177 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd004017070 con 0x7fd014072fb0 2024-04-26T20:50:59.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.207+0000 7fd010ff9640 1 -- 172.21.15.73:0/2810494177 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd004011b30 con 0x7fd014072fb0 2024-04-26T20:50:59.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.207+0000 7f58298bd640 1 --2- 172.21.15.73:0/2185217826 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f580806c9c0 0x7f580806ee80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f58240ca260 tx=0x7f5814009500 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:59.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.209+0000 7f211b5c8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f21000039b0 0x7f210c0e0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.209+0000 7f211b5c8640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f210c0e4fe0 0x7f210c04dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.209+0000 7f211b5c8640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f210c0e5460 0x7f210c04e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.209+0000 7f2119364640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f21000039b0 0x7f210c0e0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:59.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.209+0000 7f2119364640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f21000039b0 0x7f210c0e0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.73:34286/0 (socket says 172.21.15.73:34286) 2024-04-26T20:50:59.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.209+0000 7f2118b63640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f210c0e4fe0 0x7f210c04dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:59.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.209+0000 7f2118b63640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f210c0e4fe0 0x7f210c04dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.73:38126/0 (socket says 172.21.15.73:38126) 2024-04-26T20:50:59.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.209+0000 7f2119364640 1 -- 172.21.15.73:0/600725927 learned_addr learned my addr 172.21.15.73:0/600725927 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:59.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.209+0000 7f211b5c8640 1 -- 172.21.15.73:0/600725927 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f210c0517e0 con 0x7f210c0e5460 2024-04-26T20:50:59.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.209+0000 7f211b5c8640 1 -- 172.21.15.73:0/600725927 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f210c055bf0 con 0x7f21000039b0 2024-04-26T20:50:59.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.209+0000 7f211b5c8640 1 -- 172.21.15.73:0/600725927 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f210c051960 con 0x7f210c0e4fe0 2024-04-26T20:50:59.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.209+0000 7f2119b65640 1 --2- 172.21.15.73:0/600725927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f210c0e5460 0x7f210c04e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:59.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.210+0000 7f2119b65640 1 -- 172.21.15.73:0/600725927 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f210c0e4fe0 msgr2=0x7f210c04dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:59.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.210+0000 7f2119b65640 1 --2- 172.21.15.73:0/600725927 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f210c0e4fe0 0x7f210c04dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.210+0000 7f2119b65640 1 -- 172.21.15.73:0/600725927 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f21000039b0 msgr2=0x7f210c0e0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:59.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.210+0000 7f2119b65640 1 --2- 172.21.15.73:0/600725927 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f21000039b0 0x7f210c0e0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.210+0000 7f2119b65640 1 -- 172.21.15.73:0/600725927 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f210c0e5a30 con 0x7f210c0e5460 2024-04-26T20:50:59.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.210+0000 7f2119364640 1 --2- 172.21.15.73:0/600725927 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f21000039b0 0x7f210c0e0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:50:59.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.210+0000 7f2119b65640 1 --2- 172.21.15.73:0/600725927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f210c0e5460 0x7f210c04e2e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f210c0e7e50 tx=0x7f211000c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:59.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.214+0000 7f2109ffb640 1 -- 172.21.15.73:0/600725927 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2110011800 con 0x7f210c0e5460 2024-04-26T20:50:59.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.214+0000 7f211b5c8640 1 -- 172.21.15.73:0/600725927 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f210c0e5d10 con 0x7f210c0e5460 2024-04-26T20:50:59.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.215+0000 7f581affd640 1 -- 172.21.15.73:0/2185217826 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f582003c3a0 con 0x7f58240cd030 2024-04-26T20:50:59.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.215+0000 7f211b5c8640 1 -- 172.21.15.73:0/600725927 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f210c0e62d0 con 0x7f210c0e5460 2024-04-26T20:50:59.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.216+0000 7f2109ffb640 1 -- 172.21.15.73:0/600725927 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2110018070 con 0x7f210c0e5460 2024-04-26T20:50:59.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.217+0000 7f2109ffb640 1 -- 172.21.15.73:0/600725927 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2110004220 con 0x7f210c0e5460 2024-04-26T20:50:59.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.217+0000 7fd010ff9640 1 -- 172.21.15.73:0/2810494177 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd004011cd0 con 0x7fd014072fb0 2024-04-26T20:50:59.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.218+0000 7fd010ff9640 1 --2- 172.21.15.73:0/2810494177 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd00006c8f0 0x7fd00006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.218+0000 7f9b44ff9640 1 -- 172.21.15.73:0/3536493397 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9b3401a070 con 0x7f9b400defb0 2024-04-26T20:50:59.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.218+0000 7fd010ff9640 1 -- 172.21.15.73:0/2810494177 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd00408c6e0 con 0x7fd014072fb0 2024-04-26T20:50:59.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.218+0000 7f9b44ff9640 1 --2- 172.21.15.73:0/3536493397 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f9b2c06c8f0 0x7f9b2c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.218+0000 7f9b44ff9640 1 -- 172.21.15.73:0/3536493397 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f9b3408b5d0 con 0x7f9b400defb0 2024-04-26T20:50:59.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.219+0000 7f9b47fff640 1 --2- 172.21.15.73:0/3536493397 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f9b2c06c8f0 0x7f9b2c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:59.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.219+0000 7f9b47fff640 1 --2- 172.21.15.73:0/3536493397 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f9b2c06c8f0 0x7f9b2c06edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c034000 tx=0x7f9b3c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:59.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.220+0000 7fd013fff640 1 --2- 172.21.15.73:0/2810494177 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd00006c8f0 0x7fd00006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:59.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.220+0000 7f211b5c8640 1 -- 172.21.15.73:0/600725927 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20f4005290 con 0x7f210c0e5460 2024-04-26T20:50:59.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.220+0000 7fd013fff640 1 --2- 172.21.15.73:0/2810494177 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd00006c8f0 0x7fd00006edb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd008007fd0 tx=0x7fd008007de0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:59.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.226+0000 7f9b44ff9640 1 -- 172.21.15.73:0/3536493397 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9b34021020 con 0x7f9b400defb0 2024-04-26T20:50:59.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.228+0000 7f2109ffb640 1 -- 172.21.15.73:0/600725927 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f21100119a0 con 0x7f210c0e5460 2024-04-26T20:50:59.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.228+0000 7f2109ffb640 1 --2- 172.21.15.73:0/600725927 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f20f806c920 0x7f20f806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.229+0000 7f2119364640 1 --2- 172.21.15.73:0/600725927 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f20f806c920 0x7f20f806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:59.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.230+0000 7f2119364640 1 --2- 172.21.15.73:0/600725927 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f20f806c920 0x7f20f806ede0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f211406b960 tx=0x7f21140ac000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:59.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.236+0000 7fd010ff9640 1 -- 172.21.15.73:0/2810494177 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd00405a8b0 con 0x7fd014072fb0 2024-04-26T20:50:59.256 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.242+0000 7f2109ffb640 1 -- 172.21.15.73:0/600725927 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f2110057a80 con 0x7f210c0e5460 2024-04-26T20:50:59.256 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.245+0000 7f2109ffb640 1 -- 172.21.15.73:0/600725927 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2110022020 con 0x7f210c0e5460 2024-04-26T20:50:59.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.542+0000 7f4fed899640 1 -- 172.21.15.73:0/3457945439 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f4fc8005850 con 0x7f4fe814ad40 2024-04-26T20:50:59.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.543+0000 7f4fcffff640 1 -- 172.21.15.73:0/3457945439 <== mon.1 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4fd807ddf0 con 0x7f4fe814ad40 2024-04-26T20:50:59.544 INFO:teuthology.orchestra.run.smithi073.stdout:55834574858 2024-04-26T20:50:59.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.546+0000 7f4fcdffb640 1 -- 172.21.15.73:0/3457945439 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f4fd406c9c0 msgr2=0x7f4fd406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:59.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.546+0000 7f4fcdffb640 1 --2- 172.21.15.73:0/3457945439 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f4fd406c9c0 0x7f4fd406ee80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4fe0007fd0 tx=0x7f4fe0009040 comp rx=0 tx=0).stop 2024-04-26T20:50:59.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.546+0000 7f4fcdffb640 1 -- 172.21.15.73:0/3457945439 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4fe814ad40 msgr2=0x7f4fe8153140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:59.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.546+0000 7f4fcdffb640 1 --2- 172.21.15.73:0/3457945439 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4fe814ad40 0x7f4fe8153140 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4fd800f220 tx=0x7f4fd8004730 comp rx=0 tx=0).stop 2024-04-26T20:50:59.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.547+0000 7f4fcdffb640 1 -- 172.21.15.73:0/3457945439 shutdown_connections 2024-04-26T20:50:59.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.547+0000 7f4fcdffb640 1 --2- 172.21.15.73:0/3457945439 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4fe80c88d0 0x7f4fe81fd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.547+0000 7f4fcdffb640 1 --2- 172.21.15.73:0/3457945439 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f4fd406c9c0 0x7f4fd406ee80 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.547+0000 7f4fcdffb640 1 --2- 172.21.15.73:0/3457945439 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4fe8153680 0x7f4fe80c8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.547+0000 7f4fcdffb640 1 --2- 172.21.15.73:0/3457945439 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4fe814ad40 0x7f4fe8153140 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.547+0000 7f4fcdffb640 1 -- 172.21.15.73:0/3457945439 >> 172.21.15.73:0/3457945439 conn(0x7f4fe80d09f0 msgr2=0x7f4fe80d0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:59.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.547+0000 7f4fcdffb640 1 -- 172.21.15.73:0/3457945439 shutdown_connections 2024-04-26T20:50:59.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.547+0000 7f4fcdffb640 1 -- 172.21.15.73:0/3457945439 wait complete. 2024-04-26T20:50:59.565 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.0 2024-04-26T20:50:59.565 DEBUG:teuthology.parallel:result is None 2024-04-26T20:50:59.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.599+0000 7fd01a685640 1 -- 172.21.15.73:0/2810494177 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fd014159b40 con 0x7fd014072fb0 2024-04-26T20:50:59.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.600+0000 7fd010ff9640 1 -- 172.21.15.73:0/2810494177 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd00405a1b0 con 0x7fd014072fb0 2024-04-26T20:50:59.601 INFO:teuthology.orchestra.run.smithi073.stdout:55834574859 2024-04-26T20:50:59.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.601+0000 7fd01a685640 1 -- 172.21.15.73:0/2810494177 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd00006c8f0 msgr2=0x7fd00006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:59.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.602+0000 7fd01a685640 1 --2- 172.21.15.73:0/2810494177 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd00006c8f0 0x7fd00006edb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd008007fd0 tx=0x7fd008007de0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.602+0000 7fd01a685640 1 -- 172.21.15.73:0/2810494177 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd014072fb0 msgr2=0x7fd014158140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:59.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.602+0000 7fd01a685640 1 --2- 172.21.15.73:0/2810494177 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd014072fb0 0x7fd014158140 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd00400c440 tx=0x7fd00400c900 comp rx=0 tx=0).stop 2024-04-26T20:50:59.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.602+0000 7fd01a685640 1 -- 172.21.15.73:0/2810494177 shutdown_connections 2024-04-26T20:50:59.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.602+0000 7fd01a685640 1 --2- 172.21.15.73:0/2810494177 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd014072fb0 0x7fd014158140 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.602+0000 7fd01a685640 1 --2- 172.21.15.73:0/2810494177 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd00006c8f0 0x7fd00006edb0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.602+0000 7fd01a685640 1 --2- 172.21.15.73:0/2810494177 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd0140b56c0 0x7fd01415ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.602+0000 7fd01a685640 1 --2- 172.21.15.73:0/2810494177 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd014070750 0x7fd0141ee310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.602+0000 7fd01a685640 1 -- 172.21.15.73:0/2810494177 >> 172.21.15.73:0/2810494177 conn(0x7fd0140b39b0 msgr2=0x7fd0140b4aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:59.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.602+0000 7fd01a685640 1 -- 172.21.15.73:0/2810494177 shutdown_connections 2024-04-26T20:50:59.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.602+0000 7fd01a685640 1 -- 172.21.15.73:0/2810494177 wait complete. 2024-04-26T20:50:59.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.606+0000 7f52b4ff9640 1 -- 172.21.15.73:0/3677451779 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f52c00d2de0 con 0x7f52c00df390 2024-04-26T20:50:59.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.606+0000 7f45ecf3b640 1 -- 172.21.15.73:0/4267575452 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f45c8005850 con 0x7f45d00052e0 2024-04-26T20:50:59.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.606+0000 7f52b6ffd640 1 -- 172.21.15.73:0/3677451779 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f52bc058cf0 con 0x7f52c00df390 2024-04-26T20:50:59.607 INFO:teuthology.orchestra.run.smithi073.stdout:55834574859 2024-04-26T20:50:59.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.606+0000 7f45cffff640 1 -- 172.21.15.73:0/4267575452 <== mon.2 v2:172.21.15.120:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f45d805a4c0 con 0x7f45d00052e0 2024-04-26T20:50:59.608 INFO:teuthology.orchestra.run.smithi073.stdout:55834574858 2024-04-26T20:50:59.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.610+0000 7f45ecf3b640 1 -- 172.21.15.73:0/4267575452 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f45d406c9f0 msgr2=0x7f45d406eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:59.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.610+0000 7f45ecf3b640 1 --2- 172.21.15.73:0/4267575452 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f45d406c9f0 0x7f45d406eeb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f45e8166890 tx=0x7f45dc008040 comp rx=0 tx=0).stop 2024-04-26T20:50:59.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.610+0000 7f45ecf3b640 1 -- 172.21.15.73:0/4267575452 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f45d00052e0 msgr2=0x7f45e81ecc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:59.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.610+0000 7f45ecf3b640 1 --2- 172.21.15.73:0/4267575452 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f45d00052e0 0x7f45e81ecc80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f45d800e870 tx=0x7f45d800ed30 comp rx=0 tx=0).stop 2024-04-26T20:50:59.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.610+0000 7f45ecf3b640 1 -- 172.21.15.73:0/4267575452 shutdown_connections 2024-04-26T20:50:59.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.610+0000 7f45ecf3b640 1 --2- 172.21.15.73:0/4267575452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45e8205770 0x7f45e8204dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.610+0000 7f45ecf3b640 1 --2- 172.21.15.73:0/4267575452 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f45d406c9f0 0x7f45d406eeb0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.610+0000 7f45ecf3b640 1 --2- 172.21.15.73:0/4267575452 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f45e82008a0 0x7f45e8200d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.610+0000 7f45ecf3b640 1 --2- 172.21.15.73:0/4267575452 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f45d00052e0 0x7f45e81ecc80 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.610+0000 7f45ecf3b640 1 -- 172.21.15.73:0/4267575452 >> 172.21.15.73:0/4267575452 conn(0x7f45e80d4dd0 msgr2=0x7f45e80b9f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:59.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.610+0000 7f52b4ff9640 1 -- 172.21.15.73:0/3677451779 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f52a806c9c0 msgr2=0x7f52a806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:59.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.610+0000 7f52b4ff9640 1 --2- 172.21.15.73:0/3677451779 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f52a806c9c0 0x7f52a806ee80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f52c0008680 tx=0x7f52c80ab000 comp rx=0 tx=0).stop 2024-04-26T20:50:59.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.610+0000 7f52b4ff9640 1 -- 172.21.15.73:0/3677451779 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52c00df390 msgr2=0x7f52c00d10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:59.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.610+0000 7f52b4ff9640 1 --2- 172.21.15.73:0/3677451779 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52c00df390 0x7f52c00d10a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f52bc00c410 tx=0x7f52bc00c8d0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.611+0000 7f45ecf3b640 1 -- 172.21.15.73:0/4267575452 shutdown_connections 2024-04-26T20:50:59.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.611+0000 7f52b4ff9640 1 -- 172.21.15.73:0/3677451779 shutdown_connections 2024-04-26T20:50:59.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.611+0000 7f52b4ff9640 1 --2- 172.21.15.73:0/3677451779 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52c00df390 0x7f52c00d10a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.611+0000 7f52b4ff9640 1 --2- 172.21.15.73:0/3677451779 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f52a806c9c0 0x7f52a806ee80 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.611+0000 7f52b4ff9640 1 --2- 172.21.15.73:0/3677451779 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f52c00382c0 0x7f52c004d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.611+0000 7f52b4ff9640 1 --2- 172.21.15.73:0/3677451779 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f52c000aaa0 0x7f52c00e0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.611+0000 7f52b4ff9640 1 -- 172.21.15.73:0/3677451779 >> 172.21.15.73:0/3677451779 conn(0x7f52c0009fb0 msgr2=0x7f52c0052e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:59.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.611+0000 7f45ecf3b640 1 -- 172.21.15.73:0/4267575452 wait complete. 2024-04-26T20:50:59.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.611+0000 7f52b4ff9640 1 -- 172.21.15.73:0/3677451779 shutdown_connections 2024-04-26T20:50:59.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.611+0000 7f52b4ff9640 1 -- 172.21.15.73:0/3677451779 wait complete. 2024-04-26T20:50:59.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.616+0000 7f8cd89c6640 1 -- 172.21.15.73:0/478819180 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f8cd40694f0 con 0x7f8cd40b02b0 2024-04-26T20:50:59.618 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.3 2024-04-26T20:50:59.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.617+0000 7f8cc2ffd640 1 -- 172.21.15.73:0/478819180 <== mon.1 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8ccc021020 con 0x7f8cd40b02b0 2024-04-26T20:50:59.618 DEBUG:teuthology.parallel:result is None 2024-04-26T20:50:59.619 INFO:teuthology.orchestra.run.smithi073.stdout:55834574858 2024-04-26T20:50:59.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.620+0000 7f8cd89c6640 1 -- 172.21.15.73:0/478819180 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f8cb406c8f0 msgr2=0x7f8cb406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:59.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.620+0000 7f8cd89c6640 1 --2- 172.21.15.73:0/478819180 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f8cb406c8f0 0x7f8cb406edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8cc403d000 tx=0x7f8cc4037000 comp rx=0 tx=0).stop 2024-04-26T20:50:59.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.620+0000 7f8cd89c6640 1 -- 172.21.15.73:0/478819180 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8cd40b02b0 msgr2=0x7f8cd40b5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:59.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.620+0000 7f8cd89c6640 1 --2- 172.21.15.73:0/478819180 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8cd40b02b0 0x7f8cd40b5890 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc00cb20 tx=0x7f8ccc0020d0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.620+0000 7f8cd89c6640 1 -- 172.21.15.73:0/478819180 shutdown_connections 2024-04-26T20:50:59.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.620+0000 7f8cd89c6640 1 --2- 172.21.15.73:0/478819180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8cd40cb230 0x7f8cd40b5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.620+0000 7f8cd89c6640 1 --2- 172.21.15.73:0/478819180 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f8cb406c8f0 0x7f8cb406edb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.620+0000 7f8cd89c6640 1 --2- 172.21.15.73:0/478819180 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8cd40b02b0 0x7f8cd40b5890 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.620+0000 7f8cd89c6640 1 --2- 172.21.15.73:0/478819180 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8cd4070350 0x7f8cd408c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.620+0000 7f8cd89c6640 1 -- 172.21.15.73:0/478819180 >> 172.21.15.73:0/478819180 conn(0x7f8cd40d0be0 msgr2=0x7f8cd408ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:59.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.620+0000 7f8cd89c6640 1 -- 172.21.15.73:0/478819180 shutdown_connections 2024-04-26T20:50:59.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.620+0000 7f8cd89c6640 1 -- 172.21.15.73:0/478819180 wait complete. 2024-04-26T20:50:59.626 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.4 2024-04-26T20:50:59.626 DEBUG:teuthology.parallel:result is None 2024-04-26T20:50:59.630 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.1 2024-04-26T20:50:59.630 DEBUG:teuthology.parallel:result is None 2024-04-26T20:50:59.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.634+0000 7f582c322640 1 -- 172.21.15.73:0/2185217826 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f58240d0ed0 con 0x7f58240cd030 2024-04-26T20:50:59.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.634+0000 7f9b4d4fd640 1 -- 172.21.15.73:0/3536493397 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f9b400057e0 con 0x7f9b400defb0 2024-04-26T20:50:59.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.634+0000 7f9b44ff9640 1 -- 172.21.15.73:0/3536493397 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9b340590e0 con 0x7f9b400defb0 2024-04-26T20:50:59.635 INFO:teuthology.orchestra.run.smithi073.stdout:55834574859 2024-04-26T20:50:59.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.634+0000 7f581affd640 1 -- 172.21.15.73:0/2185217826 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f582007b310 con 0x7f58240cd030 2024-04-26T20:50:59.636 INFO:teuthology.orchestra.run.smithi073.stdout:55834574859 2024-04-26T20:50:59.637 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.7 2024-04-26T20:50:59.638 DEBUG:teuthology.parallel:result is None 2024-04-26T20:50:59.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.636+0000 7f9b4d4fd640 1 -- 172.21.15.73:0/3536493397 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f9b2c06c8f0 msgr2=0x7f9b2c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:59.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.636+0000 7f9b4d4fd640 1 --2- 172.21.15.73:0/3536493397 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f9b2c06c8f0 0x7f9b2c06edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c034000 tx=0x7f9b3c02e000 comp rx=0 tx=0).stop 2024-04-26T20:50:59.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.636+0000 7f9b4d4fd640 1 -- 172.21.15.73:0/3536493397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b400defb0 msgr2=0x7f9b400d6d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:59.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.636+0000 7f9b4d4fd640 1 --2- 172.21.15.73:0/3536493397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b400defb0 0x7f9b400d6d60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9b3400c4d0 tx=0x7f9b3400c990 comp rx=0 tx=0).stop 2024-04-26T20:50:59.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.637+0000 7f9b4d4fd640 1 -- 172.21.15.73:0/3536493397 shutdown_connections 2024-04-26T20:50:59.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.637+0000 7f9b4d4fd640 1 --2- 172.21.15.73:0/3536493397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b400defb0 0x7f9b400d6d60 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.637+0000 7f9b4d4fd640 1 --2- 172.21.15.73:0/3536493397 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f9b2c06c8f0 0x7f9b2c06edb0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.637+0000 7f9b4d4fd640 1 --2- 172.21.15.73:0/3536493397 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9b400e3490 0x7f9b400df8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.637+0000 7f9b4d4fd640 1 --2- 172.21.15.73:0/3536493397 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9b40004b00 0x7f9b40037380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.637+0000 7f9b4d4fd640 1 -- 172.21.15.73:0/3536493397 >> 172.21.15.73:0/3536493397 conn(0x7f9b4000a400 msgr2=0x7f9b400e5320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:59.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.637+0000 7f9b4d4fd640 1 -- 172.21.15.73:0/3536493397 shutdown_connections 2024-04-26T20:50:59.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.637+0000 7f9b4d4fd640 1 -- 172.21.15.73:0/3536493397 wait complete. 2024-04-26T20:50:59.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.637+0000 7f582c322640 1 -- 172.21.15.73:0/2185217826 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f580806c9c0 msgr2=0x7f580806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:59.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.637+0000 7f582c322640 1 --2- 172.21.15.73:0/2185217826 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f580806c9c0 0x7f580806ee80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f58240ca260 tx=0x7f5814009500 comp rx=0 tx=0).stop 2024-04-26T20:50:59.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.637+0000 7f582c322640 1 -- 172.21.15.73:0/2185217826 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58240cd030 msgr2=0x7f582414f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:59.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.637+0000 7f582c322640 1 --2- 172.21.15.73:0/2185217826 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58240cd030 0x7f582414f590 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5820031000 tx=0x7f58200046c0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.637+0000 7f582c322640 1 -- 172.21.15.73:0/2185217826 shutdown_connections 2024-04-26T20:50:59.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.637+0000 7f582c322640 1 --2- 172.21.15.73:0/2185217826 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f580806c9c0 0x7f580806ee80 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.637+0000 7f582c322640 1 --2- 172.21.15.73:0/2185217826 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5824166f60 0x7f58240b52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.637+0000 7f582c322640 1 --2- 172.21.15.73:0/2185217826 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5824163920 0x7f582414e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.637+0000 7f582c322640 1 --2- 172.21.15.73:0/2185217826 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58240cd030 0x7f582414f590 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.637+0000 7f582c322640 1 -- 172.21.15.73:0/2185217826 >> 172.21.15.73:0/2185217826 conn(0x7f58240d09f0 msgr2=0x7f58240c5b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:59.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.638+0000 7f582c322640 1 -- 172.21.15.73:0/2185217826 shutdown_connections 2024-04-26T20:50:59.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.638+0000 7f582c322640 1 -- 172.21.15.73:0/2185217826 wait complete. 2024-04-26T20:50:59.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.640+0000 7f211b5c8640 1 -- 172.21.15.73:0/600725927 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f20f4005850 con 0x7f210c0e5460 2024-04-26T20:50:59.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.640+0000 7f2109ffb640 1 -- 172.21.15.73:0/600725927 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f21100599e0 con 0x7f210c0e5460 2024-04-26T20:50:59.642 INFO:teuthology.orchestra.run.smithi073.stdout:55834574858 2024-04-26T20:50:59.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.644+0000 7f211b5c8640 1 -- 172.21.15.73:0/600725927 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f20f806c920 msgr2=0x7f20f806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:59.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.644+0000 7f211b5c8640 1 --2- 172.21.15.73:0/600725927 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f20f806c920 0x7f20f806ede0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f211406b960 tx=0x7f21140ac000 comp rx=0 tx=0).stop 2024-04-26T20:50:59.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.644+0000 7f211b5c8640 1 -- 172.21.15.73:0/600725927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f210c0e5460 msgr2=0x7f210c04e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:59.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.644+0000 7f211b5c8640 1 --2- 172.21.15.73:0/600725927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f210c0e5460 0x7f210c04e2e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f210c0e7e50 tx=0x7f211000c870 comp rx=0 tx=0).stop 2024-04-26T20:50:59.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.644+0000 7f211b5c8640 1 -- 172.21.15.73:0/600725927 shutdown_connections 2024-04-26T20:50:59.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.644+0000 7f211b5c8640 1 --2- 172.21.15.73:0/600725927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f210c0e5460 0x7f210c04e2e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.644+0000 7f211b5c8640 1 --2- 172.21.15.73:0/600725927 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f20f806c920 0x7f20f806ede0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.644+0000 7f211b5c8640 1 --2- 172.21.15.73:0/600725927 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f210c0e4fe0 0x7f210c04dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.644+0000 7f211b5c8640 1 --2- 172.21.15.73:0/600725927 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f21000039b0 0x7f210c0e0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.644+0000 7f211b5c8640 1 -- 172.21.15.73:0/600725927 >> 172.21.15.73:0/600725927 conn(0x7f210c053750 msgr2=0x7f210c0e7ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:59.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.644+0000 7f211b5c8640 1 -- 172.21.15.73:0/600725927 shutdown_connections 2024-04-26T20:50:59.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.644+0000 7f211b5c8640 1 -- 172.21.15.73:0/600725927 wait complete. 2024-04-26T20:50:59.651 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.5 2024-04-26T20:50:59.651 DEBUG:teuthology.parallel:result is None 2024-04-26T20:50:59.654 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.6 2024-04-26T20:50:59.654 DEBUG:teuthology.parallel:result is None 2024-04-26T20:50:59.658 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.2 2024-04-26T20:50:59.658 DEBUG:teuthology.parallel:result is None 2024-04-26T20:50:59.658 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-26T20:50:59.659 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:50:59.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.795+0000 7f8b1f015640 1 Processor -- start 2024-04-26T20:50:59.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.795+0000 7f8b1f015640 1 -- start start 2024-04-26T20:50:59.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.796+0000 7f8b1f015640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8b1815f1d0 0x7f8b1806f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.796+0000 7f8b1f015640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f8b1806c240 0x7f8b1806c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.796+0000 7f8b1f015640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8b180726e0 0x7f8b18072b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.796+0000 7f8b1f015640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f8b1806cbe0 con 0x7f8b1815f1d0 2024-04-26T20:50:59.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.796+0000 7f8b1f015640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f8b180ae8c0 con 0x7f8b1806c240 2024-04-26T20:50:59.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.796+0000 7f8b1f015640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f8b180b4480 con 0x7f8b180726e0 2024-04-26T20:50:59.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.796+0000 7f8b1cdb1640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8b1815f1d0 0x7f8b1806f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:59.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.796+0000 7f8b1cdb1640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8b1815f1d0 0x7f8b1806f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42560/0 (socket says 172.21.15.73:42560) 2024-04-26T20:50:59.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.796+0000 7f8b1cdb1640 1 -- 172.21.15.73:0/2871137030 learned_addr learned my addr 172.21.15.73:0/2871137030 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:59.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.796+0000 7f8b1d5b2640 1 --2- 172.21.15.73:0/2871137030 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8b180726e0 0x7f8b18072b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:59.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.796+0000 7f8b1cdb1640 1 -- 172.21.15.73:0/2871137030 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8b180726e0 msgr2=0x7f8b18072b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:59.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.796+0000 7f8b17fff640 1 --2- 172.21.15.73:0/2871137030 >> v2:172.21.15.120:3300/0 conn(0x7f8b1806c240 0x7f8b1806c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:59.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.796+0000 7f8b1cdb1640 1 --2- 172.21.15.73:0/2871137030 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8b180726e0 0x7f8b18072b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.796+0000 7f8b1cdb1640 1 -- 172.21.15.73:0/2871137030 >> v2:172.21.15.120:3300/0 conn(0x7f8b1806c240 msgr2=0x7f8b1806c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:50:59.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.796+0000 7f8b1cdb1640 1 --2- 172.21.15.73:0/2871137030 >> v2:172.21.15.120:3300/0 conn(0x7f8b1806c240 0x7f8b1806c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.796+0000 7f8b1cdb1640 1 -- 172.21.15.73:0/2871137030 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b180b4720 con 0x7f8b1815f1d0 2024-04-26T20:50:59.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.796+0000 7f8b1d5b2640 1 --2- 172.21.15.73:0/2871137030 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8b180726e0 0x7f8b18072b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:50:59.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.796+0000 7f8b17fff640 1 --2- 172.21.15.73:0/2871137030 >> v2:172.21.15.120:3300/0 conn(0x7f8b1806c240 0x7f8b1806c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:50:59.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.797+0000 7f8b1cdb1640 1 --2- 172.21.15.73:0/2871137030 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8b1815f1d0 0x7f8b1806f170 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8b18074060 tx=0x7f8b0c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f68920416f28a674 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:59.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.797+0000 7f8b177fe640 1 -- 172.21.15.73:0/2871137030 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c037070 con 0x7f8b1815f1d0 2024-04-26T20:50:59.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.797+0000 7f8b177fe640 1 -- 172.21.15.73:0/2871137030 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b0c032440 con 0x7f8b1815f1d0 2024-04-26T20:50:59.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.797+0000 7f8b177fe640 1 -- 172.21.15.73:0/2871137030 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c032810 con 0x7f8b1815f1d0 2024-04-26T20:50:59.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.797+0000 7f8b1f015640 1 -- 172.21.15.73:0/2871137030 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8b1815f1d0 msgr2=0x7f8b1806f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:59.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.797+0000 7f8b1f015640 1 --2- 172.21.15.73:0/2871137030 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8b1815f1d0 0x7f8b1806f170 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8b18074060 tx=0x7f8b0c02a1e0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.797+0000 7f8b1f015640 1 -- 172.21.15.73:0/2871137030 shutdown_connections 2024-04-26T20:50:59.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.797+0000 7f8b1f015640 1 --2- 172.21.15.73:0/2871137030 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8b180726e0 0x7f8b18072b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.797+0000 7f8b1f015640 1 --2- 172.21.15.73:0/2871137030 >> v2:172.21.15.120:3300/0 conn(0x7f8b1806c240 0x7f8b1806c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.797+0000 7f8b1f015640 1 --2- 172.21.15.73:0/2871137030 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8b1815f1d0 0x7f8b1806f170 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.797+0000 7f8b1f015640 1 -- 172.21.15.73:0/2871137030 >> 172.21.15.73:0/2871137030 conn(0x7f8b180b3180 msgr2=0x7f8b180b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:59.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.797+0000 7f8b1f015640 1 -- 172.21.15.73:0/2871137030 shutdown_connections 2024-04-26T20:50:59.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.797+0000 7f8b1f015640 1 -- 172.21.15.73:0/2871137030 wait complete. 2024-04-26T20:50:59.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.798+0000 7f8b1f015640 1 Processor -- start 2024-04-26T20:50:59.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.799+0000 7f8b1f015640 1 -- start start 2024-04-26T20:50:59.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.799+0000 7f8b1f015640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8b1806c240 0x7f8b181e7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.799+0000 7f8b1f015640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8b180726e0 0x7f8b18160b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.799+0000 7f8b1f015640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8b1815f1d0 0x7f8b1815aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.799+0000 7f8b1f015640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8b180b4fc0 con 0x7f8b1815f1d0 2024-04-26T20:50:59.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.799+0000 7f8b1f015640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8b180b4ce0 con 0x7f8b1806c240 2024-04-26T20:50:59.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.799+0000 7f8b1f015640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f8b1806f6b0 con 0x7f8b180726e0 2024-04-26T20:50:59.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.799+0000 7f8b1d5b2640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8b1815f1d0 0x7f8b1815aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:59.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.800+0000 7f8b1d5b2640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8b1815f1d0 0x7f8b1815aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42574/0 (socket says 172.21.15.73:42574) 2024-04-26T20:50:59.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.800+0000 7f8b1d5b2640 1 -- 172.21.15.73:0/2856160832 learned_addr learned my addr 172.21.15.73:0/2856160832 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:50:59.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.800+0000 7f8b1cdb1640 1 --2- 172.21.15.73:0/2856160832 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8b1806c240 0x7f8b181e7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:59.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.800+0000 7f8b17fff640 1 --2- 172.21.15.73:0/2856160832 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8b180726e0 0x7f8b18160b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:59.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.800+0000 7f8b1d5b2640 1 -- 172.21.15.73:0/2856160832 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8b180726e0 msgr2=0x7f8b18160b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:59.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.800+0000 7f8b1d5b2640 1 --2- 172.21.15.73:0/2856160832 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8b180726e0 0x7f8b18160b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.800+0000 7f8b1d5b2640 1 -- 172.21.15.73:0/2856160832 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8b1806c240 msgr2=0x7f8b181e7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:59.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.800+0000 7f8b1d5b2640 1 --2- 172.21.15.73:0/2856160832 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8b1806c240 0x7f8b181e7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.800+0000 7f8b1d5b2640 1 -- 172.21.15.73:0/2856160832 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b181593a0 con 0x7f8b1815f1d0 2024-04-26T20:50:59.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.800+0000 7f8b1d5b2640 1 --2- 172.21.15.73:0/2856160832 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8b1815f1d0 0x7f8b1815aa50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8b1000cef0 tx=0x7f8b1000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:59.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.800+0000 7f8b157fa640 1 -- 172.21.15.73:0/2856160832 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b1001a070 con 0x7f8b1815f1d0 2024-04-26T20:50:59.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.800+0000 7f8b157fa640 1 -- 172.21.15.73:0/2856160832 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b1000d860 con 0x7f8b1815f1d0 2024-04-26T20:50:59.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.800+0000 7f8b1f015640 1 -- 172.21.15.73:0/2856160832 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b18159680 con 0x7f8b1815f1d0 2024-04-26T20:50:59.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.800+0000 7f8b157fa640 1 -- 172.21.15.73:0/2856160832 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b1000dc50 con 0x7f8b1815f1d0 2024-04-26T20:50:59.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.800+0000 7f8b1f015640 1 -- 172.21.15.73:0/2856160832 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b18159c40 con 0x7f8b1815f1d0 2024-04-26T20:50:59.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.801+0000 7f8b1f015640 1 -- 172.21.15.73:0/2856160832 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b180694f0 con 0x7f8b1815f1d0 2024-04-26T20:50:59.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.802+0000 7f8b157fa640 1 -- 172.21.15.73:0/2856160832 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8b1001d070 con 0x7f8b1815f1d0 2024-04-26T20:50:59.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.802+0000 7f8b157fa640 1 --2- 172.21.15.73:0/2856160832 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f8afc06c8f0 0x7f8afc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:59.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.802+0000 7f8b157fa640 1 -- 172.21.15.73:0/2856160832 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f8b1008c710 con 0x7f8b1815f1d0 2024-04-26T20:50:59.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.802+0000 7f8b1cdb1640 1 --2- 172.21.15.73:0/2856160832 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f8afc06c8f0 0x7f8afc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:59.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.802+0000 7f8b1cdb1640 1 --2- 172.21.15.73:0/2856160832 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f8afc06c8f0 0x7f8afc06edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c034000 tx=0x7f8b0c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:59.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.806+0000 7f8b157fa640 1 -- 172.21.15.73:0/2856160832 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8b1005a8c0 con 0x7f8b1815f1d0 2024-04-26T20:50:59.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.979+0000 7f8b1f015640 1 -- 172.21.15.73:0/2856160832 --> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8b18069b20 con 0x7f8afc06c8f0 2024-04-26T20:50:59.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.987+0000 7f8b157fa640 1 -- 172.21.15.73:0/2856160832 <== mgr.4107 v2:172.21.15.73:6832/666905549 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332533 (secure 0 0 0) 0x7f8b18069b20 con 0x7f8afc06c8f0 2024-04-26T20:50:59.992 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:50:59.994 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-26T20:50:59.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.991+0000 7f8b1f015640 1 -- 172.21.15.73:0/2856160832 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f8afc06c8f0 msgr2=0x7f8afc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:59.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.991+0000 7f8b1f015640 1 --2- 172.21.15.73:0/2856160832 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f8afc06c8f0 0x7f8afc06edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c034000 tx=0x7f8b0c02e000 comp rx=0 tx=0).stop 2024-04-26T20:50:59.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.991+0000 7f8b1f015640 1 -- 172.21.15.73:0/2856160832 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8b1815f1d0 msgr2=0x7f8b1815aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:50:59.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.991+0000 7f8b1f015640 1 --2- 172.21.15.73:0/2856160832 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8b1815f1d0 0x7f8b1815aa50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8b1000cef0 tx=0x7f8b1000a710 comp rx=0 tx=0).stop 2024-04-26T20:50:59.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.991+0000 7f8b1f015640 1 -- 172.21.15.73:0/2856160832 shutdown_connections 2024-04-26T20:50:59.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.991+0000 7f8b1f015640 1 --2- 172.21.15.73:0/2856160832 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8b1815f1d0 0x7f8b1815aa50 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.991+0000 7f8b1f015640 1 --2- 172.21.15.73:0/2856160832 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f8afc06c8f0 0x7f8afc06edb0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.991+0000 7f8b1f015640 1 --2- 172.21.15.73:0/2856160832 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8b180726e0 0x7f8b18160b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.991+0000 7f8b1f015640 1 --2- 172.21.15.73:0/2856160832 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8b1806c240 0x7f8b181e7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:59.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.991+0000 7f8b1f015640 1 -- 172.21.15.73:0/2856160832 >> 172.21.15.73:0/2856160832 conn(0x7f8b180b3180 msgr2=0x7f8b180d1d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:50:59.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.991+0000 7f8b1f015640 1 -- 172.21.15.73:0/2856160832 shutdown_connections 2024-04-26T20:50:59.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:50:59.991+0000 7f8b1f015640 1 -- 172.21.15.73:0/2856160832 wait complete. 2024-04-26T20:51:00.003 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2024-04-26T20:50:57.984407+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219468,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755252,"statfs":{"total":773094113280,"available":772869378048,"internally_reserved":0,"allocated":3383296,"data_stored":2049160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002736"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803145+0000","last_change":"2024-04-26T20:50:31.630770+0000","last_active":"2024-04-26T20:50:38.803145+0000","last_peered":"2024-04-26T20:50:38.803145+0000","last_clean":"2024-04-26T20:50:38.803145+0000","last_became_active":"2024-04-26T20:50:31.630518+0000","last_became_peered":"2024-04-26T20:50:31.630518+0000","last_unstale":"2024-04-26T20:50:38.803145+0000","last_undegraded":"2024-04-26T20:50:38.803145+0000","last_fullsized":"2024-04-26T20:50:38.803145+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:20:49.853735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803210+0000","last_change":"2024-04-26T20:50:32.649145+0000","last_active":"2024-04-26T20:50:38.803210+0000","last_peered":"2024-04-26T20:50:38.803210+0000","last_clean":"2024-04-26T20:50:38.803210+0000","last_became_active":"2024-04-26T20:50:32.648873+0000","last_became_peered":"2024-04-26T20:50:32.648873+0000","last_unstale":"2024-04-26T20:50:38.803210+0000","last_undegraded":"2024-04-26T20:50:38.803210+0000","last_fullsized":"2024-04-26T20:50:38.803210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:16:25.308786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802643+0000","last_change":"2024-04-26T20:50:37.296160+0000","last_active":"2024-04-26T20:50:38.802643+0000","last_peered":"2024-04-26T20:50:38.802643+0000","last_clean":"2024-04-26T20:50:38.802643+0000","last_became_active":"2024-04-26T20:50:37.295441+0000","last_became_peered":"2024-04-26T20:50:37.295441+0000","last_unstale":"2024-04-26T20:50:38.802643+0000","last_undegraded":"2024-04-26T20:50:38.802643+0000","last_fullsized":"2024-04-26T20:50:38.802643+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:20:48.160586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803389+0000","last_change":"2024-04-26T20:50:31.634354+0000","last_active":"2024-04-26T20:50:38.803389+0000","last_peered":"2024-04-26T20:50:38.803389+0000","last_clean":"2024-04-26T20:50:38.803389+0000","last_became_active":"2024-04-26T20:50:31.634084+0000","last_became_peered":"2024-04-26T20:50:31.634084+0000","last_unstale":"2024-04-26T20:50:38.803389+0000","last_undegraded":"2024-04-26T20:50:38.803389+0000","last_fullsized":"2024-04-26T20:50:38.803389+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:49:28.099373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802875+0000","last_change":"2024-04-26T20:50:32.645829+0000","last_active":"2024-04-26T20:50:38.802875+0000","last_peered":"2024-04-26T20:50:38.802875+0000","last_clean":"2024-04-26T20:50:38.802875+0000","last_became_active":"2024-04-26T20:50:32.645589+0000","last_became_peered":"2024-04-26T20:50:32.645589+0000","last_unstale":"2024-04-26T20:50:38.802875+0000","last_undegraded":"2024-04-26T20:50:38.802875+0000","last_fullsized":"2024-04-26T20:50:38.802875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:36.651960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661856+0000","last_change":"2024-04-26T20:50:36.675389+0000","last_active":"2024-04-26T20:50:38.661856+0000","last_peered":"2024-04-26T20:50:38.661856+0000","last_clean":"2024-04-26T20:50:38.661856+0000","last_became_active":"2024-04-26T20:50:36.675027+0000","last_became_peered":"2024-04-26T20:50:36.675027+0000","last_unstale":"2024-04-26T20:50:38.661856+0000","last_undegraded":"2024-04-26T20:50:38.661856+0000","last_fullsized":"2024-04-26T20:50:38.661856+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:13:14.645783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800942+0000","last_change":"2024-04-26T20:50:31.631388+0000","last_active":"2024-04-26T20:50:38.800942+0000","last_peered":"2024-04-26T20:50:38.800942+0000","last_clean":"2024-04-26T20:50:38.800942+0000","last_became_active":"2024-04-26T20:50:31.631121+0000","last_became_peered":"2024-04-26T20:50:31.631121+0000","last_unstale":"2024-04-26T20:50:38.800942+0000","last_undegraded":"2024-04-26T20:50:38.800942+0000","last_fullsized":"2024-04-26T20:50:38.800942+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:46:19.858595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800306+0000","last_change":"2024-04-26T20:50:32.644708+0000","last_active":"2024-04-26T20:50:38.800306+0000","last_peered":"2024-04-26T20:50:38.800306+0000","last_clean":"2024-04-26T20:50:38.800306+0000","last_became_active":"2024-04-26T20:50:32.644440+0000","last_became_peered":"2024-04-26T20:50:32.644440+0000","last_unstale":"2024-04-26T20:50:38.800306+0000","last_undegraded":"2024-04-26T20:50:38.800306+0000","last_fullsized":"2024-04-26T20:50:38.800306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:40:19.711175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.662138+0000","last_change":"2024-04-26T20:50:36.676949+0000","last_active":"2024-04-26T20:50:38.662138+0000","last_peered":"2024-04-26T20:50:38.662138+0000","last_clean":"2024-04-26T20:50:38.662138+0000","last_became_active":"2024-04-26T20:50:36.676533+0000","last_became_peered":"2024-04-26T20:50:36.676533+0000","last_unstale":"2024-04-26T20:50:38.662138+0000","last_undegraded":"2024-04-26T20:50:38.662138+0000","last_fullsized":"2024-04-26T20:50:38.662138+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:59:08.998553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800828+0000","last_change":"2024-04-26T20:50:31.632017+0000","last_active":"2024-04-26T20:50:38.800828+0000","last_peered":"2024-04-26T20:50:38.800828+0000","last_clean":"2024-04-26T20:50:38.800828+0000","last_became_active":"2024-04-26T20:50:31.631717+0000","last_became_peered":"2024-04-26T20:50:31.631717+0000","last_unstale":"2024-04-26T20:50:38.800828+0000","last_undegraded":"2024-04-26T20:50:38.800828+0000","last_fullsized":"2024-04-26T20:50:38.800828+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:32:17.604576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800366+0000","last_change":"2024-04-26T20:50:32.644081+0000","last_active":"2024-04-26T20:50:38.800366+0000","last_peered":"2024-04-26T20:50:38.800366+0000","last_clean":"2024-04-26T20:50:38.800366+0000","last_became_active":"2024-04-26T20:50:32.643809+0000","last_became_peered":"2024-04-26T20:50:32.643809+0000","last_unstale":"2024-04-26T20:50:38.800366+0000","last_undegraded":"2024-04-26T20:50:38.800366+0000","last_fullsized":"2024-04-26T20:50:38.800366+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:16:25.308786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.662248+0000","last_change":"2024-04-26T20:50:36.679187+0000","last_active":"2024-04-26T20:50:38.662248+0000","last_peered":"2024-04-26T20:50:38.662248+0000","last_clean":"2024-04-26T20:50:38.662248+0000","last_became_active":"2024-04-26T20:50:36.678782+0000","last_became_peered":"2024-04-26T20:50:36.678782+0000","last_unstale":"2024-04-26T20:50:38.662248+0000","last_undegraded":"2024-04-26T20:50:38.662248+0000","last_fullsized":"2024-04-26T20:50:38.662248+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:12.140650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.717057+0000","last_change":"2024-04-26T20:50:32.646429+0000","last_active":"2024-04-26T20:50:38.717057+0000","last_peered":"2024-04-26T20:50:38.717057+0000","last_clean":"2024-04-26T20:50:38.717057+0000","last_became_active":"2024-04-26T20:50:32.646134+0000","last_became_peered":"2024-04-26T20:50:32.646134+0000","last_unstale":"2024-04-26T20:50:38.717057+0000","last_undegraded":"2024-04-26T20:50:38.717057+0000","last_fullsized":"2024-04-26T20:50:38.717057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:09:36.737379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800514+0000","last_change":"2024-04-26T20:50:31.631250+0000","last_active":"2024-04-26T20:50:38.800514+0000","last_peered":"2024-04-26T20:50:38.800514+0000","last_clean":"2024-04-26T20:50:38.800514+0000","last_became_active":"2024-04-26T20:50:31.630277+0000","last_became_peered":"2024-04-26T20:50:31.630277+0000","last_unstale":"2024-04-26T20:50:38.800514+0000","last_undegraded":"2024-04-26T20:50:38.800514+0000","last_fullsized":"2024-04-26T20:50:38.800514+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:06:39.630339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.662616+0000","last_change":"2024-04-26T20:50:36.676056+0000","last_active":"2024-04-26T20:50:38.662616+0000","last_peered":"2024-04-26T20:50:38.662616+0000","last_clean":"2024-04-26T20:50:38.662616+0000","last_became_active":"2024-04-26T20:50:36.675706+0000","last_became_peered":"2024-04-26T20:50:36.675706+0000","last_unstale":"2024-04-26T20:50:38.662616+0000","last_undegraded":"2024-04-26T20:50:38.662616+0000","last_fullsized":"2024-04-26T20:50:38.662616+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:13:08.850173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802373+0000","last_change":"2024-04-26T20:50:32.650473+0000","last_active":"2024-04-26T20:50:38.802373+0000","last_peered":"2024-04-26T20:50:38.802373+0000","last_clean":"2024-04-26T20:50:38.802373+0000","last_became_active":"2024-04-26T20:50:32.649711+0000","last_became_peered":"2024-04-26T20:50:32.649711+0000","last_unstale":"2024-04-26T20:50:38.802373+0000","last_undegraded":"2024-04-26T20:50:38.802373+0000","last_fullsized":"2024-04-26T20:50:38.802373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:36.651960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661208+0000","last_change":"2024-04-26T20:50:31.629572+0000","last_active":"2024-04-26T20:50:38.661208+0000","last_peered":"2024-04-26T20:50:38.661208+0000","last_clean":"2024-04-26T20:50:38.661208+0000","last_became_active":"2024-04-26T20:50:31.629287+0000","last_became_peered":"2024-04-26T20:50:31.629287+0000","last_unstale":"2024-04-26T20:50:38.661208+0000","last_undegraded":"2024-04-26T20:50:38.661208+0000","last_fullsized":"2024-04-26T20:50:38.661208+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:36.499651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.662538+0000","last_change":"2024-04-26T20:50:36.681910+0000","last_active":"2024-04-26T20:50:38.662538+0000","last_peered":"2024-04-26T20:50:38.662538+0000","last_clean":"2024-04-26T20:50:38.662538+0000","last_became_active":"2024-04-26T20:50:36.681432+0000","last_became_peered":"2024-04-26T20:50:36.681432+0000","last_unstale":"2024-04-26T20:50:38.662538+0000","last_undegraded":"2024-04-26T20:50:38.662538+0000","last_fullsized":"2024-04-26T20:50:38.662538+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:27:18.035795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802910+0000","last_change":"2024-04-26T20:50:32.647018+0000","last_active":"2024-04-26T20:50:38.802910+0000","last_peered":"2024-04-26T20:50:38.802910+0000","last_clean":"2024-04-26T20:50:38.802910+0000","last_became_active":"2024-04-26T20:50:32.646748+0000","last_became_peered":"2024-04-26T20:50:32.646748+0000","last_unstale":"2024-04-26T20:50:38.802910+0000","last_undegraded":"2024-04-26T20:50:38.802910+0000","last_fullsized":"2024-04-26T20:50:38.802910+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:56:51.926886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.660717+0000","last_change":"2024-04-26T20:50:31.631760+0000","last_active":"2024-04-26T20:50:38.660717+0000","last_peered":"2024-04-26T20:50:38.660717+0000","last_clean":"2024-04-26T20:50:38.660717+0000","last_became_active":"2024-04-26T20:50:31.629437+0000","last_became_peered":"2024-04-26T20:50:31.629437+0000","last_unstale":"2024-04-26T20:50:38.660717+0000","last_undegraded":"2024-04-26T20:50:38.660717+0000","last_fullsized":"2024-04-26T20:50:38.660717+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:23:29.866794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661242+0000","last_change":"2024-04-26T20:50:37.295810+0000","last_active":"2024-04-26T20:50:38.661242+0000","last_peered":"2024-04-26T20:50:38.661242+0000","last_clean":"2024-04-26T20:50:38.661242+0000","last_became_active":"2024-04-26T20:50:37.295425+0000","last_became_peered":"2024-04-26T20:50:37.295425+0000","last_unstale":"2024-04-26T20:50:38.661242+0000","last_undegraded":"2024-04-26T20:50:38.661242+0000","last_fullsized":"2024-04-26T20:50:38.661242+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:31:21.850102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.662187+0000","last_change":"2024-04-26T20:50:32.638584+0000","last_active":"2024-04-26T20:50:38.662187+0000","last_peered":"2024-04-26T20:50:38.662187+0000","last_clean":"2024-04-26T20:50:38.662187+0000","last_became_active":"2024-04-26T20:50:32.637734+0000","last_became_peered":"2024-04-26T20:50:32.637734+0000","last_unstale":"2024-04-26T20:50:38.662187+0000","last_undegraded":"2024-04-26T20:50:38.662187+0000","last_fullsized":"2024-04-26T20:50:38.662187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:37.504098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800397+0000","last_change":"2024-04-26T20:50:31.634926+0000","last_active":"2024-04-26T20:50:38.800397+0000","last_peered":"2024-04-26T20:50:38.800397+0000","last_clean":"2024-04-26T20:50:38.800397+0000","last_became_active":"2024-04-26T20:50:31.634278+0000","last_became_peered":"2024-04-26T20:50:31.634278+0000","last_unstale":"2024-04-26T20:50:38.800397+0000","last_undegraded":"2024-04-26T20:50:38.800397+0000","last_fullsized":"2024-04-26T20:50:38.800397+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:36.499651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.662287+0000","last_change":"2024-04-26T20:50:36.678377+0000","last_active":"2024-04-26T20:50:38.662287+0000","last_peered":"2024-04-26T20:50:38.662287+0000","last_clean":"2024-04-26T20:50:38.662287+0000","last_became_active":"2024-04-26T20:50:36.677946+0000","last_became_peered":"2024-04-26T20:50:36.677946+0000","last_unstale":"2024-04-26T20:50:38.662287+0000","last_undegraded":"2024-04-26T20:50:38.662287+0000","last_fullsized":"2024-04-26T20:50:38.662287+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:31:21.850102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803961+0000","last_change":"2024-04-26T20:50:32.637345+0000","last_active":"2024-04-26T20:50:38.803961+0000","last_peered":"2024-04-26T20:50:38.803961+0000","last_clean":"2024-04-26T20:50:38.803961+0000","last_became_active":"2024-04-26T20:50:32.637071+0000","last_became_peered":"2024-04-26T20:50:32.637071+0000","last_unstale":"2024-04-26T20:50:38.803961+0000","last_undegraded":"2024-04-26T20:50:38.803961+0000","last_fullsized":"2024-04-26T20:50:38.803961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:47:14.715739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.717764+0000","last_change":"2024-04-26T20:50:31.631119+0000","last_active":"2024-04-26T20:50:38.717764+0000","last_peered":"2024-04-26T20:50:38.717764+0000","last_clean":"2024-04-26T20:50:38.717764+0000","last_became_active":"2024-04-26T20:50:31.630825+0000","last_became_peered":"2024-04-26T20:50:31.630825+0000","last_unstale":"2024-04-26T20:50:38.717764+0000","last_undegraded":"2024-04-26T20:50:38.717764+0000","last_fullsized":"2024-04-26T20:50:38.717764+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:26.601021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661961+0000","last_change":"2024-04-26T20:50:37.295783+0000","last_active":"2024-04-26T20:50:38.661961+0000","last_peered":"2024-04-26T20:50:38.661961+0000","last_clean":"2024-04-26T20:50:38.661961+0000","last_became_active":"2024-04-26T20:50:37.295382+0000","last_became_peered":"2024-04-26T20:50:37.295382+0000","last_unstale":"2024-04-26T20:50:38.661961+0000","last_undegraded":"2024-04-26T20:50:38.661961+0000","last_fullsized":"2024-04-26T20:50:38.661961+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:58:22.587813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803938+0000","last_change":"2024-04-26T20:50:32.649792+0000","last_active":"2024-04-26T20:50:38.803938+0000","last_peered":"2024-04-26T20:50:38.803938+0000","last_clean":"2024-04-26T20:50:38.803938+0000","last_became_active":"2024-04-26T20:50:32.649510+0000","last_became_peered":"2024-04-26T20:50:32.649510+0000","last_unstale":"2024-04-26T20:50:38.803938+0000","last_undegraded":"2024-04-26T20:50:38.803938+0000","last_fullsized":"2024-04-26T20:50:38.803938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:35.639518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661244+0000","last_change":"2024-04-26T20:50:31.631623+0000","last_active":"2024-04-26T20:50:38.661244+0000","last_peered":"2024-04-26T20:50:38.661244+0000","last_clean":"2024-04-26T20:50:38.661244+0000","last_became_active":"2024-04-26T20:50:31.631312+0000","last_became_peered":"2024-04-26T20:50:31.631312+0000","last_unstale":"2024-04-26T20:50:38.661244+0000","last_undegraded":"2024-04-26T20:50:38.661244+0000","last_fullsized":"2024-04-26T20:50:38.661244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:46:19.858595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802802+0000","last_change":"2024-04-26T20:50:37.295508+0000","last_active":"2024-04-26T20:50:38.802802+0000","last_peered":"2024-04-26T20:50:38.802802+0000","last_clean":"2024-04-26T20:50:38.802802+0000","last_became_active":"2024-04-26T20:50:37.294858+0000","last_became_peered":"2024-04-26T20:50:37.294858+0000","last_unstale":"2024-04-26T20:50:38.802802+0000","last_undegraded":"2024-04-26T20:50:38.802802+0000","last_fullsized":"2024-04-26T20:50:38.802802+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:42:14.026063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661292+0000","last_change":"2024-04-26T20:50:32.640053+0000","last_active":"2024-04-26T20:50:38.661292+0000","last_peered":"2024-04-26T20:50:38.661292+0000","last_clean":"2024-04-26T20:50:38.661292+0000","last_became_active":"2024-04-26T20:50:32.639778+0000","last_became_peered":"2024-04-26T20:50:32.639778+0000","last_unstale":"2024-04-26T20:50:38.661292+0000","last_undegraded":"2024-04-26T20:50:38.661292+0000","last_fullsized":"2024-04-26T20:50:38.661292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:09:36.737379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.801056+0000","last_change":"2024-04-26T20:50:31.634409+0000","last_active":"2024-04-26T20:50:38.801056+0000","last_peered":"2024-04-26T20:50:38.801056+0000","last_clean":"2024-04-26T20:50:38.801056+0000","last_became_active":"2024-04-26T20:50:31.634103+0000","last_became_peered":"2024-04-26T20:50:31.634103+0000","last_unstale":"2024-04-26T20:50:38.801056+0000","last_undegraded":"2024-04-26T20:50:38.801056+0000","last_fullsized":"2024-04-26T20:50:38.801056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:40:15.520084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803602+0000","last_change":"2024-04-26T20:50:37.292899+0000","last_active":"2024-04-26T20:50:38.803602+0000","last_peered":"2024-04-26T20:50:38.803602+0000","last_clean":"2024-04-26T20:50:38.803602+0000","last_became_active":"2024-04-26T20:50:37.292379+0000","last_became_peered":"2024-04-26T20:50:37.292379+0000","last_unstale":"2024-04-26T20:50:38.803602+0000","last_undegraded":"2024-04-26T20:50:38.803602+0000","last_fullsized":"2024-04-26T20:50:38.803602+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:28:04.995069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800475+0000","last_change":"2024-04-26T20:50:32.649145+0000","last_active":"2024-04-26T20:50:38.800475+0000","last_peered":"2024-04-26T20:50:38.800475+0000","last_clean":"2024-04-26T20:50:38.800475+0000","last_became_active":"2024-04-26T20:50:32.648860+0000","last_became_peered":"2024-04-26T20:50:32.648860+0000","last_unstale":"2024-04-26T20:50:38.800475+0000","last_undegraded":"2024-04-26T20:50:38.800475+0000","last_fullsized":"2024-04-26T20:50:38.800475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:45:44.950496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661412+0000","last_change":"2024-04-26T20:50:31.627675+0000","last_active":"2024-04-26T20:50:38.661412+0000","last_peered":"2024-04-26T20:50:38.661412+0000","last_clean":"2024-04-26T20:50:38.661412+0000","last_became_active":"2024-04-26T20:50:31.627373+0000","last_became_peered":"2024-04-26T20:50:31.627373+0000","last_unstale":"2024-04-26T20:50:38.661412+0000","last_undegraded":"2024-04-26T20:50:38.661412+0000","last_fullsized":"2024-04-26T20:50:38.661412+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:26.601021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.660956+0000","last_change":"2024-04-26T20:50:36.679728+0000","last_active":"2024-04-26T20:50:38.660956+0000","last_peered":"2024-04-26T20:50:38.660956+0000","last_clean":"2024-04-26T20:50:38.660956+0000","last_became_active":"2024-04-26T20:50:36.678881+0000","last_became_peered":"2024-04-26T20:50:36.678881+0000","last_unstale":"2024-04-26T20:50:38.660956+0000","last_undegraded":"2024-04-26T20:50:38.660956+0000","last_fullsized":"2024-04-26T20:50:38.660956+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:59:08.998553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.662645+0000","last_change":"2024-04-26T20:50:32.633552+0000","last_active":"2024-04-26T20:50:38.662645+0000","last_peered":"2024-04-26T20:50:38.662645+0000","last_clean":"2024-04-26T20:50:38.662645+0000","last_became_active":"2024-04-26T20:50:32.632649+0000","last_became_peered":"2024-04-26T20:50:32.632649+0000","last_unstale":"2024-04-26T20:50:38.662645+0000","last_undegraded":"2024-04-26T20:50:38.662645+0000","last_fullsized":"2024-04-26T20:50:38.662645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:32:18.609023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800518+0000","last_change":"2024-04-26T20:50:31.622898+0000","last_active":"2024-04-26T20:50:38.800518+0000","last_peered":"2024-04-26T20:50:38.800518+0000","last_clean":"2024-04-26T20:50:38.800518+0000","last_became_active":"2024-04-26T20:50:31.622583+0000","last_became_peered":"2024-04-26T20:50:31.622583+0000","last_unstale":"2024-04-26T20:50:38.800518+0000","last_undegraded":"2024-04-26T20:50:38.800518+0000","last_fullsized":"2024-04-26T20:50:38.800518+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:53:10.645804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661250+0000","last_change":"2024-04-26T20:50:36.681863+0000","last_active":"2024-04-26T20:50:38.661250+0000","last_peered":"2024-04-26T20:50:38.661250+0000","last_clean":"2024-04-26T20:50:38.661250+0000","last_became_active":"2024-04-26T20:50:36.681433+0000","last_became_peered":"2024-04-26T20:50:36.681433+0000","last_unstale":"2024-04-26T20:50:38.661250+0000","last_undegraded":"2024-04-26T20:50:38.661250+0000","last_fullsized":"2024-04-26T20:50:38.661250+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:28:04.995069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661447+0000","last_change":"2024-04-26T20:50:32.640962+0000","last_active":"2024-04-26T20:50:38.661447+0000","last_peered":"2024-04-26T20:50:38.661447+0000","last_clean":"2024-04-26T20:50:38.661447+0000","last_became_active":"2024-04-26T20:50:32.640670+0000","last_became_peered":"2024-04-26T20:50:32.640670+0000","last_unstale":"2024-04-26T20:50:38.661447+0000","last_undegraded":"2024-04-26T20:50:38.661447+0000","last_fullsized":"2024-04-26T20:50:38.661447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:54.967490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802635+0000","last_change":"2024-04-26T20:50:31.631694+0000","last_active":"2024-04-26T20:50:38.802635+0000","last_peered":"2024-04-26T20:50:38.802635+0000","last_clean":"2024-04-26T20:50:38.802635+0000","last_became_active":"2024-04-26T20:50:31.631330+0000","last_became_peered":"2024-04-26T20:50:31.631330+0000","last_unstale":"2024-04-26T20:50:38.802635+0000","last_undegraded":"2024-04-26T20:50:38.802635+0000","last_fullsized":"2024-04-26T20:50:38.802635+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:49:28.099373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661389+0000","last_change":"2024-04-26T20:50:36.682799+0000","last_active":"2024-04-26T20:50:38.661389+0000","last_peered":"2024-04-26T20:50:38.661389+0000","last_clean":"2024-04-26T20:50:38.661389+0000","last_became_active":"2024-04-26T20:50:36.682271+0000","last_became_peered":"2024-04-26T20:50:36.682271+0000","last_unstale":"2024-04-26T20:50:38.661389+0000","last_undegraded":"2024-04-26T20:50:38.661389+0000","last_fullsized":"2024-04-26T20:50:38.661389+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:27:18.035795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802689+0000","last_change":"2024-04-26T20:50:32.647241+0000","last_active":"2024-04-26T20:50:38.802689+0000","last_peered":"2024-04-26T20:50:38.802689+0000","last_clean":"2024-04-26T20:50:38.802689+0000","last_became_active":"2024-04-26T20:50:32.646907+0000","last_became_peered":"2024-04-26T20:50:32.646907+0000","last_unstale":"2024-04-26T20:50:38.802689+0000","last_undegraded":"2024-04-26T20:50:38.802689+0000","last_fullsized":"2024-04-26T20:50:38.802689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:57:40.604842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800494+0000","last_change":"2024-04-26T20:50:31.633583+0000","last_active":"2024-04-26T20:50:38.800494+0000","last_peered":"2024-04-26T20:50:38.800494+0000","last_clean":"2024-04-26T20:50:38.800494+0000","last_became_active":"2024-04-26T20:50:31.633272+0000","last_became_peered":"2024-04-26T20:50:31.633272+0000","last_unstale":"2024-04-26T20:50:38.800494+0000","last_undegraded":"2024-04-26T20:50:38.800494+0000","last_fullsized":"2024-04-26T20:50:38.800494+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:53:10.645804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803808+0000","last_change":"2024-04-26T20:50:37.291410+0000","last_active":"2024-04-26T20:50:38.803808+0000","last_peered":"2024-04-26T20:50:38.803808+0000","last_clean":"2024-04-26T20:50:38.803808+0000","last_became_active":"2024-04-26T20:50:37.291016+0000","last_became_peered":"2024-04-26T20:50:37.291016+0000","last_unstale":"2024-04-26T20:50:38.803808+0000","last_undegraded":"2024-04-26T20:50:38.803808+0000","last_fullsized":"2024-04-26T20:50:38.803808+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:13:08.850173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802348+0000","last_change":"2024-04-26T20:50:32.653314+0000","last_active":"2024-04-26T20:50:38.802348+0000","last_peered":"2024-04-26T20:50:38.802348+0000","last_clean":"2024-04-26T20:50:38.802348+0000","last_became_active":"2024-04-26T20:50:32.652980+0000","last_became_peered":"2024-04-26T20:50:32.652980+0000","last_unstale":"2024-04-26T20:50:38.802348+0000","last_undegraded":"2024-04-26T20:50:38.802348+0000","last_fullsized":"2024-04-26T20:50:38.802348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:15:55.563522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800028+0000","last_change":"2024-04-26T20:50:31.629049+0000","last_active":"2024-04-26T20:50:38.800028+0000","last_peered":"2024-04-26T20:50:38.800028+0000","last_clean":"2024-04-26T20:50:38.800028+0000","last_became_active":"2024-04-26T20:50:31.628788+0000","last_became_peered":"2024-04-26T20:50:31.628788+0000","last_unstale":"2024-04-26T20:50:38.800028+0000","last_undegraded":"2024-04-26T20:50:38.800028+0000","last_fullsized":"2024-04-26T20:50:38.800028+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:26.601021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":179,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800588+0000","last_change":"2024-04-26T20:50:21.150132+0000","last_active":"2024-04-26T20:50:38.800588+0000","last_peered":"2024-04-26T20:50:38.800588+0000","last_clean":"2024-04-26T20:50:38.800588+0000","last_became_active":"2024-04-26T20:50:21.147439+0000","last_became_peered":"2024-04-26T20:50:21.147439+0000","last_unstale":"2024-04-26T20:50:38.800588+0000","last_undegraded":"2024-04-26T20:50:38.800588+0000","last_fullsized":"2024-04-26T20:50:38.800588+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:20.084499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:20.084499+0000","last_clean_scrub_stamp":"2024-04-26T20:50:20.084499+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:19:29.395477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.717442+0000","last_change":"2024-04-26T20:50:37.295209+0000","last_active":"2024-04-26T20:50:38.717442+0000","last_peered":"2024-04-26T20:50:38.717442+0000","last_clean":"2024-04-26T20:50:38.717442+0000","last_became_active":"2024-04-26T20:50:37.294715+0000","last_became_peered":"2024-04-26T20:50:37.294715+0000","last_unstale":"2024-04-26T20:50:38.717442+0000","last_undegraded":"2024-04-26T20:50:38.717442+0000","last_fullsized":"2024-04-26T20:50:38.717442+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:13:14.645783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800810+0000","last_change":"2024-04-26T20:50:32.648081+0000","last_active":"2024-04-26T20:50:38.800810+0000","last_peered":"2024-04-26T20:50:38.800810+0000","last_clean":"2024-04-26T20:50:38.800810+0000","last_became_active":"2024-04-26T20:50:32.644812+0000","last_became_peered":"2024-04-26T20:50:32.644812+0000","last_unstale":"2024-04-26T20:50:38.800810+0000","last_undegraded":"2024-04-26T20:50:38.800810+0000","last_fullsized":"2024-04-26T20:50:38.800810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:45:44.950496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803401+0000","last_change":"2024-04-26T20:50:31.630301+0000","last_active":"2024-04-26T20:50:38.803401+0000","last_peered":"2024-04-26T20:50:38.803401+0000","last_clean":"2024-04-26T20:50:38.803401+0000","last_became_active":"2024-04-26T20:50:31.630039+0000","last_became_peered":"2024-04-26T20:50:31.630039+0000","last_unstale":"2024-04-26T20:50:38.803401+0000","last_undegraded":"2024-04-26T20:50:38.803401+0000","last_fullsized":"2024-04-26T20:50:38.803401+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:21:09.963731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.717176+0000","last_change":"2024-04-26T20:50:37.291889+0000","last_active":"2024-04-26T20:50:38.717176+0000","last_peered":"2024-04-26T20:50:38.717176+0000","last_clean":"2024-04-26T20:50:38.717176+0000","last_became_active":"2024-04-26T20:50:37.291401+0000","last_became_peered":"2024-04-26T20:50:37.291401+0000","last_unstale":"2024-04-26T20:50:38.717176+0000","last_undegraded":"2024-04-26T20:50:38.717176+0000","last_fullsized":"2024-04-26T20:50:38.717176+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:28:29.639328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800412+0000","last_change":"2024-04-26T20:50:32.655082+0000","last_active":"2024-04-26T20:50:38.800412+0000","last_peered":"2024-04-26T20:50:38.800412+0000","last_clean":"2024-04-26T20:50:38.800412+0000","last_became_active":"2024-04-26T20:50:32.654751+0000","last_became_peered":"2024-04-26T20:50:32.654751+0000","last_unstale":"2024-04-26T20:50:38.800412+0000","last_undegraded":"2024-04-26T20:50:38.800412+0000","last_fullsized":"2024-04-26T20:50:38.800412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:27:59.302826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661274+0000","last_change":"2024-04-26T20:50:31.628865+0000","last_active":"2024-04-26T20:50:38.661274+0000","last_peered":"2024-04-26T20:50:38.661274+0000","last_clean":"2024-04-26T20:50:38.661274+0000","last_became_active":"2024-04-26T20:50:31.627789+0000","last_became_peered":"2024-04-26T20:50:31.627789+0000","last_unstale":"2024-04-26T20:50:38.661274+0000","last_undegraded":"2024-04-26T20:50:38.661274+0000","last_fullsized":"2024-04-26T20:50:38.661274+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:53:10.645804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803847+0000","last_change":"2024-04-26T20:50:36.679869+0000","last_active":"2024-04-26T20:50:38.803847+0000","last_peered":"2024-04-26T20:50:38.803847+0000","last_clean":"2024-04-26T20:50:38.803847+0000","last_became_active":"2024-04-26T20:50:36.678678+0000","last_became_peered":"2024-04-26T20:50:36.678678+0000","last_unstale":"2024-04-26T20:50:38.803847+0000","last_undegraded":"2024-04-26T20:50:38.803847+0000","last_fullsized":"2024-04-26T20:50:38.803847+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:54.249138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.801078+0000","last_change":"2024-04-26T20:50:32.655912+0000","last_active":"2024-04-26T20:50:38.801078+0000","last_peered":"2024-04-26T20:50:38.801078+0000","last_clean":"2024-04-26T20:50:38.801078+0000","last_became_active":"2024-04-26T20:50:32.655649+0000","last_became_peered":"2024-04-26T20:50:32.655649+0000","last_unstale":"2024-04-26T20:50:38.801078+0000","last_undegraded":"2024-04-26T20:50:38.801078+0000","last_fullsized":"2024-04-26T20:50:38.801078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:59:04.962815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803582+0000","last_change":"2024-04-26T20:50:31.631670+0000","last_active":"2024-04-26T20:50:38.803582+0000","last_peered":"2024-04-26T20:50:38.803582+0000","last_clean":"2024-04-26T20:50:38.803582+0000","last_became_active":"2024-04-26T20:50:31.631380+0000","last_became_peered":"2024-04-26T20:50:31.631380+0000","last_unstale":"2024-04-26T20:50:38.803582+0000","last_undegraded":"2024-04-26T20:50:38.803582+0000","last_fullsized":"2024-04-26T20:50:38.803582+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:23:29.866794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.799715+0000","last_change":"2024-04-26T20:50:37.291812+0000","last_active":"2024-04-26T20:50:38.799715+0000","last_peered":"2024-04-26T20:50:38.799715+0000","last_clean":"2024-04-26T20:50:38.799715+0000","last_became_active":"2024-04-26T20:50:37.291408+0000","last_became_peered":"2024-04-26T20:50:37.291408+0000","last_unstale":"2024-04-26T20:50:38.799715+0000","last_undegraded":"2024-04-26T20:50:38.799715+0000","last_fullsized":"2024-04-26T20:50:38.799715+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:29:58.077073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800564+0000","last_change":"2024-04-26T20:50:32.649031+0000","last_active":"2024-04-26T20:50:38.800564+0000","last_peered":"2024-04-26T20:50:38.800564+0000","last_clean":"2024-04-26T20:50:38.800564+0000","last_became_active":"2024-04-26T20:50:32.648769+0000","last_became_peered":"2024-04-26T20:50:32.648769+0000","last_unstale":"2024-04-26T20:50:38.800564+0000","last_undegraded":"2024-04-26T20:50:38.800564+0000","last_fullsized":"2024-04-26T20:50:38.800564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:16:25.308786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.662578+0000","last_change":"2024-04-26T20:50:31.627514+0000","last_active":"2024-04-26T20:50:38.662578+0000","last_peered":"2024-04-26T20:50:38.662578+0000","last_clean":"2024-04-26T20:50:38.662578+0000","last_became_active":"2024-04-26T20:50:31.627177+0000","last_became_peered":"2024-04-26T20:50:31.627177+0000","last_unstale":"2024-04-26T20:50:38.662578+0000","last_undegraded":"2024-04-26T20:50:38.662578+0000","last_fullsized":"2024-04-26T20:50:38.662578+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:23:21.503395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661857+0000","last_change":"2024-04-26T20:50:36.678980+0000","last_active":"2024-04-26T20:50:38.661857+0000","last_peered":"2024-04-26T20:50:38.661857+0000","last_clean":"2024-04-26T20:50:38.661857+0000","last_became_active":"2024-04-26T20:50:36.678581+0000","last_became_peered":"2024-04-26T20:50:36.678581+0000","last_unstale":"2024-04-26T20:50:38.661857+0000","last_undegraded":"2024-04-26T20:50:38.661857+0000","last_fullsized":"2024-04-26T20:50:38.661857+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:28:29.639328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800927+0000","last_change":"2024-04-26T20:50:32.647060+0000","last_active":"2024-04-26T20:50:38.800927+0000","last_peered":"2024-04-26T20:50:38.800927+0000","last_clean":"2024-04-26T20:50:38.800927+0000","last_became_active":"2024-04-26T20:50:32.646761+0000","last_became_peered":"2024-04-26T20:50:32.646761+0000","last_unstale":"2024-04-26T20:50:38.800927+0000","last_undegraded":"2024-04-26T20:50:38.800927+0000","last_fullsized":"2024-04-26T20:50:38.800927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:35.639518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802576+0000","last_change":"2024-04-26T20:50:31.627228+0000","last_active":"2024-04-26T20:50:38.802576+0000","last_peered":"2024-04-26T20:50:38.802576+0000","last_clean":"2024-04-26T20:50:38.802576+0000","last_became_active":"2024-04-26T20:50:31.626931+0000","last_became_peered":"2024-04-26T20:50:31.626931+0000","last_unstale":"2024-04-26T20:50:38.802576+0000","last_undegraded":"2024-04-26T20:50:38.802576+0000","last_fullsized":"2024-04-26T20:50:38.802576+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:26.601021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803663+0000","last_change":"2024-04-26T20:50:36.684013+0000","last_active":"2024-04-26T20:50:38.803663+0000","last_peered":"2024-04-26T20:50:38.803663+0000","last_clean":"2024-04-26T20:50:38.803663+0000","last_became_active":"2024-04-26T20:50:36.683653+0000","last_became_peered":"2024-04-26T20:50:36.683653+0000","last_unstale":"2024-04-26T20:50:38.803663+0000","last_undegraded":"2024-04-26T20:50:38.803663+0000","last_fullsized":"2024-04-26T20:50:38.803663+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:27:18.035795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.717061+0000","last_change":"2024-04-26T20:50:32.645444+0000","last_active":"2024-04-26T20:50:38.717061+0000","last_peered":"2024-04-26T20:50:38.717061+0000","last_clean":"2024-04-26T20:50:38.717061+0000","last_became_active":"2024-04-26T20:50:32.645155+0000","last_became_peered":"2024-04-26T20:50:32.645155+0000","last_unstale":"2024-04-26T20:50:38.717061+0000","last_undegraded":"2024-04-26T20:50:38.717061+0000","last_fullsized":"2024-04-26T20:50:38.717061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:49:57.427154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.717091+0000","last_change":"2024-04-26T20:50:31.629590+0000","last_active":"2024-04-26T20:50:38.717091+0000","last_peered":"2024-04-26T20:50:38.717091+0000","last_clean":"2024-04-26T20:50:38.717091+0000","last_became_active":"2024-04-26T20:50:31.629324+0000","last_became_peered":"2024-04-26T20:50:31.629324+0000","last_unstale":"2024-04-26T20:50:38.717091+0000","last_undegraded":"2024-04-26T20:50:38.717091+0000","last_fullsized":"2024-04-26T20:50:38.717091+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:49:28.099373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803803+0000","last_change":"2024-04-26T20:50:37.291611+0000","last_active":"2024-04-26T20:50:38.803803+0000","last_peered":"2024-04-26T20:50:38.803803+0000","last_clean":"2024-04-26T20:50:38.803803+0000","last_became_active":"2024-04-26T20:50:37.291068+0000","last_became_peered":"2024-04-26T20:50:37.291068+0000","last_unstale":"2024-04-26T20:50:38.803803+0000","last_undegraded":"2024-04-26T20:50:38.803803+0000","last_fullsized":"2024-04-26T20:50:38.803803+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:31:09.384259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800427+0000","last_change":"2024-04-26T20:50:32.649304+0000","last_active":"2024-04-26T20:50:38.800427+0000","last_peered":"2024-04-26T20:50:38.800427+0000","last_clean":"2024-04-26T20:50:38.800427+0000","last_became_active":"2024-04-26T20:50:32.647714+0000","last_became_peered":"2024-04-26T20:50:32.647714+0000","last_unstale":"2024-04-26T20:50:38.800427+0000","last_undegraded":"2024-04-26T20:50:38.800427+0000","last_fullsized":"2024-04-26T20:50:38.800427+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:27:59.302826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800065+0000","last_change":"2024-04-26T20:50:31.626907+0000","last_active":"2024-04-26T20:50:38.800065+0000","last_peered":"2024-04-26T20:50:38.800065+0000","last_clean":"2024-04-26T20:50:38.800065+0000","last_became_active":"2024-04-26T20:50:31.625955+0000","last_became_peered":"2024-04-26T20:50:31.625955+0000","last_unstale":"2024-04-26T20:50:38.800065+0000","last_undegraded":"2024-04-26T20:50:38.800065+0000","last_fullsized":"2024-04-26T20:50:38.800065+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:46:19.858595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803391+0000","last_change":"2024-04-26T20:50:37.292652+0000","last_active":"2024-04-26T20:50:38.803391+0000","last_peered":"2024-04-26T20:50:38.803391+0000","last_clean":"2024-04-26T20:50:38.803391+0000","last_became_active":"2024-04-26T20:50:37.291717+0000","last_became_peered":"2024-04-26T20:50:37.291717+0000","last_unstale":"2024-04-26T20:50:38.803391+0000","last_undegraded":"2024-04-26T20:50:38.803391+0000","last_fullsized":"2024-04-26T20:50:38.803391+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:31:19.069050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803321+0000","last_change":"2024-04-26T20:50:32.643484+0000","last_active":"2024-04-26T20:50:38.803321+0000","last_peered":"2024-04-26T20:50:38.803321+0000","last_clean":"2024-04-26T20:50:38.803321+0000","last_became_active":"2024-04-26T20:50:32.643225+0000","last_became_peered":"2024-04-26T20:50:32.643225+0000","last_unstale":"2024-04-26T20:50:38.803321+0000","last_undegraded":"2024-04-26T20:50:38.803321+0000","last_fullsized":"2024-04-26T20:50:38.803321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:49:57.427154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803958+0000","last_change":"2024-04-26T20:50:31.627130+0000","last_active":"2024-04-26T20:50:38.803958+0000","last_peered":"2024-04-26T20:50:38.803958+0000","last_clean":"2024-04-26T20:50:38.803958+0000","last_became_active":"2024-04-26T20:50:31.626697+0000","last_became_peered":"2024-04-26T20:50:31.626697+0000","last_unstale":"2024-04-26T20:50:38.803958+0000","last_undegraded":"2024-04-26T20:50:38.803958+0000","last_fullsized":"2024-04-26T20:50:38.803958+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:21:09.963731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800487+0000","last_change":"2024-04-26T20:50:37.294808+0000","last_active":"2024-04-26T20:50:38.800487+0000","last_peered":"2024-04-26T20:50:38.800487+0000","last_clean":"2024-04-26T20:50:38.800487+0000","last_became_active":"2024-04-26T20:50:37.294404+0000","last_became_peered":"2024-04-26T20:50:37.294404+0000","last_unstale":"2024-04-26T20:50:38.800487+0000","last_undegraded":"2024-04-26T20:50:38.800487+0000","last_fullsized":"2024-04-26T20:50:38.800487+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:29:17.876006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800773+0000","last_change":"2024-04-26T20:50:32.649505+0000","last_active":"2024-04-26T20:50:38.800773+0000","last_peered":"2024-04-26T20:50:38.800773+0000","last_clean":"2024-04-26T20:50:38.800773+0000","last_became_active":"2024-04-26T20:50:32.648538+0000","last_became_peered":"2024-04-26T20:50:32.648538+0000","last_unstale":"2024-04-26T20:50:38.800773+0000","last_undegraded":"2024-04-26T20:50:38.800773+0000","last_fullsized":"2024-04-26T20:50:38.800773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:33:28.969815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:41.974976+0000","last_change":"2024-04-26T20:50:31.628758+0000","last_active":"2024-04-26T20:50:41.974976+0000","last_peered":"2024-04-26T20:50:41.974976+0000","last_clean":"2024-04-26T20:50:41.974976+0000","last_became_active":"2024-04-26T20:50:31.628431+0000","last_became_peered":"2024-04-26T20:50:31.628431+0000","last_unstale":"2024-04-26T20:50:41.974976+0000","last_undegraded":"2024-04-26T20:50:41.974976+0000","last_fullsized":"2024-04-26T20:50:41.974976+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:36.499651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661093+0000","last_change":"2024-04-26T20:50:36.679403+0000","last_active":"2024-04-26T20:50:38.661093+0000","last_peered":"2024-04-26T20:50:38.661093+0000","last_clean":"2024-04-26T20:50:38.661093+0000","last_became_active":"2024-04-26T20:50:36.678945+0000","last_became_peered":"2024-04-26T20:50:36.678945+0000","last_unstale":"2024-04-26T20:50:38.661093+0000","last_undegraded":"2024-04-26T20:50:38.661093+0000","last_fullsized":"2024-04-26T20:50:38.661093+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:12.140650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.662571+0000","last_change":"2024-04-26T20:50:32.636718+0000","last_active":"2024-04-26T20:50:38.662571+0000","last_peered":"2024-04-26T20:50:38.662571+0000","last_clean":"2024-04-26T20:50:38.662571+0000","last_became_active":"2024-04-26T20:50:32.636419+0000","last_became_peered":"2024-04-26T20:50:32.636419+0000","last_unstale":"2024-04-26T20:50:38.662571+0000","last_undegraded":"2024-04-26T20:50:38.662571+0000","last_fullsized":"2024-04-26T20:50:38.662571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:06:40.634786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661155+0000","last_change":"2024-04-26T20:50:31.630575+0000","last_active":"2024-04-26T20:50:38.661155+0000","last_peered":"2024-04-26T20:50:38.661155+0000","last_clean":"2024-04-26T20:50:38.661155+0000","last_became_active":"2024-04-26T20:50:31.630304+0000","last_became_peered":"2024-04-26T20:50:31.630304+0000","last_unstale":"2024-04-26T20:50:38.661155+0000","last_undegraded":"2024-04-26T20:50:38.661155+0000","last_fullsized":"2024-04-26T20:50:38.661155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:32:17.604576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.662550+0000","last_change":"2024-04-26T20:50:36.672365+0000","last_active":"2024-04-26T20:50:38.662550+0000","last_peered":"2024-04-26T20:50:38.662550+0000","last_clean":"2024-04-26T20:50:38.662550+0000","last_became_active":"2024-04-26T20:50:36.671917+0000","last_became_peered":"2024-04-26T20:50:36.671917+0000","last_unstale":"2024-04-26T20:50:38.662550+0000","last_undegraded":"2024-04-26T20:50:38.662550+0000","last_fullsized":"2024-04-26T20:50:38.662550+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:28:04.995069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803056+0000","last_change":"2024-04-26T20:50:32.645616+0000","last_active":"2024-04-26T20:50:38.803056+0000","last_peered":"2024-04-26T20:50:38.803056+0000","last_clean":"2024-04-26T20:50:38.803056+0000","last_became_active":"2024-04-26T20:50:32.645372+0000","last_became_peered":"2024-04-26T20:50:32.645372+0000","last_unstale":"2024-04-26T20:50:38.803056+0000","last_undegraded":"2024-04-26T20:50:38.803056+0000","last_fullsized":"2024-04-26T20:50:38.803056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:56:51.926886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800505+0000","last_change":"2024-04-26T20:50:31.632113+0000","last_active":"2024-04-26T20:50:38.800505+0000","last_peered":"2024-04-26T20:50:38.800505+0000","last_clean":"2024-04-26T20:50:38.800505+0000","last_became_active":"2024-04-26T20:50:31.631815+0000","last_became_peered":"2024-04-26T20:50:31.631815+0000","last_unstale":"2024-04-26T20:50:38.800505+0000","last_undegraded":"2024-04-26T20:50:38.800505+0000","last_fullsized":"2024-04-26T20:50:38.800505+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:36.499651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802860+0000","last_change":"2024-04-26T20:50:36.676688+0000","last_active":"2024-04-26T20:50:38.802860+0000","last_peered":"2024-04-26T20:50:38.802860+0000","last_clean":"2024-04-26T20:50:38.802860+0000","last_became_active":"2024-04-26T20:50:36.675819+0000","last_became_peered":"2024-04-26T20:50:36.675819+0000","last_unstale":"2024-04-26T20:50:38.802860+0000","last_undegraded":"2024-04-26T20:50:38.802860+0000","last_fullsized":"2024-04-26T20:50:38.802860+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:12:04.255812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800398+0000","last_change":"2024-04-26T20:50:32.642860+0000","last_active":"2024-04-26T20:50:38.800398+0000","last_peered":"2024-04-26T20:50:38.800398+0000","last_clean":"2024-04-26T20:50:38.800398+0000","last_became_active":"2024-04-26T20:50:32.642567+0000","last_became_peered":"2024-04-26T20:50:32.642567+0000","last_unstale":"2024-04-26T20:50:38.800398+0000","last_undegraded":"2024-04-26T20:50:38.800398+0000","last_fullsized":"2024-04-26T20:50:38.800398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:00:12.394204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.717653+0000","last_change":"2024-04-26T20:50:31.632961+0000","last_active":"2024-04-26T20:50:38.717653+0000","last_peered":"2024-04-26T20:50:38.717653+0000","last_clean":"2024-04-26T20:50:38.717653+0000","last_became_active":"2024-04-26T20:50:31.632044+0000","last_became_peered":"2024-04-26T20:50:31.632044+0000","last_unstale":"2024-04-26T20:50:38.717653+0000","last_undegraded":"2024-04-26T20:50:38.717653+0000","last_fullsized":"2024-04-26T20:50:38.717653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:23:29.866794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802614+0000","last_change":"2024-04-26T20:50:37.293243+0000","last_active":"2024-04-26T20:50:38.802614+0000","last_peered":"2024-04-26T20:50:38.802614+0000","last_clean":"2024-04-26T20:50:38.802614+0000","last_became_active":"2024-04-26T20:50:37.292840+0000","last_became_peered":"2024-04-26T20:50:37.292840+0000","last_unstale":"2024-04-26T20:50:38.802614+0000","last_undegraded":"2024-04-26T20:50:38.802614+0000","last_fullsized":"2024-04-26T20:50:38.802614+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:05:40.897275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803514+0000","last_change":"2024-04-26T20:50:37.294767+0000","last_active":"2024-04-26T20:50:38.803514+0000","last_peered":"2024-04-26T20:50:38.803514+0000","last_clean":"2024-04-26T20:50:38.803514+0000","last_became_active":"2024-04-26T20:50:37.294351+0000","last_became_peered":"2024-04-26T20:50:37.294351+0000","last_unstale":"2024-04-26T20:50:38.803514+0000","last_undegraded":"2024-04-26T20:50:38.803514+0000","last_fullsized":"2024-04-26T20:50:38.803514+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:31:21.850102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.660600+0000","last_change":"2024-04-26T20:50:32.648105+0000","last_active":"2024-04-26T20:50:38.660600+0000","last_peered":"2024-04-26T20:50:38.660600+0000","last_clean":"2024-04-26T20:50:38.660600+0000","last_became_active":"2024-04-26T20:50:32.647833+0000","last_became_peered":"2024-04-26T20:50:32.647833+0000","last_unstale":"2024-04-26T20:50:38.660600+0000","last_undegraded":"2024-04-26T20:50:38.660600+0000","last_fullsized":"2024-04-26T20:50:38.660600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:55.634765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.799847+0000","last_change":"2024-04-26T20:50:31.622923+0000","last_active":"2024-04-26T20:50:38.799847+0000","last_peered":"2024-04-26T20:50:38.799847+0000","last_clean":"2024-04-26T20:50:38.799847+0000","last_became_active":"2024-04-26T20:50:31.622533+0000","last_became_peered":"2024-04-26T20:50:31.622533+0000","last_unstale":"2024-04-26T20:50:38.799847+0000","last_undegraded":"2024-04-26T20:50:38.799847+0000","last_fullsized":"2024-04-26T20:50:38.799847+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:32:17.604576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.717573+0000","last_change":"2024-04-26T20:50:36.680804+0000","last_active":"2024-04-26T20:50:38.717573+0000","last_peered":"2024-04-26T20:50:38.717573+0000","last_clean":"2024-04-26T20:50:38.717573+0000","last_became_active":"2024-04-26T20:50:36.680438+0000","last_became_peered":"2024-04-26T20:50:36.680438+0000","last_unstale":"2024-04-26T20:50:38.717573+0000","last_undegraded":"2024-04-26T20:50:38.717573+0000","last_fullsized":"2024-04-26T20:50:38.717573+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:58:22.587813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802852+0000","last_change":"2024-04-26T20:50:32.649026+0000","last_active":"2024-04-26T20:50:38.802852+0000","last_peered":"2024-04-26T20:50:38.802852+0000","last_clean":"2024-04-26T20:50:38.802852+0000","last_became_active":"2024-04-26T20:50:32.648737+0000","last_became_peered":"2024-04-26T20:50:32.648737+0000","last_unstale":"2024-04-26T20:50:38.802852+0000","last_undegraded":"2024-04-26T20:50:38.802852+0000","last_fullsized":"2024-04-26T20:50:38.802852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:47:14.715739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802756+0000","last_change":"2024-04-26T20:50:31.631362+0000","last_active":"2024-04-26T20:50:38.802756+0000","last_peered":"2024-04-26T20:50:38.802756+0000","last_clean":"2024-04-26T20:50:38.802756+0000","last_became_active":"2024-04-26T20:50:31.631099+0000","last_became_peered":"2024-04-26T20:50:31.631099+0000","last_unstale":"2024-04-26T20:50:38.802756+0000","last_undegraded":"2024-04-26T20:50:38.802756+0000","last_fullsized":"2024-04-26T20:50:38.802756+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:20:49.853735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.717285+0000","last_change":"2024-04-26T20:50:31.629712+0000","last_active":"2024-04-26T20:50:38.717285+0000","last_peered":"2024-04-26T20:50:38.717285+0000","last_clean":"2024-04-26T20:50:38.717285+0000","last_became_active":"2024-04-26T20:50:31.629412+0000","last_became_peered":"2024-04-26T20:50:31.629412+0000","last_unstale":"2024-04-26T20:50:38.717285+0000","last_undegraded":"2024-04-26T20:50:38.717285+0000","last_fullsized":"2024-04-26T20:50:38.717285+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:32:17.604576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803849+0000","last_change":"2024-04-26T20:50:32.638185+0000","last_active":"2024-04-26T20:50:38.803849+0000","last_peered":"2024-04-26T20:50:38.803849+0000","last_clean":"2024-04-26T20:50:38.803849+0000","last_became_active":"2024-04-26T20:50:32.637917+0000","last_became_peered":"2024-04-26T20:50:32.637917+0000","last_unstale":"2024-04-26T20:50:38.803849+0000","last_undegraded":"2024-04-26T20:50:38.803849+0000","last_fullsized":"2024-04-26T20:50:38.803849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:00:12.394204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803278+0000","last_change":"2024-04-26T20:50:36.682087+0000","last_active":"2024-04-26T20:50:38.803278+0000","last_peered":"2024-04-26T20:50:38.803278+0000","last_clean":"2024-04-26T20:50:38.803278+0000","last_became_active":"2024-04-26T20:50:36.681543+0000","last_became_peered":"2024-04-26T20:50:36.681543+0000","last_unstale":"2024-04-26T20:50:38.803278+0000","last_undegraded":"2024-04-26T20:50:38.803278+0000","last_fullsized":"2024-04-26T20:50:38.803278+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:57:44.640580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802464+0000","last_change":"2024-04-26T20:50:31.632761+0000","last_active":"2024-04-26T20:50:38.802464+0000","last_peered":"2024-04-26T20:50:38.802464+0000","last_clean":"2024-04-26T20:50:38.802464+0000","last_became_active":"2024-04-26T20:50:31.632381+0000","last_became_peered":"2024-04-26T20:50:31.632381+0000","last_unstale":"2024-04-26T20:50:38.802464+0000","last_undegraded":"2024-04-26T20:50:38.802464+0000","last_fullsized":"2024-04-26T20:50:38.802464+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:21:09.963731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.662120+0000","last_change":"2024-04-26T20:50:32.637920+0000","last_active":"2024-04-26T20:50:38.662120+0000","last_peered":"2024-04-26T20:50:38.662120+0000","last_clean":"2024-04-26T20:50:38.662120+0000","last_became_active":"2024-04-26T20:50:32.637618+0000","last_became_peered":"2024-04-26T20:50:32.637618+0000","last_unstale":"2024-04-26T20:50:38.662120+0000","last_undegraded":"2024-04-26T20:50:38.662120+0000","last_fullsized":"2024-04-26T20:50:38.662120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:46:20.863042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803215+0000","last_change":"2024-04-26T20:50:36.678219+0000","last_active":"2024-04-26T20:50:38.803215+0000","last_peered":"2024-04-26T20:50:38.803215+0000","last_clean":"2024-04-26T20:50:38.803215+0000","last_became_active":"2024-04-26T20:50:36.677732+0000","last_became_peered":"2024-04-26T20:50:36.677732+0000","last_unstale":"2024-04-26T20:50:38.803215+0000","last_undegraded":"2024-04-26T20:50:38.803215+0000","last_fullsized":"2024-04-26T20:50:38.803215+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:03:12.186398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.716924+0000","last_change":"2024-04-26T20:50:32.638997+0000","last_active":"2024-04-26T20:50:38.716924+0000","last_peered":"2024-04-26T20:50:38.716924+0000","last_clean":"2024-04-26T20:50:38.716924+0000","last_became_active":"2024-04-26T20:50:32.638708+0000","last_became_peered":"2024-04-26T20:50:32.638708+0000","last_unstale":"2024-04-26T20:50:38.716924+0000","last_undegraded":"2024-04-26T20:50:38.716924+0000","last_fullsized":"2024-04-26T20:50:38.716924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:36.651960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802480+0000","last_change":"2024-04-26T20:50:32.648305+0000","last_active":"2024-04-26T20:50:38.802480+0000","last_peered":"2024-04-26T20:50:38.802480+0000","last_clean":"2024-04-26T20:50:38.802480+0000","last_became_active":"2024-04-26T20:50:32.648021+0000","last_became_peered":"2024-04-26T20:50:32.648021+0000","last_unstale":"2024-04-26T20:50:38.802480+0000","last_undegraded":"2024-04-26T20:50:38.802480+0000","last_fullsized":"2024-04-26T20:50:38.802480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:46:20.863042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800030+0000","last_change":"2024-04-26T20:50:32.649133+0000","last_active":"2024-04-26T20:50:38.800030+0000","last_peered":"2024-04-26T20:50:38.800030+0000","last_clean":"2024-04-26T20:50:38.800030+0000","last_became_active":"2024-04-26T20:50:32.648855+0000","last_became_peered":"2024-04-26T20:50:32.648855+0000","last_unstale":"2024-04-26T20:50:38.800030+0000","last_undegraded":"2024-04-26T20:50:38.800030+0000","last_fullsized":"2024-04-26T20:50:38.800030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:58:18.552075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803763+0000","last_change":"2024-04-26T20:50:32.650322+0000","last_active":"2024-04-26T20:50:38.803763+0000","last_peered":"2024-04-26T20:50:38.803763+0000","last_clean":"2024-04-26T20:50:38.803763+0000","last_became_active":"2024-04-26T20:50:32.650011+0000","last_became_peered":"2024-04-26T20:50:32.650011+0000","last_unstale":"2024-04-26T20:50:38.803763+0000","last_undegraded":"2024-04-26T20:50:38.803763+0000","last_fullsized":"2024-04-26T20:50:38.803763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:15:55.563522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803550+0000","last_change":"2024-04-26T20:50:32.649914+0000","last_active":"2024-04-26T20:50:38.803550+0000","last_peered":"2024-04-26T20:50:38.803550+0000","last_clean":"2024-04-26T20:50:38.803550+0000","last_became_active":"2024-04-26T20:50:32.649439+0000","last_became_peered":"2024-04-26T20:50:32.649439+0000","last_unstale":"2024-04-26T20:50:38.803550+0000","last_undegraded":"2024-04-26T20:50:38.803550+0000","last_fullsized":"2024-04-26T20:50:38.803550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:33:28.969815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661177+0000","last_change":"2024-04-26T20:50:32.648570+0000","last_active":"2024-04-26T20:50:38.661177+0000","last_peered":"2024-04-26T20:50:38.661177+0000","last_clean":"2024-04-26T20:50:38.661177+0000","last_became_active":"2024-04-26T20:50:32.646951+0000","last_became_peered":"2024-04-26T20:50:32.646951+0000","last_unstale":"2024-04-26T20:50:38.661177+0000","last_undegraded":"2024-04-26T20:50:38.661177+0000","last_fullsized":"2024-04-26T20:50:38.661177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:15:55.563522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.717644+0000","last_change":"2024-04-26T20:50:32.640208+0000","last_active":"2024-04-26T20:50:38.717644+0000","last_peered":"2024-04-26T20:50:38.717644+0000","last_clean":"2024-04-26T20:50:38.717644+0000","last_became_active":"2024-04-26T20:50:32.639869+0000","last_became_peered":"2024-04-26T20:50:32.639869+0000","last_unstale":"2024-04-26T20:50:38.717644+0000","last_undegraded":"2024-04-26T20:50:38.717644+0000","last_fullsized":"2024-04-26T20:50:38.717644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:15:55.563522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803850+0000","last_change":"2024-04-26T20:50:32.650173+0000","last_active":"2024-04-26T20:50:38.803850+0000","last_peered":"2024-04-26T20:50:38.803850+0000","last_clean":"2024-04-26T20:50:38.803850+0000","last_became_active":"2024-04-26T20:50:32.649876+0000","last_became_peered":"2024-04-26T20:50:32.649876+0000","last_unstale":"2024-04-26T20:50:38.803850+0000","last_undegraded":"2024-04-26T20:50:38.803850+0000","last_fullsized":"2024-04-26T20:50:38.803850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:00:12.394204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.799788+0000","last_change":"2024-04-26T20:50:32.651129+0000","last_active":"2024-04-26T20:50:38.799788+0000","last_peered":"2024-04-26T20:50:38.799788+0000","last_clean":"2024-04-26T20:50:38.799788+0000","last_became_active":"2024-04-26T20:50:32.650853+0000","last_became_peered":"2024-04-26T20:50:32.650853+0000","last_unstale":"2024-04-26T20:50:38.799788+0000","last_undegraded":"2024-04-26T20:50:38.799788+0000","last_fullsized":"2024-04-26T20:50:38.799788+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:35.639518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802726+0000","last_change":"2024-04-26T20:50:32.650192+0000","last_active":"2024-04-26T20:50:38.802726+0000","last_peered":"2024-04-26T20:50:38.802726+0000","last_clean":"2024-04-26T20:50:38.802726+0000","last_became_active":"2024-04-26T20:50:32.649911+0000","last_became_peered":"2024-04-26T20:50:32.649911+0000","last_unstale":"2024-04-26T20:50:38.802726+0000","last_undegraded":"2024-04-26T20:50:38.802726+0000","last_fullsized":"2024-04-26T20:50:38.802726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:09:36.737379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.799890+0000","last_change":"2024-04-26T20:50:32.641183+0000","last_active":"2024-04-26T20:50:38.799890+0000","last_peered":"2024-04-26T20:50:38.799890+0000","last_clean":"2024-04-26T20:50:38.799890+0000","last_became_active":"2024-04-26T20:50:32.640584+0000","last_became_peered":"2024-04-26T20:50:32.640584+0000","last_unstale":"2024-04-26T20:50:38.799890+0000","last_undegraded":"2024-04-26T20:50:38.799890+0000","last_fullsized":"2024-04-26T20:50:38.799890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:13:10.610044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803653+0000","last_change":"2024-04-26T20:50:32.651661+0000","last_active":"2024-04-26T20:50:38.803653+0000","last_peered":"2024-04-26T20:50:38.803653+0000","last_clean":"2024-04-26T20:50:38.803653+0000","last_became_active":"2024-04-26T20:50:32.651316+0000","last_became_peered":"2024-04-26T20:50:32.651316+0000","last_unstale":"2024-04-26T20:50:38.803653+0000","last_undegraded":"2024-04-26T20:50:38.803653+0000","last_fullsized":"2024-04-26T20:50:38.803653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:09:36.737379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.717426+0000","last_change":"2024-04-26T20:50:32.645619+0000","last_active":"2024-04-26T20:50:38.717426+0000","last_peered":"2024-04-26T20:50:38.717426+0000","last_clean":"2024-04-26T20:50:38.717426+0000","last_became_active":"2024-04-26T20:50:32.644645+0000","last_became_peered":"2024-04-26T20:50:32.644645+0000","last_unstale":"2024-04-26T20:50:38.717426+0000","last_undegraded":"2024-04-26T20:50:38.717426+0000","last_fullsized":"2024-04-26T20:50:38.717426+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:54.967490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.801055+0000","last_change":"2024-04-26T20:50:32.646646+0000","last_active":"2024-04-26T20:50:38.801055+0000","last_peered":"2024-04-26T20:50:38.801055+0000","last_clean":"2024-04-26T20:50:38.801055+0000","last_became_active":"2024-04-26T20:50:32.645404+0000","last_became_peered":"2024-04-26T20:50:32.645404+0000","last_unstale":"2024-04-26T20:50:38.801055+0000","last_undegraded":"2024-04-26T20:50:38.801055+0000","last_fullsized":"2024-04-26T20:50:38.801055+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:58:18.552075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803547+0000","last_change":"2024-04-26T20:50:32.646750+0000","last_active":"2024-04-26T20:50:38.803547+0000","last_peered":"2024-04-26T20:50:38.803547+0000","last_clean":"2024-04-26T20:50:38.803547+0000","last_became_active":"2024-04-26T20:50:32.646447+0000","last_became_peered":"2024-04-26T20:50:32.646447+0000","last_unstale":"2024-04-26T20:50:38.803547+0000","last_undegraded":"2024-04-26T20:50:38.803547+0000","last_fullsized":"2024-04-26T20:50:38.803547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:53:11.650251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800765+0000","last_change":"2024-04-26T20:50:32.642132+0000","last_active":"2024-04-26T20:50:38.800765+0000","last_peered":"2024-04-26T20:50:38.800765+0000","last_clean":"2024-04-26T20:50:38.800765+0000","last_became_active":"2024-04-26T20:50:32.641690+0000","last_became_peered":"2024-04-26T20:50:32.641690+0000","last_unstale":"2024-04-26T20:50:38.800765+0000","last_undegraded":"2024-04-26T20:50:38.800765+0000","last_fullsized":"2024-04-26T20:50:38.800765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:59:04.962815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.799661+0000","last_change":"2024-04-26T20:50:32.644332+0000","last_active":"2024-04-26T20:50:38.799661+0000","last_peered":"2024-04-26T20:50:38.799661+0000","last_clean":"2024-04-26T20:50:38.799661+0000","last_became_active":"2024-04-26T20:50:32.643783+0000","last_became_peered":"2024-04-26T20:50:32.643783+0000","last_unstale":"2024-04-26T20:50:38.799661+0000","last_undegraded":"2024-04-26T20:50:38.799661+0000","last_fullsized":"2024-04-26T20:50:38.799661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:56:51.926886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.660696+0000","last_change":"2024-04-26T20:50:32.638177+0000","last_active":"2024-04-26T20:50:38.660696+0000","last_peered":"2024-04-26T20:50:38.660696+0000","last_clean":"2024-04-26T20:50:38.660696+0000","last_became_active":"2024-04-26T20:50:32.637905+0000","last_became_peered":"2024-04-26T20:50:32.637905+0000","last_unstale":"2024-04-26T20:50:38.660696+0000","last_undegraded":"2024-04-26T20:50:38.660696+0000","last_fullsized":"2024-04-26T20:50:38.660696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:36.651960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803236+0000","last_change":"2024-04-26T20:50:32.637456+0000","last_active":"2024-04-26T20:50:38.803236+0000","last_peered":"2024-04-26T20:50:38.803236+0000","last_clean":"2024-04-26T20:50:38.803236+0000","last_became_active":"2024-04-26T20:50:32.637188+0000","last_became_peered":"2024-04-26T20:50:32.637188+0000","last_unstale":"2024-04-26T20:50:38.803236+0000","last_undegraded":"2024-04-26T20:50:38.803236+0000","last_fullsized":"2024-04-26T20:50:38.803236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:54.967490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802268+0000","last_change":"2024-04-26T20:50:32.649659+0000","last_active":"2024-04-26T20:50:38.802268+0000","last_peered":"2024-04-26T20:50:38.802268+0000","last_clean":"2024-04-26T20:50:38.802268+0000","last_became_active":"2024-04-26T20:50:32.645837+0000","last_became_peered":"2024-04-26T20:50:32.645837+0000","last_unstale":"2024-04-26T20:50:38.802268+0000","last_undegraded":"2024-04-26T20:50:38.802268+0000","last_fullsized":"2024-04-26T20:50:38.802268+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:55.634765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.662458+0000","last_change":"2024-04-26T20:50:32.643463+0000","last_active":"2024-04-26T20:50:38.662458+0000","last_peered":"2024-04-26T20:50:38.662458+0000","last_clean":"2024-04-26T20:50:38.662458+0000","last_became_active":"2024-04-26T20:50:32.642783+0000","last_became_peered":"2024-04-26T20:50:32.642783+0000","last_unstale":"2024-04-26T20:50:38.662458+0000","last_undegraded":"2024-04-26T20:50:38.662458+0000","last_fullsized":"2024-04-26T20:50:38.662458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:53:11.650251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.717503+0000","last_change":"2024-04-26T20:50:32.643690+0000","last_active":"2024-04-26T20:50:38.717503+0000","last_peered":"2024-04-26T20:50:38.717503+0000","last_clean":"2024-04-26T20:50:38.717503+0000","last_became_active":"2024-04-26T20:50:32.643412+0000","last_became_peered":"2024-04-26T20:50:32.643412+0000","last_unstale":"2024-04-26T20:50:38.717503+0000","last_undegraded":"2024-04-26T20:50:38.717503+0000","last_fullsized":"2024-04-26T20:50:38.717503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:55.634765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.717608+0000","last_change":"2024-04-26T20:50:32.643116+0000","last_active":"2024-04-26T20:50:38.717608+0000","last_peered":"2024-04-26T20:50:38.717608+0000","last_clean":"2024-04-26T20:50:38.717608+0000","last_became_active":"2024-04-26T20:50:32.642598+0000","last_became_peered":"2024-04-26T20:50:32.642598+0000","last_unstale":"2024-04-26T20:50:38.717608+0000","last_undegraded":"2024-04-26T20:50:38.717608+0000","last_fullsized":"2024-04-26T20:50:38.717608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:47:14.715739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.801040+0000","last_change":"2024-04-26T20:50:32.654019+0000","last_active":"2024-04-26T20:50:38.801040+0000","last_peered":"2024-04-26T20:50:38.801040+0000","last_clean":"2024-04-26T20:50:38.801040+0000","last_became_active":"2024-04-26T20:50:32.653569+0000","last_became_peered":"2024-04-26T20:50:32.653569+0000","last_unstale":"2024-04-26T20:50:38.801040+0000","last_undegraded":"2024-04-26T20:50:38.801040+0000","last_fullsized":"2024-04-26T20:50:38.801040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:13:10.610044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802660+0000","last_change":"2024-04-26T20:50:32.646837+0000","last_active":"2024-04-26T20:50:38.802660+0000","last_peered":"2024-04-26T20:50:38.802660+0000","last_clean":"2024-04-26T20:50:38.802660+0000","last_became_active":"2024-04-26T20:50:32.646528+0000","last_became_peered":"2024-04-26T20:50:32.646528+0000","last_unstale":"2024-04-26T20:50:38.802660+0000","last_undegraded":"2024-04-26T20:50:38.802660+0000","last_fullsized":"2024-04-26T20:50:38.802660+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:54.967490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800781+0000","last_change":"2024-04-26T20:50:32.654006+0000","last_active":"2024-04-26T20:50:38.800781+0000","last_peered":"2024-04-26T20:50:38.800781+0000","last_clean":"2024-04-26T20:50:38.800781+0000","last_became_active":"2024-04-26T20:50:32.653553+0000","last_became_peered":"2024-04-26T20:50:32.653553+0000","last_unstale":"2024-04-26T20:50:38.800781+0000","last_undegraded":"2024-04-26T20:50:38.800781+0000","last_fullsized":"2024-04-26T20:50:38.800781+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:00:12.394204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.799949+0000","last_change":"2024-04-26T20:50:32.641407+0000","last_active":"2024-04-26T20:50:38.799949+0000","last_peered":"2024-04-26T20:50:38.799949+0000","last_clean":"2024-04-26T20:50:38.799949+0000","last_became_active":"2024-04-26T20:50:32.640889+0000","last_became_peered":"2024-04-26T20:50:32.640889+0000","last_unstale":"2024-04-26T20:50:38.799949+0000","last_undegraded":"2024-04-26T20:50:38.799949+0000","last_fullsized":"2024-04-26T20:50:38.799949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:28:25.603590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800212+0000","last_change":"2024-04-26T20:50:32.645181+0000","last_active":"2024-04-26T20:50:38.800212+0000","last_peered":"2024-04-26T20:50:38.800212+0000","last_clean":"2024-04-26T20:50:38.800212+0000","last_became_active":"2024-04-26T20:50:32.644919+0000","last_became_peered":"2024-04-26T20:50:32.644919+0000","last_unstale":"2024-04-26T20:50:38.800212+0000","last_undegraded":"2024-04-26T20:50:38.800212+0000","last_fullsized":"2024-04-26T20:50:38.800212+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:40:19.711175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800200+0000","last_change":"2024-04-26T20:50:32.655095+0000","last_active":"2024-04-26T20:50:38.800200+0000","last_peered":"2024-04-26T20:50:38.800200+0000","last_clean":"2024-04-26T20:50:38.800200+0000","last_became_active":"2024-04-26T20:50:32.654821+0000","last_became_peered":"2024-04-26T20:50:32.654821+0000","last_unstale":"2024-04-26T20:50:38.800200+0000","last_undegraded":"2024-04-26T20:50:38.800200+0000","last_fullsized":"2024-04-26T20:50:38.800200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:33:28.969815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802460+0000","last_change":"2024-04-26T20:50:32.642769+0000","last_active":"2024-04-26T20:50:38.802460+0000","last_peered":"2024-04-26T20:50:38.802460+0000","last_clean":"2024-04-26T20:50:38.802460+0000","last_became_active":"2024-04-26T20:50:32.642506+0000","last_became_peered":"2024-04-26T20:50:32.642506+0000","last_unstale":"2024-04-26T20:50:38.802460+0000","last_undegraded":"2024-04-26T20:50:38.802460+0000","last_fullsized":"2024-04-26T20:50:38.802460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:55.634765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661146+0000","last_change":"2024-04-26T20:50:32.641048+0000","last_active":"2024-04-26T20:50:38.661146+0000","last_peered":"2024-04-26T20:50:38.661146+0000","last_clean":"2024-04-26T20:50:38.661146+0000","last_became_active":"2024-04-26T20:50:32.640572+0000","last_became_peered":"2024-04-26T20:50:32.640572+0000","last_unstale":"2024-04-26T20:50:38.661146+0000","last_undegraded":"2024-04-26T20:50:38.661146+0000","last_fullsized":"2024-04-26T20:50:38.661146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:47:14.715739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800925+0000","last_change":"2024-04-26T20:50:32.650971+0000","last_active":"2024-04-26T20:50:38.800925+0000","last_peered":"2024-04-26T20:50:38.800925+0000","last_clean":"2024-04-26T20:50:38.800925+0000","last_became_active":"2024-04-26T20:50:32.650713+0000","last_became_peered":"2024-04-26T20:50:32.650713+0000","last_unstale":"2024-04-26T20:50:38.800925+0000","last_undegraded":"2024-04-26T20:50:38.800925+0000","last_fullsized":"2024-04-26T20:50:38.800925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:28:25.603590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":864256,"data_stored":698343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":107483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":107519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":107393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":107553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":872448,"data_stored":698399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:51:00.008 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:51:00.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.140+0000 7f149e1f0640 1 Processor -- start 2024-04-26T20:51:00.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.140+0000 7f149e1f0640 1 -- start start 2024-04-26T20:51:00.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.141+0000 7f149e1f0640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f14980b2340 0x7f14980cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:00.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.141+0000 7f149e1f0640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f14980cd930 0x7f14980d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:00.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.141+0000 7f149e1f0640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14980d10d0 con 0x7f1498162810 2024-04-26T20:51:00.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.141+0000 7f149e1f0640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f14980d0750 con 0x7f14980cd930 2024-04-26T20:51:00.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.141+0000 7f149e1f0640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f14980d16a0 con 0x7f14980b2340 2024-04-26T20:51:00.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.141+0000 7f1497fff640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f14980cd930 0x7f14980d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:00.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.141+0000 7f14977fe640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f1498162810 0x7f1498070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:42014/0 (socket says 172.21.15.73:42014) 2024-04-26T20:51:00.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.141+0000 7f14977fe640 1 -- 172.21.15.73:0/1591608056 learned_addr learned my addr 172.21.15.73:0/1591608056 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:00.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.141+0000 7f1496ffd640 1 --2- 172.21.15.73:0/1591608056 >> v2:172.21.15.120:3300/0 conn(0x7f14980b2340 0x7f14980cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:00.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.141+0000 7f1497fff640 1 -- 172.21.15.73:0/1591608056 >> v2:172.21.15.120:3300/0 conn(0x7f14980b2340 msgr2=0x7f14980cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:00.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.141+0000 7f1497fff640 1 --2- 172.21.15.73:0/1591608056 >> v2:172.21.15.120:3300/0 conn(0x7f14980b2340 0x7f14980cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:00.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.141+0000 7f1497fff640 1 -- 172.21.15.73:0/1591608056 >> v1:172.21.15.73:6789/0 conn(0x7f1498162810 legacy=0x7f1498070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:00.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.141+0000 7f1497fff640 1 -- 172.21.15.73:0/1591608056 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14980d1370 con 0x7f14980cd930 2024-04-26T20:51:00.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.141+0000 7f14967fc640 1 -- 172.21.15.73:0/1591608056 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1875250611 0 0) 0x7f14980d10d0 con 0x7f1498162810 2024-04-26T20:51:00.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.142+0000 7f1497fff640 1 --2- 172.21.15.73:0/1591608056 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14980cd930 0x7f14980d1b30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f14980cd0a0 tx=0x7f149002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2bd980f0fa15c0d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:00.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.142+0000 7f14967fc640 1 -- 172.21.15.73:0/1591608056 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1490007740 con 0x7f14980cd930 2024-04-26T20:51:00.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.142+0000 7f14967fc640 1 -- 172.21.15.73:0/1591608056 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14900078e0 con 0x7f14980cd930 2024-04-26T20:51:00.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.142+0000 7f14967fc640 1 -- 172.21.15.73:0/1591608056 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1490007cb0 con 0x7f14980cd930 2024-04-26T20:51:00.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.143+0000 7f149e1f0640 1 -- 172.21.15.73:0/1591608056 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14980cd930 msgr2=0x7f14980d1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:00.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.143+0000 7f149e1f0640 1 --2- 172.21.15.73:0/1591608056 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14980cd930 0x7f14980d1b30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f14980cd0a0 tx=0x7f149002a680 comp rx=0 tx=0).stop 2024-04-26T20:51:00.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.143+0000 7f149e1f0640 1 -- 172.21.15.73:0/1591608056 shutdown_connections 2024-04-26T20:51:00.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.143+0000 7f149e1f0640 1 --2- 172.21.15.73:0/1591608056 >> v2:172.21.15.120:3300/0 conn(0x7f14980b2340 0x7f14980cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:00.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.143+0000 7f149e1f0640 1 --2- 172.21.15.73:0/1591608056 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14980cd930 0x7f14980d1b30 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:00.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.143+0000 7f149e1f0640 1 -- 172.21.15.73:0/1591608056 >> 172.21.15.73:0/1591608056 conn(0x7f14980d2da0 msgr2=0x7f14980d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:00.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.143+0000 7f149e1f0640 1 -- 172.21.15.73:0/1591608056 shutdown_connections 2024-04-26T20:51:00.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.143+0000 7f149e1f0640 1 -- 172.21.15.73:0/1591608056 wait complete. 2024-04-26T20:51:00.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.144+0000 7f149e1f0640 1 Processor -- start 2024-04-26T20:51:00.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.144+0000 7f149e1f0640 1 -- start start 2024-04-26T20:51:00.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.145+0000 7f149e1f0640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f14980b2340 0x7f14980b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:00.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.145+0000 7f149e1f0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14980cd930 0x7f14980ca430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:00.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.145+0000 7f149e1f0640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1498162810 0x7f14980cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:00.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.145+0000 7f149e1f0640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f14980d2070 con 0x7f1498162810 2024-04-26T20:51:00.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.145+0000 7f149e1f0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f14980d0470 con 0x7f14980cd930 2024-04-26T20:51:00.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.145+0000 7f149e1f0640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f14980d21f0 con 0x7f14980b2340 2024-04-26T20:51:00.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.145+0000 7f1497fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1498162810 0x7f14980cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:00.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.145+0000 7f1497fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1498162810 0x7f14980cedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42596/0 (socket says 172.21.15.73:42596) 2024-04-26T20:51:00.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.145+0000 7f1497fff640 1 -- 172.21.15.73:0/3640574326 learned_addr learned my addr 172.21.15.73:0/3640574326 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:00.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.145+0000 7f14977fe640 1 --2- 172.21.15.73:0/3640574326 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f14980b2340 0x7f14980b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:00.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.145+0000 7f1496ffd640 1 --2- 172.21.15.73:0/3640574326 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14980cd930 0x7f14980ca430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:00.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.145+0000 7f1497fff640 1 -- 172.21.15.73:0/3640574326 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f14980b2340 msgr2=0x7f14980b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:00.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.145+0000 7f1497fff640 1 --2- 172.21.15.73:0/3640574326 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f14980b2340 0x7f14980b3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:00.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.145+0000 7f1497fff640 1 -- 172.21.15.73:0/3640574326 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14980cd930 msgr2=0x7f14980ca430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:00.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.145+0000 7f1497fff640 1 --2- 172.21.15.73:0/3640574326 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14980cd930 0x7f14980ca430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:00.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.145+0000 7f1497fff640 1 -- 172.21.15.73:0/3640574326 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14980c6d00 con 0x7f1498162810 2024-04-26T20:51:00.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.145+0000 7f1497fff640 1 --2- 172.21.15.73:0/3640574326 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1498162810 0x7f14980cedb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f149002ac70 tx=0x7f1490004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:00.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.146+0000 7f147ffff640 1 -- 172.21.15.73:0/3640574326 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1490040070 con 0x7f1498162810 2024-04-26T20:51:00.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.146+0000 7f147ffff640 1 -- 172.21.15.73:0/3640574326 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1490031bc0 con 0x7f1498162810 2024-04-26T20:51:00.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.146+0000 7f149e1f0640 1 -- 172.21.15.73:0/3640574326 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14980c6fe0 con 0x7f1498162810 2024-04-26T20:51:00.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.146+0000 7f147ffff640 1 -- 172.21.15.73:0/3640574326 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1490045460 con 0x7f1498162810 2024-04-26T20:51:00.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.146+0000 7f149e1f0640 1 -- 172.21.15.73:0/3640574326 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14980c75a0 con 0x7f1498162810 2024-04-26T20:51:00.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.147+0000 7f149e1f0640 1 -- 172.21.15.73:0/3640574326 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14980694f0 con 0x7f1498162810 2024-04-26T20:51:00.151 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.147+0000 7f147ffff640 1 -- 172.21.15.73:0/3640574326 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1490039040 con 0x7f1498162810 2024-04-26T20:51:00.151 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.147+0000 7f147ffff640 1 --2- 172.21.15.73:0/3640574326 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f148406c8f0 0x7f148406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:00.151 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.148+0000 7f147ffff640 1 -- 172.21.15.73:0/3640574326 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f14900ad720 con 0x7f1498162810 2024-04-26T20:51:00.151 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.148+0000 7f14977fe640 1 --2- 172.21.15.73:0/3640574326 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f148406c8f0 0x7f148406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:00.151 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.148+0000 7f14977fe640 1 --2- 172.21.15.73:0/3640574326 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f148406c8f0 0x7f148406edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1488000f60 tx=0x7f1488007c00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:00.152 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.152+0000 7f147ffff640 1 -- 172.21.15.73:0/3640574326 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f149007b9c0 con 0x7f1498162810 2024-04-26T20:51:00.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.323+0000 7f149e1f0640 1 -- 172.21.15.73:0/3640574326 --> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f14980d27b0 con 0x7f148406c8f0 2024-04-26T20:51:00.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.331+0000 7f147ffff640 1 -- 172.21.15.73:0/3640574326 <== mgr.4107 v2:172.21.15.73:6832/666905549 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332533 (secure 0 0 0) 0x7f14980d27b0 con 0x7f148406c8f0 2024-04-26T20:51:00.336 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:51:00.338 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-26T20:51:00.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.334+0000 7f149e1f0640 1 -- 172.21.15.73:0/3640574326 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f148406c8f0 msgr2=0x7f148406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:00.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.334+0000 7f149e1f0640 1 --2- 172.21.15.73:0/3640574326 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f148406c8f0 0x7f148406edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1488000f60 tx=0x7f1488007c00 comp rx=0 tx=0).stop 2024-04-26T20:51:00.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.334+0000 7f149e1f0640 1 -- 172.21.15.73:0/3640574326 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1498162810 msgr2=0x7f14980cedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:00.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.334+0000 7f149e1f0640 1 --2- 172.21.15.73:0/3640574326 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1498162810 0x7f14980cedb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f149002ac70 tx=0x7f1490004a40 comp rx=0 tx=0).stop 2024-04-26T20:51:00.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.334+0000 7f149e1f0640 1 -- 172.21.15.73:0/3640574326 shutdown_connections 2024-04-26T20:51:00.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.334+0000 7f149e1f0640 1 --2- 172.21.15.73:0/3640574326 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1498162810 0x7f14980cedb0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:00.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.334+0000 7f149e1f0640 1 --2- 172.21.15.73:0/3640574326 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f148406c8f0 0x7f148406edb0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:00.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.334+0000 7f149e1f0640 1 --2- 172.21.15.73:0/3640574326 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14980cd930 0x7f14980ca430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:00.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.334+0000 7f149e1f0640 1 --2- 172.21.15.73:0/3640574326 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f14980b2340 0x7f14980b3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:00.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.334+0000 7f149e1f0640 1 -- 172.21.15.73:0/3640574326 >> 172.21.15.73:0/3640574326 conn(0x7f14980d2da0 msgr2=0x7f1498070b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:00.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.334+0000 7f149e1f0640 1 -- 172.21.15.73:0/3640574326 shutdown_connections 2024-04-26T20:51:00.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.334+0000 7f149e1f0640 1 -- 172.21.15.73:0/3640574326 wait complete. 2024-04-26T20:51:00.346 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-04-26T20:50:59.988321+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219468,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755252,"statfs":{"total":773094113280,"available":772869378048,"internally_reserved":0,"allocated":3383296,"data_stored":2049160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006060"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803145+0000","last_change":"2024-04-26T20:50:31.630770+0000","last_active":"2024-04-26T20:50:38.803145+0000","last_peered":"2024-04-26T20:50:38.803145+0000","last_clean":"2024-04-26T20:50:38.803145+0000","last_became_active":"2024-04-26T20:50:31.630518+0000","last_became_peered":"2024-04-26T20:50:31.630518+0000","last_unstale":"2024-04-26T20:50:38.803145+0000","last_undegraded":"2024-04-26T20:50:38.803145+0000","last_fullsized":"2024-04-26T20:50:38.803145+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:20:49.853735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803210+0000","last_change":"2024-04-26T20:50:32.649145+0000","last_active":"2024-04-26T20:50:38.803210+0000","last_peered":"2024-04-26T20:50:38.803210+0000","last_clean":"2024-04-26T20:50:38.803210+0000","last_became_active":"2024-04-26T20:50:32.648873+0000","last_became_peered":"2024-04-26T20:50:32.648873+0000","last_unstale":"2024-04-26T20:50:38.803210+0000","last_undegraded":"2024-04-26T20:50:38.803210+0000","last_fullsized":"2024-04-26T20:50:38.803210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:16:25.308786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802643+0000","last_change":"2024-04-26T20:50:37.296160+0000","last_active":"2024-04-26T20:50:38.802643+0000","last_peered":"2024-04-26T20:50:38.802643+0000","last_clean":"2024-04-26T20:50:38.802643+0000","last_became_active":"2024-04-26T20:50:37.295441+0000","last_became_peered":"2024-04-26T20:50:37.295441+0000","last_unstale":"2024-04-26T20:50:38.802643+0000","last_undegraded":"2024-04-26T20:50:38.802643+0000","last_fullsized":"2024-04-26T20:50:38.802643+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:20:48.160586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803389+0000","last_change":"2024-04-26T20:50:31.634354+0000","last_active":"2024-04-26T20:50:38.803389+0000","last_peered":"2024-04-26T20:50:38.803389+0000","last_clean":"2024-04-26T20:50:38.803389+0000","last_became_active":"2024-04-26T20:50:31.634084+0000","last_became_peered":"2024-04-26T20:50:31.634084+0000","last_unstale":"2024-04-26T20:50:38.803389+0000","last_undegraded":"2024-04-26T20:50:38.803389+0000","last_fullsized":"2024-04-26T20:50:38.803389+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:49:28.099373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802875+0000","last_change":"2024-04-26T20:50:32.645829+0000","last_active":"2024-04-26T20:50:38.802875+0000","last_peered":"2024-04-26T20:50:38.802875+0000","last_clean":"2024-04-26T20:50:38.802875+0000","last_became_active":"2024-04-26T20:50:32.645589+0000","last_became_peered":"2024-04-26T20:50:32.645589+0000","last_unstale":"2024-04-26T20:50:38.802875+0000","last_undegraded":"2024-04-26T20:50:38.802875+0000","last_fullsized":"2024-04-26T20:50:38.802875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:36.651960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661856+0000","last_change":"2024-04-26T20:50:36.675389+0000","last_active":"2024-04-26T20:50:38.661856+0000","last_peered":"2024-04-26T20:50:38.661856+0000","last_clean":"2024-04-26T20:50:38.661856+0000","last_became_active":"2024-04-26T20:50:36.675027+0000","last_became_peered":"2024-04-26T20:50:36.675027+0000","last_unstale":"2024-04-26T20:50:38.661856+0000","last_undegraded":"2024-04-26T20:50:38.661856+0000","last_fullsized":"2024-04-26T20:50:38.661856+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:13:14.645783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800942+0000","last_change":"2024-04-26T20:50:31.631388+0000","last_active":"2024-04-26T20:50:38.800942+0000","last_peered":"2024-04-26T20:50:38.800942+0000","last_clean":"2024-04-26T20:50:38.800942+0000","last_became_active":"2024-04-26T20:50:31.631121+0000","last_became_peered":"2024-04-26T20:50:31.631121+0000","last_unstale":"2024-04-26T20:50:38.800942+0000","last_undegraded":"2024-04-26T20:50:38.800942+0000","last_fullsized":"2024-04-26T20:50:38.800942+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:46:19.858595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800306+0000","last_change":"2024-04-26T20:50:32.644708+0000","last_active":"2024-04-26T20:50:38.800306+0000","last_peered":"2024-04-26T20:50:38.800306+0000","last_clean":"2024-04-26T20:50:38.800306+0000","last_became_active":"2024-04-26T20:50:32.644440+0000","last_became_peered":"2024-04-26T20:50:32.644440+0000","last_unstale":"2024-04-26T20:50:38.800306+0000","last_undegraded":"2024-04-26T20:50:38.800306+0000","last_fullsized":"2024-04-26T20:50:38.800306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:40:19.711175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.662138+0000","last_change":"2024-04-26T20:50:36.676949+0000","last_active":"2024-04-26T20:50:38.662138+0000","last_peered":"2024-04-26T20:50:38.662138+0000","last_clean":"2024-04-26T20:50:38.662138+0000","last_became_active":"2024-04-26T20:50:36.676533+0000","last_became_peered":"2024-04-26T20:50:36.676533+0000","last_unstale":"2024-04-26T20:50:38.662138+0000","last_undegraded":"2024-04-26T20:50:38.662138+0000","last_fullsized":"2024-04-26T20:50:38.662138+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:59:08.998553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800828+0000","last_change":"2024-04-26T20:50:31.632017+0000","last_active":"2024-04-26T20:50:38.800828+0000","last_peered":"2024-04-26T20:50:38.800828+0000","last_clean":"2024-04-26T20:50:38.800828+0000","last_became_active":"2024-04-26T20:50:31.631717+0000","last_became_peered":"2024-04-26T20:50:31.631717+0000","last_unstale":"2024-04-26T20:50:38.800828+0000","last_undegraded":"2024-04-26T20:50:38.800828+0000","last_fullsized":"2024-04-26T20:50:38.800828+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:32:17.604576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800366+0000","last_change":"2024-04-26T20:50:32.644081+0000","last_active":"2024-04-26T20:50:38.800366+0000","last_peered":"2024-04-26T20:50:38.800366+0000","last_clean":"2024-04-26T20:50:38.800366+0000","last_became_active":"2024-04-26T20:50:32.643809+0000","last_became_peered":"2024-04-26T20:50:32.643809+0000","last_unstale":"2024-04-26T20:50:38.800366+0000","last_undegraded":"2024-04-26T20:50:38.800366+0000","last_fullsized":"2024-04-26T20:50:38.800366+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:16:25.308786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.662248+0000","last_change":"2024-04-26T20:50:36.679187+0000","last_active":"2024-04-26T20:50:38.662248+0000","last_peered":"2024-04-26T20:50:38.662248+0000","last_clean":"2024-04-26T20:50:38.662248+0000","last_became_active":"2024-04-26T20:50:36.678782+0000","last_became_peered":"2024-04-26T20:50:36.678782+0000","last_unstale":"2024-04-26T20:50:38.662248+0000","last_undegraded":"2024-04-26T20:50:38.662248+0000","last_fullsized":"2024-04-26T20:50:38.662248+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:12.140650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.717057+0000","last_change":"2024-04-26T20:50:32.646429+0000","last_active":"2024-04-26T20:50:38.717057+0000","last_peered":"2024-04-26T20:50:38.717057+0000","last_clean":"2024-04-26T20:50:38.717057+0000","last_became_active":"2024-04-26T20:50:32.646134+0000","last_became_peered":"2024-04-26T20:50:32.646134+0000","last_unstale":"2024-04-26T20:50:38.717057+0000","last_undegraded":"2024-04-26T20:50:38.717057+0000","last_fullsized":"2024-04-26T20:50:38.717057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:09:36.737379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800514+0000","last_change":"2024-04-26T20:50:31.631250+0000","last_active":"2024-04-26T20:50:38.800514+0000","last_peered":"2024-04-26T20:50:38.800514+0000","last_clean":"2024-04-26T20:50:38.800514+0000","last_became_active":"2024-04-26T20:50:31.630277+0000","last_became_peered":"2024-04-26T20:50:31.630277+0000","last_unstale":"2024-04-26T20:50:38.800514+0000","last_undegraded":"2024-04-26T20:50:38.800514+0000","last_fullsized":"2024-04-26T20:50:38.800514+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:06:39.630339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.662616+0000","last_change":"2024-04-26T20:50:36.676056+0000","last_active":"2024-04-26T20:50:38.662616+0000","last_peered":"2024-04-26T20:50:38.662616+0000","last_clean":"2024-04-26T20:50:38.662616+0000","last_became_active":"2024-04-26T20:50:36.675706+0000","last_became_peered":"2024-04-26T20:50:36.675706+0000","last_unstale":"2024-04-26T20:50:38.662616+0000","last_undegraded":"2024-04-26T20:50:38.662616+0000","last_fullsized":"2024-04-26T20:50:38.662616+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:13:08.850173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802373+0000","last_change":"2024-04-26T20:50:32.650473+0000","last_active":"2024-04-26T20:50:38.802373+0000","last_peered":"2024-04-26T20:50:38.802373+0000","last_clean":"2024-04-26T20:50:38.802373+0000","last_became_active":"2024-04-26T20:50:32.649711+0000","last_became_peered":"2024-04-26T20:50:32.649711+0000","last_unstale":"2024-04-26T20:50:38.802373+0000","last_undegraded":"2024-04-26T20:50:38.802373+0000","last_fullsized":"2024-04-26T20:50:38.802373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:36.651960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661208+0000","last_change":"2024-04-26T20:50:31.629572+0000","last_active":"2024-04-26T20:50:38.661208+0000","last_peered":"2024-04-26T20:50:38.661208+0000","last_clean":"2024-04-26T20:50:38.661208+0000","last_became_active":"2024-04-26T20:50:31.629287+0000","last_became_peered":"2024-04-26T20:50:31.629287+0000","last_unstale":"2024-04-26T20:50:38.661208+0000","last_undegraded":"2024-04-26T20:50:38.661208+0000","last_fullsized":"2024-04-26T20:50:38.661208+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:36.499651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.662538+0000","last_change":"2024-04-26T20:50:36.681910+0000","last_active":"2024-04-26T20:50:38.662538+0000","last_peered":"2024-04-26T20:50:38.662538+0000","last_clean":"2024-04-26T20:50:38.662538+0000","last_became_active":"2024-04-26T20:50:36.681432+0000","last_became_peered":"2024-04-26T20:50:36.681432+0000","last_unstale":"2024-04-26T20:50:38.662538+0000","last_undegraded":"2024-04-26T20:50:38.662538+0000","last_fullsized":"2024-04-26T20:50:38.662538+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:27:18.035795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802910+0000","last_change":"2024-04-26T20:50:32.647018+0000","last_active":"2024-04-26T20:50:38.802910+0000","last_peered":"2024-04-26T20:50:38.802910+0000","last_clean":"2024-04-26T20:50:38.802910+0000","last_became_active":"2024-04-26T20:50:32.646748+0000","last_became_peered":"2024-04-26T20:50:32.646748+0000","last_unstale":"2024-04-26T20:50:38.802910+0000","last_undegraded":"2024-04-26T20:50:38.802910+0000","last_fullsized":"2024-04-26T20:50:38.802910+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:56:51.926886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.660717+0000","last_change":"2024-04-26T20:50:31.631760+0000","last_active":"2024-04-26T20:50:38.660717+0000","last_peered":"2024-04-26T20:50:38.660717+0000","last_clean":"2024-04-26T20:50:38.660717+0000","last_became_active":"2024-04-26T20:50:31.629437+0000","last_became_peered":"2024-04-26T20:50:31.629437+0000","last_unstale":"2024-04-26T20:50:38.660717+0000","last_undegraded":"2024-04-26T20:50:38.660717+0000","last_fullsized":"2024-04-26T20:50:38.660717+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:23:29.866794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661242+0000","last_change":"2024-04-26T20:50:37.295810+0000","last_active":"2024-04-26T20:50:38.661242+0000","last_peered":"2024-04-26T20:50:38.661242+0000","last_clean":"2024-04-26T20:50:38.661242+0000","last_became_active":"2024-04-26T20:50:37.295425+0000","last_became_peered":"2024-04-26T20:50:37.295425+0000","last_unstale":"2024-04-26T20:50:38.661242+0000","last_undegraded":"2024-04-26T20:50:38.661242+0000","last_fullsized":"2024-04-26T20:50:38.661242+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:31:21.850102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.662187+0000","last_change":"2024-04-26T20:50:32.638584+0000","last_active":"2024-04-26T20:50:38.662187+0000","last_peered":"2024-04-26T20:50:38.662187+0000","last_clean":"2024-04-26T20:50:38.662187+0000","last_became_active":"2024-04-26T20:50:32.637734+0000","last_became_peered":"2024-04-26T20:50:32.637734+0000","last_unstale":"2024-04-26T20:50:38.662187+0000","last_undegraded":"2024-04-26T20:50:38.662187+0000","last_fullsized":"2024-04-26T20:50:38.662187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:37.504098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800397+0000","last_change":"2024-04-26T20:50:31.634926+0000","last_active":"2024-04-26T20:50:38.800397+0000","last_peered":"2024-04-26T20:50:38.800397+0000","last_clean":"2024-04-26T20:50:38.800397+0000","last_became_active":"2024-04-26T20:50:31.634278+0000","last_became_peered":"2024-04-26T20:50:31.634278+0000","last_unstale":"2024-04-26T20:50:38.800397+0000","last_undegraded":"2024-04-26T20:50:38.800397+0000","last_fullsized":"2024-04-26T20:50:38.800397+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:36.499651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.662287+0000","last_change":"2024-04-26T20:50:36.678377+0000","last_active":"2024-04-26T20:50:38.662287+0000","last_peered":"2024-04-26T20:50:38.662287+0000","last_clean":"2024-04-26T20:50:38.662287+0000","last_became_active":"2024-04-26T20:50:36.677946+0000","last_became_peered":"2024-04-26T20:50:36.677946+0000","last_unstale":"2024-04-26T20:50:38.662287+0000","last_undegraded":"2024-04-26T20:50:38.662287+0000","last_fullsized":"2024-04-26T20:50:38.662287+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:31:21.850102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803961+0000","last_change":"2024-04-26T20:50:32.637345+0000","last_active":"2024-04-26T20:50:38.803961+0000","last_peered":"2024-04-26T20:50:38.803961+0000","last_clean":"2024-04-26T20:50:38.803961+0000","last_became_active":"2024-04-26T20:50:32.637071+0000","last_became_peered":"2024-04-26T20:50:32.637071+0000","last_unstale":"2024-04-26T20:50:38.803961+0000","last_undegraded":"2024-04-26T20:50:38.803961+0000","last_fullsized":"2024-04-26T20:50:38.803961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:47:14.715739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.717764+0000","last_change":"2024-04-26T20:50:31.631119+0000","last_active":"2024-04-26T20:50:38.717764+0000","last_peered":"2024-04-26T20:50:38.717764+0000","last_clean":"2024-04-26T20:50:38.717764+0000","last_became_active":"2024-04-26T20:50:31.630825+0000","last_became_peered":"2024-04-26T20:50:31.630825+0000","last_unstale":"2024-04-26T20:50:38.717764+0000","last_undegraded":"2024-04-26T20:50:38.717764+0000","last_fullsized":"2024-04-26T20:50:38.717764+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:26.601021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661961+0000","last_change":"2024-04-26T20:50:37.295783+0000","last_active":"2024-04-26T20:50:38.661961+0000","last_peered":"2024-04-26T20:50:38.661961+0000","last_clean":"2024-04-26T20:50:38.661961+0000","last_became_active":"2024-04-26T20:50:37.295382+0000","last_became_peered":"2024-04-26T20:50:37.295382+0000","last_unstale":"2024-04-26T20:50:38.661961+0000","last_undegraded":"2024-04-26T20:50:38.661961+0000","last_fullsized":"2024-04-26T20:50:38.661961+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:58:22.587813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803938+0000","last_change":"2024-04-26T20:50:32.649792+0000","last_active":"2024-04-26T20:50:38.803938+0000","last_peered":"2024-04-26T20:50:38.803938+0000","last_clean":"2024-04-26T20:50:38.803938+0000","last_became_active":"2024-04-26T20:50:32.649510+0000","last_became_peered":"2024-04-26T20:50:32.649510+0000","last_unstale":"2024-04-26T20:50:38.803938+0000","last_undegraded":"2024-04-26T20:50:38.803938+0000","last_fullsized":"2024-04-26T20:50:38.803938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:35.639518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661244+0000","last_change":"2024-04-26T20:50:31.631623+0000","last_active":"2024-04-26T20:50:38.661244+0000","last_peered":"2024-04-26T20:50:38.661244+0000","last_clean":"2024-04-26T20:50:38.661244+0000","last_became_active":"2024-04-26T20:50:31.631312+0000","last_became_peered":"2024-04-26T20:50:31.631312+0000","last_unstale":"2024-04-26T20:50:38.661244+0000","last_undegraded":"2024-04-26T20:50:38.661244+0000","last_fullsized":"2024-04-26T20:50:38.661244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:46:19.858595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802802+0000","last_change":"2024-04-26T20:50:37.295508+0000","last_active":"2024-04-26T20:50:38.802802+0000","last_peered":"2024-04-26T20:50:38.802802+0000","last_clean":"2024-04-26T20:50:38.802802+0000","last_became_active":"2024-04-26T20:50:37.294858+0000","last_became_peered":"2024-04-26T20:50:37.294858+0000","last_unstale":"2024-04-26T20:50:38.802802+0000","last_undegraded":"2024-04-26T20:50:38.802802+0000","last_fullsized":"2024-04-26T20:50:38.802802+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:42:14.026063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661292+0000","last_change":"2024-04-26T20:50:32.640053+0000","last_active":"2024-04-26T20:50:38.661292+0000","last_peered":"2024-04-26T20:50:38.661292+0000","last_clean":"2024-04-26T20:50:38.661292+0000","last_became_active":"2024-04-26T20:50:32.639778+0000","last_became_peered":"2024-04-26T20:50:32.639778+0000","last_unstale":"2024-04-26T20:50:38.661292+0000","last_undegraded":"2024-04-26T20:50:38.661292+0000","last_fullsized":"2024-04-26T20:50:38.661292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:09:36.737379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.801056+0000","last_change":"2024-04-26T20:50:31.634409+0000","last_active":"2024-04-26T20:50:38.801056+0000","last_peered":"2024-04-26T20:50:38.801056+0000","last_clean":"2024-04-26T20:50:38.801056+0000","last_became_active":"2024-04-26T20:50:31.634103+0000","last_became_peered":"2024-04-26T20:50:31.634103+0000","last_unstale":"2024-04-26T20:50:38.801056+0000","last_undegraded":"2024-04-26T20:50:38.801056+0000","last_fullsized":"2024-04-26T20:50:38.801056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:40:15.520084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803602+0000","last_change":"2024-04-26T20:50:37.292899+0000","last_active":"2024-04-26T20:50:38.803602+0000","last_peered":"2024-04-26T20:50:38.803602+0000","last_clean":"2024-04-26T20:50:38.803602+0000","last_became_active":"2024-04-26T20:50:37.292379+0000","last_became_peered":"2024-04-26T20:50:37.292379+0000","last_unstale":"2024-04-26T20:50:38.803602+0000","last_undegraded":"2024-04-26T20:50:38.803602+0000","last_fullsized":"2024-04-26T20:50:38.803602+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:28:04.995069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800475+0000","last_change":"2024-04-26T20:50:32.649145+0000","last_active":"2024-04-26T20:50:38.800475+0000","last_peered":"2024-04-26T20:50:38.800475+0000","last_clean":"2024-04-26T20:50:38.800475+0000","last_became_active":"2024-04-26T20:50:32.648860+0000","last_became_peered":"2024-04-26T20:50:32.648860+0000","last_unstale":"2024-04-26T20:50:38.800475+0000","last_undegraded":"2024-04-26T20:50:38.800475+0000","last_fullsized":"2024-04-26T20:50:38.800475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:45:44.950496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661412+0000","last_change":"2024-04-26T20:50:31.627675+0000","last_active":"2024-04-26T20:50:38.661412+0000","last_peered":"2024-04-26T20:50:38.661412+0000","last_clean":"2024-04-26T20:50:38.661412+0000","last_became_active":"2024-04-26T20:50:31.627373+0000","last_became_peered":"2024-04-26T20:50:31.627373+0000","last_unstale":"2024-04-26T20:50:38.661412+0000","last_undegraded":"2024-04-26T20:50:38.661412+0000","last_fullsized":"2024-04-26T20:50:38.661412+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:26.601021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.660956+0000","last_change":"2024-04-26T20:50:36.679728+0000","last_active":"2024-04-26T20:50:38.660956+0000","last_peered":"2024-04-26T20:50:38.660956+0000","last_clean":"2024-04-26T20:50:38.660956+0000","last_became_active":"2024-04-26T20:50:36.678881+0000","last_became_peered":"2024-04-26T20:50:36.678881+0000","last_unstale":"2024-04-26T20:50:38.660956+0000","last_undegraded":"2024-04-26T20:50:38.660956+0000","last_fullsized":"2024-04-26T20:50:38.660956+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:59:08.998553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.662645+0000","last_change":"2024-04-26T20:50:32.633552+0000","last_active":"2024-04-26T20:50:38.662645+0000","last_peered":"2024-04-26T20:50:38.662645+0000","last_clean":"2024-04-26T20:50:38.662645+0000","last_became_active":"2024-04-26T20:50:32.632649+0000","last_became_peered":"2024-04-26T20:50:32.632649+0000","last_unstale":"2024-04-26T20:50:38.662645+0000","last_undegraded":"2024-04-26T20:50:38.662645+0000","last_fullsized":"2024-04-26T20:50:38.662645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:32:18.609023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800518+0000","last_change":"2024-04-26T20:50:31.622898+0000","last_active":"2024-04-26T20:50:38.800518+0000","last_peered":"2024-04-26T20:50:38.800518+0000","last_clean":"2024-04-26T20:50:38.800518+0000","last_became_active":"2024-04-26T20:50:31.622583+0000","last_became_peered":"2024-04-26T20:50:31.622583+0000","last_unstale":"2024-04-26T20:50:38.800518+0000","last_undegraded":"2024-04-26T20:50:38.800518+0000","last_fullsized":"2024-04-26T20:50:38.800518+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:53:10.645804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661250+0000","last_change":"2024-04-26T20:50:36.681863+0000","last_active":"2024-04-26T20:50:38.661250+0000","last_peered":"2024-04-26T20:50:38.661250+0000","last_clean":"2024-04-26T20:50:38.661250+0000","last_became_active":"2024-04-26T20:50:36.681433+0000","last_became_peered":"2024-04-26T20:50:36.681433+0000","last_unstale":"2024-04-26T20:50:38.661250+0000","last_undegraded":"2024-04-26T20:50:38.661250+0000","last_fullsized":"2024-04-26T20:50:38.661250+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:28:04.995069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661447+0000","last_change":"2024-04-26T20:50:32.640962+0000","last_active":"2024-04-26T20:50:38.661447+0000","last_peered":"2024-04-26T20:50:38.661447+0000","last_clean":"2024-04-26T20:50:38.661447+0000","last_became_active":"2024-04-26T20:50:32.640670+0000","last_became_peered":"2024-04-26T20:50:32.640670+0000","last_unstale":"2024-04-26T20:50:38.661447+0000","last_undegraded":"2024-04-26T20:50:38.661447+0000","last_fullsized":"2024-04-26T20:50:38.661447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:54.967490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802635+0000","last_change":"2024-04-26T20:50:31.631694+0000","last_active":"2024-04-26T20:50:38.802635+0000","last_peered":"2024-04-26T20:50:38.802635+0000","last_clean":"2024-04-26T20:50:38.802635+0000","last_became_active":"2024-04-26T20:50:31.631330+0000","last_became_peered":"2024-04-26T20:50:31.631330+0000","last_unstale":"2024-04-26T20:50:38.802635+0000","last_undegraded":"2024-04-26T20:50:38.802635+0000","last_fullsized":"2024-04-26T20:50:38.802635+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:49:28.099373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661389+0000","last_change":"2024-04-26T20:50:36.682799+0000","last_active":"2024-04-26T20:50:38.661389+0000","last_peered":"2024-04-26T20:50:38.661389+0000","last_clean":"2024-04-26T20:50:38.661389+0000","last_became_active":"2024-04-26T20:50:36.682271+0000","last_became_peered":"2024-04-26T20:50:36.682271+0000","last_unstale":"2024-04-26T20:50:38.661389+0000","last_undegraded":"2024-04-26T20:50:38.661389+0000","last_fullsized":"2024-04-26T20:50:38.661389+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:27:18.035795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802689+0000","last_change":"2024-04-26T20:50:32.647241+0000","last_active":"2024-04-26T20:50:38.802689+0000","last_peered":"2024-04-26T20:50:38.802689+0000","last_clean":"2024-04-26T20:50:38.802689+0000","last_became_active":"2024-04-26T20:50:32.646907+0000","last_became_peered":"2024-04-26T20:50:32.646907+0000","last_unstale":"2024-04-26T20:50:38.802689+0000","last_undegraded":"2024-04-26T20:50:38.802689+0000","last_fullsized":"2024-04-26T20:50:38.802689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:57:40.604842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800494+0000","last_change":"2024-04-26T20:50:31.633583+0000","last_active":"2024-04-26T20:50:38.800494+0000","last_peered":"2024-04-26T20:50:38.800494+0000","last_clean":"2024-04-26T20:50:38.800494+0000","last_became_active":"2024-04-26T20:50:31.633272+0000","last_became_peered":"2024-04-26T20:50:31.633272+0000","last_unstale":"2024-04-26T20:50:38.800494+0000","last_undegraded":"2024-04-26T20:50:38.800494+0000","last_fullsized":"2024-04-26T20:50:38.800494+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:53:10.645804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803808+0000","last_change":"2024-04-26T20:50:37.291410+0000","last_active":"2024-04-26T20:50:38.803808+0000","last_peered":"2024-04-26T20:50:38.803808+0000","last_clean":"2024-04-26T20:50:38.803808+0000","last_became_active":"2024-04-26T20:50:37.291016+0000","last_became_peered":"2024-04-26T20:50:37.291016+0000","last_unstale":"2024-04-26T20:50:38.803808+0000","last_undegraded":"2024-04-26T20:50:38.803808+0000","last_fullsized":"2024-04-26T20:50:38.803808+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:13:08.850173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802348+0000","last_change":"2024-04-26T20:50:32.653314+0000","last_active":"2024-04-26T20:50:38.802348+0000","last_peered":"2024-04-26T20:50:38.802348+0000","last_clean":"2024-04-26T20:50:38.802348+0000","last_became_active":"2024-04-26T20:50:32.652980+0000","last_became_peered":"2024-04-26T20:50:32.652980+0000","last_unstale":"2024-04-26T20:50:38.802348+0000","last_undegraded":"2024-04-26T20:50:38.802348+0000","last_fullsized":"2024-04-26T20:50:38.802348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:15:55.563522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800028+0000","last_change":"2024-04-26T20:50:31.629049+0000","last_active":"2024-04-26T20:50:38.800028+0000","last_peered":"2024-04-26T20:50:38.800028+0000","last_clean":"2024-04-26T20:50:38.800028+0000","last_became_active":"2024-04-26T20:50:31.628788+0000","last_became_peered":"2024-04-26T20:50:31.628788+0000","last_unstale":"2024-04-26T20:50:38.800028+0000","last_undegraded":"2024-04-26T20:50:38.800028+0000","last_fullsized":"2024-04-26T20:50:38.800028+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:26.601021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":179,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800588+0000","last_change":"2024-04-26T20:50:21.150132+0000","last_active":"2024-04-26T20:50:38.800588+0000","last_peered":"2024-04-26T20:50:38.800588+0000","last_clean":"2024-04-26T20:50:38.800588+0000","last_became_active":"2024-04-26T20:50:21.147439+0000","last_became_peered":"2024-04-26T20:50:21.147439+0000","last_unstale":"2024-04-26T20:50:38.800588+0000","last_undegraded":"2024-04-26T20:50:38.800588+0000","last_fullsized":"2024-04-26T20:50:38.800588+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:20.084499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:20.084499+0000","last_clean_scrub_stamp":"2024-04-26T20:50:20.084499+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:19:29.395477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.717442+0000","last_change":"2024-04-26T20:50:37.295209+0000","last_active":"2024-04-26T20:50:38.717442+0000","last_peered":"2024-04-26T20:50:38.717442+0000","last_clean":"2024-04-26T20:50:38.717442+0000","last_became_active":"2024-04-26T20:50:37.294715+0000","last_became_peered":"2024-04-26T20:50:37.294715+0000","last_unstale":"2024-04-26T20:50:38.717442+0000","last_undegraded":"2024-04-26T20:50:38.717442+0000","last_fullsized":"2024-04-26T20:50:38.717442+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:13:14.645783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800810+0000","last_change":"2024-04-26T20:50:32.648081+0000","last_active":"2024-04-26T20:50:38.800810+0000","last_peered":"2024-04-26T20:50:38.800810+0000","last_clean":"2024-04-26T20:50:38.800810+0000","last_became_active":"2024-04-26T20:50:32.644812+0000","last_became_peered":"2024-04-26T20:50:32.644812+0000","last_unstale":"2024-04-26T20:50:38.800810+0000","last_undegraded":"2024-04-26T20:50:38.800810+0000","last_fullsized":"2024-04-26T20:50:38.800810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:45:44.950496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803401+0000","last_change":"2024-04-26T20:50:31.630301+0000","last_active":"2024-04-26T20:50:38.803401+0000","last_peered":"2024-04-26T20:50:38.803401+0000","last_clean":"2024-04-26T20:50:38.803401+0000","last_became_active":"2024-04-26T20:50:31.630039+0000","last_became_peered":"2024-04-26T20:50:31.630039+0000","last_unstale":"2024-04-26T20:50:38.803401+0000","last_undegraded":"2024-04-26T20:50:38.803401+0000","last_fullsized":"2024-04-26T20:50:38.803401+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:21:09.963731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.717176+0000","last_change":"2024-04-26T20:50:37.291889+0000","last_active":"2024-04-26T20:50:38.717176+0000","last_peered":"2024-04-26T20:50:38.717176+0000","last_clean":"2024-04-26T20:50:38.717176+0000","last_became_active":"2024-04-26T20:50:37.291401+0000","last_became_peered":"2024-04-26T20:50:37.291401+0000","last_unstale":"2024-04-26T20:50:38.717176+0000","last_undegraded":"2024-04-26T20:50:38.717176+0000","last_fullsized":"2024-04-26T20:50:38.717176+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:28:29.639328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800412+0000","last_change":"2024-04-26T20:50:32.655082+0000","last_active":"2024-04-26T20:50:38.800412+0000","last_peered":"2024-04-26T20:50:38.800412+0000","last_clean":"2024-04-26T20:50:38.800412+0000","last_became_active":"2024-04-26T20:50:32.654751+0000","last_became_peered":"2024-04-26T20:50:32.654751+0000","last_unstale":"2024-04-26T20:50:38.800412+0000","last_undegraded":"2024-04-26T20:50:38.800412+0000","last_fullsized":"2024-04-26T20:50:38.800412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:27:59.302826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661274+0000","last_change":"2024-04-26T20:50:31.628865+0000","last_active":"2024-04-26T20:50:38.661274+0000","last_peered":"2024-04-26T20:50:38.661274+0000","last_clean":"2024-04-26T20:50:38.661274+0000","last_became_active":"2024-04-26T20:50:31.627789+0000","last_became_peered":"2024-04-26T20:50:31.627789+0000","last_unstale":"2024-04-26T20:50:38.661274+0000","last_undegraded":"2024-04-26T20:50:38.661274+0000","last_fullsized":"2024-04-26T20:50:38.661274+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:53:10.645804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803847+0000","last_change":"2024-04-26T20:50:36.679869+0000","last_active":"2024-04-26T20:50:38.803847+0000","last_peered":"2024-04-26T20:50:38.803847+0000","last_clean":"2024-04-26T20:50:38.803847+0000","last_became_active":"2024-04-26T20:50:36.678678+0000","last_became_peered":"2024-04-26T20:50:36.678678+0000","last_unstale":"2024-04-26T20:50:38.803847+0000","last_undegraded":"2024-04-26T20:50:38.803847+0000","last_fullsized":"2024-04-26T20:50:38.803847+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:54.249138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.801078+0000","last_change":"2024-04-26T20:50:32.655912+0000","last_active":"2024-04-26T20:50:38.801078+0000","last_peered":"2024-04-26T20:50:38.801078+0000","last_clean":"2024-04-26T20:50:38.801078+0000","last_became_active":"2024-04-26T20:50:32.655649+0000","last_became_peered":"2024-04-26T20:50:32.655649+0000","last_unstale":"2024-04-26T20:50:38.801078+0000","last_undegraded":"2024-04-26T20:50:38.801078+0000","last_fullsized":"2024-04-26T20:50:38.801078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:59:04.962815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803582+0000","last_change":"2024-04-26T20:50:31.631670+0000","last_active":"2024-04-26T20:50:38.803582+0000","last_peered":"2024-04-26T20:50:38.803582+0000","last_clean":"2024-04-26T20:50:38.803582+0000","last_became_active":"2024-04-26T20:50:31.631380+0000","last_became_peered":"2024-04-26T20:50:31.631380+0000","last_unstale":"2024-04-26T20:50:38.803582+0000","last_undegraded":"2024-04-26T20:50:38.803582+0000","last_fullsized":"2024-04-26T20:50:38.803582+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:23:29.866794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.799715+0000","last_change":"2024-04-26T20:50:37.291812+0000","last_active":"2024-04-26T20:50:38.799715+0000","last_peered":"2024-04-26T20:50:38.799715+0000","last_clean":"2024-04-26T20:50:38.799715+0000","last_became_active":"2024-04-26T20:50:37.291408+0000","last_became_peered":"2024-04-26T20:50:37.291408+0000","last_unstale":"2024-04-26T20:50:38.799715+0000","last_undegraded":"2024-04-26T20:50:38.799715+0000","last_fullsized":"2024-04-26T20:50:38.799715+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:29:58.077073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800564+0000","last_change":"2024-04-26T20:50:32.649031+0000","last_active":"2024-04-26T20:50:38.800564+0000","last_peered":"2024-04-26T20:50:38.800564+0000","last_clean":"2024-04-26T20:50:38.800564+0000","last_became_active":"2024-04-26T20:50:32.648769+0000","last_became_peered":"2024-04-26T20:50:32.648769+0000","last_unstale":"2024-04-26T20:50:38.800564+0000","last_undegraded":"2024-04-26T20:50:38.800564+0000","last_fullsized":"2024-04-26T20:50:38.800564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:16:25.308786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.662578+0000","last_change":"2024-04-26T20:50:31.627514+0000","last_active":"2024-04-26T20:50:38.662578+0000","last_peered":"2024-04-26T20:50:38.662578+0000","last_clean":"2024-04-26T20:50:38.662578+0000","last_became_active":"2024-04-26T20:50:31.627177+0000","last_became_peered":"2024-04-26T20:50:31.627177+0000","last_unstale":"2024-04-26T20:50:38.662578+0000","last_undegraded":"2024-04-26T20:50:38.662578+0000","last_fullsized":"2024-04-26T20:50:38.662578+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:23:21.503395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661857+0000","last_change":"2024-04-26T20:50:36.678980+0000","last_active":"2024-04-26T20:50:38.661857+0000","last_peered":"2024-04-26T20:50:38.661857+0000","last_clean":"2024-04-26T20:50:38.661857+0000","last_became_active":"2024-04-26T20:50:36.678581+0000","last_became_peered":"2024-04-26T20:50:36.678581+0000","last_unstale":"2024-04-26T20:50:38.661857+0000","last_undegraded":"2024-04-26T20:50:38.661857+0000","last_fullsized":"2024-04-26T20:50:38.661857+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:28:29.639328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800927+0000","last_change":"2024-04-26T20:50:32.647060+0000","last_active":"2024-04-26T20:50:38.800927+0000","last_peered":"2024-04-26T20:50:38.800927+0000","last_clean":"2024-04-26T20:50:38.800927+0000","last_became_active":"2024-04-26T20:50:32.646761+0000","last_became_peered":"2024-04-26T20:50:32.646761+0000","last_unstale":"2024-04-26T20:50:38.800927+0000","last_undegraded":"2024-04-26T20:50:38.800927+0000","last_fullsized":"2024-04-26T20:50:38.800927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:35.639518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802576+0000","last_change":"2024-04-26T20:50:31.627228+0000","last_active":"2024-04-26T20:50:38.802576+0000","last_peered":"2024-04-26T20:50:38.802576+0000","last_clean":"2024-04-26T20:50:38.802576+0000","last_became_active":"2024-04-26T20:50:31.626931+0000","last_became_peered":"2024-04-26T20:50:31.626931+0000","last_unstale":"2024-04-26T20:50:38.802576+0000","last_undegraded":"2024-04-26T20:50:38.802576+0000","last_fullsized":"2024-04-26T20:50:38.802576+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:19:26.601021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803663+0000","last_change":"2024-04-26T20:50:36.684013+0000","last_active":"2024-04-26T20:50:38.803663+0000","last_peered":"2024-04-26T20:50:38.803663+0000","last_clean":"2024-04-26T20:50:38.803663+0000","last_became_active":"2024-04-26T20:50:36.683653+0000","last_became_peered":"2024-04-26T20:50:36.683653+0000","last_unstale":"2024-04-26T20:50:38.803663+0000","last_undegraded":"2024-04-26T20:50:38.803663+0000","last_fullsized":"2024-04-26T20:50:38.803663+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:27:18.035795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.717061+0000","last_change":"2024-04-26T20:50:32.645444+0000","last_active":"2024-04-26T20:50:38.717061+0000","last_peered":"2024-04-26T20:50:38.717061+0000","last_clean":"2024-04-26T20:50:38.717061+0000","last_became_active":"2024-04-26T20:50:32.645155+0000","last_became_peered":"2024-04-26T20:50:32.645155+0000","last_unstale":"2024-04-26T20:50:38.717061+0000","last_undegraded":"2024-04-26T20:50:38.717061+0000","last_fullsized":"2024-04-26T20:50:38.717061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:49:57.427154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.717091+0000","last_change":"2024-04-26T20:50:31.629590+0000","last_active":"2024-04-26T20:50:38.717091+0000","last_peered":"2024-04-26T20:50:38.717091+0000","last_clean":"2024-04-26T20:50:38.717091+0000","last_became_active":"2024-04-26T20:50:31.629324+0000","last_became_peered":"2024-04-26T20:50:31.629324+0000","last_unstale":"2024-04-26T20:50:38.717091+0000","last_undegraded":"2024-04-26T20:50:38.717091+0000","last_fullsized":"2024-04-26T20:50:38.717091+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:49:28.099373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803803+0000","last_change":"2024-04-26T20:50:37.291611+0000","last_active":"2024-04-26T20:50:38.803803+0000","last_peered":"2024-04-26T20:50:38.803803+0000","last_clean":"2024-04-26T20:50:38.803803+0000","last_became_active":"2024-04-26T20:50:37.291068+0000","last_became_peered":"2024-04-26T20:50:37.291068+0000","last_unstale":"2024-04-26T20:50:38.803803+0000","last_undegraded":"2024-04-26T20:50:38.803803+0000","last_fullsized":"2024-04-26T20:50:38.803803+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:31:09.384259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800427+0000","last_change":"2024-04-26T20:50:32.649304+0000","last_active":"2024-04-26T20:50:38.800427+0000","last_peered":"2024-04-26T20:50:38.800427+0000","last_clean":"2024-04-26T20:50:38.800427+0000","last_became_active":"2024-04-26T20:50:32.647714+0000","last_became_peered":"2024-04-26T20:50:32.647714+0000","last_unstale":"2024-04-26T20:50:38.800427+0000","last_undegraded":"2024-04-26T20:50:38.800427+0000","last_fullsized":"2024-04-26T20:50:38.800427+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:27:59.302826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800065+0000","last_change":"2024-04-26T20:50:31.626907+0000","last_active":"2024-04-26T20:50:38.800065+0000","last_peered":"2024-04-26T20:50:38.800065+0000","last_clean":"2024-04-26T20:50:38.800065+0000","last_became_active":"2024-04-26T20:50:31.625955+0000","last_became_peered":"2024-04-26T20:50:31.625955+0000","last_unstale":"2024-04-26T20:50:38.800065+0000","last_undegraded":"2024-04-26T20:50:38.800065+0000","last_fullsized":"2024-04-26T20:50:38.800065+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:46:19.858595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803391+0000","last_change":"2024-04-26T20:50:37.292652+0000","last_active":"2024-04-26T20:50:38.803391+0000","last_peered":"2024-04-26T20:50:38.803391+0000","last_clean":"2024-04-26T20:50:38.803391+0000","last_became_active":"2024-04-26T20:50:37.291717+0000","last_became_peered":"2024-04-26T20:50:37.291717+0000","last_unstale":"2024-04-26T20:50:38.803391+0000","last_undegraded":"2024-04-26T20:50:38.803391+0000","last_fullsized":"2024-04-26T20:50:38.803391+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:31:19.069050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803321+0000","last_change":"2024-04-26T20:50:32.643484+0000","last_active":"2024-04-26T20:50:38.803321+0000","last_peered":"2024-04-26T20:50:38.803321+0000","last_clean":"2024-04-26T20:50:38.803321+0000","last_became_active":"2024-04-26T20:50:32.643225+0000","last_became_peered":"2024-04-26T20:50:32.643225+0000","last_unstale":"2024-04-26T20:50:38.803321+0000","last_undegraded":"2024-04-26T20:50:38.803321+0000","last_fullsized":"2024-04-26T20:50:38.803321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:49:57.427154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803958+0000","last_change":"2024-04-26T20:50:31.627130+0000","last_active":"2024-04-26T20:50:38.803958+0000","last_peered":"2024-04-26T20:50:38.803958+0000","last_clean":"2024-04-26T20:50:38.803958+0000","last_became_active":"2024-04-26T20:50:31.626697+0000","last_became_peered":"2024-04-26T20:50:31.626697+0000","last_unstale":"2024-04-26T20:50:38.803958+0000","last_undegraded":"2024-04-26T20:50:38.803958+0000","last_fullsized":"2024-04-26T20:50:38.803958+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:21:09.963731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800487+0000","last_change":"2024-04-26T20:50:37.294808+0000","last_active":"2024-04-26T20:50:38.800487+0000","last_peered":"2024-04-26T20:50:38.800487+0000","last_clean":"2024-04-26T20:50:38.800487+0000","last_became_active":"2024-04-26T20:50:37.294404+0000","last_became_peered":"2024-04-26T20:50:37.294404+0000","last_unstale":"2024-04-26T20:50:38.800487+0000","last_undegraded":"2024-04-26T20:50:38.800487+0000","last_fullsized":"2024-04-26T20:50:38.800487+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:29:17.876006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800773+0000","last_change":"2024-04-26T20:50:32.649505+0000","last_active":"2024-04-26T20:50:38.800773+0000","last_peered":"2024-04-26T20:50:38.800773+0000","last_clean":"2024-04-26T20:50:38.800773+0000","last_became_active":"2024-04-26T20:50:32.648538+0000","last_became_peered":"2024-04-26T20:50:32.648538+0000","last_unstale":"2024-04-26T20:50:38.800773+0000","last_undegraded":"2024-04-26T20:50:38.800773+0000","last_fullsized":"2024-04-26T20:50:38.800773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:33:28.969815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:41.974976+0000","last_change":"2024-04-26T20:50:31.628758+0000","last_active":"2024-04-26T20:50:41.974976+0000","last_peered":"2024-04-26T20:50:41.974976+0000","last_clean":"2024-04-26T20:50:41.974976+0000","last_became_active":"2024-04-26T20:50:31.628431+0000","last_became_peered":"2024-04-26T20:50:31.628431+0000","last_unstale":"2024-04-26T20:50:41.974976+0000","last_undegraded":"2024-04-26T20:50:41.974976+0000","last_fullsized":"2024-04-26T20:50:41.974976+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:36.499651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661093+0000","last_change":"2024-04-26T20:50:36.679403+0000","last_active":"2024-04-26T20:50:38.661093+0000","last_peered":"2024-04-26T20:50:38.661093+0000","last_clean":"2024-04-26T20:50:38.661093+0000","last_became_active":"2024-04-26T20:50:36.678945+0000","last_became_peered":"2024-04-26T20:50:36.678945+0000","last_unstale":"2024-04-26T20:50:38.661093+0000","last_undegraded":"2024-04-26T20:50:38.661093+0000","last_fullsized":"2024-04-26T20:50:38.661093+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:12.140650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.662571+0000","last_change":"2024-04-26T20:50:32.636718+0000","last_active":"2024-04-26T20:50:38.662571+0000","last_peered":"2024-04-26T20:50:38.662571+0000","last_clean":"2024-04-26T20:50:38.662571+0000","last_became_active":"2024-04-26T20:50:32.636419+0000","last_became_peered":"2024-04-26T20:50:32.636419+0000","last_unstale":"2024-04-26T20:50:38.662571+0000","last_undegraded":"2024-04-26T20:50:38.662571+0000","last_fullsized":"2024-04-26T20:50:38.662571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:06:40.634786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661155+0000","last_change":"2024-04-26T20:50:31.630575+0000","last_active":"2024-04-26T20:50:38.661155+0000","last_peered":"2024-04-26T20:50:38.661155+0000","last_clean":"2024-04-26T20:50:38.661155+0000","last_became_active":"2024-04-26T20:50:31.630304+0000","last_became_peered":"2024-04-26T20:50:31.630304+0000","last_unstale":"2024-04-26T20:50:38.661155+0000","last_undegraded":"2024-04-26T20:50:38.661155+0000","last_fullsized":"2024-04-26T20:50:38.661155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:32:17.604576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.662550+0000","last_change":"2024-04-26T20:50:36.672365+0000","last_active":"2024-04-26T20:50:38.662550+0000","last_peered":"2024-04-26T20:50:38.662550+0000","last_clean":"2024-04-26T20:50:38.662550+0000","last_became_active":"2024-04-26T20:50:36.671917+0000","last_became_peered":"2024-04-26T20:50:36.671917+0000","last_unstale":"2024-04-26T20:50:38.662550+0000","last_undegraded":"2024-04-26T20:50:38.662550+0000","last_fullsized":"2024-04-26T20:50:38.662550+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:28:04.995069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803056+0000","last_change":"2024-04-26T20:50:32.645616+0000","last_active":"2024-04-26T20:50:38.803056+0000","last_peered":"2024-04-26T20:50:38.803056+0000","last_clean":"2024-04-26T20:50:38.803056+0000","last_became_active":"2024-04-26T20:50:32.645372+0000","last_became_peered":"2024-04-26T20:50:32.645372+0000","last_unstale":"2024-04-26T20:50:38.803056+0000","last_undegraded":"2024-04-26T20:50:38.803056+0000","last_fullsized":"2024-04-26T20:50:38.803056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:56:51.926886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800505+0000","last_change":"2024-04-26T20:50:31.632113+0000","last_active":"2024-04-26T20:50:38.800505+0000","last_peered":"2024-04-26T20:50:38.800505+0000","last_clean":"2024-04-26T20:50:38.800505+0000","last_became_active":"2024-04-26T20:50:31.631815+0000","last_became_peered":"2024-04-26T20:50:31.631815+0000","last_unstale":"2024-04-26T20:50:38.800505+0000","last_undegraded":"2024-04-26T20:50:38.800505+0000","last_fullsized":"2024-04-26T20:50:38.800505+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:36.499651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802860+0000","last_change":"2024-04-26T20:50:36.676688+0000","last_active":"2024-04-26T20:50:38.802860+0000","last_peered":"2024-04-26T20:50:38.802860+0000","last_clean":"2024-04-26T20:50:38.802860+0000","last_became_active":"2024-04-26T20:50:36.675819+0000","last_became_peered":"2024-04-26T20:50:36.675819+0000","last_unstale":"2024-04-26T20:50:38.802860+0000","last_undegraded":"2024-04-26T20:50:38.802860+0000","last_fullsized":"2024-04-26T20:50:38.802860+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:12:04.255812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800398+0000","last_change":"2024-04-26T20:50:32.642860+0000","last_active":"2024-04-26T20:50:38.800398+0000","last_peered":"2024-04-26T20:50:38.800398+0000","last_clean":"2024-04-26T20:50:38.800398+0000","last_became_active":"2024-04-26T20:50:32.642567+0000","last_became_peered":"2024-04-26T20:50:32.642567+0000","last_unstale":"2024-04-26T20:50:38.800398+0000","last_undegraded":"2024-04-26T20:50:38.800398+0000","last_fullsized":"2024-04-26T20:50:38.800398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:00:12.394204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.717653+0000","last_change":"2024-04-26T20:50:31.632961+0000","last_active":"2024-04-26T20:50:38.717653+0000","last_peered":"2024-04-26T20:50:38.717653+0000","last_clean":"2024-04-26T20:50:38.717653+0000","last_became_active":"2024-04-26T20:50:31.632044+0000","last_became_peered":"2024-04-26T20:50:31.632044+0000","last_unstale":"2024-04-26T20:50:38.717653+0000","last_undegraded":"2024-04-26T20:50:38.717653+0000","last_fullsized":"2024-04-26T20:50:38.717653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:23:29.866794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802614+0000","last_change":"2024-04-26T20:50:37.293243+0000","last_active":"2024-04-26T20:50:38.802614+0000","last_peered":"2024-04-26T20:50:38.802614+0000","last_clean":"2024-04-26T20:50:38.802614+0000","last_became_active":"2024-04-26T20:50:37.292840+0000","last_became_peered":"2024-04-26T20:50:37.292840+0000","last_unstale":"2024-04-26T20:50:38.802614+0000","last_undegraded":"2024-04-26T20:50:38.802614+0000","last_fullsized":"2024-04-26T20:50:38.802614+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:05:40.897275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803514+0000","last_change":"2024-04-26T20:50:37.294767+0000","last_active":"2024-04-26T20:50:38.803514+0000","last_peered":"2024-04-26T20:50:38.803514+0000","last_clean":"2024-04-26T20:50:38.803514+0000","last_became_active":"2024-04-26T20:50:37.294351+0000","last_became_peered":"2024-04-26T20:50:37.294351+0000","last_unstale":"2024-04-26T20:50:38.803514+0000","last_undegraded":"2024-04-26T20:50:38.803514+0000","last_fullsized":"2024-04-26T20:50:38.803514+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:31:21.850102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.660600+0000","last_change":"2024-04-26T20:50:32.648105+0000","last_active":"2024-04-26T20:50:38.660600+0000","last_peered":"2024-04-26T20:50:38.660600+0000","last_clean":"2024-04-26T20:50:38.660600+0000","last_became_active":"2024-04-26T20:50:32.647833+0000","last_became_peered":"2024-04-26T20:50:32.647833+0000","last_unstale":"2024-04-26T20:50:38.660600+0000","last_undegraded":"2024-04-26T20:50:38.660600+0000","last_fullsized":"2024-04-26T20:50:38.660600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:55.634765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.799847+0000","last_change":"2024-04-26T20:50:31.622923+0000","last_active":"2024-04-26T20:50:38.799847+0000","last_peered":"2024-04-26T20:50:38.799847+0000","last_clean":"2024-04-26T20:50:38.799847+0000","last_became_active":"2024-04-26T20:50:31.622533+0000","last_became_peered":"2024-04-26T20:50:31.622533+0000","last_unstale":"2024-04-26T20:50:38.799847+0000","last_undegraded":"2024-04-26T20:50:38.799847+0000","last_fullsized":"2024-04-26T20:50:38.799847+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:32:17.604576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.717573+0000","last_change":"2024-04-26T20:50:36.680804+0000","last_active":"2024-04-26T20:50:38.717573+0000","last_peered":"2024-04-26T20:50:38.717573+0000","last_clean":"2024-04-26T20:50:38.717573+0000","last_became_active":"2024-04-26T20:50:36.680438+0000","last_became_peered":"2024-04-26T20:50:36.680438+0000","last_unstale":"2024-04-26T20:50:38.717573+0000","last_undegraded":"2024-04-26T20:50:38.717573+0000","last_fullsized":"2024-04-26T20:50:38.717573+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:58:22.587813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802852+0000","last_change":"2024-04-26T20:50:32.649026+0000","last_active":"2024-04-26T20:50:38.802852+0000","last_peered":"2024-04-26T20:50:38.802852+0000","last_clean":"2024-04-26T20:50:38.802852+0000","last_became_active":"2024-04-26T20:50:32.648737+0000","last_became_peered":"2024-04-26T20:50:32.648737+0000","last_unstale":"2024-04-26T20:50:38.802852+0000","last_undegraded":"2024-04-26T20:50:38.802852+0000","last_fullsized":"2024-04-26T20:50:38.802852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:47:14.715739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802756+0000","last_change":"2024-04-26T20:50:31.631362+0000","last_active":"2024-04-26T20:50:38.802756+0000","last_peered":"2024-04-26T20:50:38.802756+0000","last_clean":"2024-04-26T20:50:38.802756+0000","last_became_active":"2024-04-26T20:50:31.631099+0000","last_became_peered":"2024-04-26T20:50:31.631099+0000","last_unstale":"2024-04-26T20:50:38.802756+0000","last_undegraded":"2024-04-26T20:50:38.802756+0000","last_fullsized":"2024-04-26T20:50:38.802756+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:20:49.853735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.717285+0000","last_change":"2024-04-26T20:50:31.629712+0000","last_active":"2024-04-26T20:50:38.717285+0000","last_peered":"2024-04-26T20:50:38.717285+0000","last_clean":"2024-04-26T20:50:38.717285+0000","last_became_active":"2024-04-26T20:50:31.629412+0000","last_became_peered":"2024-04-26T20:50:31.629412+0000","last_unstale":"2024-04-26T20:50:38.717285+0000","last_undegraded":"2024-04-26T20:50:38.717285+0000","last_fullsized":"2024-04-26T20:50:38.717285+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:32:17.604576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803849+0000","last_change":"2024-04-26T20:50:32.638185+0000","last_active":"2024-04-26T20:50:38.803849+0000","last_peered":"2024-04-26T20:50:38.803849+0000","last_clean":"2024-04-26T20:50:38.803849+0000","last_became_active":"2024-04-26T20:50:32.637917+0000","last_became_peered":"2024-04-26T20:50:32.637917+0000","last_unstale":"2024-04-26T20:50:38.803849+0000","last_undegraded":"2024-04-26T20:50:38.803849+0000","last_fullsized":"2024-04-26T20:50:38.803849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:00:12.394204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803278+0000","last_change":"2024-04-26T20:50:36.682087+0000","last_active":"2024-04-26T20:50:38.803278+0000","last_peered":"2024-04-26T20:50:38.803278+0000","last_clean":"2024-04-26T20:50:38.803278+0000","last_became_active":"2024-04-26T20:50:36.681543+0000","last_became_peered":"2024-04-26T20:50:36.681543+0000","last_unstale":"2024-04-26T20:50:38.803278+0000","last_undegraded":"2024-04-26T20:50:38.803278+0000","last_fullsized":"2024-04-26T20:50:38.803278+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:57:44.640580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802464+0000","last_change":"2024-04-26T20:50:31.632761+0000","last_active":"2024-04-26T20:50:38.802464+0000","last_peered":"2024-04-26T20:50:38.802464+0000","last_clean":"2024-04-26T20:50:38.802464+0000","last_became_active":"2024-04-26T20:50:31.632381+0000","last_became_peered":"2024-04-26T20:50:31.632381+0000","last_unstale":"2024-04-26T20:50:38.802464+0000","last_undegraded":"2024-04-26T20:50:38.802464+0000","last_fullsized":"2024-04-26T20:50:38.802464+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:30.604506+0000","last_clean_scrub_stamp":"2024-04-26T20:50:30.604506+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:21:09.963731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.662120+0000","last_change":"2024-04-26T20:50:32.637920+0000","last_active":"2024-04-26T20:50:38.662120+0000","last_peered":"2024-04-26T20:50:38.662120+0000","last_clean":"2024-04-26T20:50:38.662120+0000","last_became_active":"2024-04-26T20:50:32.637618+0000","last_became_peered":"2024-04-26T20:50:32.637618+0000","last_unstale":"2024-04-26T20:50:38.662120+0000","last_undegraded":"2024-04-26T20:50:38.662120+0000","last_fullsized":"2024-04-26T20:50:38.662120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:46:20.863042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803215+0000","last_change":"2024-04-26T20:50:36.678219+0000","last_active":"2024-04-26T20:50:38.803215+0000","last_peered":"2024-04-26T20:50:38.803215+0000","last_clean":"2024-04-26T20:50:38.803215+0000","last_became_active":"2024-04-26T20:50:36.677732+0000","last_became_peered":"2024-04-26T20:50:36.677732+0000","last_unstale":"2024-04-26T20:50:38.803215+0000","last_undegraded":"2024-04-26T20:50:38.803215+0000","last_fullsized":"2024-04-26T20:50:38.803215+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:35.644691+0000","last_clean_scrub_stamp":"2024-04-26T20:50:35.644691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:03:12.186398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.716924+0000","last_change":"2024-04-26T20:50:32.638997+0000","last_active":"2024-04-26T20:50:38.716924+0000","last_peered":"2024-04-26T20:50:38.716924+0000","last_clean":"2024-04-26T20:50:38.716924+0000","last_became_active":"2024-04-26T20:50:32.638708+0000","last_became_peered":"2024-04-26T20:50:32.638708+0000","last_unstale":"2024-04-26T20:50:38.716924+0000","last_undegraded":"2024-04-26T20:50:38.716924+0000","last_fullsized":"2024-04-26T20:50:38.716924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:36.651960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802480+0000","last_change":"2024-04-26T20:50:32.648305+0000","last_active":"2024-04-26T20:50:38.802480+0000","last_peered":"2024-04-26T20:50:38.802480+0000","last_clean":"2024-04-26T20:50:38.802480+0000","last_became_active":"2024-04-26T20:50:32.648021+0000","last_became_peered":"2024-04-26T20:50:32.648021+0000","last_unstale":"2024-04-26T20:50:38.802480+0000","last_undegraded":"2024-04-26T20:50:38.802480+0000","last_fullsized":"2024-04-26T20:50:38.802480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:46:20.863042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800030+0000","last_change":"2024-04-26T20:50:32.649133+0000","last_active":"2024-04-26T20:50:38.800030+0000","last_peered":"2024-04-26T20:50:38.800030+0000","last_clean":"2024-04-26T20:50:38.800030+0000","last_became_active":"2024-04-26T20:50:32.648855+0000","last_became_peered":"2024-04-26T20:50:32.648855+0000","last_unstale":"2024-04-26T20:50:38.800030+0000","last_undegraded":"2024-04-26T20:50:38.800030+0000","last_fullsized":"2024-04-26T20:50:38.800030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:58:18.552075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803763+0000","last_change":"2024-04-26T20:50:32.650322+0000","last_active":"2024-04-26T20:50:38.803763+0000","last_peered":"2024-04-26T20:50:38.803763+0000","last_clean":"2024-04-26T20:50:38.803763+0000","last_became_active":"2024-04-26T20:50:32.650011+0000","last_became_peered":"2024-04-26T20:50:32.650011+0000","last_unstale":"2024-04-26T20:50:38.803763+0000","last_undegraded":"2024-04-26T20:50:38.803763+0000","last_fullsized":"2024-04-26T20:50:38.803763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:15:55.563522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803550+0000","last_change":"2024-04-26T20:50:32.649914+0000","last_active":"2024-04-26T20:50:38.803550+0000","last_peered":"2024-04-26T20:50:38.803550+0000","last_clean":"2024-04-26T20:50:38.803550+0000","last_became_active":"2024-04-26T20:50:32.649439+0000","last_became_peered":"2024-04-26T20:50:32.649439+0000","last_unstale":"2024-04-26T20:50:38.803550+0000","last_undegraded":"2024-04-26T20:50:38.803550+0000","last_fullsized":"2024-04-26T20:50:38.803550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:33:28.969815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661177+0000","last_change":"2024-04-26T20:50:32.648570+0000","last_active":"2024-04-26T20:50:38.661177+0000","last_peered":"2024-04-26T20:50:38.661177+0000","last_clean":"2024-04-26T20:50:38.661177+0000","last_became_active":"2024-04-26T20:50:32.646951+0000","last_became_peered":"2024-04-26T20:50:32.646951+0000","last_unstale":"2024-04-26T20:50:38.661177+0000","last_undegraded":"2024-04-26T20:50:38.661177+0000","last_fullsized":"2024-04-26T20:50:38.661177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:15:55.563522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.717644+0000","last_change":"2024-04-26T20:50:32.640208+0000","last_active":"2024-04-26T20:50:38.717644+0000","last_peered":"2024-04-26T20:50:38.717644+0000","last_clean":"2024-04-26T20:50:38.717644+0000","last_became_active":"2024-04-26T20:50:32.639869+0000","last_became_peered":"2024-04-26T20:50:32.639869+0000","last_unstale":"2024-04-26T20:50:38.717644+0000","last_undegraded":"2024-04-26T20:50:38.717644+0000","last_fullsized":"2024-04-26T20:50:38.717644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:15:55.563522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803850+0000","last_change":"2024-04-26T20:50:32.650173+0000","last_active":"2024-04-26T20:50:38.803850+0000","last_peered":"2024-04-26T20:50:38.803850+0000","last_clean":"2024-04-26T20:50:38.803850+0000","last_became_active":"2024-04-26T20:50:32.649876+0000","last_became_peered":"2024-04-26T20:50:32.649876+0000","last_unstale":"2024-04-26T20:50:38.803850+0000","last_undegraded":"2024-04-26T20:50:38.803850+0000","last_fullsized":"2024-04-26T20:50:38.803850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:00:12.394204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.799788+0000","last_change":"2024-04-26T20:50:32.651129+0000","last_active":"2024-04-26T20:50:38.799788+0000","last_peered":"2024-04-26T20:50:38.799788+0000","last_clean":"2024-04-26T20:50:38.799788+0000","last_became_active":"2024-04-26T20:50:32.650853+0000","last_became_peered":"2024-04-26T20:50:32.650853+0000","last_unstale":"2024-04-26T20:50:38.799788+0000","last_undegraded":"2024-04-26T20:50:38.799788+0000","last_fullsized":"2024-04-26T20:50:38.799788+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:35.639518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802726+0000","last_change":"2024-04-26T20:50:32.650192+0000","last_active":"2024-04-26T20:50:38.802726+0000","last_peered":"2024-04-26T20:50:38.802726+0000","last_clean":"2024-04-26T20:50:38.802726+0000","last_became_active":"2024-04-26T20:50:32.649911+0000","last_became_peered":"2024-04-26T20:50:32.649911+0000","last_unstale":"2024-04-26T20:50:38.802726+0000","last_undegraded":"2024-04-26T20:50:38.802726+0000","last_fullsized":"2024-04-26T20:50:38.802726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:09:36.737379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.799890+0000","last_change":"2024-04-26T20:50:32.641183+0000","last_active":"2024-04-26T20:50:38.799890+0000","last_peered":"2024-04-26T20:50:38.799890+0000","last_clean":"2024-04-26T20:50:38.799890+0000","last_became_active":"2024-04-26T20:50:32.640584+0000","last_became_peered":"2024-04-26T20:50:32.640584+0000","last_unstale":"2024-04-26T20:50:38.799890+0000","last_undegraded":"2024-04-26T20:50:38.799890+0000","last_fullsized":"2024-04-26T20:50:38.799890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:13:10.610044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803653+0000","last_change":"2024-04-26T20:50:32.651661+0000","last_active":"2024-04-26T20:50:38.803653+0000","last_peered":"2024-04-26T20:50:38.803653+0000","last_clean":"2024-04-26T20:50:38.803653+0000","last_became_active":"2024-04-26T20:50:32.651316+0000","last_became_peered":"2024-04-26T20:50:32.651316+0000","last_unstale":"2024-04-26T20:50:38.803653+0000","last_undegraded":"2024-04-26T20:50:38.803653+0000","last_fullsized":"2024-04-26T20:50:38.803653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:09:36.737379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.717426+0000","last_change":"2024-04-26T20:50:32.645619+0000","last_active":"2024-04-26T20:50:38.717426+0000","last_peered":"2024-04-26T20:50:38.717426+0000","last_clean":"2024-04-26T20:50:38.717426+0000","last_became_active":"2024-04-26T20:50:32.644645+0000","last_became_peered":"2024-04-26T20:50:32.644645+0000","last_unstale":"2024-04-26T20:50:38.717426+0000","last_undegraded":"2024-04-26T20:50:38.717426+0000","last_fullsized":"2024-04-26T20:50:38.717426+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:54.967490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.801055+0000","last_change":"2024-04-26T20:50:32.646646+0000","last_active":"2024-04-26T20:50:38.801055+0000","last_peered":"2024-04-26T20:50:38.801055+0000","last_clean":"2024-04-26T20:50:38.801055+0000","last_became_active":"2024-04-26T20:50:32.645404+0000","last_became_peered":"2024-04-26T20:50:32.645404+0000","last_unstale":"2024-04-26T20:50:38.801055+0000","last_undegraded":"2024-04-26T20:50:38.801055+0000","last_fullsized":"2024-04-26T20:50:38.801055+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:58:18.552075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803547+0000","last_change":"2024-04-26T20:50:32.646750+0000","last_active":"2024-04-26T20:50:38.803547+0000","last_peered":"2024-04-26T20:50:38.803547+0000","last_clean":"2024-04-26T20:50:38.803547+0000","last_became_active":"2024-04-26T20:50:32.646447+0000","last_became_peered":"2024-04-26T20:50:32.646447+0000","last_unstale":"2024-04-26T20:50:38.803547+0000","last_undegraded":"2024-04-26T20:50:38.803547+0000","last_fullsized":"2024-04-26T20:50:38.803547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:53:11.650251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800765+0000","last_change":"2024-04-26T20:50:32.642132+0000","last_active":"2024-04-26T20:50:38.800765+0000","last_peered":"2024-04-26T20:50:38.800765+0000","last_clean":"2024-04-26T20:50:38.800765+0000","last_became_active":"2024-04-26T20:50:32.641690+0000","last_became_peered":"2024-04-26T20:50:32.641690+0000","last_unstale":"2024-04-26T20:50:38.800765+0000","last_undegraded":"2024-04-26T20:50:38.800765+0000","last_fullsized":"2024-04-26T20:50:38.800765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:59:04.962815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.799661+0000","last_change":"2024-04-26T20:50:32.644332+0000","last_active":"2024-04-26T20:50:38.799661+0000","last_peered":"2024-04-26T20:50:38.799661+0000","last_clean":"2024-04-26T20:50:38.799661+0000","last_became_active":"2024-04-26T20:50:32.643783+0000","last_became_peered":"2024-04-26T20:50:32.643783+0000","last_unstale":"2024-04-26T20:50:38.799661+0000","last_undegraded":"2024-04-26T20:50:38.799661+0000","last_fullsized":"2024-04-26T20:50:38.799661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:56:51.926886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.660696+0000","last_change":"2024-04-26T20:50:32.638177+0000","last_active":"2024-04-26T20:50:38.660696+0000","last_peered":"2024-04-26T20:50:38.660696+0000","last_clean":"2024-04-26T20:50:38.660696+0000","last_became_active":"2024-04-26T20:50:32.637905+0000","last_became_peered":"2024-04-26T20:50:32.637905+0000","last_unstale":"2024-04-26T20:50:38.660696+0000","last_undegraded":"2024-04-26T20:50:38.660696+0000","last_fullsized":"2024-04-26T20:50:38.660696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:36.651960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.803236+0000","last_change":"2024-04-26T20:50:32.637456+0000","last_active":"2024-04-26T20:50:38.803236+0000","last_peered":"2024-04-26T20:50:38.803236+0000","last_clean":"2024-04-26T20:50:38.803236+0000","last_became_active":"2024-04-26T20:50:32.637188+0000","last_became_peered":"2024-04-26T20:50:32.637188+0000","last_unstale":"2024-04-26T20:50:38.803236+0000","last_undegraded":"2024-04-26T20:50:38.803236+0000","last_fullsized":"2024-04-26T20:50:38.803236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:54.967490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802268+0000","last_change":"2024-04-26T20:50:32.649659+0000","last_active":"2024-04-26T20:50:38.802268+0000","last_peered":"2024-04-26T20:50:38.802268+0000","last_clean":"2024-04-26T20:50:38.802268+0000","last_became_active":"2024-04-26T20:50:32.645837+0000","last_became_peered":"2024-04-26T20:50:32.645837+0000","last_unstale":"2024-04-26T20:50:38.802268+0000","last_undegraded":"2024-04-26T20:50:38.802268+0000","last_fullsized":"2024-04-26T20:50:38.802268+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:55.634765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.662458+0000","last_change":"2024-04-26T20:50:32.643463+0000","last_active":"2024-04-26T20:50:38.662458+0000","last_peered":"2024-04-26T20:50:38.662458+0000","last_clean":"2024-04-26T20:50:38.662458+0000","last_became_active":"2024-04-26T20:50:32.642783+0000","last_became_peered":"2024-04-26T20:50:32.642783+0000","last_unstale":"2024-04-26T20:50:38.662458+0000","last_undegraded":"2024-04-26T20:50:38.662458+0000","last_fullsized":"2024-04-26T20:50:38.662458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:53:11.650251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.717503+0000","last_change":"2024-04-26T20:50:32.643690+0000","last_active":"2024-04-26T20:50:38.717503+0000","last_peered":"2024-04-26T20:50:38.717503+0000","last_clean":"2024-04-26T20:50:38.717503+0000","last_became_active":"2024-04-26T20:50:32.643412+0000","last_became_peered":"2024-04-26T20:50:32.643412+0000","last_unstale":"2024-04-26T20:50:38.717503+0000","last_undegraded":"2024-04-26T20:50:38.717503+0000","last_fullsized":"2024-04-26T20:50:38.717503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:55.634765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.717608+0000","last_change":"2024-04-26T20:50:32.643116+0000","last_active":"2024-04-26T20:50:38.717608+0000","last_peered":"2024-04-26T20:50:38.717608+0000","last_clean":"2024-04-26T20:50:38.717608+0000","last_became_active":"2024-04-26T20:50:32.642598+0000","last_became_peered":"2024-04-26T20:50:32.642598+0000","last_unstale":"2024-04-26T20:50:38.717608+0000","last_undegraded":"2024-04-26T20:50:38.717608+0000","last_fullsized":"2024-04-26T20:50:38.717608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:47:14.715739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.801040+0000","last_change":"2024-04-26T20:50:32.654019+0000","last_active":"2024-04-26T20:50:38.801040+0000","last_peered":"2024-04-26T20:50:38.801040+0000","last_clean":"2024-04-26T20:50:38.801040+0000","last_became_active":"2024-04-26T20:50:32.653569+0000","last_became_peered":"2024-04-26T20:50:32.653569+0000","last_unstale":"2024-04-26T20:50:38.801040+0000","last_undegraded":"2024-04-26T20:50:38.801040+0000","last_fullsized":"2024-04-26T20:50:38.801040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:13:10.610044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802660+0000","last_change":"2024-04-26T20:50:32.646837+0000","last_active":"2024-04-26T20:50:38.802660+0000","last_peered":"2024-04-26T20:50:38.802660+0000","last_clean":"2024-04-26T20:50:38.802660+0000","last_became_active":"2024-04-26T20:50:32.646528+0000","last_became_peered":"2024-04-26T20:50:32.646528+0000","last_unstale":"2024-04-26T20:50:38.802660+0000","last_undegraded":"2024-04-26T20:50:38.802660+0000","last_fullsized":"2024-04-26T20:50:38.802660+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:54.967490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800781+0000","last_change":"2024-04-26T20:50:32.654006+0000","last_active":"2024-04-26T20:50:38.800781+0000","last_peered":"2024-04-26T20:50:38.800781+0000","last_clean":"2024-04-26T20:50:38.800781+0000","last_became_active":"2024-04-26T20:50:32.653553+0000","last_became_peered":"2024-04-26T20:50:32.653553+0000","last_unstale":"2024-04-26T20:50:38.800781+0000","last_undegraded":"2024-04-26T20:50:38.800781+0000","last_fullsized":"2024-04-26T20:50:38.800781+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:00:12.394204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.799949+0000","last_change":"2024-04-26T20:50:32.641407+0000","last_active":"2024-04-26T20:50:38.799949+0000","last_peered":"2024-04-26T20:50:38.799949+0000","last_clean":"2024-04-26T20:50:38.799949+0000","last_became_active":"2024-04-26T20:50:32.640889+0000","last_became_peered":"2024-04-26T20:50:32.640889+0000","last_unstale":"2024-04-26T20:50:38.799949+0000","last_undegraded":"2024-04-26T20:50:38.799949+0000","last_fullsized":"2024-04-26T20:50:38.799949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:28:25.603590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800212+0000","last_change":"2024-04-26T20:50:32.645181+0000","last_active":"2024-04-26T20:50:38.800212+0000","last_peered":"2024-04-26T20:50:38.800212+0000","last_clean":"2024-04-26T20:50:38.800212+0000","last_became_active":"2024-04-26T20:50:32.644919+0000","last_became_peered":"2024-04-26T20:50:32.644919+0000","last_unstale":"2024-04-26T20:50:38.800212+0000","last_undegraded":"2024-04-26T20:50:38.800212+0000","last_fullsized":"2024-04-26T20:50:38.800212+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:40:19.711175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800200+0000","last_change":"2024-04-26T20:50:32.655095+0000","last_active":"2024-04-26T20:50:38.800200+0000","last_peered":"2024-04-26T20:50:38.800200+0000","last_clean":"2024-04-26T20:50:38.800200+0000","last_became_active":"2024-04-26T20:50:32.654821+0000","last_became_peered":"2024-04-26T20:50:32.654821+0000","last_unstale":"2024-04-26T20:50:38.800200+0000","last_undegraded":"2024-04-26T20:50:38.800200+0000","last_fullsized":"2024-04-26T20:50:38.800200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:33:28.969815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.802460+0000","last_change":"2024-04-26T20:50:32.642769+0000","last_active":"2024-04-26T20:50:38.802460+0000","last_peered":"2024-04-26T20:50:38.802460+0000","last_clean":"2024-04-26T20:50:38.802460+0000","last_became_active":"2024-04-26T20:50:32.642506+0000","last_became_peered":"2024-04-26T20:50:32.642506+0000","last_unstale":"2024-04-26T20:50:38.802460+0000","last_undegraded":"2024-04-26T20:50:38.802460+0000","last_fullsized":"2024-04-26T20:50:38.802460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:32:55.634765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.661146+0000","last_change":"2024-04-26T20:50:32.641048+0000","last_active":"2024-04-26T20:50:38.661146+0000","last_peered":"2024-04-26T20:50:38.661146+0000","last_clean":"2024-04-26T20:50:38.661146+0000","last_became_active":"2024-04-26T20:50:32.640572+0000","last_became_peered":"2024-04-26T20:50:32.640572+0000","last_unstale":"2024-04-26T20:50:38.661146+0000","last_undegraded":"2024-04-26T20:50:38.661146+0000","last_fullsized":"2024-04-26T20:50:38.661146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:47:14.715739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T20:50:38.800925+0000","last_change":"2024-04-26T20:50:32.650971+0000","last_active":"2024-04-26T20:50:38.800925+0000","last_peered":"2024-04-26T20:50:38.800925+0000","last_clean":"2024-04-26T20:50:38.800925+0000","last_became_active":"2024-04-26T20:50:32.650713+0000","last_became_peered":"2024-04-26T20:50:32.650713+0000","last_unstale":"2024-04-26T20:50:38.800925+0000","last_undegraded":"2024-04-26T20:50:38.800925+0000","last_fullsized":"2024-04-26T20:50:38.800925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:31.608953+0000","last_clean_scrub_stamp":"2024-04-26T20:50:31.608953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:28:25.603590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":864256,"data_stored":698343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":107483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":107519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":107393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":107553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":872448,"data_stored":698399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:51:00.353 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-26T20:51:00.353 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-26T20:51:00.353 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-26T20:51:00.354 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-26T20:51:00.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.480+0000 7f82723bf640 1 Processor -- start 2024-04-26T20:51:00.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.480+0000 7f82723bf640 1 -- start start 2024-04-26T20:51:00.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.481+0000 7f82723bf640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f826c15f3a0 0x7f826c0d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:00.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.481+0000 7f82723bf640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f826c0d2330 0x7f826c0b2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:00.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.481+0000 7f82723bf640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f826c0d0e20 con 0x7f826c0d1770 2024-04-26T20:51:00.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.481+0000 7f82723bf640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f826c0b1d70 con 0x7f826c15f3a0 2024-04-26T20:51:00.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.481+0000 7f82723bf640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f826c0b2050 con 0x7f826c0d2330 2024-04-26T20:51:00.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.481+0000 7f826bfff640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f826c15f3a0 0x7f826c0d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:00.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.481+0000 7f826bfff640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f826c15f3a0 0x7f826c0d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42608/0 (socket says 172.21.15.73:42608) 2024-04-26T20:51:00.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.481+0000 7f826bfff640 1 -- 172.21.15.73:0/137639454 learned_addr learned my addr 172.21.15.73:0/137639454 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:00.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.482+0000 7f827095c640 1 --2- 172.21.15.73:0/137639454 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f826c0d2330 0x7f826c0b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:00.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.482+0000 7f826bfff640 1 -- 172.21.15.73:0/137639454 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f826c0d2330 msgr2=0x7f826c0b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:00.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.482+0000 7f826bfff640 1 --2- 172.21.15.73:0/137639454 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f826c0d2330 0x7f826c0b2850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:00.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.482+0000 7f826bfff640 1 -- 172.21.15.73:0/137639454 >> v1:172.21.15.120:6789/0 conn(0x7f826c0d1770 legacy=0x7f826c0d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:00.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.482+0000 7f826bfff640 1 -- 172.21.15.73:0/137639454 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f826c0b2330 con 0x7f826c15f3a0 2024-04-26T20:51:00.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.482+0000 7f826bfff640 1 --2- 172.21.15.73:0/137639454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f826c15f3a0 0x7f826c0d06f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f825c009870 tx=0x7f825c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bc7342a174848f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:00.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.482+0000 7f826affd640 1 -- 172.21.15.73:0/137639454 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825c037070 con 0x7f826c15f3a0 2024-04-26T20:51:00.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.483+0000 7f826affd640 1 -- 172.21.15.73:0/137639454 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f825c032440 con 0x7f826c15f3a0 2024-04-26T20:51:00.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.483+0000 7f826affd640 1 -- 172.21.15.73:0/137639454 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825c032810 con 0x7f826c15f3a0 2024-04-26T20:51:00.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.483+0000 7f82723bf640 1 -- 172.21.15.73:0/137639454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f826c15f3a0 msgr2=0x7f826c0d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:00.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.483+0000 7f82723bf640 1 --2- 172.21.15.73:0/137639454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f826c15f3a0 0x7f826c0d06f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f825c009870 tx=0x7f825c02a210 comp rx=0 tx=0).stop 2024-04-26T20:51:00.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.483+0000 7f82723bf640 1 -- 172.21.15.73:0/137639454 shutdown_connections 2024-04-26T20:51:00.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.483+0000 7f82723bf640 1 --2- 172.21.15.73:0/137639454 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f826c0d2330 0x7f826c0b2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:00.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.483+0000 7f82723bf640 1 --2- 172.21.15.73:0/137639454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f826c15f3a0 0x7f826c0d06f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:00.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.483+0000 7f82723bf640 1 -- 172.21.15.73:0/137639454 >> 172.21.15.73:0/137639454 conn(0x7f826c0d4f90 msgr2=0x7f826c0d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:00.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.483+0000 7f82723bf640 1 -- 172.21.15.73:0/137639454 shutdown_connections 2024-04-26T20:51:00.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.483+0000 7f82723bf640 1 -- 172.21.15.73:0/137639454 wait complete. 2024-04-26T20:51:00.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.484+0000 7f82723bf640 1 Processor -- start 2024-04-26T20:51:00.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.484+0000 7f82723bf640 1 -- start start 2024-04-26T20:51:00.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.485+0000 7f82723bf640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f826c0d1770 0x7f826c153480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:00.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.485+0000 7f82723bf640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f826c0d2330 0x7f826c15f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:00.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.485+0000 7f82723bf640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f826c15f3a0 0x7f826c0ce610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:00.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.485+0000 7f82723bf640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f826c0b2f10 con 0x7f826c0d2330 2024-04-26T20:51:00.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.485+0000 7f82723bf640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f826c0b2d90 con 0x7f826c15f3a0 2024-04-26T20:51:00.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.485+0000 7f82723bf640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f826c0b3090 con 0x7f826c0d1770 2024-04-26T20:51:00.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.485+0000 7f826b7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f826c0d2330 0x7f826c15f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:00.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.485+0000 7f826b7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f826c0d2330 0x7f826c15f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42612/0 (socket says 172.21.15.73:42612) 2024-04-26T20:51:00.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.485+0000 7f826b7fe640 1 -- 172.21.15.73:0/2875109032 learned_addr learned my addr 172.21.15.73:0/2875109032 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:00.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.485+0000 7f826bfff640 1 --2- 172.21.15.73:0/2875109032 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f826c0d1770 0x7f826c153480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:00.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.485+0000 7f826b7fe640 1 -- 172.21.15.73:0/2875109032 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f826c0d1770 msgr2=0x7f826c153480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:00.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.485+0000 7f826b7fe640 1 --2- 172.21.15.73:0/2875109032 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f826c0d1770 0x7f826c153480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:00.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.485+0000 7f826b7fe640 1 -- 172.21.15.73:0/2875109032 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f826c15f3a0 msgr2=0x7f826c0ce610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:00.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.485+0000 7f827095c640 1 --2- 172.21.15.73:0/2875109032 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f826c15f3a0 0x7f826c0ce610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:00.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.486+0000 7f826b7fe640 1 --2- 172.21.15.73:0/2875109032 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f826c15f3a0 0x7f826c0ce610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:00.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.486+0000 7f826b7fe640 1 -- 172.21.15.73:0/2875109032 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f826c0cefd0 con 0x7f826c0d2330 2024-04-26T20:51:00.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.486+0000 7f827095c640 1 --2- 172.21.15.73:0/2875109032 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f826c15f3a0 0x7f826c0ce610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:00.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.486+0000 7f826b7fe640 1 --2- 172.21.15.73:0/2875109032 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f826c0d2330 0x7f826c15f990 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f826000c430 tx=0x7f826000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:00.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.486+0000 7f8268ff9640 1 -- 172.21.15.73:0/2875109032 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8260011790 con 0x7f826c0d2330 2024-04-26T20:51:00.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.486+0000 7f8268ff9640 1 -- 172.21.15.73:0/2875109032 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8260017070 con 0x7f826c0d2330 2024-04-26T20:51:00.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.486+0000 7f8268ff9640 1 -- 172.21.15.73:0/2875109032 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8260011bb0 con 0x7f826c0d2330 2024-04-26T20:51:00.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.486+0000 7f82723bf640 1 -- 172.21.15.73:0/2875109032 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f826c0cc100 con 0x7f826c0d2330 2024-04-26T20:51:00.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.486+0000 7f82723bf640 1 -- 172.21.15.73:0/2875109032 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f826c0cc6c0 con 0x7f826c0d2330 2024-04-26T20:51:00.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.487+0000 7f82723bf640 1 -- 172.21.15.73:0/2875109032 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f826c0694f0 con 0x7f826c0d2330 2024-04-26T20:51:00.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.487+0000 7f8268ff9640 1 -- 172.21.15.73:0/2875109032 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f826001c460 con 0x7f826c0d2330 2024-04-26T20:51:00.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.488+0000 7f8268ff9640 1 --2- 172.21.15.73:0/2875109032 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f825806c920 0x7f825806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:00.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.488+0000 7f8268ff9640 1 -- 172.21.15.73:0/2875109032 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f826008bcb0 con 0x7f826c0d2330 2024-04-26T20:51:00.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.488+0000 7f826bfff640 1 --2- 172.21.15.73:0/2875109032 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f825806c920 0x7f825806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:00.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.488+0000 7f826bfff640 1 --2- 172.21.15.73:0/2875109032 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f825806c920 0x7f825806ede0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f825c034000 tx=0x7f825c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:00.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.492+0000 7f8268ff9640 1 -- 172.21.15.73:0/2875109032 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8260059da0 con 0x7f826c0d2330 2024-04-26T20:51:00.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.707+0000 7f82723bf640 1 -- 172.21.15.73:0/2875109032 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f826c0b6860 con 0x7f826c0d2330 2024-04-26T20:51:00.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.708+0000 7f8268ff9640 1 -- 172.21.15.73:0/2875109032 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f82600596a0 con 0x7f826c0d2330 2024-04-26T20:51:00.709 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:51:00.709 INFO:teuthology.orchestra.run.smithi073.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-26T20:51:00.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.710+0000 7f82723bf640 1 -- 172.21.15.73:0/2875109032 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f825806c920 msgr2=0x7f825806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:00.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.710+0000 7f82723bf640 1 --2- 172.21.15.73:0/2875109032 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f825806c920 0x7f825806ede0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f825c034000 tx=0x7f825c02e000 comp rx=0 tx=0).stop 2024-04-26T20:51:00.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.710+0000 7f82723bf640 1 -- 172.21.15.73:0/2875109032 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f826c0d2330 msgr2=0x7f826c15f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:00.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.710+0000 7f82723bf640 1 --2- 172.21.15.73:0/2875109032 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f826c0d2330 0x7f826c15f990 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f826000c430 tx=0x7f826000c8f0 comp rx=0 tx=0).stop 2024-04-26T20:51:00.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.710+0000 7f82723bf640 1 -- 172.21.15.73:0/2875109032 shutdown_connections 2024-04-26T20:51:00.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.710+0000 7f82723bf640 1 --2- 172.21.15.73:0/2875109032 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f826c0d2330 0x7f826c15f990 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:00.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.710+0000 7f82723bf640 1 --2- 172.21.15.73:0/2875109032 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f825806c920 0x7f825806ede0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:00.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.710+0000 7f82723bf640 1 --2- 172.21.15.73:0/2875109032 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f826c15f3a0 0x7f826c0ce610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:00.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.710+0000 7f82723bf640 1 --2- 172.21.15.73:0/2875109032 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f826c0d1770 0x7f826c153480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:00.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.710+0000 7f82723bf640 1 -- 172.21.15.73:0/2875109032 >> 172.21.15.73:0/2875109032 conn(0x7f826c0d4f90 msgr2=0x7f826c0d2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:00.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.710+0000 7f82723bf640 1 -- 172.21.15.73:0/2875109032 shutdown_connections 2024-04-26T20:51:00.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.710+0000 7f82723bf640 1 -- 172.21.15.73:0/2875109032 wait complete. 2024-04-26T20:51:00.721 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-26T20:51:00.722 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:51:00.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.817+0000 7fc5e26b1640 1 Processor -- start 2024-04-26T20:51:00.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.817+0000 7fc5e26b1640 1 -- start start 2024-04-26T20:51:00.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.818+0000 7fc5e26b1640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc5dc162880 0x7fc5dc0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:00.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.818+0000 7fc5e26b1640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5dc0b7e50 con 0x7fc5dc0d3bc0 2024-04-26T20:51:00.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.818+0000 7fc5e26b1640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5dc0d1800 con 0x7fc5dc0b5f20 2024-04-26T20:51:00.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.818+0000 7fc5e26b1640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7fc5dc0d0580 con 0x7fc5dc162880 2024-04-26T20:51:00.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.818+0000 7fc5dbfff640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc5dc162880 0x7fc5dc0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:00.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.818+0000 7fc5e0c4e640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fc5dc0d3bc0 0x7fc5dc0b8300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:42028/0 (socket says 172.21.15.73:42028) 2024-04-26T20:51:00.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.818+0000 7fc5e0c4e640 1 -- 172.21.15.73:0/1838046588 learned_addr learned my addr 172.21.15.73:0/1838046588 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:00.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.818+0000 7fc5dbfff640 1 -- 172.21.15.73:0/1838046588 >> v1:172.21.15.120:6789/0 conn(0x7fc5dc0b5f20 legacy=0x7fc5dc0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:00.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.818+0000 7fc5dbfff640 1 -- 172.21.15.73:0/1838046588 >> v1:172.21.15.73:6789/0 conn(0x7fc5dc0d3bc0 legacy=0x7fc5dc0b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:00.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.818+0000 7fc5dbfff640 1 -- 172.21.15.73:0/1838046588 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5dc0d07d0 con 0x7fc5dc162880 2024-04-26T20:51:00.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.819+0000 7fc5dbfff640 1 --2- 172.21.15.73:0/1838046588 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5dc162880 0x7fc5dc0d2cb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc0097a0 tx=0x7fc5cc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49c2d42f04e6b4d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:00.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.819+0000 7fc5daffd640 1 -- 172.21.15.73:0/1838046588 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc037070 con 0x7fc5dc162880 2024-04-26T20:51:00.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.819+0000 7fc5daffd640 1 -- 172.21.15.73:0/1838046588 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5cc032440 con 0x7fc5dc162880 2024-04-26T20:51:00.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.819+0000 7fc5daffd640 1 -- 172.21.15.73:0/1838046588 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc032810 con 0x7fc5dc162880 2024-04-26T20:51:00.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.820+0000 7fc5e26b1640 1 -- 172.21.15.73:0/1838046588 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5dc162880 msgr2=0x7fc5dc0d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:00.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.820+0000 7fc5e26b1640 1 --2- 172.21.15.73:0/1838046588 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5dc162880 0x7fc5dc0d2cb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc0097a0 tx=0x7fc5cc02a1e0 comp rx=0 tx=0).stop 2024-04-26T20:51:00.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.820+0000 7fc5e26b1640 1 -- 172.21.15.73:0/1838046588 shutdown_connections 2024-04-26T20:51:00.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.820+0000 7fc5e26b1640 1 --2- 172.21.15.73:0/1838046588 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5dc162880 0x7fc5dc0d2cb0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:00.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.820+0000 7fc5e26b1640 1 -- 172.21.15.73:0/1838046588 >> 172.21.15.73:0/1838046588 conn(0x7fc5dc0d4f60 msgr2=0x7fc5dc0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:00.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.820+0000 7fc5e26b1640 1 -- 172.21.15.73:0/1838046588 shutdown_connections 2024-04-26T20:51:00.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.820+0000 7fc5e26b1640 1 -- 172.21.15.73:0/1838046588 wait complete. 2024-04-26T20:51:00.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.821+0000 7fc5e26b1640 1 Processor -- start 2024-04-26T20:51:00.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.821+0000 7fc5e26b1640 1 -- start start 2024-04-26T20:51:00.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.822+0000 7fc5e26b1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5dc0b5f20 0x7fc5dc159d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:00.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.822+0000 7fc5e26b1640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc5dc0d3bc0 0x7fc5dc159520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:00.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.822+0000 7fc5e26b1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5dc162880 0x7fc5dc0ba110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:00.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.822+0000 7fc5e26b1640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc162c60 con 0x7fc5dc162880 2024-04-26T20:51:00.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.822+0000 7fc5e26b1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc0d0d90 con 0x7fc5dc0b5f20 2024-04-26T20:51:00.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.822+0000 7fc5e26b1640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fc5dc162de0 con 0x7fc5dc0d3bc0 2024-04-26T20:51:00.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.822+0000 7fc5e0c4e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5dc162880 0x7fc5dc0ba110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:00.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.822+0000 7fc5e0c4e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5dc162880 0x7fc5dc0ba110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42626/0 (socket says 172.21.15.73:42626) 2024-04-26T20:51:00.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.822+0000 7fc5e0c4e640 1 -- 172.21.15.73:0/2149935351 learned_addr learned my addr 172.21.15.73:0/2149935351 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:00.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.822+0000 7fc5dbfff640 1 --2- 172.21.15.73:0/2149935351 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5dc0b5f20 0x7fc5dc159d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:00.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.822+0000 7fc5db7fe640 1 --2- 172.21.15.73:0/2149935351 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc5dc0d3bc0 0x7fc5dc159520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:00.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.822+0000 7fc5e0c4e640 1 -- 172.21.15.73:0/2149935351 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc5dc0d3bc0 msgr2=0x7fc5dc159520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:00.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.822+0000 7fc5e0c4e640 1 --2- 172.21.15.73:0/2149935351 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc5dc0d3bc0 0x7fc5dc159520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:00.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.822+0000 7fc5e0c4e640 1 -- 172.21.15.73:0/2149935351 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5dc0b5f20 msgr2=0x7fc5dc159d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:00.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.822+0000 7fc5e0c4e640 1 --2- 172.21.15.73:0/2149935351 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5dc0b5f20 0x7fc5dc159d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:00.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.822+0000 7fc5e0c4e640 1 -- 172.21.15.73:0/2149935351 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5dc1f4eb0 con 0x7fc5dc162880 2024-04-26T20:51:00.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.822+0000 7fc5e0c4e640 1 --2- 172.21.15.73:0/2149935351 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5dc162880 0x7fc5dc0ba110 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc5d400caf0 tx=0x7fc5d400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:00.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.823+0000 7fc5d8ff9640 1 -- 172.21.15.73:0/2149935351 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d4017070 con 0x7fc5dc162880 2024-04-26T20:51:00.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.823+0000 7fc5d8ff9640 1 -- 172.21.15.73:0/2149935351 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5d400aca0 con 0x7fc5dc162880 2024-04-26T20:51:00.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.823+0000 7fc5e26b1640 1 -- 172.21.15.73:0/2149935351 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5dc1f5190 con 0x7fc5dc162880 2024-04-26T20:51:00.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.823+0000 7fc5d8ff9640 1 -- 172.21.15.73:0/2149935351 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d4012600 con 0x7fc5dc162880 2024-04-26T20:51:00.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.823+0000 7fc5e26b1640 1 -- 172.21.15.73:0/2149935351 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5dc1f5750 con 0x7fc5dc162880 2024-04-26T20:51:00.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.824+0000 7fc5e26b1640 1 -- 172.21.15.73:0/2149935351 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5dc0694f0 con 0x7fc5dc162880 2024-04-26T20:51:00.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.824+0000 7fc5d8ff9640 1 -- 172.21.15.73:0/2149935351 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc5d401a070 con 0x7fc5dc162880 2024-04-26T20:51:00.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.825+0000 7fc5d8ff9640 1 --2- 172.21.15.73:0/2149935351 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc5c806c8f0 0x7fc5c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:00.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.825+0000 7fc5dbfff640 1 --2- 172.21.15.73:0/2149935351 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc5c806c8f0 0x7fc5c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:00.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.825+0000 7fc5d8ff9640 1 -- 172.21.15.73:0/2149935351 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc5d408bac0 con 0x7fc5dc162880 2024-04-26T20:51:00.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.825+0000 7fc5dbfff640 1 --2- 172.21.15.73:0/2149935351 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc5c806c8f0 0x7fc5c806edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc034000 tx=0x7fc5cc02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:00.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:00.829+0000 7fc5d8ff9640 1 -- 172.21.15.73:0/2149935351 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc5d4059560 con 0x7fc5dc162880 2024-04-26T20:51:01.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.002+0000 7fc5e26b1640 1 -- 172.21.15.73:0/2149935351 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc5dc0008d0 con 0x7fc5dc162880 2024-04-26T20:51:01.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.004+0000 7fc5d8ff9640 1 -- 172.21.15.73:0/2149935351 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18514 (secure 0 0 0) 0x7fc5d405d860 con 0x7fc5dc162880 2024-04-26T20:51:01.005 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:51:01.005 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":25,"fsid":"fbd3f548-b547-4713-856b-09f98f5d1764","created":"2024-04-26T20:50:07.876786+0000","modified":"2024-04-26T20:50:38.657510+0000","last_up_change":"2024-04-26T20:50:19.082741+0000","last_in_change":"2024-04-26T20:50:10.018400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:50:19.993731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:50:29.964420+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:50:30.920082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:50:34.650207+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"72d6f4f3-214a-4392-a02f-c33cbcab3c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6809","nonce":3461705124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6811","nonce":3461705124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6815","nonce":3461705124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6813","nonce":3461705124}]},"public_addr":"172.21.15.73:6809/3461705124","cluster_addr":"172.21.15.73:6811/3461705124","heartbeat_back_addr":"172.21.15.73:6815/3461705124","heartbeat_front_addr":"172.21.15.73:6813/3461705124","state":["exists","up"]},{"osd":1,"uuid":"786bf8ca-6ddc-4b3c-be6c-9152398bd5fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6817","nonce":2827619403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6819","nonce":2827619403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6825","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6827","nonce":2827619403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6821","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6823","nonce":2827619403}]},"public_addr":"172.21.15.73:6817/2827619403","cluster_addr":"172.21.15.73:6819/2827619403","heartbeat_back_addr":"172.21.15.73:6827/2827619403","heartbeat_front_addr":"172.21.15.73:6823/2827619403","state":["exists","up"]},{"osd":2,"uuid":"90963b1a-5437-4ef8-ad8b-6b74c534eeb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6801","nonce":2542681064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6803","nonce":2542681064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6808","nonce":2542681064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6805","nonce":2542681064}]},"public_addr":"172.21.15.73:6801/2542681064","cluster_addr":"172.21.15.73:6803/2542681064","heartbeat_back_addr":"172.21.15.73:6808/2542681064","heartbeat_front_addr":"172.21.15.73:6805/2542681064","state":["exists","up"]},{"osd":3,"uuid":"bd3d2148-6e38-41b4-a453-fe6e3b8b4ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6822","nonce":1604042718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6826","nonce":1604042718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6831","nonce":1604042718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6829","nonce":1604042718}]},"public_addr":"172.21.15.73:6822/1604042718","cluster_addr":"172.21.15.73:6826/1604042718","heartbeat_back_addr":"172.21.15.73:6831/1604042718","heartbeat_front_addr":"172.21.15.73:6829/1604042718","state":["exists","up"]},{"osd":4,"uuid":"e8668dc8-a23f-46d2-8311-cb882da08e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6817","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6819","nonce":3591055561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6824","nonce":3591055561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6831","nonce":3591055561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6828","nonce":3591055561}]},"public_addr":"172.21.15.120:6819/3591055561","cluster_addr":"172.21.15.120:6824/3591055561","heartbeat_back_addr":"172.21.15.120:6831/3591055561","heartbeat_front_addr":"172.21.15.120:6828/3591055561","state":["exists","up"]},{"osd":5,"uuid":"cbac0bdb-96bd-4852-bcce-e564495c3fcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6809","nonce":819450433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6811","nonce":819450433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6815","nonce":819450433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6813","nonce":819450433}]},"public_addr":"172.21.15.120:6809/819450433","cluster_addr":"172.21.15.120:6811/819450433","heartbeat_back_addr":"172.21.15.120:6815/819450433","heartbeat_front_addr":"172.21.15.120:6813/819450433","state":["exists","up"]},{"osd":6,"uuid":"1884b2a8-569f-4272-96a9-f25a09364dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6801","nonce":3946210727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6803","nonce":3946210727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6807","nonce":3946210727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6805","nonce":3946210727}]},"public_addr":"172.21.15.120:6801/3946210727","cluster_addr":"172.21.15.120:6803/3946210727","heartbeat_back_addr":"172.21.15.120:6807/3946210727","heartbeat_front_addr":"172.21.15.120:6805/3946210727","state":["exists","up"]},{"osd":7,"uuid":"c0c3bf6d-a4c5-4f1e-aa2d-330951830666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6818","nonce":2915574674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6821","nonce":2915574674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6827","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6829","nonce":2915574674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6823","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6825","nonce":2915574674}]},"public_addr":"172.21.15.120:6818/2915574674","cluster_addr":"172.21.15.120:6821/2915574674","heartbeat_back_addr":"172.21.15.120:6829/2915574674","heartbeat_front_addr":"172.21.15.120:6825/2915574674","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.218216+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.208710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.112705+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.182789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.164324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.174212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.225396+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.261559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:51:01.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.005+0000 7fc5e26b1640 1 -- 172.21.15.73:0/2149935351 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc5c806c8f0 msgr2=0x7fc5c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:01.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.005+0000 7fc5e26b1640 1 --2- 172.21.15.73:0/2149935351 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc5c806c8f0 0x7fc5c806edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc034000 tx=0x7fc5cc02e000 comp rx=0 tx=0).stop 2024-04-26T20:51:01.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.005+0000 7fc5e26b1640 1 -- 172.21.15.73:0/2149935351 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5dc162880 msgr2=0x7fc5dc0ba110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:01.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.005+0000 7fc5e26b1640 1 --2- 172.21.15.73:0/2149935351 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5dc162880 0x7fc5dc0ba110 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc5d400caf0 tx=0x7fc5d400a060 comp rx=0 tx=0).stop 2024-04-26T20:51:01.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.006+0000 7fc5e26b1640 1 -- 172.21.15.73:0/2149935351 shutdown_connections 2024-04-26T20:51:01.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.006+0000 7fc5e26b1640 1 --2- 172.21.15.73:0/2149935351 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5dc162880 0x7fc5dc0ba110 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:01.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.006+0000 7fc5e26b1640 1 --2- 172.21.15.73:0/2149935351 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc5c806c8f0 0x7fc5c806edb0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:01.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.006+0000 7fc5e26b1640 1 --2- 172.21.15.73:0/2149935351 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc5dc0d3bc0 0x7fc5dc159520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:01.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.006+0000 7fc5e26b1640 1 --2- 172.21.15.73:0/2149935351 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5dc0b5f20 0x7fc5dc159d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:01.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.006+0000 7fc5e26b1640 1 -- 172.21.15.73:0/2149935351 >> 172.21.15.73:0/2149935351 conn(0x7fc5dc0d4f60 msgr2=0x7fc5dc15f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:01.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.006+0000 7fc5e26b1640 1 -- 172.21.15.73:0/2149935351 shutdown_connections 2024-04-26T20:51:01.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.006+0000 7fc5e26b1640 1 -- 172.21.15.73:0/2149935351 wait complete. 2024-04-26T20:51:01.018 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:50:19.993731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:50:29.964420+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:50:30.920082+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T20:50:34.650207+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T20:51:01.018 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:51:01.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.116+0000 7fb0c51b6640 1 Processor -- start 2024-04-26T20:51:01.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.116+0000 7fb0c51b6640 1 -- start start 2024-04-26T20:51:01.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.117+0000 7fb0c51b6640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb0c015f170 0x7fb0c00d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:01.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.117+0000 7fb0c51b6640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fb0c015f550 0x7fb0c00d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:01.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.117+0000 7fb0c51b6640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0c00d1040 con 0x7fb0c00d34b0 2024-04-26T20:51:01.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.117+0000 7fb0c51b6640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fb0c00c9430 con 0x7fb0c015f550 2024-04-26T20:51:01.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.117+0000 7fb0c51b6640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fb0c00c9680 con 0x7fb0c015f170 2024-04-26T20:51:01.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.117+0000 7fb0be59c640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fb0c015f550 0x7fb0c00d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:01.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.117+0000 7fb0bed9d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb0c015f170 0x7fb0c00d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:01.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.117+0000 7fb0be59c640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fb0c015f550 0x7fb0c00d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.73:54490/0 (socket says 172.21.15.73:54490) 2024-04-26T20:51:01.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.117+0000 7fb0bf59e640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7fb0c00d34b0 0x7fb0c00cfc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.73:42458/0 (socket says 172.21.15.73:42458) 2024-04-26T20:51:01.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.117+0000 7fb0be59c640 1 -- 172.21.15.73:0/2710934521 learned_addr learned my addr 172.21.15.73:0/2710934521 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:01.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.117+0000 7fb0bed9d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb0c015f170 0x7fb0c00d5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.73:43660/0 (socket says 172.21.15.73:43660) 2024-04-26T20:51:01.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.117+0000 7fb0be59c640 1 -- 172.21.15.73:0/2710934521 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb0c015f170 msgr2=0x7fb0c00d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:01.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.117+0000 7fb0be59c640 1 --2- 172.21.15.73:0/2710934521 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb0c015f170 0x7fb0c00d5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:01.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.117+0000 7fb0be59c640 1 -- 172.21.15.73:0/2710934521 >> v1:172.21.15.120:6789/0 conn(0x7fb0c00d34b0 legacy=0x7fb0c00cfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:01.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.117+0000 7fb0be59c640 1 -- 172.21.15.73:0/2710934521 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0c00c9960 con 0x7fb0c015f550 2024-04-26T20:51:01.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.118+0000 7fb0be59c640 1 --2- 172.21.15.73:0/2710934521 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb0c015f550 0x7fb0c00d2cb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4009870 tx=0x7fb0b402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99b0f5d0faf36a9b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:01.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.118+0000 7fb0bdd9b640 1 -- 172.21.15.73:0/2710934521 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b4037070 con 0x7fb0c015f550 2024-04-26T20:51:01.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.118+0000 7fb0bdd9b640 1 -- 172.21.15.73:0/2710934521 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0b4032440 con 0x7fb0c015f550 2024-04-26T20:51:01.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.118+0000 7fb0bdd9b640 1 -- 172.21.15.73:0/2710934521 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b4032810 con 0x7fb0c015f550 2024-04-26T20:51:01.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.119+0000 7fb0c51b6640 1 -- 172.21.15.73:0/2710934521 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb0c015f550 msgr2=0x7fb0c00d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:01.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.119+0000 7fb0c51b6640 1 --2- 172.21.15.73:0/2710934521 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb0c015f550 0x7fb0c00d2cb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4009870 tx=0x7fb0b402a270 comp rx=0 tx=0).stop 2024-04-26T20:51:01.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.119+0000 7fb0c51b6640 1 -- 172.21.15.73:0/2710934521 shutdown_connections 2024-04-26T20:51:01.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.119+0000 7fb0c51b6640 1 --2- 172.21.15.73:0/2710934521 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb0c015f550 0x7fb0c00d2cb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:01.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.119+0000 7fb0c51b6640 1 --2- 172.21.15.73:0/2710934521 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb0c015f170 0x7fb0c00d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:01.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.119+0000 7fb0c51b6640 1 -- 172.21.15.73:0/2710934521 >> 172.21.15.73:0/2710934521 conn(0x7fb0c00d4f60 msgr2=0x7fb0c00d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:01.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.119+0000 7fb0c51b6640 1 -- 172.21.15.73:0/2710934521 shutdown_connections 2024-04-26T20:51:01.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.119+0000 7fb0c51b6640 1 -- 172.21.15.73:0/2710934521 wait complete. 2024-04-26T20:51:01.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.120+0000 7fb0c51b6640 1 Processor -- start 2024-04-26T20:51:01.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.120+0000 7fb0c51b6640 1 -- start start 2024-04-26T20:51:01.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.121+0000 7fb0c51b6640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb0c00d34b0 0x7fb0c014f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:01.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.121+0000 7fb0c51b6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb0c015f170 0x7fb0c014fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:01.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.121+0000 7fb0c51b6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0c0150330 0x7fb0c00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:01.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.121+0000 7fb0c51b6640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c00cb5d0 con 0x7fb0c0150330 2024-04-26T20:51:01.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.121+0000 7fb0c51b6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c00cb450 con 0x7fb0c015f170 2024-04-26T20:51:01.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.121+0000 7fb0c51b6640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fb0c00cb750 con 0x7fb0c00d34b0 2024-04-26T20:51:01.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.121+0000 7fb0bf59e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0c0150330 0x7fb0c00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:01.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.121+0000 7fb0bf59e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0c0150330 0x7fb0c00b8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42634/0 (socket says 172.21.15.73:42634) 2024-04-26T20:51:01.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.121+0000 7fb0bf59e640 1 -- 172.21.15.73:0/1457352047 learned_addr learned my addr 172.21.15.73:0/1457352047 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:01.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.121+0000 7fb0bed9d640 1 --2- 172.21.15.73:0/1457352047 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb0c00d34b0 0x7fb0c014f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:01.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.121+0000 7fb0be59c640 1 --2- 172.21.15.73:0/1457352047 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb0c015f170 0x7fb0c014fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:01.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.121+0000 7fb0bf59e640 1 -- 172.21.15.73:0/1457352047 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb0c00d34b0 msgr2=0x7fb0c014f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:01.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.121+0000 7fb0bf59e640 1 --2- 172.21.15.73:0/1457352047 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb0c00d34b0 0x7fb0c014f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:01.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.122+0000 7fb0bf59e640 1 -- 172.21.15.73:0/1457352047 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb0c015f170 msgr2=0x7fb0c014fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:01.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.122+0000 7fb0bf59e640 1 --2- 172.21.15.73:0/1457352047 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb0c015f170 0x7fb0c014fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:01.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.122+0000 7fb0bf59e640 1 -- 172.21.15.73:0/1457352047 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0c00b7dd0 con 0x7fb0c0150330 2024-04-26T20:51:01.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.122+0000 7fb0bf59e640 1 --2- 172.21.15.73:0/1457352047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0c0150330 0x7fb0c00b8b90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb0b800c780 tx=0x7fb0b800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:01.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.122+0000 7fb0a77fe640 1 -- 172.21.15.73:0/1457352047 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b8017070 con 0x7fb0c0150330 2024-04-26T20:51:01.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.122+0000 7fb0a77fe640 1 -- 172.21.15.73:0/1457352047 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0b800a640 con 0x7fb0c0150330 2024-04-26T20:51:01.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.122+0000 7fb0c51b6640 1 -- 172.21.15.73:0/1457352047 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0c00b80b0 con 0x7fb0c0150330 2024-04-26T20:51:01.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.122+0000 7fb0a77fe640 1 -- 172.21.15.73:0/1457352047 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b800aa30 con 0x7fb0c0150330 2024-04-26T20:51:01.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.122+0000 7fb0c51b6640 1 -- 172.21.15.73:0/1457352047 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0c014f7b0 con 0x7fb0c0150330 2024-04-26T20:51:01.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.123+0000 7fb0c51b6640 1 -- 172.21.15.73:0/1457352047 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0c00694f0 con 0x7fb0c0150330 2024-04-26T20:51:01.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.123+0000 7fb0a77fe640 1 -- 172.21.15.73:0/1457352047 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb0b801a070 con 0x7fb0c0150330 2024-04-26T20:51:01.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.124+0000 7fb0a77fe640 1 --2- 172.21.15.73:0/1457352047 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb0ac06c8f0 0x7fb0ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:01.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.124+0000 7fb0bed9d640 1 --2- 172.21.15.73:0/1457352047 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb0ac06c8f0 0x7fb0ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:01.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.124+0000 7fb0a77fe640 1 -- 172.21.15.73:0/1457352047 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fb0b808b410 con 0x7fb0c0150330 2024-04-26T20:51:01.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.124+0000 7fb0bed9d640 1 --2- 172.21.15.73:0/1457352047 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb0ac06c8f0 0x7fb0ac06edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0009b40 tx=0x7fb0b0009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:01.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.128+0000 7fb0a77fe640 1 -- 172.21.15.73:0/1457352047 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb0c00d1040 con 0x7fb0c0150330 2024-04-26T20:51:01.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.305+0000 7fb0c51b6640 1 -- 172.21.15.73:0/1457352047 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb0c00d0c20 con 0x7fb0c0150330 2024-04-26T20:51:01.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.306+0000 7fb0a77fe640 1 -- 172.21.15.73:0/1457352047 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7fb0b8058eb0 con 0x7fb0c0150330 2024-04-26T20:51:01.307 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-26T20:51:01.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.307+0000 7fb0c51b6640 1 -- 172.21.15.73:0/1457352047 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb0ac06c8f0 msgr2=0x7fb0ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:01.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.307+0000 7fb0c51b6640 1 --2- 172.21.15.73:0/1457352047 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb0ac06c8f0 0x7fb0ac06edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0009b40 tx=0x7fb0b0009190 comp rx=0 tx=0).stop 2024-04-26T20:51:01.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.307+0000 7fb0c51b6640 1 -- 172.21.15.73:0/1457352047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0c0150330 msgr2=0x7fb0c00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:01.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.307+0000 7fb0c51b6640 1 --2- 172.21.15.73:0/1457352047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0c0150330 0x7fb0c00b8b90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb0b800c780 tx=0x7fb0b800cc40 comp rx=0 tx=0).stop 2024-04-26T20:51:01.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.308+0000 7fb0c51b6640 1 -- 172.21.15.73:0/1457352047 shutdown_connections 2024-04-26T20:51:01.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.308+0000 7fb0c51b6640 1 --2- 172.21.15.73:0/1457352047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0c0150330 0x7fb0c00b8b90 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:01.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.308+0000 7fb0c51b6640 1 --2- 172.21.15.73:0/1457352047 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb0ac06c8f0 0x7fb0ac06edb0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:01.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.308+0000 7fb0c51b6640 1 --2- 172.21.15.73:0/1457352047 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb0c015f170 0x7fb0c014fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:01.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.308+0000 7fb0c51b6640 1 --2- 172.21.15.73:0/1457352047 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb0c00d34b0 0x7fb0c014f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:01.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.308+0000 7fb0c51b6640 1 -- 172.21.15.73:0/1457352047 >> 172.21.15.73:0/1457352047 conn(0x7fb0c00d4f60 msgr2=0x7fb0c00d38e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:01.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.308+0000 7fb0c51b6640 1 -- 172.21.15.73:0/1457352047 shutdown_connections 2024-04-26T20:51:01.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.308+0000 7fb0c51b6640 1 -- 172.21.15.73:0/1457352047 wait complete. 2024-04-26T20:51:01.320 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:51:01.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.418+0000 7f2f1bc45640 1 Processor -- start 2024-04-26T20:51:01.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.418+0000 7f2f1bc45640 1 -- start start 2024-04-26T20:51:01.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.419+0000 7f2f1bc45640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2f1415f170 0x7f2f140d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:01.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.419+0000 7f2f1bc45640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f2f1415f550 0x7f2f140d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:01.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.419+0000 7f2f1bc45640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2f140d34b0 0x7f2f140d3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:01.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.419+0000 7f2f1bc45640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f2f140d3e60 con 0x7f2f1415f170 2024-04-26T20:51:01.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.419+0000 7f2f1bc45640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f2f140c9430 con 0x7f2f1415f550 2024-04-26T20:51:01.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.419+0000 7f2f1bc45640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f2f140d1040 con 0x7f2f140d34b0 2024-04-26T20:51:01.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.419+0000 7f2f199e1640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2f1415f170 0x7f2f140d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:01.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.419+0000 7f2f199e1640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2f1415f170 0x7f2f140d5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42644/0 (socket says 172.21.15.73:42644) 2024-04-26T20:51:01.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.419+0000 7f2f199e1640 1 -- 172.21.15.73:0/3519719401 learned_addr learned my addr 172.21.15.73:0/3519719401 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:01.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.419+0000 7f2f191e0640 1 --2- 172.21.15.73:0/3519719401 >> v2:172.21.15.120:3300/0 conn(0x7f2f1415f550 0x7f2f140d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:01.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.419+0000 7f2f1a1e2640 1 --2- 172.21.15.73:0/3519719401 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2f140d34b0 0x7f2f140d3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:01.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.420+0000 7f2f199e1640 1 -- 172.21.15.73:0/3519719401 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2f140d34b0 msgr2=0x7f2f140d3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:01.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.420+0000 7f2f199e1640 1 --2- 172.21.15.73:0/3519719401 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2f140d34b0 0x7f2f140d3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:01.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.420+0000 7f2f199e1640 1 -- 172.21.15.73:0/3519719401 >> v2:172.21.15.120:3300/0 conn(0x7f2f1415f550 msgr2=0x7f2f140d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:01.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.420+0000 7f2f199e1640 1 --2- 172.21.15.73:0/3519719401 >> v2:172.21.15.120:3300/0 conn(0x7f2f1415f550 0x7f2f140d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:01.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.420+0000 7f2f199e1640 1 -- 172.21.15.73:0/3519719401 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f140c9680 con 0x7f2f1415f170 2024-04-26T20:51:01.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.420+0000 7f2f1a1e2640 1 --2- 172.21.15.73:0/3519719401 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2f140d34b0 0x7f2f140d3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:01.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.420+0000 7f2f199e1640 1 --2- 172.21.15.73:0/3519719401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f1415f170 0x7f2f140d5a80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2f10009e50 tx=0x7f2f1002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2611a7920a14c52 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:01.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.420+0000 7f2f189df640 1 -- 172.21.15.73:0/3519719401 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f10036070 con 0x7f2f1415f170 2024-04-26T20:51:01.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.420+0000 7f2f189df640 1 -- 172.21.15.73:0/3519719401 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f10031600 con 0x7f2f1415f170 2024-04-26T20:51:01.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.420+0000 7f2f189df640 1 -- 172.21.15.73:0/3519719401 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f100319d0 con 0x7f2f1415f170 2024-04-26T20:51:01.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.421+0000 7f2f1bc45640 1 -- 172.21.15.73:0/3519719401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f1415f170 msgr2=0x7f2f140d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:01.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.421+0000 7f2f1bc45640 1 --2- 172.21.15.73:0/3519719401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f1415f170 0x7f2f140d5a80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2f10009e50 tx=0x7f2f1002a6c0 comp rx=0 tx=0).stop 2024-04-26T20:51:01.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.421+0000 7f2f1bc45640 1 -- 172.21.15.73:0/3519719401 shutdown_connections 2024-04-26T20:51:01.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.421+0000 7f2f1bc45640 1 --2- 172.21.15.73:0/3519719401 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2f140d34b0 0x7f2f140d3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:01.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.421+0000 7f2f1bc45640 1 --2- 172.21.15.73:0/3519719401 >> v2:172.21.15.120:3300/0 conn(0x7f2f1415f550 0x7f2f140d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:01.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.421+0000 7f2f1bc45640 1 --2- 172.21.15.73:0/3519719401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f1415f170 0x7f2f140d5a80 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:01.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.421+0000 7f2f1bc45640 1 -- 172.21.15.73:0/3519719401 >> 172.21.15.73:0/3519719401 conn(0x7f2f140d4f60 msgr2=0x7f2f140d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:01.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.421+0000 7f2f1bc45640 1 -- 172.21.15.73:0/3519719401 shutdown_connections 2024-04-26T20:51:01.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.421+0000 7f2f1bc45640 1 -- 172.21.15.73:0/3519719401 wait complete. 2024-04-26T20:51:01.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.422+0000 7f2f1bc45640 1 Processor -- start 2024-04-26T20:51:01.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.422+0000 7f2f1bc45640 1 -- start start 2024-04-26T20:51:01.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.423+0000 7f2f1bc45640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f140d34b0 0x7f2f141e56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:01.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.423+0000 7f2f1bc45640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2f1415f170 0x7f2f141e8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:01.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.423+0000 7f2f1bc45640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2f141e90f0 0x7f2f1415b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:01.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.423+0000 7f2f1bc45640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2f140cfc50 con 0x7f2f140d34b0 2024-04-26T20:51:01.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.423+0000 7f2f1bc45640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2f140c9c40 con 0x7f2f1415f170 2024-04-26T20:51:01.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.423+0000 7f2f1bc45640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f2f140cfdd0 con 0x7f2f141e90f0 2024-04-26T20:51:01.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.423+0000 7f2f199e1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f140d34b0 0x7f2f141e56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:01.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.423+0000 7f2f199e1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f140d34b0 0x7f2f141e56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42654/0 (socket says 172.21.15.73:42654) 2024-04-26T20:51:01.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.423+0000 7f2f199e1640 1 -- 172.21.15.73:0/2744577989 learned_addr learned my addr 172.21.15.73:0/2744577989 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:01.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.423+0000 7f2f1a1e2640 1 --2- 172.21.15.73:0/2744577989 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2f141e90f0 0x7f2f1415b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:01.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.423+0000 7f2f191e0640 1 --2- 172.21.15.73:0/2744577989 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2f1415f170 0x7f2f141e8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:01.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.423+0000 7f2f199e1640 1 -- 172.21.15.73:0/2744577989 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2f141e90f0 msgr2=0x7f2f1415b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:01.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.423+0000 7f2f199e1640 1 --2- 172.21.15.73:0/2744577989 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2f141e90f0 0x7f2f1415b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:01.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.423+0000 7f2f199e1640 1 -- 172.21.15.73:0/2744577989 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2f1415f170 msgr2=0x7f2f141e8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:01.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.423+0000 7f2f199e1640 1 --2- 172.21.15.73:0/2744577989 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2f1415f170 0x7f2f141e8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:01.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.423+0000 7f2f199e1640 1 -- 172.21.15.73:0/2744577989 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f141e4650 con 0x7f2f140d34b0 2024-04-26T20:51:01.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.424+0000 7f2f199e1640 1 --2- 172.21.15.73:0/2744577989 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f140d34b0 0x7f2f141e56b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2f10034040 tx=0x7f2f10004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:01.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.424+0000 7f2f0a7fc640 1 -- 172.21.15.73:0/2744577989 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f10036050 con 0x7f2f140d34b0 2024-04-26T20:51:01.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.424+0000 7f2f0a7fc640 1 -- 172.21.15.73:0/2744577989 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f10042070 con 0x7f2f140d34b0 2024-04-26T20:51:01.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.424+0000 7f2f0a7fc640 1 -- 172.21.15.73:0/2744577989 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f10009510 con 0x7f2f140d34b0 2024-04-26T20:51:01.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.424+0000 7f2f1bc45640 1 -- 172.21.15.73:0/2744577989 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f141e48f0 con 0x7f2f140d34b0 2024-04-26T20:51:01.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.424+0000 7f2f1bc45640 1 -- 172.21.15.73:0/2744577989 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f141e4eb0 con 0x7f2f140d34b0 2024-04-26T20:51:01.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.425+0000 7f2f1bc45640 1 -- 172.21.15.73:0/2744577989 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f140694f0 con 0x7f2f140d34b0 2024-04-26T20:51:01.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.425+0000 7f2f0a7fc640 1 -- 172.21.15.73:0/2744577989 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2f10007c50 con 0x7f2f140d34b0 2024-04-26T20:51:01.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.426+0000 7f2f0a7fc640 1 --2- 172.21.15.73:0/2744577989 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f2ef806c8f0 0x7f2ef806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:01.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.426+0000 7f2f191e0640 1 --2- 172.21.15.73:0/2744577989 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f2ef806c8f0 0x7f2ef806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:01.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.426+0000 7f2f0a7fc640 1 -- 172.21.15.73:0/2744577989 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f2f100acbe0 con 0x7f2f140d34b0 2024-04-26T20:51:01.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.426+0000 7f2f191e0640 1 --2- 172.21.15.73:0/2744577989 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f2ef806c8f0 0x7f2ef806edb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2f1408e440 tx=0x7f2f04008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:01.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.430+0000 7f2f0a7fc640 1 -- 172.21.15.73:0/2744577989 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2f1007a680 con 0x7f2f140d34b0 2024-04-26T20:51:01.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.604+0000 7f2f1bc45640 1 -- 172.21.15.73:0/2744577989 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2f140d1560 con 0x7f2f140d34b0 2024-04-26T20:51:01.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.604+0000 7f2f0a7fc640 1 -- 172.21.15.73:0/2744577989 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f2f1007e980 con 0x7f2f140d34b0 2024-04-26T20:51:01.605 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-26T20:51:01.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.606+0000 7f2f1bc45640 1 -- 172.21.15.73:0/2744577989 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f2ef806c8f0 msgr2=0x7f2ef806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:01.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.606+0000 7f2f1bc45640 1 --2- 172.21.15.73:0/2744577989 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f2ef806c8f0 0x7f2ef806edb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2f1408e440 tx=0x7f2f04008040 comp rx=0 tx=0).stop 2024-04-26T20:51:01.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.606+0000 7f2f1bc45640 1 -- 172.21.15.73:0/2744577989 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f140d34b0 msgr2=0x7f2f141e56b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:01.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.606+0000 7f2f1bc45640 1 --2- 172.21.15.73:0/2744577989 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f140d34b0 0x7f2f141e56b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2f10034040 tx=0x7f2f10004a40 comp rx=0 tx=0).stop 2024-04-26T20:51:01.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.606+0000 7f2f1bc45640 1 -- 172.21.15.73:0/2744577989 shutdown_connections 2024-04-26T20:51:01.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.606+0000 7f2f1bc45640 1 --2- 172.21.15.73:0/2744577989 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f2ef806c8f0 0x7f2ef806edb0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:01.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.606+0000 7f2f1bc45640 1 --2- 172.21.15.73:0/2744577989 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2f141e90f0 0x7f2f1415b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:01.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.606+0000 7f2f1bc45640 1 --2- 172.21.15.73:0/2744577989 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2f1415f170 0x7f2f141e8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:01.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.606+0000 7f2f1bc45640 1 --2- 172.21.15.73:0/2744577989 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f140d34b0 0x7f2f141e56b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:01.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.606+0000 7f2f1bc45640 1 -- 172.21.15.73:0/2744577989 >> 172.21.15.73:0/2744577989 conn(0x7f2f140d4f60 msgr2=0x7f2f1415e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:01.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.606+0000 7f2f1bc45640 1 -- 172.21.15.73:0/2744577989 shutdown_connections 2024-04-26T20:51:01.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.606+0000 7f2f1bc45640 1 -- 172.21.15.73:0/2744577989 wait complete. 2024-04-26T20:51:01.618 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:51:01.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.716+0000 7f1bf4259640 1 Processor -- start 2024-04-26T20:51:01.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.717+0000 7f1bf4259640 1 -- start start 2024-04-26T20:51:01.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.717+0000 7f1bf4259640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1bec0d1a00 0x7f1bec0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:01.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.717+0000 7f1bf4259640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bec0d0a10 con 0x7f1bec0b40d0 2024-04-26T20:51:01.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.717+0000 7f1bf4259640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bec0d0cf0 con 0x7f1bec0d2320 2024-04-26T20:51:01.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.717+0000 7f1bf4259640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f1bec0d0fd0 con 0x7f1bec0d1a00 2024-04-26T20:51:01.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.718+0000 7f1bf27f6640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f1bec0b40d0 0x7f1bec0b4b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:42042/0 (socket says 172.21.15.73:42042) 2024-04-26T20:51:01.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.718+0000 7f1bf27f6640 1 -- 172.21.15.73:0/4137663748 learned_addr learned my addr 172.21.15.73:0/4137663748 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:01.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.718+0000 7f1bf0ff3640 1 -- 172.21.15.73:0/4137663748 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 779903152 0 0) 0x7f1bec0d0a10 con 0x7f1bec0b40d0 2024-04-26T20:51:01.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.718+0000 7f1bf1ff5640 1 --2- 172.21.15.73:0/4137663748 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1bec0d1a00 0x7f1bec0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:01.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.718+0000 7f1bf0ff3640 1 -- 172.21.15.73:0/4137663748 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1bd8003520 con 0x7f1bec0b40d0 2024-04-26T20:51:01.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.718+0000 7f1bf0ff3640 1 -- 172.21.15.73:0/4137663748 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3603755081 0 0) 0x7f1bd8003520 con 0x7f1bec0b40d0 2024-04-26T20:51:01.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.718+0000 7f1bf0ff3640 1 -- 172.21.15.73:0/4137663748 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1bec0d1a00 msgr2=0x7f1bec0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:01.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.718+0000 7f1bf0ff3640 1 --2- 172.21.15.73:0/4137663748 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1bec0d1a00 0x7f1bec0d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:01.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.718+0000 7f1bf0ff3640 1 -- 172.21.15.73:0/4137663748 >> v1:172.21.15.120:6789/0 conn(0x7f1bec0d2320 legacy=0x7f1bec1625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:01.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.718+0000 7f1bf0ff3640 1 -- 172.21.15.73:0/4137663748 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bec0d40a0 con 0x7f1bec0b40d0 2024-04-26T20:51:01.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.718+0000 7f1bf0ff3640 1 -- 172.21.15.73:0/4137663748 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7f1be80019f0 con 0x7f1bec0b40d0 2024-04-26T20:51:01.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.718+0000 7f1bf0ff3640 1 -- 172.21.15.73:0/4137663748 >> v1:172.21.15.73:6789/0 conn(0x7f1bec0b40d0 legacy=0x7f1bec0b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:01.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.718+0000 7f1bf0ff3640 1 --2- 172.21.15.73:0/4137663748 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1bd80039e0 0x7f1bd8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:01.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.718+0000 7f1bf0ff3640 1 --2- 172.21.15.73:0/4137663748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1bd8004800 0x7f1bd8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:01.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.718+0000 7f1bf0ff3640 1 --2- 172.21.15.73:0/4137663748 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1bec0d1a00 0x7f1bd8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:01.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.718+0000 7f1bf0ff3640 1 -- 172.21.15.73:0/4137663748 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1bec0d0fd0 con 0x7f1bd8004800 2024-04-26T20:51:01.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.718+0000 7f1bf0ff3640 1 -- 172.21.15.73:0/4137663748 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd800baf0 con 0x7f1bd80039e0 2024-04-26T20:51:01.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.718+0000 7f1bf0ff3640 1 -- 172.21.15.73:0/4137663748 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f1bd800bcf0 con 0x7f1bec0d1a00 2024-04-26T20:51:01.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.719+0000 7f1bf17f4640 1 --2- 172.21.15.73:0/4137663748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1bd8004800 0x7f1bd8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:01.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.719+0000 7f1bf1ff5640 1 --2- 172.21.15.73:0/4137663748 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1bd80039e0 0x7f1bd8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:01.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.719+0000 7f1bf27f6640 1 --2- 172.21.15.73:0/4137663748 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1bec0d1a00 0x7f1bd8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:01.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.719+0000 7f1bf17f4640 1 -- 172.21.15.73:0/4137663748 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1bec0d1a00 msgr2=0x7f1bd8005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:01.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.719+0000 7f1bf17f4640 1 --2- 172.21.15.73:0/4137663748 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1bec0d1a00 0x7f1bd8005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:01.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.719+0000 7f1bf17f4640 1 -- 172.21.15.73:0/4137663748 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1bd80039e0 msgr2=0x7f1bd8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:01.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.719+0000 7f1bf17f4640 1 --2- 172.21.15.73:0/4137663748 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1bd80039e0 0x7f1bd8003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:01.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.719+0000 7f1bf17f4640 1 -- 172.21.15.73:0/4137663748 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bd8003520 con 0x7f1bd8004800 2024-04-26T20:51:01.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.719+0000 7f1bf17f4640 1 --2- 172.21.15.73:0/4137663748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1bd8004800 0x7f1bd8004c80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc001750 tx=0x7f1bd8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76455a67ad511ba0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:01.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.720+0000 7f1bf0ff3640 1 -- 172.21.15.73:0/4137663748 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc001370 con 0x7f1bd8004800 2024-04-26T20:51:01.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.720+0000 7f1bf0ff3640 1 -- 172.21.15.73:0/4137663748 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bdc0052d0 con 0x7f1bd8004800 2024-04-26T20:51:01.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.720+0000 7f1bf0ff3640 1 -- 172.21.15.73:0/4137663748 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc02abe0 con 0x7f1bd8004800 2024-04-26T20:51:01.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.720+0000 7f1bf4259640 1 -- 172.21.15.73:0/4137663748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1bd8004800 msgr2=0x7f1bd8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:01.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.720+0000 7f1bf4259640 1 --2- 172.21.15.73:0/4137663748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1bd8004800 0x7f1bd8004c80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc001750 tx=0x7f1bd8005830 comp rx=0 tx=0).stop 2024-04-26T20:51:01.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.720+0000 7f1bf27f6640 1 -- 172.21.15.73:0/4137663748 reap_dead start 2024-04-26T20:51:01.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.720+0000 7f1bf4259640 1 -- 172.21.15.73:0/4137663748 shutdown_connections 2024-04-26T20:51:01.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.720+0000 7f1bf4259640 1 -- 172.21.15.73:0/4137663748 >> 172.21.15.73:0/4137663748 conn(0x7f1bec0d4fd0 msgr2=0x7f1bec0d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:01.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.720+0000 7f1bf4259640 1 -- 172.21.15.73:0/4137663748 shutdown_connections 2024-04-26T20:51:01.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.720+0000 7f1bf4259640 1 -- 172.21.15.73:0/4137663748 wait complete. 2024-04-26T20:51:01.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.721+0000 7f1bf4259640 1 Processor -- start 2024-04-26T20:51:01.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.722+0000 7f1bf4259640 1 -- start start 2024-04-26T20:51:01.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.722+0000 7f1bf4259640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1bd8004800 0x7f1bec0cb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:01.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.722+0000 7f1bf4259640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1bec14ed90 0x7f1bec0c86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:01.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.722+0000 7f1bf4259640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1bec0c8c10 0x7f1bec1f5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:01.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.722+0000 7f1bf4259640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1bec0d1de0 con 0x7f1bd8004800 2024-04-26T20:51:01.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.722+0000 7f1bf4259640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1bec162f50 con 0x7f1bec0c8c10 2024-04-26T20:51:01.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.722+0000 7f1bf4259640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f1bec0d1f60 con 0x7f1bec14ed90 2024-04-26T20:51:01.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.722+0000 7f1bf1ff5640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1bd8004800 0x7f1bec0cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:01.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.722+0000 7f1bf1ff5640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1bd8004800 0x7f1bec0cb890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42664/0 (socket says 172.21.15.73:42664) 2024-04-26T20:51:01.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.722+0000 7f1bf1ff5640 1 -- 172.21.15.73:0/232291028 learned_addr learned my addr 172.21.15.73:0/232291028 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:01.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.722+0000 7f1bf17f4640 1 --2- 172.21.15.73:0/232291028 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1bec14ed90 0x7f1bec0c86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:01.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.722+0000 7f1bf27f6640 1 --2- 172.21.15.73:0/232291028 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1bec0c8c10 0x7f1bec1f5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:01.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.723+0000 7f1bf1ff5640 1 -- 172.21.15.73:0/232291028 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1bec14ed90 msgr2=0x7f1bec0c86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:01.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.723+0000 7f1bf1ff5640 1 --2- 172.21.15.73:0/232291028 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1bec14ed90 0x7f1bec0c86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:01.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.723+0000 7f1bf1ff5640 1 -- 172.21.15.73:0/232291028 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1bec0c8c10 msgr2=0x7f1bec1f5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:01.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.723+0000 7f1bf1ff5640 1 --2- 172.21.15.73:0/232291028 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1bec0c8c10 0x7f1bec1f5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:01.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.723+0000 7f1bf1ff5640 1 -- 172.21.15.73:0/232291028 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bec0cab40 con 0x7f1bd8004800 2024-04-26T20:51:01.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.723+0000 7f1bf27f6640 1 --2- 172.21.15.73:0/232291028 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1bec0c8c10 0x7f1bec1f5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:01.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.723+0000 7f1bf1ff5640 1 --2- 172.21.15.73:0/232291028 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1bd8004800 0x7f1bec0cb890 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1be40118e0 tx=0x7f1be4011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:01.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.723+0000 7f1be27fc640 1 -- 172.21.15.73:0/232291028 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be400fa90 con 0x7f1bd8004800 2024-04-26T20:51:01.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.723+0000 7f1be27fc640 1 -- 172.21.15.73:0/232291028 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1be400fc30 con 0x7f1bd8004800 2024-04-26T20:51:01.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.723+0000 7f1bf4259640 1 -- 172.21.15.73:0/232291028 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bec0cadb0 con 0x7f1bd8004800 2024-04-26T20:51:01.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.723+0000 7f1be27fc640 1 -- 172.21.15.73:0/232291028 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be4014440 con 0x7f1bd8004800 2024-04-26T20:51:01.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.723+0000 7f1bf4259640 1 -- 172.21.15.73:0/232291028 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bec0cbdd0 con 0x7f1bd8004800 2024-04-26T20:51:01.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.724+0000 7f1bf4259640 1 -- 172.21.15.73:0/232291028 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bec0694f0 con 0x7f1bd8004800 2024-04-26T20:51:01.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.724+0000 7f1be27fc640 1 -- 172.21.15.73:0/232291028 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1be4012040 con 0x7f1bd8004800 2024-04-26T20:51:01.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.725+0000 7f1be27fc640 1 --2- 172.21.15.73:0/232291028 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1bd006c920 0x7f1bd006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:01.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.725+0000 7f1be27fc640 1 -- 172.21.15.73:0/232291028 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f1be408c410 con 0x7f1bd8004800 2024-04-26T20:51:01.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.725+0000 7f1bf17f4640 1 --2- 172.21.15.73:0/232291028 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1bd006c920 0x7f1bd006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:01.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.725+0000 7f1bf17f4640 1 --2- 172.21.15.73:0/232291028 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1bd006c920 0x7f1bd006ede0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc032000 tx=0x7f1bdc02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:01.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.729+0000 7f1be27fc640 1 -- 172.21.15.73:0/232291028 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1be4019030 con 0x7f1bd8004800 2024-04-26T20:51:01.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.905+0000 7f1bf4259640 1 -- 172.21.15.73:0/232291028 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1bec1622e0 con 0x7f1bd8004800 2024-04-26T20:51:01.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.905+0000 7f1be27fc640 1 -- 172.21.15.73:0/232291028 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f1be405a500 con 0x7f1bd8004800 2024-04-26T20:51:01.906 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-26T20:51:01.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.907+0000 7f1bf4259640 1 -- 172.21.15.73:0/232291028 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1bd006c920 msgr2=0x7f1bd006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:01.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.907+0000 7f1bf4259640 1 --2- 172.21.15.73:0/232291028 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1bd006c920 0x7f1bd006ede0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc032000 tx=0x7f1bdc02e000 comp rx=0 tx=0).stop 2024-04-26T20:51:01.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.907+0000 7f1bf4259640 1 -- 172.21.15.73:0/232291028 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1bd8004800 msgr2=0x7f1bec0cb890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:01.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.907+0000 7f1bf4259640 1 --2- 172.21.15.73:0/232291028 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1bd8004800 0x7f1bec0cb890 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1be40118e0 tx=0x7f1be4011da0 comp rx=0 tx=0).stop 2024-04-26T20:51:01.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.907+0000 7f1bf4259640 1 -- 172.21.15.73:0/232291028 shutdown_connections 2024-04-26T20:51:01.909 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.907+0000 7f1bf4259640 1 --2- 172.21.15.73:0/232291028 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1bd006c920 0x7f1bd006ede0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:01.909 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.907+0000 7f1bf4259640 1 --2- 172.21.15.73:0/232291028 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1bec0c8c10 0x7f1bec1f5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:01.909 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.907+0000 7f1bf4259640 1 --2- 172.21.15.73:0/232291028 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1bec14ed90 0x7f1bec0c86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:01.909 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.907+0000 7f1bf4259640 1 --2- 172.21.15.73:0/232291028 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1bd8004800 0x7f1bec0cb890 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:01.909 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.907+0000 7f1bf4259640 1 -- 172.21.15.73:0/232291028 >> 172.21.15.73:0/232291028 conn(0x7f1bec0d4fd0 msgr2=0x7f1bec0b6260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:01.909 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.907+0000 7f1bf4259640 1 -- 172.21.15.73:0/232291028 shutdown_connections 2024-04-26T20:51:01.909 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:01.907+0000 7f1bf4259640 1 -- 172.21.15.73:0/232291028 wait complete. 2024-04-26T20:51:01.919 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T20:51:02.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.019+0000 7fa4179e4640 1 Processor -- start 2024-04-26T20:51:02.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.019+0000 7fa4179e4640 1 -- start start 2024-04-26T20:51:02.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.020+0000 7fa4179e4640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa4100cfde0 0x7fa4100d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:02.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.020+0000 7fa4179e4640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa410071260 0x7fa41015e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:02.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.020+0000 7fa4179e4640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4100d2390 con 0x7fa410070350 2024-04-26T20:51:02.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.020+0000 7fa4179e4640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7fa4100afab0 con 0x7fa410071260 2024-04-26T20:51:02.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.020+0000 7fa4179e4640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fa41006dd20 con 0x7fa4100cfde0 2024-04-26T20:51:02.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.020+0000 7fa415f81640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa410071260 0x7fa41015e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:02.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.020+0000 7fa415f81640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa410071260 0x7fa41015e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42668/0 (socket says 172.21.15.73:42668) 2024-04-26T20:51:02.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.020+0000 7fa415f81640 1 -- 172.21.15.73:0/3546738250 learned_addr learned my addr 172.21.15.73:0/3546738250 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:02.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.020+0000 7fa415780640 1 --2- 172.21.15.73:0/3546738250 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa4100cfde0 0x7fa4100d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:02.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.020+0000 7fa415f81640 1 -- 172.21.15.73:0/3546738250 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa4100cfde0 msgr2=0x7fa4100d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:02.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.020+0000 7fa415f81640 1 --2- 172.21.15.73:0/3546738250 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa4100cfde0 0x7fa4100d1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:02.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.020+0000 7fa415f81640 1 -- 172.21.15.73:0/3546738250 >> v1:172.21.15.120:6789/0 conn(0x7fa410070350 legacy=0x7fa410070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:02.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.020+0000 7fa415f81640 1 -- 172.21.15.73:0/3546738250 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa41006d3f0 con 0x7fa410071260 2024-04-26T20:51:02.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.021+0000 7fa415f81640 1 --2- 172.21.15.73:0/3546738250 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa410071260 0x7fa41015e9f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa408009e50 tx=0x7fa40802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=657a5c17728055a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:02.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.021+0000 7fa407fff640 1 -- 172.21.15.73:0/3546738250 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa408036070 con 0x7fa410071260 2024-04-26T20:51:02.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.021+0000 7fa407fff640 1 -- 172.21.15.73:0/3546738250 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa408031600 con 0x7fa410071260 2024-04-26T20:51:02.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.021+0000 7fa407fff640 1 -- 172.21.15.73:0/3546738250 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4080319d0 con 0x7fa410071260 2024-04-26T20:51:02.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.022+0000 7fa4179e4640 1 -- 172.21.15.73:0/3546738250 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa410071260 msgr2=0x7fa41015e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:02.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.022+0000 7fa4179e4640 1 --2- 172.21.15.73:0/3546738250 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa410071260 0x7fa41015e9f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa408009e50 tx=0x7fa40802a680 comp rx=0 tx=0).stop 2024-04-26T20:51:02.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.022+0000 7fa4179e4640 1 -- 172.21.15.73:0/3546738250 shutdown_connections 2024-04-26T20:51:02.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.022+0000 7fa4179e4640 1 --2- 172.21.15.73:0/3546738250 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa410071260 0x7fa41015e9f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:02.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.022+0000 7fa4179e4640 1 --2- 172.21.15.73:0/3546738250 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa4100cfde0 0x7fa4100d1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:02.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.022+0000 7fa4179e4640 1 -- 172.21.15.73:0/3546738250 >> 172.21.15.73:0/3546738250 conn(0x7fa4100d2d80 msgr2=0x7fa4100d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:02.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.022+0000 7fa4179e4640 1 -- 172.21.15.73:0/3546738250 shutdown_connections 2024-04-26T20:51:02.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.022+0000 7fa4179e4640 1 -- 172.21.15.73:0/3546738250 wait complete. 2024-04-26T20:51:02.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.023+0000 7fa4179e4640 1 Processor -- start 2024-04-26T20:51:02.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.023+0000 7fa4179e4640 1 -- start start 2024-04-26T20:51:02.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.023+0000 7fa4179e4640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa410070350 0x7fa41014dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:02.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.024+0000 7fa4179e4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa410071260 0x7fa4100b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:02.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.024+0000 7fa4179e4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4100cfde0 0x7fa41014ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:02.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.024+0000 7fa4179e4640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa4100d20a0 con 0x7fa4100cfde0 2024-04-26T20:51:02.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.024+0000 7fa4179e4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa41006d9b0 con 0x7fa410071260 2024-04-26T20:51:02.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.024+0000 7fa4179e4640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fa41015ef30 con 0x7fa410070350 2024-04-26T20:51:02.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.024+0000 7fa415f81640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4100cfde0 0x7fa41014ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:02.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.024+0000 7fa415f81640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4100cfde0 0x7fa41014ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42678/0 (socket says 172.21.15.73:42678) 2024-04-26T20:51:02.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.024+0000 7fa415f81640 1 -- 172.21.15.73:0/2288003045 learned_addr learned my addr 172.21.15.73:0/2288003045 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:02.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.024+0000 7fa415780640 1 --2- 172.21.15.73:0/2288003045 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa410070350 0x7fa41014dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:02.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.024+0000 7fa415f81640 1 -- 172.21.15.73:0/2288003045 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa410070350 msgr2=0x7fa41014dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:02.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.024+0000 7fa415f81640 1 --2- 172.21.15.73:0/2288003045 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa410070350 0x7fa41014dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:02.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.024+0000 7fa415f81640 1 -- 172.21.15.73:0/2288003045 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa410071260 msgr2=0x7fa4100b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:02.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.024+0000 7fa414f7f640 1 --2- 172.21.15.73:0/2288003045 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa410071260 0x7fa4100b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:02.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.024+0000 7fa415f81640 1 --2- 172.21.15.73:0/2288003045 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa410071260 0x7fa4100b4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:02.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.024+0000 7fa415f81640 1 -- 172.21.15.73:0/2288003045 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa41015eb50 con 0x7fa4100cfde0 2024-04-26T20:51:02.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.024+0000 7fa415f81640 1 --2- 172.21.15.73:0/2288003045 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4100cfde0 0x7fa41014ce20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa408009e50 tx=0x7fa408004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:02.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.024+0000 7fa405ffb640 1 -- 172.21.15.73:0/2288003045 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa408036050 con 0x7fa4100cfde0 2024-04-26T20:51:02.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.024+0000 7fa405ffb640 1 -- 172.21.15.73:0/2288003045 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa408042070 con 0x7fa4100cfde0 2024-04-26T20:51:02.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.024+0000 7fa4179e4640 1 -- 172.21.15.73:0/2288003045 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa41014e280 con 0x7fa4100cfde0 2024-04-26T20:51:02.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.024+0000 7fa405ffb640 1 -- 172.21.15.73:0/2288003045 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa408009510 con 0x7fa4100cfde0 2024-04-26T20:51:02.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.025+0000 7fa4179e4640 1 -- 172.21.15.73:0/2288003045 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa41014e840 con 0x7fa4100cfde0 2024-04-26T20:51:02.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.026+0000 7fa4179e4640 1 -- 172.21.15.73:0/2288003045 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4100694f0 con 0x7fa4100cfde0 2024-04-26T20:51:02.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.026+0000 7fa405ffb640 1 -- 172.21.15.73:0/2288003045 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa408007bf0 con 0x7fa4100cfde0 2024-04-26T20:51:02.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.026+0000 7fa405ffb640 1 --2- 172.21.15.73:0/2288003045 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa3f406c920 0x7fa3f406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:02.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.026+0000 7fa405ffb640 1 -- 172.21.15.73:0/2288003045 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa4080acc80 con 0x7fa4100cfde0 2024-04-26T20:51:02.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.027+0000 7fa415780640 1 --2- 172.21.15.73:0/2288003045 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa3f406c920 0x7fa3f406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:02.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.027+0000 7fa415780640 1 --2- 172.21.15.73:0/2288003045 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa3f406c920 0x7fa3f406ede0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa40c009970 tx=0x7fa40c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:02.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.031+0000 7fa405ffb640 1 -- 172.21.15.73:0/2288003045 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa40807ad70 con 0x7fa4100cfde0 2024-04-26T20:51:02.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.204+0000 7fa4179e4640 1 -- 172.21.15.73:0/2288003045 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fa4100b4430 con 0x7fa4100cfde0 2024-04-26T20:51:02.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.204+0000 7fa405ffb640 1 -- 172.21.15.73:0/2288003045 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7fa40807a670 con 0x7fa4100cfde0 2024-04-26T20:51:02.205 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-26T20:51:02.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.206+0000 7fa4179e4640 1 -- 172.21.15.73:0/2288003045 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa3f406c920 msgr2=0x7fa3f406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:02.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.206+0000 7fa4179e4640 1 --2- 172.21.15.73:0/2288003045 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa3f406c920 0x7fa3f406ede0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa40c009970 tx=0x7fa40c008040 comp rx=0 tx=0).stop 2024-04-26T20:51:02.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.206+0000 7fa4179e4640 1 -- 172.21.15.73:0/2288003045 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4100cfde0 msgr2=0x7fa41014ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:02.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.206+0000 7fa4179e4640 1 --2- 172.21.15.73:0/2288003045 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4100cfde0 0x7fa41014ce20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa408009e50 tx=0x7fa408004a40 comp rx=0 tx=0).stop 2024-04-26T20:51:02.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.206+0000 7fa4179e4640 1 -- 172.21.15.73:0/2288003045 shutdown_connections 2024-04-26T20:51:02.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.206+0000 7fa4179e4640 1 --2- 172.21.15.73:0/2288003045 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4100cfde0 0x7fa41014ce20 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:02.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.206+0000 7fa4179e4640 1 --2- 172.21.15.73:0/2288003045 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa3f406c920 0x7fa3f406ede0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:02.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.206+0000 7fa4179e4640 1 --2- 172.21.15.73:0/2288003045 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa410071260 0x7fa4100b4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:02.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.206+0000 7fa4179e4640 1 --2- 172.21.15.73:0/2288003045 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa410070350 0x7fa41014dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:02.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.206+0000 7fa4179e4640 1 -- 172.21.15.73:0/2288003045 >> 172.21.15.73:0/2288003045 conn(0x7fa4100d2d80 msgr2=0x7fa4100c9650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:02.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.206+0000 7fa4179e4640 1 -- 172.21.15.73:0/2288003045 shutdown_connections 2024-04-26T20:51:02.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.206+0000 7fa4179e4640 1 -- 172.21.15.73:0/2288003045 wait complete. 2024-04-26T20:51:02.218 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:51:02.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.314+0000 7f10d2a62640 1 Processor -- start 2024-04-26T20:51:02.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.314+0000 7f10d2a62640 1 -- start start 2024-04-26T20:51:02.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.314+0000 7f10d2a62640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f10cc1604e0 0x7f10cc06f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:02.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.314+0000 7f10d2a62640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10cc160970 con 0x7f10cc06fcd0 2024-04-26T20:51:02.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.314+0000 7f10d2a62640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10cc072f70 con 0x7f10cc15f2d0 2024-04-26T20:51:02.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.314+0000 7f10d2a62640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f10cc074010 con 0x7f10cc1604e0 2024-04-26T20:51:02.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.315+0000 7f10cb7fe640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f10cc1604e0 0x7f10cc06f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:02.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.315+0000 7f10d0fff640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f10cc06fcd0 0x7f10cc0ce7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:42058/0 (socket says 172.21.15.73:42058) 2024-04-26T20:51:02.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.315+0000 7f10d0fff640 1 -- 172.21.15.73:0/2857967134 learned_addr learned my addr 172.21.15.73:0/2857967134 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:02.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.315+0000 7f10cb7fe640 1 -- 172.21.15.73:0/2857967134 >> v1:172.21.15.120:6789/0 conn(0x7f10cc15f2d0 legacy=0x7f10cc072700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:02.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.315+0000 7f10cb7fe640 1 -- 172.21.15.73:0/2857967134 >> v1:172.21.15.73:6789/0 conn(0x7f10cc06fcd0 legacy=0x7f10cc0ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:02.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.315+0000 7f10cb7fe640 1 -- 172.21.15.73:0/2857967134 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10cc073250 con 0x7f10cc1604e0 2024-04-26T20:51:02.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.316+0000 7f10cb7fe640 1 --2- 172.21.15.73:0/2857967134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10cc1604e0 0x7f10cc06f790 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f10c00097a0 tx=0x7f10c002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d528741b5134567 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:02.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.316+0000 7f10caffd640 1 -- 172.21.15.73:0/2857967134 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c0037070 con 0x7f10cc1604e0 2024-04-26T20:51:02.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.316+0000 7f10caffd640 1 -- 172.21.15.73:0/2857967134 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10c0032440 con 0x7f10cc1604e0 2024-04-26T20:51:02.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.316+0000 7f10caffd640 1 -- 172.21.15.73:0/2857967134 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c0032810 con 0x7f10cc1604e0 2024-04-26T20:51:02.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.316+0000 7f10d2a62640 1 -- 172.21.15.73:0/2857967134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10cc1604e0 msgr2=0x7f10cc06f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:02.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.316+0000 7f10d2a62640 1 --2- 172.21.15.73:0/2857967134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10cc1604e0 0x7f10cc06f790 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f10c00097a0 tx=0x7f10c002a230 comp rx=0 tx=0).stop 2024-04-26T20:51:02.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.316+0000 7f10d2a62640 1 -- 172.21.15.73:0/2857967134 shutdown_connections 2024-04-26T20:51:02.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.316+0000 7f10d2a62640 1 --2- 172.21.15.73:0/2857967134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10cc1604e0 0x7f10cc06f790 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:02.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.316+0000 7f10d2a62640 1 -- 172.21.15.73:0/2857967134 >> 172.21.15.73:0/2857967134 conn(0x7f10cc0d0be0 msgr2=0x7f10cc0d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:02.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.316+0000 7f10d2a62640 1 -- 172.21.15.73:0/2857967134 shutdown_connections 2024-04-26T20:51:02.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.316+0000 7f10d2a62640 1 -- 172.21.15.73:0/2857967134 wait complete. 2024-04-26T20:51:02.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.317+0000 7f10d2a62640 1 Processor -- start 2024-04-26T20:51:02.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.318+0000 7f10d2a62640 1 -- start start 2024-04-26T20:51:02.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.318+0000 7f10d2a62640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10cc06fcd0 0x7f10cc15b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:02.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.318+0000 7f10d2a62640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10cc15f2d0 0x7f10cc157f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:02.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.318+0000 7f10d2a62640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f10cc1604e0 0x7f10cc15e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:02.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.318+0000 7f10d2a62640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc073810 con 0x7f10cc15f2d0 2024-04-26T20:51:02.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.318+0000 7f10d2a62640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc070100 con 0x7f10cc06fcd0 2024-04-26T20:51:02.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.318+0000 7f10d2a62640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f10cc073af0 con 0x7f10cc1604e0 2024-04-26T20:51:02.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.318+0000 7f10cb7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10cc15f2d0 0x7f10cc157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:02.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.319+0000 7f10cb7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10cc15f2d0 0x7f10cc157f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42698/0 (socket says 172.21.15.73:42698) 2024-04-26T20:51:02.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.319+0000 7f10cb7fe640 1 -- 172.21.15.73:0/3859515937 learned_addr learned my addr 172.21.15.73:0/3859515937 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:02.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.319+0000 7f10cbfff640 1 --2- 172.21.15.73:0/3859515937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10cc06fcd0 0x7f10cc15b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:02.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.319+0000 7f10d0fff640 1 --2- 172.21.15.73:0/3859515937 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f10cc1604e0 0x7f10cc15e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:02.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.319+0000 7f10cb7fe640 1 -- 172.21.15.73:0/3859515937 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f10cc1604e0 msgr2=0x7f10cc15e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:02.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.319+0000 7f10cb7fe640 1 --2- 172.21.15.73:0/3859515937 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f10cc1604e0 0x7f10cc15e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:02.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.319+0000 7f10cb7fe640 1 -- 172.21.15.73:0/3859515937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10cc06fcd0 msgr2=0x7f10cc15b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:02.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.319+0000 7f10cb7fe640 1 --2- 172.21.15.73:0/3859515937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10cc06fcd0 0x7f10cc15b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:02.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.319+0000 7f10cb7fe640 1 -- 172.21.15.73:0/3859515937 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10cc15bc60 con 0x7f10cc15f2d0 2024-04-26T20:51:02.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.319+0000 7f10cbfff640 1 --2- 172.21.15.73:0/3859515937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10cc06fcd0 0x7f10cc15b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:02.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.319+0000 7f10cb7fe640 1 --2- 172.21.15.73:0/3859515937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10cc15f2d0 0x7f10cc157f70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f10c00098c0 tx=0x7f10c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:02.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.319+0000 7f10c8ff9640 1 -- 172.21.15.73:0/3859515937 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c0037040 con 0x7f10cc15f2d0 2024-04-26T20:51:02.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.319+0000 7f10c8ff9640 1 -- 172.21.15.73:0/3859515937 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10c0030770 con 0x7f10cc15f2d0 2024-04-26T20:51:02.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.319+0000 7f10d2a62640 1 -- 172.21.15.73:0/3859515937 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10cc072b90 con 0x7f10cc15f2d0 2024-04-26T20:51:02.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.319+0000 7f10c8ff9640 1 -- 172.21.15.73:0/3859515937 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c0030b40 con 0x7f10cc15f2d0 2024-04-26T20:51:02.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.319+0000 7f10d2a62640 1 -- 172.21.15.73:0/3859515937 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10cc159870 con 0x7f10cc15f2d0 2024-04-26T20:51:02.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.320+0000 7f10d2a62640 1 -- 172.21.15.73:0/3859515937 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10cc0694f0 con 0x7f10cc15f2d0 2024-04-26T20:51:02.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.320+0000 7f10c8ff9640 1 -- 172.21.15.73:0/3859515937 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f10c0044020 con 0x7f10cc15f2d0 2024-04-26T20:51:02.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.321+0000 7f10c8ff9640 1 --2- 172.21.15.73:0/3859515937 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f10b806c8f0 0x7f10b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:02.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.321+0000 7f10c8ff9640 1 -- 172.21.15.73:0/3859515937 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f10c00ac470 con 0x7f10cc15f2d0 2024-04-26T20:51:02.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.321+0000 7f10cbfff640 1 --2- 172.21.15.73:0/3859515937 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f10b806c8f0 0x7f10b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:02.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.321+0000 7f10cbfff640 1 --2- 172.21.15.73:0/3859515937 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f10b806c8f0 0x7f10b806edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f10bc001700 tx=0x7f10bc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:02.326 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.325+0000 7f10c8ff9640 1 -- 172.21.15.73:0/3859515937 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f10c0032e20 con 0x7f10cc15f2d0 2024-04-26T20:51:02.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.534+0000 7f10d2a62640 1 -- 172.21.15.73:0/3859515937 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f10cc15bf10 con 0x7f10cc15f2d0 2024-04-26T20:51:02.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.535+0000 7f10c8ff9640 1 -- 172.21.15.73:0/3859515937 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5180 (secure 0 0 0) 0x7f10c00313a0 con 0x7f10cc15f2d0 2024-04-26T20:51:02.536 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:51:02.536 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/4263167680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6833","nonce":4263167680},{"type":"v1","addr":"172.21.15.120:6835","nonce":4263167680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6834/1614342510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1614342510},{"type":"v1","addr":"172.21.15.120:6834","nonce":1614342510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2146325725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2146325725},{"type":"v1","addr":"172.21.15.73:6835","nonce":2146325725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:50:31.964317+0000","modified":"2024-04-26T20:50:54.049029+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/181202544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":181202544},{"type":"v1","addr":"172.21.15.73:6837","nonce":181202544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4281,"qdb_cluster":[4281]},"id":1}]} 2024-04-26T20:51:02.536 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 8 2024-04-26T20:51:02.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.537+0000 7f10d2a62640 1 -- 172.21.15.73:0/3859515937 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f10b806c8f0 msgr2=0x7f10b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:02.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.537+0000 7f10d2a62640 1 --2- 172.21.15.73:0/3859515937 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f10b806c8f0 0x7f10b806edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f10bc001700 tx=0x7f10bc009040 comp rx=0 tx=0).stop 2024-04-26T20:51:02.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.537+0000 7f10d2a62640 1 -- 172.21.15.73:0/3859515937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10cc15f2d0 msgr2=0x7f10cc157f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:02.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.537+0000 7f10d2a62640 1 --2- 172.21.15.73:0/3859515937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10cc15f2d0 0x7f10cc157f70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f10c00098c0 tx=0x7f10c00046c0 comp rx=0 tx=0).stop 2024-04-26T20:51:02.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.537+0000 7f10d2a62640 1 -- 172.21.15.73:0/3859515937 shutdown_connections 2024-04-26T20:51:02.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.537+0000 7f10d2a62640 1 --2- 172.21.15.73:0/3859515937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10cc15f2d0 0x7f10cc157f70 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:02.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.537+0000 7f10d2a62640 1 --2- 172.21.15.73:0/3859515937 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f10b806c8f0 0x7f10b806edb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:02.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.537+0000 7f10d2a62640 1 --2- 172.21.15.73:0/3859515937 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f10cc1604e0 0x7f10cc15e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:02.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.537+0000 7f10d2a62640 1 --2- 172.21.15.73:0/3859515937 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10cc06fcd0 0x7f10cc15b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:02.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.537+0000 7f10d2a62640 1 -- 172.21.15.73:0/3859515937 >> 172.21.15.73:0/3859515937 conn(0x7f10cc0d0be0 msgr2=0x7f10cc150ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:02.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.537+0000 7f10d2a62640 1 -- 172.21.15.73:0/3859515937 shutdown_connections 2024-04-26T20:51:02.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.537+0000 7f10d2a62640 1 -- 172.21.15.73:0/3859515937 wait complete. 2024-04-26T20:51:02.549 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:51:02.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.645+0000 7ff3b7ab0640 1 Processor -- start 2024-04-26T20:51:02.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.645+0000 7ff3b7ab0640 1 -- start start 2024-04-26T20:51:02.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.646+0000 7ff3b7ab0640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff3b015f3a0 0x7ff3b00d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:02.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.646+0000 7ff3b7ab0640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7ff3b00d2330 0x7ff3b00b2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:02.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.646+0000 7ff3b7ab0640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3b00d0e20 con 0x7ff3b00d1770 2024-04-26T20:51:02.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.646+0000 7ff3b7ab0640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7ff3b00b1d70 con 0x7ff3b015f3a0 2024-04-26T20:51:02.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.646+0000 7ff3b7ab0640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7ff3b00b2050 con 0x7ff3b00d2330 2024-04-26T20:51:02.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.646+0000 7ff3b584c640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff3b015f3a0 0x7ff3b00d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:02.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.646+0000 7ff3b584c640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff3b015f3a0 0x7ff3b00d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42714/0 (socket says 172.21.15.73:42714) 2024-04-26T20:51:02.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.646+0000 7ff3b584c640 1 -- 172.21.15.73:0/2920834158 learned_addr learned my addr 172.21.15.73:0/2920834158 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:02.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.646+0000 7ff3b604d640 1 --2- 172.21.15.73:0/2920834158 >> v2:172.21.15.120:3300/0 conn(0x7ff3b00d2330 0x7ff3b00b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:02.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.647+0000 7ff3b584c640 1 -- 172.21.15.73:0/2920834158 >> v2:172.21.15.120:3300/0 conn(0x7ff3b00d2330 msgr2=0x7ff3b00b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:02.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.647+0000 7ff3b584c640 1 --2- 172.21.15.73:0/2920834158 >> v2:172.21.15.120:3300/0 conn(0x7ff3b00d2330 0x7ff3b00b2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:02.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.647+0000 7ff3b584c640 1 -- 172.21.15.73:0/2920834158 >> v1:172.21.15.120:6789/0 conn(0x7ff3b00d1770 legacy=0x7ff3b00d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:02.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.647+0000 7ff3b584c640 1 -- 172.21.15.73:0/2920834158 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3b00b2330 con 0x7ff3b015f3a0 2024-04-26T20:51:02.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.647+0000 7ff3b584c640 1 --2- 172.21.15.73:0/2920834158 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3b015f3a0 0x7ff3b00d06f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0073b50 tx=0x7ff3ac02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dc2ac92b8968244 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:02.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.647+0000 7ff3b484a640 1 -- 172.21.15.73:0/2920834158 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac036070 con 0x7ff3b015f3a0 2024-04-26T20:51:02.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.647+0000 7ff3b484a640 1 -- 172.21.15.73:0/2920834158 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3ac031600 con 0x7ff3b015f3a0 2024-04-26T20:51:02.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.647+0000 7ff3b484a640 1 -- 172.21.15.73:0/2920834158 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac0319d0 con 0x7ff3b015f3a0 2024-04-26T20:51:02.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.648+0000 7ff3b7ab0640 1 -- 172.21.15.73:0/2920834158 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3b015f3a0 msgr2=0x7ff3b00d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:02.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.648+0000 7ff3b7ab0640 1 --2- 172.21.15.73:0/2920834158 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3b015f3a0 0x7ff3b00d06f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0073b50 tx=0x7ff3ac02a6c0 comp rx=0 tx=0).stop 2024-04-26T20:51:02.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.648+0000 7ff3b7ab0640 1 -- 172.21.15.73:0/2920834158 shutdown_connections 2024-04-26T20:51:02.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.648+0000 7ff3b7ab0640 1 --2- 172.21.15.73:0/2920834158 >> v2:172.21.15.120:3300/0 conn(0x7ff3b00d2330 0x7ff3b00b2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:02.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.648+0000 7ff3b7ab0640 1 --2- 172.21.15.73:0/2920834158 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3b015f3a0 0x7ff3b00d06f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:02.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.648+0000 7ff3b7ab0640 1 -- 172.21.15.73:0/2920834158 >> 172.21.15.73:0/2920834158 conn(0x7ff3b00d4f90 msgr2=0x7ff3b00d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:02.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.648+0000 7ff3b7ab0640 1 -- 172.21.15.73:0/2920834158 shutdown_connections 2024-04-26T20:51:02.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.648+0000 7ff3b7ab0640 1 -- 172.21.15.73:0/2920834158 wait complete. 2024-04-26T20:51:02.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.649+0000 7ff3b7ab0640 1 Processor -- start 2024-04-26T20:51:02.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.649+0000 7ff3b7ab0640 1 -- start start 2024-04-26T20:51:02.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.650+0000 7ff3b7ab0640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3b00d1770 0x7ff3b0159720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:02.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.650+0000 7ff3b7ab0640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff3b00d2330 0x7ff3b015f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:02.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.650+0000 7ff3b7ab0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3b015f3a0 0x7ff3b00b9f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:02.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.650+0000 7ff3b7ab0640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b00b2f10 con 0x7ff3b00d1770 2024-04-26T20:51:02.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.650+0000 7ff3b7ab0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff3b00b2d90 con 0x7ff3b015f3a0 2024-04-26T20:51:02.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.650+0000 7ff3b7ab0640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7ff3b00b3090 con 0x7ff3b00d2330 2024-04-26T20:51:02.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.650+0000 7ff3b584c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3b00d1770 0x7ff3b0159720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:02.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.650+0000 7ff3b584c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3b00d1770 0x7ff3b0159720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42720/0 (socket says 172.21.15.73:42720) 2024-04-26T20:51:02.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.650+0000 7ff3b584c640 1 -- 172.21.15.73:0/3269840092 learned_addr learned my addr 172.21.15.73:0/3269840092 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:02.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.650+0000 7ff3b504b640 1 --2- 172.21.15.73:0/3269840092 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff3b00d2330 0x7ff3b015f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:02.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.650+0000 7ff3b604d640 1 --2- 172.21.15.73:0/3269840092 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3b015f3a0 0x7ff3b00b9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:02.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.650+0000 7ff3b584c640 1 -- 172.21.15.73:0/3269840092 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff3b00d2330 msgr2=0x7ff3b015f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:02.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.650+0000 7ff3b584c640 1 --2- 172.21.15.73:0/3269840092 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff3b00d2330 0x7ff3b015f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:02.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.650+0000 7ff3b584c640 1 -- 172.21.15.73:0/3269840092 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3b015f3a0 msgr2=0x7ff3b00b9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:02.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.650+0000 7ff3b584c640 1 --2- 172.21.15.73:0/3269840092 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3b015f3a0 0x7ff3b00b9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:02.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.650+0000 7ff3b584c640 1 -- 172.21.15.73:0/3269840092 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3b0159c60 con 0x7ff3b00d1770 2024-04-26T20:51:02.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.650+0000 7ff3b504b640 1 --2- 172.21.15.73:0/3269840092 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff3b00d2330 0x7ff3b015f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:02.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.650+0000 7ff3b604d640 1 --2- 172.21.15.73:0/3269840092 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3b015f3a0 0x7ff3b00b9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:02.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.650+0000 7ff3b584c640 1 --2- 172.21.15.73:0/3269840092 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3b00d1770 0x7ff3b0159720 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac034040 tx=0x7ff3ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:02.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.651+0000 7ff3a67fc640 1 -- 172.21.15.73:0/3269840092 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac036050 con 0x7ff3b00d1770 2024-04-26T20:51:02.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.651+0000 7ff3a67fc640 1 -- 172.21.15.73:0/3269840092 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3ac042070 con 0x7ff3b00d1770 2024-04-26T20:51:02.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.651+0000 7ff3b7ab0640 1 -- 172.21.15.73:0/3269840092 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3b0160300 con 0x7ff3b00d1770 2024-04-26T20:51:02.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.651+0000 7ff3a67fc640 1 -- 172.21.15.73:0/3269840092 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac009510 con 0x7ff3b00d1770 2024-04-26T20:51:02.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.651+0000 7ff3b7ab0640 1 -- 172.21.15.73:0/3269840092 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3b01608c0 con 0x7ff3b00d1770 2024-04-26T20:51:02.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.652+0000 7ff3b7ab0640 1 -- 172.21.15.73:0/3269840092 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3b00694f0 con 0x7ff3b00d1770 2024-04-26T20:51:02.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.652+0000 7ff3a67fc640 1 -- 172.21.15.73:0/3269840092 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff3ac007c70 con 0x7ff3b00d1770 2024-04-26T20:51:02.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.652+0000 7ff3a67fc640 1 --2- 172.21.15.73:0/3269840092 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff39406c8f0 0x7ff39406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:02.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.653+0000 7ff3b504b640 1 --2- 172.21.15.73:0/3269840092 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff39406c8f0 0x7ff39406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:02.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.653+0000 7ff3a67fc640 1 -- 172.21.15.73:0/3269840092 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff3ac0acca0 con 0x7ff3b00d1770 2024-04-26T20:51:02.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.653+0000 7ff3b504b640 1 --2- 172.21.15.73:0/3269840092 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff39406c8f0 0x7ff39406edb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff3b00b6030 tx=0x7ff3a0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:02.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.657+0000 7ff3a67fc640 1 -- 172.21.15.73:0/3269840092 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff3ac045360 con 0x7ff3b00d1770 2024-04-26T20:51:02.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.835+0000 7ff3b7ab0640 1 -- 172.21.15.73:0/3269840092 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff3b00d1e70 con 0x7ff3b00d1770 2024-04-26T20:51:02.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.836+0000 7ff3a67fc640 1 -- 172.21.15.73:0/3269840092 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18514 (secure 0 0 0) 0x7ff3ac07ae20 con 0x7ff3b00d1770 2024-04-26T20:51:02.837 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:51:02.837 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":25,"fsid":"fbd3f548-b547-4713-856b-09f98f5d1764","created":"2024-04-26T20:50:07.876786+0000","modified":"2024-04-26T20:50:38.657510+0000","last_up_change":"2024-04-26T20:50:19.082741+0000","last_in_change":"2024-04-26T20:50:10.018400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:50:19.993731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:50:29.964420+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:50:30.920082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:50:34.650207+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"72d6f4f3-214a-4392-a02f-c33cbcab3c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6809","nonce":3461705124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6811","nonce":3461705124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6815","nonce":3461705124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6813","nonce":3461705124}]},"public_addr":"172.21.15.73:6809/3461705124","cluster_addr":"172.21.15.73:6811/3461705124","heartbeat_back_addr":"172.21.15.73:6815/3461705124","heartbeat_front_addr":"172.21.15.73:6813/3461705124","state":["exists","up"]},{"osd":1,"uuid":"786bf8ca-6ddc-4b3c-be6c-9152398bd5fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6817","nonce":2827619403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6819","nonce":2827619403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6825","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6827","nonce":2827619403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6821","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6823","nonce":2827619403}]},"public_addr":"172.21.15.73:6817/2827619403","cluster_addr":"172.21.15.73:6819/2827619403","heartbeat_back_addr":"172.21.15.73:6827/2827619403","heartbeat_front_addr":"172.21.15.73:6823/2827619403","state":["exists","up"]},{"osd":2,"uuid":"90963b1a-5437-4ef8-ad8b-6b74c534eeb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6801","nonce":2542681064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6803","nonce":2542681064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6808","nonce":2542681064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6805","nonce":2542681064}]},"public_addr":"172.21.15.73:6801/2542681064","cluster_addr":"172.21.15.73:6803/2542681064","heartbeat_back_addr":"172.21.15.73:6808/2542681064","heartbeat_front_addr":"172.21.15.73:6805/2542681064","state":["exists","up"]},{"osd":3,"uuid":"bd3d2148-6e38-41b4-a453-fe6e3b8b4ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6822","nonce":1604042718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6826","nonce":1604042718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6831","nonce":1604042718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6829","nonce":1604042718}]},"public_addr":"172.21.15.73:6822/1604042718","cluster_addr":"172.21.15.73:6826/1604042718","heartbeat_back_addr":"172.21.15.73:6831/1604042718","heartbeat_front_addr":"172.21.15.73:6829/1604042718","state":["exists","up"]},{"osd":4,"uuid":"e8668dc8-a23f-46d2-8311-cb882da08e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6817","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6819","nonce":3591055561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6824","nonce":3591055561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6831","nonce":3591055561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6828","nonce":3591055561}]},"public_addr":"172.21.15.120:6819/3591055561","cluster_addr":"172.21.15.120:6824/3591055561","heartbeat_back_addr":"172.21.15.120:6831/3591055561","heartbeat_front_addr":"172.21.15.120:6828/3591055561","state":["exists","up"]},{"osd":5,"uuid":"cbac0bdb-96bd-4852-bcce-e564495c3fcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6809","nonce":819450433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6811","nonce":819450433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6815","nonce":819450433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6813","nonce":819450433}]},"public_addr":"172.21.15.120:6809/819450433","cluster_addr":"172.21.15.120:6811/819450433","heartbeat_back_addr":"172.21.15.120:6815/819450433","heartbeat_front_addr":"172.21.15.120:6813/819450433","state":["exists","up"]},{"osd":6,"uuid":"1884b2a8-569f-4272-96a9-f25a09364dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6801","nonce":3946210727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6803","nonce":3946210727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6807","nonce":3946210727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6805","nonce":3946210727}]},"public_addr":"172.21.15.120:6801/3946210727","cluster_addr":"172.21.15.120:6803/3946210727","heartbeat_back_addr":"172.21.15.120:6807/3946210727","heartbeat_front_addr":"172.21.15.120:6805/3946210727","state":["exists","up"]},{"osd":7,"uuid":"c0c3bf6d-a4c5-4f1e-aa2d-330951830666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6818","nonce":2915574674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6821","nonce":2915574674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6827","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6829","nonce":2915574674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6823","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6825","nonce":2915574674}]},"public_addr":"172.21.15.120:6818/2915574674","cluster_addr":"172.21.15.120:6821/2915574674","heartbeat_back_addr":"172.21.15.120:6829/2915574674","heartbeat_front_addr":"172.21.15.120:6825/2915574674","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.218216+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.208710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.112705+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.182789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.164324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.174212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.225396+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.261559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:51:02.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.838+0000 7ff3b7ab0640 1 -- 172.21.15.73:0/3269840092 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff39406c8f0 msgr2=0x7ff39406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:02.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.838+0000 7ff3b7ab0640 1 --2- 172.21.15.73:0/3269840092 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff39406c8f0 0x7ff39406edb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff3b00b6030 tx=0x7ff3a0009040 comp rx=0 tx=0).stop 2024-04-26T20:51:02.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.838+0000 7ff3b7ab0640 1 -- 172.21.15.73:0/3269840092 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3b00d1770 msgr2=0x7ff3b0159720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:02.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.838+0000 7ff3b7ab0640 1 --2- 172.21.15.73:0/3269840092 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3b00d1770 0x7ff3b0159720 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac034040 tx=0x7ff3ac004a40 comp rx=0 tx=0).stop 2024-04-26T20:51:02.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.838+0000 7ff3b7ab0640 1 -- 172.21.15.73:0/3269840092 shutdown_connections 2024-04-26T20:51:02.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.838+0000 7ff3b7ab0640 1 --2- 172.21.15.73:0/3269840092 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff39406c8f0 0x7ff39406edb0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:02.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.838+0000 7ff3b7ab0640 1 --2- 172.21.15.73:0/3269840092 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff3b015f3a0 0x7ff3b00b9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:02.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.838+0000 7ff3b7ab0640 1 --2- 172.21.15.73:0/3269840092 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff3b00d2330 0x7ff3b015f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:02.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.838+0000 7ff3b7ab0640 1 --2- 172.21.15.73:0/3269840092 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3b00d1770 0x7ff3b0159720 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:02.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.838+0000 7ff3b7ab0640 1 -- 172.21.15.73:0/3269840092 >> 172.21.15.73:0/3269840092 conn(0x7ff3b00d4f90 msgr2=0x7ff3b006d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:02.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.838+0000 7ff3b7ab0640 1 -- 172.21.15.73:0/3269840092 shutdown_connections 2024-04-26T20:51:02.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.838+0000 7ff3b7ab0640 1 -- 172.21.15.73:0/3269840092 wait complete. 2024-04-26T20:51:02.850 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:50:19.993731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:50:29.964420+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:50:30.920082+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T20:50:34.650207+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T20:51:02.850 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:51:02.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.950+0000 7f4e3e378640 1 Processor -- start 2024-04-26T20:51:02.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.950+0000 7f4e3e378640 1 -- start start 2024-04-26T20:51:02.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.950+0000 7f4e3e378640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f4e380b0b70 0x7f4e380b0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:02.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.950+0000 7f4e3e378640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e380b1490 con 0x7f4e380aed90 2024-04-26T20:51:02.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.950+0000 7f4e3e378640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e38072fb0 con 0x7f4e380b4d40 2024-04-26T20:51:02.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.950+0000 7f4e3e378640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f4e38073bc0 con 0x7f4e380b0b70 2024-04-26T20:51:02.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.951+0000 7f4e3c915640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f4e380aed90 0x7f4e380af240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:42060/0 (socket says 172.21.15.73:42060) 2024-04-26T20:51:02.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.951+0000 7f4e3c915640 1 -- 172.21.15.73:0/3714761235 learned_addr learned my addr 172.21.15.73:0/3714761235 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:02.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.951+0000 7f4e36ffd640 1 -- 172.21.15.73:0/3714761235 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4120853463 0 0) 0x7f4e380b1490 con 0x7f4e380aed90 2024-04-26T20:51:02.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.951+0000 7f4e36ffd640 1 -- 172.21.15.73:0/3714761235 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e20003520 con 0x7f4e380aed90 2024-04-26T20:51:02.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.951+0000 7f4e37fff640 1 --2- 172.21.15.73:0/3714761235 >> v2:172.21.15.120:3300/0 conn(0x7f4e380b0b70 0x7f4e380b0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:02.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.951+0000 7f4e36ffd640 1 -- 172.21.15.73:0/3714761235 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3541136556 0 0) 0x7f4e20003520 con 0x7f4e380aed90 2024-04-26T20:51:02.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.951+0000 7f4e36ffd640 1 -- 172.21.15.73:0/3714761235 >> v2:172.21.15.120:3300/0 conn(0x7f4e380b0b70 msgr2=0x7f4e380b0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:02.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.951+0000 7f4e36ffd640 1 --2- 172.21.15.73:0/3714761235 >> v2:172.21.15.120:3300/0 conn(0x7f4e380b0b70 0x7f4e380b0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:02.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.951+0000 7f4e36ffd640 1 -- 172.21.15.73:0/3714761235 >> v1:172.21.15.120:6789/0 conn(0x7f4e380b4d40 legacy=0x7f4e380b6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:02.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.951+0000 7f4e36ffd640 1 -- 172.21.15.73:0/3714761235 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e38073290 con 0x7f4e380aed90 2024-04-26T20:51:02.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.951+0000 7f4e36ffd640 1 -- 172.21.15.73:0/3714761235 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7f4e300019f0 con 0x7f4e380aed90 2024-04-26T20:51:02.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.951+0000 7f4e36ffd640 1 -- 172.21.15.73:0/3714761235 >> v1:172.21.15.73:6789/0 conn(0x7f4e380aed90 legacy=0x7f4e380af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:02.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.951+0000 7f4e36ffd640 1 --2- 172.21.15.73:0/3714761235 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e200039b0 0x7f4e20003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:02.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.951+0000 7f4e36ffd640 1 --2- 172.21.15.73:0/3714761235 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e20004800 0x7f4e20004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:02.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.951+0000 7f4e36ffd640 1 --2- 172.21.15.73:0/3714761235 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4e200051c0 0x7f4e2000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:02.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.951+0000 7f4e36ffd640 1 -- 172.21.15.73:0/3714761235 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4e38073bc0 con 0x7f4e200039b0 2024-04-26T20:51:02.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.951+0000 7f4e36ffd640 1 -- 172.21.15.73:0/3714761235 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2000c060 con 0x7f4e20004800 2024-04-26T20:51:02.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.951+0000 7f4e36ffd640 1 -- 172.21.15.73:0/3714761235 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f4e2000c260 con 0x7f4e200051c0 2024-04-26T20:51:02.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.951+0000 7f4e37fff640 1 --2- 172.21.15.73:0/3714761235 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e200039b0 0x7f4e20003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:02.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.951+0000 7f4e377fe640 1 --2- 172.21.15.73:0/3714761235 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e20004800 0x7f4e20004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:02.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.952+0000 7f4e3c915640 1 --2- 172.21.15.73:0/3714761235 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4e200051c0 0x7f4e2000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:02.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.952+0000 7f4e37fff640 1 -- 172.21.15.73:0/3714761235 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4e200051c0 msgr2=0x7f4e2000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:02.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.952+0000 7f4e37fff640 1 --2- 172.21.15.73:0/3714761235 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4e200051c0 0x7f4e2000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:02.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.952+0000 7f4e37fff640 1 -- 172.21.15.73:0/3714761235 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e20004800 msgr2=0x7f4e20004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:02.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.952+0000 7f4e37fff640 1 --2- 172.21.15.73:0/3714761235 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e20004800 0x7f4e20004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:02.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.952+0000 7f4e37fff640 1 -- 172.21.15.73:0/3714761235 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e20003520 con 0x7f4e200039b0 2024-04-26T20:51:02.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.952+0000 7f4e3c915640 1 -- 172.21.15.73:0/3714761235 reap_dead start 2024-04-26T20:51:02.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.952+0000 7f4e37fff640 1 --2- 172.21.15.73:0/3714761235 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e200039b0 0x7f4e20003dd0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4e28009ac0 tx=0x7f4e20005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edc388eec99ba81c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:02.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.952+0000 7f4e36ffd640 1 -- 172.21.15.73:0/3714761235 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e28036070 con 0x7f4e200039b0 2024-04-26T20:51:02.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.953+0000 7f4e36ffd640 1 -- 172.21.15.73:0/3714761235 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e28031400 con 0x7f4e200039b0 2024-04-26T20:51:02.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.953+0000 7f4e36ffd640 1 -- 172.21.15.73:0/3714761235 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e28009690 con 0x7f4e200039b0 2024-04-26T20:51:02.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.953+0000 7f4e3e378640 1 -- 172.21.15.73:0/3714761235 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e200039b0 msgr2=0x7f4e20003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:02.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.953+0000 7f4e3e378640 1 --2- 172.21.15.73:0/3714761235 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e200039b0 0x7f4e20003dd0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4e28009ac0 tx=0x7f4e20005830 comp rx=0 tx=0).stop 2024-04-26T20:51:02.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.953+0000 7f4e3e378640 1 -- 172.21.15.73:0/3714761235 shutdown_connections 2024-04-26T20:51:02.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.953+0000 7f4e3e378640 1 --2- 172.21.15.73:0/3714761235 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e200039b0 0x7f4e20003dd0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:02.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.953+0000 7f4e3e378640 1 -- 172.21.15.73:0/3714761235 >> 172.21.15.73:0/3714761235 conn(0x7f4e380b39b0 msgr2=0x7f4e380b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:02.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.953+0000 7f4e3e378640 1 -- 172.21.15.73:0/3714761235 shutdown_connections 2024-04-26T20:51:02.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.953+0000 7f4e3e378640 1 -- 172.21.15.73:0/3714761235 wait complete. 2024-04-26T20:51:02.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.955+0000 7f4e3e378640 1 Processor -- start 2024-04-26T20:51:02.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.955+0000 7f4e3e378640 1 -- start start 2024-04-26T20:51:02.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.955+0000 7f4e3e378640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4e200039b0 0x7f4e38154120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:02.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.955+0000 7f4e3e378640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e38156f50 0x7f4e381497d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:02.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.955+0000 7f4e3e378640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e381f4b50 0x7f4e381f4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:02.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.955+0000 7f4e3e378640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4e380b0f50 con 0x7f4e381f4b50 2024-04-26T20:51:02.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.955+0000 7f4e3e378640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4e38073850 con 0x7f4e38156f50 2024-04-26T20:51:02.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.955+0000 7f4e3e378640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f4e380b10d0 con 0x7f4e200039b0 2024-04-26T20:51:02.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.956+0000 7f4e3c915640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e381f4b50 0x7f4e381f4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:02.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.956+0000 7f4e3c915640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e381f4b50 0x7f4e381f4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42748/0 (socket says 172.21.15.73:42748) 2024-04-26T20:51:02.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.956+0000 7f4e3c915640 1 -- 172.21.15.73:0/3924254192 learned_addr learned my addr 172.21.15.73:0/3924254192 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:02.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.956+0000 7f4e37fff640 1 --2- 172.21.15.73:0/3924254192 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4e200039b0 0x7f4e38154120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:02.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.956+0000 7f4e377fe640 1 --2- 172.21.15.73:0/3924254192 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e38156f50 0x7f4e381497d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:02.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.956+0000 7f4e3c915640 1 -- 172.21.15.73:0/3924254192 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4e200039b0 msgr2=0x7f4e38154120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:02.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.956+0000 7f4e3c915640 1 --2- 172.21.15.73:0/3924254192 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4e200039b0 0x7f4e38154120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:02.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.956+0000 7f4e3c915640 1 -- 172.21.15.73:0/3924254192 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e38156f50 msgr2=0x7f4e381497d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:02.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.956+0000 7f4e3c915640 1 --2- 172.21.15.73:0/3924254192 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e38156f50 0x7f4e381497d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:02.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.956+0000 7f4e3c915640 1 -- 172.21.15.73:0/3924254192 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e38149d10 con 0x7f4e381f4b50 2024-04-26T20:51:02.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.956+0000 7f4e377fe640 1 --2- 172.21.15.73:0/3924254192 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e38156f50 0x7f4e381497d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:02.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.956+0000 7f4e3c915640 1 --2- 172.21.15.73:0/3924254192 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e381f4b50 0x7f4e381f4fd0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4e30001f00 tx=0x7f4e30007b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:02.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.956+0000 7f4e34ff9640 1 -- 172.21.15.73:0/3924254192 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e3000ab60 con 0x7f4e381f4b50 2024-04-26T20:51:02.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.956+0000 7f4e34ff9640 1 -- 172.21.15.73:0/3924254192 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e30019070 con 0x7f4e381f4b50 2024-04-26T20:51:02.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.956+0000 7f4e34ff9640 1 -- 172.21.15.73:0/3924254192 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e300133f0 con 0x7f4e381f4b50 2024-04-26T20:51:02.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.956+0000 7f4e3e378640 1 -- 172.21.15.73:0/3924254192 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e381592d0 con 0x7f4e381f4b50 2024-04-26T20:51:02.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.956+0000 7f4e3e378640 1 -- 172.21.15.73:0/3924254192 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e38159890 con 0x7f4e381f4b50 2024-04-26T20:51:02.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.958+0000 7f4e3e378640 1 -- 172.21.15.73:0/3924254192 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e380694f0 con 0x7f4e381f4b50 2024-04-26T20:51:02.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.958+0000 7f4e34ff9640 1 -- 172.21.15.73:0/3924254192 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4e30013590 con 0x7f4e381f4b50 2024-04-26T20:51:02.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.958+0000 7f4e34ff9640 1 --2- 172.21.15.73:0/3924254192 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f4e2406c8f0 0x7f4e2406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:02.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.958+0000 7f4e34ff9640 1 -- 172.21.15.73:0/3924254192 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f4e3008c440 con 0x7f4e381f4b50 2024-04-26T20:51:02.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.958+0000 7f4e37fff640 1 --2- 172.21.15.73:0/3924254192 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f4e2406c8f0 0x7f4e2406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:02.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.959+0000 7f4e37fff640 1 --2- 172.21.15.73:0/3924254192 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f4e2406c8f0 0x7f4e2406edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4e28033000 tx=0x7f4e28008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:02.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:02.963+0000 7f4e34ff9640 1 -- 172.21.15.73:0/3924254192 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4e38149d10 con 0x7f4e381f4b50 2024-04-26T20:51:03.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.140+0000 7f4e3e378640 1 -- 172.21.15.73:0/3924254192 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4e380b3ee0 con 0x7f4e381f4b50 2024-04-26T20:51:03.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.140+0000 7f4e34ff9640 1 -- 172.21.15.73:0/3924254192 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f4e3005a530 con 0x7f4e381f4b50 2024-04-26T20:51:03.142 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-26T20:51:03.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.142+0000 7f4e3e378640 1 -- 172.21.15.73:0/3924254192 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f4e2406c8f0 msgr2=0x7f4e2406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:03.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.142+0000 7f4e3e378640 1 --2- 172.21.15.73:0/3924254192 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f4e2406c8f0 0x7f4e2406edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4e28033000 tx=0x7f4e28008030 comp rx=0 tx=0).stop 2024-04-26T20:51:03.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.142+0000 7f4e3e378640 1 -- 172.21.15.73:0/3924254192 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e381f4b50 msgr2=0x7f4e381f4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:03.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.142+0000 7f4e3e378640 1 --2- 172.21.15.73:0/3924254192 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e381f4b50 0x7f4e381f4fd0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4e30001f00 tx=0x7f4e30007b20 comp rx=0 tx=0).stop 2024-04-26T20:51:03.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.142+0000 7f4e3e378640 1 -- 172.21.15.73:0/3924254192 shutdown_connections 2024-04-26T20:51:03.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.142+0000 7f4e3e378640 1 --2- 172.21.15.73:0/3924254192 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e381f4b50 0x7f4e381f4fd0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:03.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.142+0000 7f4e3e378640 1 --2- 172.21.15.73:0/3924254192 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f4e2406c8f0 0x7f4e2406edb0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:03.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.142+0000 7f4e3e378640 1 --2- 172.21.15.73:0/3924254192 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e38156f50 0x7f4e381497d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:03.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.142+0000 7f4e3e378640 1 --2- 172.21.15.73:0/3924254192 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4e200039b0 0x7f4e38154120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:03.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.142+0000 7f4e3e378640 1 -- 172.21.15.73:0/3924254192 >> 172.21.15.73:0/3924254192 conn(0x7f4e380b39b0 msgr2=0x7f4e380afa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:03.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.142+0000 7f4e3e378640 1 -- 172.21.15.73:0/3924254192 shutdown_connections 2024-04-26T20:51:03.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.143+0000 7f4e3e378640 1 -- 172.21.15.73:0/3924254192 wait complete. 2024-04-26T20:51:03.154 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:51:03.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.253+0000 7f48ca978640 1 Processor -- start 2024-04-26T20:51:03.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.253+0000 7f48ca978640 1 -- start start 2024-04-26T20:51:03.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.254+0000 7f48ca978640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f48c4071e60 0x7f48c406f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:03.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.254+0000 7f48ca978640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f48c406fe20 0x7f48c4072240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:03.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.254+0000 7f48ca978640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f48c4072780 0x7f48c40cb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:03.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.254+0000 7f48ca978640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f48c415d7c0 con 0x7f48c406fe20 2024-04-26T20:51:03.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.254+0000 7f48ca978640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f48c415daa0 con 0x7f48c4071e60 2024-04-26T20:51:03.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.254+0000 7f48ca978640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f48c415dd80 con 0x7f48c4072780 2024-04-26T20:51:03.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.254+0000 7f48c37fe640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f48c406fe20 0x7f48c4072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:03.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.254+0000 7f48c37fe640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f48c406fe20 0x7f48c4072240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42750/0 (socket says 172.21.15.73:42750) 2024-04-26T20:51:03.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.254+0000 7f48c37fe640 1 -- 172.21.15.73:0/774343204 learned_addr learned my addr 172.21.15.73:0/774343204 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:03.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.254+0000 7f48c8f15640 1 --2- 172.21.15.73:0/774343204 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f48c4072780 0x7f48c40cb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:03.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.254+0000 7f48c3fff640 1 --2- 172.21.15.73:0/774343204 >> v2:172.21.15.120:3300/0 conn(0x7f48c4071e60 0x7f48c406f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:03.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.254+0000 7f48c37fe640 1 -- 172.21.15.73:0/774343204 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f48c4072780 msgr2=0x7f48c40cb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:03.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.254+0000 7f48c37fe640 1 --2- 172.21.15.73:0/774343204 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f48c4072780 0x7f48c40cb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:03.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.254+0000 7f48c37fe640 1 -- 172.21.15.73:0/774343204 >> v2:172.21.15.120:3300/0 conn(0x7f48c4071e60 msgr2=0x7f48c406f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:03.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.254+0000 7f48c37fe640 1 --2- 172.21.15.73:0/774343204 >> v2:172.21.15.120:3300/0 conn(0x7f48c4071e60 0x7f48c406f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:03.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.254+0000 7f48c37fe640 1 -- 172.21.15.73:0/774343204 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48c415e060 con 0x7f48c406fe20 2024-04-26T20:51:03.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.254+0000 7f48c8f15640 1 --2- 172.21.15.73:0/774343204 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f48c4072780 0x7f48c40cb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:03.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.254+0000 7f48c3fff640 1 --2- 172.21.15.73:0/774343204 >> v2:172.21.15.120:3300/0 conn(0x7f48c4071e60 0x7f48c406f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:03.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.255+0000 7f48c37fe640 1 --2- 172.21.15.73:0/774343204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48c406fe20 0x7f48c4072240 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f48c40b3700 tx=0x7f48b802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4d157ff448da2f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:03.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.255+0000 7f48c2ffd640 1 -- 172.21.15.73:0/774343204 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b8037070 con 0x7f48c406fe20 2024-04-26T20:51:03.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.255+0000 7f48c2ffd640 1 -- 172.21.15.73:0/774343204 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48b8032440 con 0x7f48c406fe20 2024-04-26T20:51:03.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.255+0000 7f48c2ffd640 1 -- 172.21.15.73:0/774343204 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b8032810 con 0x7f48c406fe20 2024-04-26T20:51:03.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.256+0000 7f48ca978640 1 -- 172.21.15.73:0/774343204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48c406fe20 msgr2=0x7f48c4072240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:03.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.256+0000 7f48ca978640 1 --2- 172.21.15.73:0/774343204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48c406fe20 0x7f48c4072240 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f48c40b3700 tx=0x7f48b802a1e0 comp rx=0 tx=0).stop 2024-04-26T20:51:03.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.256+0000 7f48ca978640 1 -- 172.21.15.73:0/774343204 shutdown_connections 2024-04-26T20:51:03.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.256+0000 7f48ca978640 1 --2- 172.21.15.73:0/774343204 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f48c4072780 0x7f48c40cb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:03.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.256+0000 7f48ca978640 1 --2- 172.21.15.73:0/774343204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48c406fe20 0x7f48c4072240 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:03.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.256+0000 7f48ca978640 1 --2- 172.21.15.73:0/774343204 >> v2:172.21.15.120:3300/0 conn(0x7f48c4071e60 0x7f48c406f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:03.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.256+0000 7f48ca978640 1 -- 172.21.15.73:0/774343204 >> 172.21.15.73:0/774343204 conn(0x7f48c40d0be0 msgr2=0x7f48c40d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:03.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.256+0000 7f48ca978640 1 -- 172.21.15.73:0/774343204 shutdown_connections 2024-04-26T20:51:03.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.256+0000 7f48ca978640 1 -- 172.21.15.73:0/774343204 wait complete. 2024-04-26T20:51:03.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.257+0000 7f48ca978640 1 Processor -- start 2024-04-26T20:51:03.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.257+0000 7f48ca978640 1 -- start start 2024-04-26T20:51:03.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.258+0000 7f48ca978640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f48c406fe20 0x7f48c40ca5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:03.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.258+0000 7f48ca978640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f48c4071e60 0x7f48c41490e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:03.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.258+0000 7f48ca978640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48c4072780 0x7f48c40b6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:03.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.258+0000 7f48ca978640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f48c415d3f0 con 0x7f48c4072780 2024-04-26T20:51:03.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.258+0000 7f48ca978640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f48c4072c10 con 0x7f48c406fe20 2024-04-26T20:51:03.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.258+0000 7f48ca978640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f48c415d570 con 0x7f48c4071e60 2024-04-26T20:51:03.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.258+0000 7f48c8f15640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48c4072780 0x7f48c40b6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:03.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.258+0000 7f48c8f15640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48c4072780 0x7f48c40b6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42756/0 (socket says 172.21.15.73:42756) 2024-04-26T20:51:03.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.258+0000 7f48c8f15640 1 -- 172.21.15.73:0/3418581630 learned_addr learned my addr 172.21.15.73:0/3418581630 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:03.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.258+0000 7f48c37fe640 1 --2- 172.21.15.73:0/3418581630 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f48c4071e60 0x7f48c41490e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:03.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.258+0000 7f48c3fff640 1 --2- 172.21.15.73:0/3418581630 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f48c406fe20 0x7f48c40ca5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:03.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.258+0000 7f48c8f15640 1 -- 172.21.15.73:0/3418581630 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f48c4071e60 msgr2=0x7f48c41490e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:03.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.258+0000 7f48c8f15640 1 --2- 172.21.15.73:0/3418581630 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f48c4071e60 0x7f48c41490e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:03.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.258+0000 7f48c8f15640 1 -- 172.21.15.73:0/3418581630 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f48c406fe20 msgr2=0x7f48c40ca5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:03.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.258+0000 7f48c8f15640 1 --2- 172.21.15.73:0/3418581630 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f48c406fe20 0x7f48c40ca5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:03.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.258+0000 7f48c8f15640 1 -- 172.21.15.73:0/3418581630 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48c40b5390 con 0x7f48c4072780 2024-04-26T20:51:03.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.258+0000 7f48c37fe640 1 --2- 172.21.15.73:0/3418581630 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f48c4071e60 0x7f48c41490e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:03.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.259+0000 7f48c8f15640 1 --2- 172.21.15.73:0/3418581630 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48c4072780 0x7f48c40b6430 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f48bc00cd50 tx=0x7f48bc00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:03.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.259+0000 7f48c0ff9640 1 -- 172.21.15.73:0/3418581630 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc00f680 con 0x7f48c4072780 2024-04-26T20:51:03.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.259+0000 7f48c0ff9640 1 -- 172.21.15.73:0/3418581630 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48bc00f840 con 0x7f48c4072780 2024-04-26T20:51:03.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.259+0000 7f48c0ff9640 1 -- 172.21.15.73:0/3418581630 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc00fc30 con 0x7f48c4072780 2024-04-26T20:51:03.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.259+0000 7f48ca978640 1 -- 172.21.15.73:0/3418581630 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48c40b5670 con 0x7f48c4072780 2024-04-26T20:51:03.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.259+0000 7f48ca978640 1 -- 172.21.15.73:0/3418581630 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48c40b5c30 con 0x7f48c4072780 2024-04-26T20:51:03.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.260+0000 7f48ca978640 1 -- 172.21.15.73:0/3418581630 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48c40694f0 con 0x7f48c4072780 2024-04-26T20:51:03.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.260+0000 7f48c0ff9640 1 -- 172.21.15.73:0/3418581630 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f48bc0213f0 con 0x7f48c4072780 2024-04-26T20:51:03.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.261+0000 7f48c0ff9640 1 --2- 172.21.15.73:0/3418581630 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f48b006c8f0 0x7f48b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:03.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.261+0000 7f48c3fff640 1 --2- 172.21.15.73:0/3418581630 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f48b006c8f0 0x7f48b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:03.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.261+0000 7f48c0ff9640 1 -- 172.21.15.73:0/3418581630 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f48bc08ccc0 con 0x7f48c4072780 2024-04-26T20:51:03.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.261+0000 7f48c3fff640 1 --2- 172.21.15.73:0/3418581630 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f48b006c8f0 0x7f48b006edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f48b4006fd0 tx=0x7f48b4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:03.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.265+0000 7f48c0ff9640 1 -- 172.21.15.73:0/3418581630 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f48bc014030 con 0x7f48c4072780 2024-04-26T20:51:03.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.440+0000 7f48ca978640 1 -- 172.21.15.73:0/3418581630 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f48c40cc7a0 con 0x7f48c4072780 2024-04-26T20:51:03.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.440+0000 7f48c0ff9640 1 -- 172.21.15.73:0/3418581630 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f48bc05ad20 con 0x7f48c4072780 2024-04-26T20:51:03.441 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-26T20:51:03.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.442+0000 7f48ca978640 1 -- 172.21.15.73:0/3418581630 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f48b006c8f0 msgr2=0x7f48b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:03.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.442+0000 7f48ca978640 1 --2- 172.21.15.73:0/3418581630 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f48b006c8f0 0x7f48b006edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f48b4006fd0 tx=0x7f48b4008040 comp rx=0 tx=0).stop 2024-04-26T20:51:03.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.442+0000 7f48ca978640 1 -- 172.21.15.73:0/3418581630 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48c4072780 msgr2=0x7f48c40b6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:03.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.442+0000 7f48ca978640 1 --2- 172.21.15.73:0/3418581630 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48c4072780 0x7f48c40b6430 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f48bc00cd50 tx=0x7f48bc00a650 comp rx=0 tx=0).stop 2024-04-26T20:51:03.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.442+0000 7f48ca978640 1 -- 172.21.15.73:0/3418581630 shutdown_connections 2024-04-26T20:51:03.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.442+0000 7f48ca978640 1 --2- 172.21.15.73:0/3418581630 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48c4072780 0x7f48c40b6430 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:03.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.442+0000 7f48ca978640 1 --2- 172.21.15.73:0/3418581630 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f48b006c8f0 0x7f48b006edb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:03.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.442+0000 7f48ca978640 1 --2- 172.21.15.73:0/3418581630 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f48c4071e60 0x7f48c41490e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:03.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.442+0000 7f48ca978640 1 --2- 172.21.15.73:0/3418581630 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f48c406fe20 0x7f48c40ca5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:03.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.442+0000 7f48ca978640 1 -- 172.21.15.73:0/3418581630 >> 172.21.15.73:0/3418581630 conn(0x7f48c40d0be0 msgr2=0x7f48c40cb2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:03.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.442+0000 7f48ca978640 1 -- 172.21.15.73:0/3418581630 shutdown_connections 2024-04-26T20:51:03.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.443+0000 7f48ca978640 1 -- 172.21.15.73:0/3418581630 wait complete. 2024-04-26T20:51:03.454 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:51:03.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.557+0000 7f11d1420640 1 Processor -- start 2024-04-26T20:51:03.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.557+0000 7f11d1420640 1 -- start start 2024-04-26T20:51:03.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.558+0000 7f11d1420640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f11cc0b0380 0x7f11cc0ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:03.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.558+0000 7f11d1420640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f11cc06c950 0x7f11cc15e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:03.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.558+0000 7f11d1420640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11cc0cfd80 con 0x7f11cc0b10b0 2024-04-26T20:51:03.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.558+0000 7f11d1420640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f11cc0af8e0 con 0x7f11cc06c950 2024-04-26T20:51:03.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.558+0000 7f11d1420640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f11cc0d0020 con 0x7f11cc0b0380 2024-04-26T20:51:03.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.558+0000 7f11caffd640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f11cc0b0380 0x7f11cc0ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:03.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.558+0000 7f11cb7fe640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f11cc06c950 0x7f11cc15e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:03.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.558+0000 7f11caffd640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f11cc0b0380 0x7f11cc0ccd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.73:43788/0 (socket says 172.21.15.73:43788) 2024-04-26T20:51:03.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.558+0000 7f11caffd640 1 -- 172.21.15.73:0/3834928658 learned_addr learned my addr 172.21.15.73:0/3834928658 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:03.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.558+0000 7f11c3fff640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f11cc0b10b0 0x7f11cc06c240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.73:42516/0 (socket says 172.21.15.73:42516) 2024-04-26T20:51:03.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.558+0000 7f11caffd640 1 -- 172.21.15.73:0/3834928658 >> v2:172.21.15.120:3300/0 conn(0x7f11cc06c950 msgr2=0x7f11cc15e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:03.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.558+0000 7f11caffd640 1 --2- 172.21.15.73:0/3834928658 >> v2:172.21.15.120:3300/0 conn(0x7f11cc06c950 0x7f11cc15e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:03.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.558+0000 7f11caffd640 1 -- 172.21.15.73:0/3834928658 >> v1:172.21.15.120:6789/0 conn(0x7f11cc0b10b0 legacy=0x7f11cc06c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:03.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.558+0000 7f11caffd640 1 -- 172.21.15.73:0/3834928658 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11cc0d0300 con 0x7f11cc0b0380 2024-04-26T20:51:03.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.559+0000 7f11caffd640 1 --2- 172.21.15.73:0/3834928658 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f11cc0b0380 0x7f11cc0ccd50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f11cc0afa60 tx=0x7f11bc02a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94c8d05a44acedd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:03.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.559+0000 7f11ca7fc640 1 -- 172.21.15.73:0/3834928658 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11bc037070 con 0x7f11cc0b0380 2024-04-26T20:51:03.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.559+0000 7f11ca7fc640 1 -- 172.21.15.73:0/3834928658 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11bc032440 con 0x7f11cc0b0380 2024-04-26T20:51:03.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.559+0000 7f11ca7fc640 1 -- 172.21.15.73:0/3834928658 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11bc032810 con 0x7f11cc0b0380 2024-04-26T20:51:03.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.560+0000 7f11d1420640 1 -- 172.21.15.73:0/3834928658 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f11cc0b0380 msgr2=0x7f11cc0ccd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:03.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.560+0000 7f11d1420640 1 --2- 172.21.15.73:0/3834928658 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f11cc0b0380 0x7f11cc0ccd50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f11cc0afa60 tx=0x7f11bc02a1c0 comp rx=0 tx=0).stop 2024-04-26T20:51:03.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.560+0000 7f11d1420640 1 -- 172.21.15.73:0/3834928658 shutdown_connections 2024-04-26T20:51:03.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.560+0000 7f11d1420640 1 --2- 172.21.15.73:0/3834928658 >> v2:172.21.15.120:3300/0 conn(0x7f11cc06c950 0x7f11cc15e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:03.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.560+0000 7f11d1420640 1 --2- 172.21.15.73:0/3834928658 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f11cc0b0380 0x7f11cc0ccd50 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:03.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.560+0000 7f11d1420640 1 -- 172.21.15.73:0/3834928658 >> 172.21.15.73:0/3834928658 conn(0x7f11cc0d1cf0 msgr2=0x7f11cc0d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:03.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.560+0000 7f11d1420640 1 -- 172.21.15.73:0/3834928658 shutdown_connections 2024-04-26T20:51:03.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.560+0000 7f11d1420640 1 -- 172.21.15.73:0/3834928658 wait complete. 2024-04-26T20:51:03.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.561+0000 7f11d1420640 1 Processor -- start 2024-04-26T20:51:03.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.561+0000 7f11d1420640 1 -- start start 2024-04-26T20:51:03.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.562+0000 7f11d1420640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11cc06c950 0x7f11cc162040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:03.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.562+0000 7f11d1420640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11cc0b0380 0x7f11cc0b7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:03.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.562+0000 7f11d1420640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f11cc0b10b0 0x7f11cc0b7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:03.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.562+0000 7f11d1420640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc0b0760 con 0x7f11cc06c950 2024-04-26T20:51:03.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.562+0000 7f11d1420640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc0d1980 con 0x7f11cc0b0380 2024-04-26T20:51:03.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.562+0000 7f11d1420640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f11cc0d15c0 con 0x7f11cc0b10b0 2024-04-26T20:51:03.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.562+0000 7f11caffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11cc06c950 0x7f11cc162040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:03.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.562+0000 7f11caffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11cc06c950 0x7f11cc162040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42768/0 (socket says 172.21.15.73:42768) 2024-04-26T20:51:03.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.562+0000 7f11caffd640 1 -- 172.21.15.73:0/2530194906 learned_addr learned my addr 172.21.15.73:0/2530194906 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:03.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.562+0000 7f11c3fff640 1 --2- 172.21.15.73:0/2530194906 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11cc0b0380 0x7f11cc0b7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:03.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.562+0000 7f11cb7fe640 1 --2- 172.21.15.73:0/2530194906 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f11cc0b10b0 0x7f11cc0b7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:03.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.562+0000 7f11caffd640 1 -- 172.21.15.73:0/2530194906 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f11cc0b10b0 msgr2=0x7f11cc0b7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:03.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.562+0000 7f11caffd640 1 --2- 172.21.15.73:0/2530194906 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f11cc0b10b0 0x7f11cc0b7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:03.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.562+0000 7f11caffd640 1 -- 172.21.15.73:0/2530194906 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11cc0b0380 msgr2=0x7f11cc0b7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:03.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.562+0000 7f11caffd640 1 --2- 172.21.15.73:0/2530194906 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11cc0b0380 0x7f11cc0b7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:03.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.562+0000 7f11caffd640 1 -- 172.21.15.73:0/2530194906 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11cc162580 con 0x7f11cc06c950 2024-04-26T20:51:03.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.562+0000 7f11caffd640 1 --2- 172.21.15.73:0/2530194906 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11cc06c950 0x7f11cc162040 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f11bc02a200 tx=0x7f11bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:03.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.563+0000 7f11c37fe640 1 -- 172.21.15.73:0/2530194906 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11bc037040 con 0x7f11cc06c950 2024-04-26T20:51:03.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.563+0000 7f11c37fe640 1 -- 172.21.15.73:0/2530194906 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11bc042070 con 0x7f11cc06c950 2024-04-26T20:51:03.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.563+0000 7f11d1420640 1 -- 172.21.15.73:0/2530194906 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11cc0ca8c0 con 0x7f11cc06c950 2024-04-26T20:51:03.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.563+0000 7f11c37fe640 1 -- 172.21.15.73:0/2530194906 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11bc030880 con 0x7f11cc06c950 2024-04-26T20:51:03.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.563+0000 7f11d1420640 1 -- 172.21.15.73:0/2530194906 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11cc0c7990 con 0x7f11cc06c950 2024-04-26T20:51:03.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.564+0000 7f11d1420640 1 -- 172.21.15.73:0/2530194906 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11cc0694f0 con 0x7f11cc06c950 2024-04-26T20:51:03.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.564+0000 7f11c37fe640 1 -- 172.21.15.73:0/2530194906 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f11bc02ad10 con 0x7f11cc06c950 2024-04-26T20:51:03.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.564+0000 7f11c37fe640 1 --2- 172.21.15.73:0/2530194906 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f11ac06c8f0 0x7f11ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:03.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.564+0000 7f11c37fe640 1 -- 172.21.15.73:0/2530194906 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f11bc0ac450 con 0x7f11cc06c950 2024-04-26T20:51:03.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.564+0000 7f11c3fff640 1 --2- 172.21.15.73:0/2530194906 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f11ac06c8f0 0x7f11ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:03.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.565+0000 7f11c3fff640 1 --2- 172.21.15.73:0/2530194906 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f11ac06c8f0 0x7f11ac06edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f11cc0b7c10 tx=0x7f11b4007ce0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:03.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.569+0000 7f11c37fe640 1 -- 172.21.15.73:0/2530194906 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f11bc07a550 con 0x7f11cc06c950 2024-04-26T20:51:03.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.748+0000 7f11d1420640 1 -- 172.21.15.73:0/2530194906 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f11cc06df70 con 0x7f11cc06c950 2024-04-26T20:51:03.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.748+0000 7f11c37fe640 1 -- 172.21.15.73:0/2530194906 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f11bc079e50 con 0x7f11cc06c950 2024-04-26T20:51:03.749 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-26T20:51:03.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.750+0000 7f11d1420640 1 -- 172.21.15.73:0/2530194906 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f11ac06c8f0 msgr2=0x7f11ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:03.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.750+0000 7f11d1420640 1 --2- 172.21.15.73:0/2530194906 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f11ac06c8f0 0x7f11ac06edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f11cc0b7c10 tx=0x7f11b4007ce0 comp rx=0 tx=0).stop 2024-04-26T20:51:03.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.750+0000 7f11d1420640 1 -- 172.21.15.73:0/2530194906 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11cc06c950 msgr2=0x7f11cc162040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:03.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.750+0000 7f11d1420640 1 --2- 172.21.15.73:0/2530194906 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11cc06c950 0x7f11cc162040 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f11bc02a200 tx=0x7f11bc0046c0 comp rx=0 tx=0).stop 2024-04-26T20:51:03.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.750+0000 7f11d1420640 1 -- 172.21.15.73:0/2530194906 shutdown_connections 2024-04-26T20:51:03.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.750+0000 7f11d1420640 1 --2- 172.21.15.73:0/2530194906 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f11ac06c8f0 0x7f11ac06edb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:03.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.750+0000 7f11d1420640 1 --2- 172.21.15.73:0/2530194906 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f11cc0b10b0 0x7f11cc0b7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:03.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.750+0000 7f11d1420640 1 --2- 172.21.15.73:0/2530194906 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f11cc0b0380 0x7f11cc0b7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:03.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.750+0000 7f11d1420640 1 --2- 172.21.15.73:0/2530194906 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11cc06c950 0x7f11cc162040 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:03.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.750+0000 7f11d1420640 1 -- 172.21.15.73:0/2530194906 >> 172.21.15.73:0/2530194906 conn(0x7f11cc0d1cf0 msgr2=0x7f11cc0ceff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:03.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.750+0000 7f11d1420640 1 -- 172.21.15.73:0/2530194906 shutdown_connections 2024-04-26T20:51:03.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.750+0000 7f11d1420640 1 -- 172.21.15.73:0/2530194906 wait complete. 2024-04-26T20:51:03.762 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T20:51:03.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.860+0000 7fb7c99e4640 1 Processor -- start 2024-04-26T20:51:03.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.860+0000 7fb7c99e4640 1 -- start start 2024-04-26T20:51:03.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.861+0000 7fb7c99e4640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb7c41628f0 0x7fb7c40d1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:03.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.861+0000 7fb7c99e4640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb7c4162cd0 0x7fb7c40d1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:03.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.861+0000 7fb7c99e4640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7c40d0890 con 0x7fb7c40d1550 2024-04-26T20:51:03.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.861+0000 7fb7c99e4640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7fb7c40d1130 con 0x7fb7c4162cd0 2024-04-26T20:51:03.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.861+0000 7fb7c99e4640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fb7c40d0b70 con 0x7fb7c41628f0 2024-04-26T20:51:03.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.861+0000 7fb7c27fc640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb7c4162cd0 0x7fb7c40d1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:03.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.861+0000 7fb7c27fc640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb7c4162cd0 0x7fb7c40d1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42770/0 (socket says 172.21.15.73:42770) 2024-04-26T20:51:03.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.861+0000 7fb7c27fc640 1 -- 172.21.15.73:0/2903305915 learned_addr learned my addr 172.21.15.73:0/2903305915 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:03.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.862+0000 7fb7c2ffd640 1 --2- 172.21.15.73:0/2903305915 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb7c41628f0 0x7fb7c40d1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:03.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.862+0000 7fb7c27fc640 1 -- 172.21.15.73:0/2903305915 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb7c41628f0 msgr2=0x7fb7c40d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:03.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.862+0000 7fb7c27fc640 1 --2- 172.21.15.73:0/2903305915 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb7c41628f0 0x7fb7c40d1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:03.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.862+0000 7fb7c27fc640 1 -- 172.21.15.73:0/2903305915 >> v1:172.21.15.120:6789/0 conn(0x7fb7c40d1550 legacy=0x7fb7c40d2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:03.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.862+0000 7fb7c27fc640 1 -- 172.21.15.73:0/2903305915 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7c40d0e50 con 0x7fb7c4162cd0 2024-04-26T20:51:03.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.862+0000 7fb7c27fc640 1 --2- 172.21.15.73:0/2903305915 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb7c4162cd0 0x7fb7c40d1f40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8009870 tx=0x7fb7b802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8db6f114144148b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:03.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.862+0000 7fb7c1ffb640 1 -- 172.21.15.73:0/2903305915 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b8037070 con 0x7fb7c4162cd0 2024-04-26T20:51:03.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.863+0000 7fb7c1ffb640 1 -- 172.21.15.73:0/2903305915 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7b8032440 con 0x7fb7c4162cd0 2024-04-26T20:51:03.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.863+0000 7fb7c1ffb640 1 -- 172.21.15.73:0/2903305915 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b8032810 con 0x7fb7c4162cd0 2024-04-26T20:51:03.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.863+0000 7fb7c99e4640 1 -- 172.21.15.73:0/2903305915 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb7c4162cd0 msgr2=0x7fb7c40d1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:03.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.863+0000 7fb7c99e4640 1 --2- 172.21.15.73:0/2903305915 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb7c4162cd0 0x7fb7c40d1f40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8009870 tx=0x7fb7b802a180 comp rx=0 tx=0).stop 2024-04-26T20:51:03.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.863+0000 7fb7c99e4640 1 -- 172.21.15.73:0/2903305915 shutdown_connections 2024-04-26T20:51:03.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.863+0000 7fb7c99e4640 1 --2- 172.21.15.73:0/2903305915 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb7c4162cd0 0x7fb7c40d1f40 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:03.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.863+0000 7fb7c99e4640 1 --2- 172.21.15.73:0/2903305915 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb7c41628f0 0x7fb7c40d1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:03.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.863+0000 7fb7c99e4640 1 -- 172.21.15.73:0/2903305915 >> 172.21.15.73:0/2903305915 conn(0x7fb7c40d4ff0 msgr2=0x7fb7c40d5400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:03.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.863+0000 7fb7c99e4640 1 -- 172.21.15.73:0/2903305915 shutdown_connections 2024-04-26T20:51:03.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.863+0000 7fb7c99e4640 1 -- 172.21.15.73:0/2903305915 wait complete. 2024-04-26T20:51:03.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.864+0000 7fb7c99e4640 1 Processor -- start 2024-04-26T20:51:03.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.864+0000 7fb7c99e4640 1 -- start start 2024-04-26T20:51:03.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.865+0000 7fb7c99e4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb7c40d1550 0x7fb7c415d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:03.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.865+0000 7fb7c99e4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb7c41628f0 0x7fb7c41f5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:03.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.865+0000 7fb7c99e4640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb7c4162cd0 0x7fb7c40b99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:03.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.865+0000 7fb7c99e4640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c40b5df0 con 0x7fb7c41628f0 2024-04-26T20:51:03.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.865+0000 7fb7c99e4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c40b5c70 con 0x7fb7c40d1550 2024-04-26T20:51:03.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.865+0000 7fb7c99e4640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fb7c40b5f70 con 0x7fb7c4162cd0 2024-04-26T20:51:03.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.865+0000 7fb7c27fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb7c41628f0 0x7fb7c41f5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:03.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.865+0000 7fb7c27fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb7c41628f0 0x7fb7c41f5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42782/0 (socket says 172.21.15.73:42782) 2024-04-26T20:51:03.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.865+0000 7fb7c27fc640 1 -- 172.21.15.73:0/2672834601 learned_addr learned my addr 172.21.15.73:0/2672834601 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:03.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.865+0000 7fb7c2ffd640 1 --2- 172.21.15.73:0/2672834601 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb7c40d1550 0x7fb7c415d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:03.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.865+0000 7fb7c37fe640 1 --2- 172.21.15.73:0/2672834601 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb7c4162cd0 0x7fb7c40b99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:03.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.866+0000 7fb7c27fc640 1 -- 172.21.15.73:0/2672834601 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb7c4162cd0 msgr2=0x7fb7c40b99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:03.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.866+0000 7fb7c27fc640 1 --2- 172.21.15.73:0/2672834601 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb7c4162cd0 0x7fb7c40b99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:03.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.866+0000 7fb7c27fc640 1 -- 172.21.15.73:0/2672834601 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb7c40d1550 msgr2=0x7fb7c415d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:03.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.866+0000 7fb7c27fc640 1 --2- 172.21.15.73:0/2672834601 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb7c40d1550 0x7fb7c415d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:03.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.866+0000 7fb7c27fc640 1 -- 172.21.15.73:0/2672834601 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7c40cc160 con 0x7fb7c41628f0 2024-04-26T20:51:03.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.866+0000 7fb7c27fc640 1 --2- 172.21.15.73:0/2672834601 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb7c41628f0 0x7fb7c41f5f20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb7b80090b0 tx=0x7fb7b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:03.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.866+0000 7fb7abfff640 1 -- 172.21.15.73:0/2672834601 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b8041020 con 0x7fb7c41628f0 2024-04-26T20:51:03.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.866+0000 7fb7abfff640 1 -- 172.21.15.73:0/2672834601 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7b8030990 con 0x7fb7c41628f0 2024-04-26T20:51:03.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.866+0000 7fb7c99e4640 1 -- 172.21.15.73:0/2672834601 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7c40cc440 con 0x7fb7c41628f0 2024-04-26T20:51:03.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.866+0000 7fb7abfff640 1 -- 172.21.15.73:0/2672834601 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b8037040 con 0x7fb7c41628f0 2024-04-26T20:51:03.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.866+0000 7fb7c99e4640 1 -- 172.21.15.73:0/2672834601 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7c40cca00 con 0x7fb7c41628f0 2024-04-26T20:51:03.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.867+0000 7fb7c99e4640 1 -- 172.21.15.73:0/2672834601 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7c40694f0 con 0x7fb7c41628f0 2024-04-26T20:51:03.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.867+0000 7fb7abfff640 1 -- 172.21.15.73:0/2672834601 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb7b8044020 con 0x7fb7c41628f0 2024-04-26T20:51:03.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.868+0000 7fb7abfff640 1 --2- 172.21.15.73:0/2672834601 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb7b006c8f0 0x7fb7b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:03.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.868+0000 7fb7c2ffd640 1 --2- 172.21.15.73:0/2672834601 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb7b006c8f0 0x7fb7b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:03.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.868+0000 7fb7abfff640 1 -- 172.21.15.73:0/2672834601 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fb7b80ac7b0 con 0x7fb7c41628f0 2024-04-26T20:51:03.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.868+0000 7fb7c2ffd640 1 --2- 172.21.15.73:0/2672834601 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb7b006c8f0 0x7fb7b006edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb7b400f510 tx=0x7fb7b40077b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:03.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:03.872+0000 7fb7abfff640 1 -- 172.21.15.73:0/2672834601 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb7b804b030 con 0x7fb7c41628f0 2024-04-26T20:51:04.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.047+0000 7fb7c99e4640 1 -- 172.21.15.73:0/2672834601 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fb7c40745d0 con 0x7fb7c41628f0 2024-04-26T20:51:04.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.047+0000 7fb7abfff640 1 -- 172.21.15.73:0/2672834601 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7fb7b807a250 con 0x7fb7c41628f0 2024-04-26T20:51:04.049 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-26T20:51:04.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.049+0000 7fb7c99e4640 1 -- 172.21.15.73:0/2672834601 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb7b006c8f0 msgr2=0x7fb7b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:04.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.049+0000 7fb7c99e4640 1 --2- 172.21.15.73:0/2672834601 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb7b006c8f0 0x7fb7b006edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb7b400f510 tx=0x7fb7b40077b0 comp rx=0 tx=0).stop 2024-04-26T20:51:04.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.049+0000 7fb7c99e4640 1 -- 172.21.15.73:0/2672834601 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb7c41628f0 msgr2=0x7fb7c41f5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:04.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.049+0000 7fb7c99e4640 1 --2- 172.21.15.73:0/2672834601 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb7c41628f0 0x7fb7c41f5f20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb7b80090b0 tx=0x7fb7b80046c0 comp rx=0 tx=0).stop 2024-04-26T20:51:04.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.049+0000 7fb7c99e4640 1 -- 172.21.15.73:0/2672834601 shutdown_connections 2024-04-26T20:51:04.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.049+0000 7fb7c99e4640 1 --2- 172.21.15.73:0/2672834601 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb7c41628f0 0x7fb7c41f5f20 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:04.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.049+0000 7fb7c99e4640 1 --2- 172.21.15.73:0/2672834601 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb7b006c8f0 0x7fb7b006edb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:04.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.049+0000 7fb7c99e4640 1 --2- 172.21.15.73:0/2672834601 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb7c4162cd0 0x7fb7c40b99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:04.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.049+0000 7fb7c99e4640 1 --2- 172.21.15.73:0/2672834601 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb7c40d1550 0x7fb7c415d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:04.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.049+0000 7fb7c99e4640 1 -- 172.21.15.73:0/2672834601 >> 172.21.15.73:0/2672834601 conn(0x7fb7c40d4ff0 msgr2=0x7fb7c40d2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:04.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.050+0000 7fb7c99e4640 1 -- 172.21.15.73:0/2672834601 shutdown_connections 2024-04-26T20:51:04.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.050+0000 7fb7c99e4640 1 -- 172.21.15.73:0/2672834601 wait complete. 2024-04-26T20:51:04.061 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:51:04.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.157+0000 7f66d57af640 1 Processor -- start 2024-04-26T20:51:04.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.157+0000 7f66d57af640 1 -- start start 2024-04-26T20:51:04.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.158+0000 7f66d57af640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f66d00d3bc0 0x7f66d00d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:04.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.158+0000 7f66d57af640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f66d00b5f20 0x7f66d00b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:04.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.158+0000 7f66d57af640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66d00b7e50 con 0x7f66d0162880 2024-04-26T20:51:04.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.158+0000 7f66d57af640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f66d00d0580 con 0x7f66d00d3bc0 2024-04-26T20:51:04.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.158+0000 7f66d57af640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f66d00d1800 con 0x7f66d00b5f20 2024-04-26T20:51:04.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.158+0000 7f66ceffd640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f66d0162880 0x7f66d00d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:42076/0 (socket says 172.21.15.73:42076) 2024-04-26T20:51:04.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.158+0000 7f66ceffd640 1 -- 172.21.15.73:0/3663447690 learned_addr learned my addr 172.21.15.73:0/3663447690 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:04.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.158+0000 7f66ce7fc640 1 --2- 172.21.15.73:0/3663447690 >> v2:172.21.15.73:3300/0 conn(0x7f66d00d3bc0 0x7f66d00d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:04.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.158+0000 7f66cdffb640 1 -- 172.21.15.73:0/3663447690 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 978846277 0 0) 0x7f66d00b7e50 con 0x7f66d0162880 2024-04-26T20:51:04.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.158+0000 7f66cf7fe640 1 --2- 172.21.15.73:0/3663447690 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f66d00b5f20 0x7f66d00b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:04.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.158+0000 7f66cdffb640 1 -- 172.21.15.73:0/3663447690 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f66b8003520 con 0x7f66d0162880 2024-04-26T20:51:04.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.158+0000 7f66ce7fc640 1 -- 172.21.15.73:0/3663447690 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f66d00b5f20 msgr2=0x7f66d00b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:04.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.158+0000 7f66ce7fc640 1 --2- 172.21.15.73:0/3663447690 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f66d00b5f20 0x7f66d00b8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:04.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.158+0000 7f66ce7fc640 1 -- 172.21.15.73:0/3663447690 >> v1:172.21.15.73:6789/0 conn(0x7f66d0162880 legacy=0x7f66d00d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:04.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.158+0000 7f66ce7fc640 1 -- 172.21.15.73:0/3663447690 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66d00d07d0 con 0x7f66d00d3bc0 2024-04-26T20:51:04.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.159+0000 7f66ce7fc640 1 --2- 172.21.15.73:0/3663447690 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66d00d3bc0 0x7f66d00d2cb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f66c40097a0 tx=0x7f66c402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e9e92399eaa643d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:04.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.159+0000 7f66cdffb640 1 -- 172.21.15.73:0/3663447690 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c4037070 con 0x7f66d00d3bc0 2024-04-26T20:51:04.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.159+0000 7f66cdffb640 1 -- 172.21.15.73:0/3663447690 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66c4032440 con 0x7f66d00d3bc0 2024-04-26T20:51:04.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.159+0000 7f66cdffb640 1 -- 172.21.15.73:0/3663447690 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c4032810 con 0x7f66d00d3bc0 2024-04-26T20:51:04.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.160+0000 7f66d57af640 1 -- 172.21.15.73:0/3663447690 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66d00d3bc0 msgr2=0x7f66d00d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:04.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.160+0000 7f66d57af640 1 --2- 172.21.15.73:0/3663447690 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66d00d3bc0 0x7f66d00d2cb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f66c40097a0 tx=0x7f66c402a180 comp rx=0 tx=0).stop 2024-04-26T20:51:04.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.160+0000 7f66d57af640 1 -- 172.21.15.73:0/3663447690 shutdown_connections 2024-04-26T20:51:04.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.160+0000 7f66d57af640 1 --2- 172.21.15.73:0/3663447690 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f66d00b5f20 0x7f66d00b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:04.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.160+0000 7f66d57af640 1 --2- 172.21.15.73:0/3663447690 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66d00d3bc0 0x7f66d00d2cb0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:04.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.160+0000 7f66d57af640 1 -- 172.21.15.73:0/3663447690 >> 172.21.15.73:0/3663447690 conn(0x7f66d00d4f60 msgr2=0x7f66d00d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:04.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.160+0000 7f66d57af640 1 -- 172.21.15.73:0/3663447690 shutdown_connections 2024-04-26T20:51:04.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.160+0000 7f66d57af640 1 -- 172.21.15.73:0/3663447690 wait complete. 2024-04-26T20:51:04.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.161+0000 7f66d57af640 1 Processor -- start 2024-04-26T20:51:04.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.161+0000 7f66d57af640 1 -- start start 2024-04-26T20:51:04.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.162+0000 7f66d57af640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66d00b5f20 0x7f66d00b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:04.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.162+0000 7f66d57af640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f66d00d3bc0 0x7f66d00cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:04.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.162+0000 7f66d57af640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f66d0162880 0x7f66d015c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:04.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.162+0000 7f66d57af640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f66d0162c60 con 0x7f66d00b5f20 2024-04-26T20:51:04.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.162+0000 7f66d57af640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f66d00d0d90 con 0x7f66d00d3bc0 2024-04-26T20:51:04.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.162+0000 7f66d57af640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f66d0162de0 con 0x7f66d0162880 2024-04-26T20:51:04.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.162+0000 7f66ceffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66d00b5f20 0x7f66d00b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:04.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.162+0000 7f66ceffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66d00b5f20 0x7f66d00b1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42796/0 (socket says 172.21.15.73:42796) 2024-04-26T20:51:04.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.162+0000 7f66ceffd640 1 -- 172.21.15.73:0/1745947969 learned_addr learned my addr 172.21.15.73:0/1745947969 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:04.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.162+0000 7f66ce7fc640 1 --2- 172.21.15.73:0/1745947969 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f66d00d3bc0 0x7f66d00cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:04.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.162+0000 7f66cf7fe640 1 --2- 172.21.15.73:0/1745947969 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f66d0162880 0x7f66d015c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:04.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.162+0000 7f66ceffd640 1 -- 172.21.15.73:0/1745947969 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f66d0162880 msgr2=0x7f66d015c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:04.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.162+0000 7f66ceffd640 1 --2- 172.21.15.73:0/1745947969 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f66d0162880 0x7f66d015c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:04.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.162+0000 7f66ceffd640 1 -- 172.21.15.73:0/1745947969 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f66d00d3bc0 msgr2=0x7f66d00cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:04.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.162+0000 7f66ceffd640 1 --2- 172.21.15.73:0/1745947969 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f66d00d3bc0 0x7f66d00cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:04.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.162+0000 7f66ceffd640 1 -- 172.21.15.73:0/1745947969 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66d015c880 con 0x7f66d00b5f20 2024-04-26T20:51:04.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.162+0000 7f66ce7fc640 1 --2- 172.21.15.73:0/1745947969 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f66d00d3bc0 0x7f66d00cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:04.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.162+0000 7f66cf7fe640 1 --2- 172.21.15.73:0/1745947969 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f66d0162880 0x7f66d015c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:04.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.162+0000 7f66ceffd640 1 --2- 172.21.15.73:0/1745947969 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66d00b5f20 0x7f66d00b1b50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f66c000cb40 tx=0x7f66c000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:04.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.163+0000 7f66b77fe640 1 -- 172.21.15.73:0/1745947969 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c0017070 con 0x7f66d00b5f20 2024-04-26T20:51:04.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.163+0000 7f66b77fe640 1 -- 172.21.15.73:0/1745947969 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66c000aca0 con 0x7f66d00b5f20 2024-04-26T20:51:04.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.163+0000 7f66d57af640 1 -- 172.21.15.73:0/1745947969 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66d00b21c0 con 0x7f66d00b5f20 2024-04-26T20:51:04.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.163+0000 7f66b77fe640 1 -- 172.21.15.73:0/1745947969 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c0012620 con 0x7f66d00b5f20 2024-04-26T20:51:04.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.163+0000 7f66d57af640 1 -- 172.21.15.73:0/1745947969 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66d00b2780 con 0x7f66d00b5f20 2024-04-26T20:51:04.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.164+0000 7f66d57af640 1 -- 172.21.15.73:0/1745947969 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66d00b2f50 con 0x7f66d00b5f20 2024-04-26T20:51:04.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.164+0000 7f66b77fe640 1 -- 172.21.15.73:0/1745947969 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f66c001a070 con 0x7f66d00b5f20 2024-04-26T20:51:04.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.164+0000 7f66b77fe640 1 --2- 172.21.15.73:0/1745947969 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f66bc06c8f0 0x7f66bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:04.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.165+0000 7f66b77fe640 1 -- 172.21.15.73:0/1745947969 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f66c008bc90 con 0x7f66d00b5f20 2024-04-26T20:51:04.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.165+0000 7f66ce7fc640 1 --2- 172.21.15.73:0/1745947969 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f66bc06c8f0 0x7f66bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:04.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.165+0000 7f66ce7fc640 1 --2- 172.21.15.73:0/1745947969 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f66bc06c8f0 0x7f66bc06edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f66c4034000 tx=0x7f66c402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:04.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.169+0000 7f66b77fe640 1 -- 172.21.15.73:0/1745947969 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f66c0059e10 con 0x7f66d00b5f20 2024-04-26T20:51:04.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.382+0000 7f66d57af640 1 -- 172.21.15.73:0/1745947969 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66d00d3750 con 0x7f66d00b5f20 2024-04-26T20:51:04.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.383+0000 7f66b77fe640 1 -- 172.21.15.73:0/1745947969 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5180 (secure 0 0 0) 0x7f66c0059710 con 0x7f66d00b5f20 2024-04-26T20:51:04.384 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:51:04.384 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/4263167680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6833","nonce":4263167680},{"type":"v1","addr":"172.21.15.120:6835","nonce":4263167680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6834/1614342510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1614342510},{"type":"v1","addr":"172.21.15.120:6834","nonce":1614342510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2146325725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2146325725},{"type":"v1","addr":"172.21.15.73:6835","nonce":2146325725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:50:31.964317+0000","modified":"2024-04-26T20:50:54.049029+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/181202544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":181202544},{"type":"v1","addr":"172.21.15.73:6837","nonce":181202544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4281,"qdb_cluster":[4281]},"id":1}]} 2024-04-26T20:51:04.384 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 8 2024-04-26T20:51:04.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.384+0000 7f66d57af640 1 -- 172.21.15.73:0/1745947969 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f66bc06c8f0 msgr2=0x7f66bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:04.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.384+0000 7f66d57af640 1 --2- 172.21.15.73:0/1745947969 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f66bc06c8f0 0x7f66bc06edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f66c4034000 tx=0x7f66c402e000 comp rx=0 tx=0).stop 2024-04-26T20:51:04.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.384+0000 7f66d57af640 1 -- 172.21.15.73:0/1745947969 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66d00b5f20 msgr2=0x7f66d00b1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:04.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.384+0000 7f66d57af640 1 --2- 172.21.15.73:0/1745947969 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66d00b5f20 0x7f66d00b1b50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f66c000cb40 tx=0x7f66c000a060 comp rx=0 tx=0).stop 2024-04-26T20:51:04.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.384+0000 7f66d57af640 1 -- 172.21.15.73:0/1745947969 shutdown_connections 2024-04-26T20:51:04.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.384+0000 7f66d57af640 1 --2- 172.21.15.73:0/1745947969 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f66bc06c8f0 0x7f66bc06edb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:04.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.385+0000 7f66d57af640 1 --2- 172.21.15.73:0/1745947969 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f66d0162880 0x7f66d015c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:04.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.385+0000 7f66d57af640 1 --2- 172.21.15.73:0/1745947969 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f66d00d3bc0 0x7f66d00cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:04.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.385+0000 7f66d57af640 1 --2- 172.21.15.73:0/1745947969 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66d00b5f20 0x7f66d00b1b50 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:04.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.385+0000 7f66d57af640 1 -- 172.21.15.73:0/1745947969 >> 172.21.15.73:0/1745947969 conn(0x7f66d00d4f60 msgr2=0x7f66d00b69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:04.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.385+0000 7f66d57af640 1 -- 172.21.15.73:0/1745947969 shutdown_connections 2024-04-26T20:51:04.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.385+0000 7f66d57af640 1 -- 172.21.15.73:0/1745947969 wait complete. 2024-04-26T20:51:04.397 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:51:04.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.492+0000 7f920bff0640 1 Processor -- start 2024-04-26T20:51:04.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.493+0000 7f920bff0640 1 -- start start 2024-04-26T20:51:04.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.493+0000 7f920bff0640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f92040b5f20 0x7f92040d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:04.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.493+0000 7f920bff0640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92040b7e50 con 0x7f9204162880 2024-04-26T20:51:04.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.493+0000 7f920bff0640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92040d1800 con 0x7f92040d3bc0 2024-04-26T20:51:04.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.493+0000 7f920bff0640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f92040d0580 con 0x7f92040b5f20 2024-04-26T20:51:04.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.494+0000 7f9209d8c640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f9204162880 0x7f92040d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:42080/0 (socket says 172.21.15.73:42080) 2024-04-26T20:51:04.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.494+0000 7f9209d8c640 1 -- 172.21.15.73:0/2305584451 learned_addr learned my addr 172.21.15.73:0/2305584451 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:04.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.494+0000 7f920a58d640 1 --2- 172.21.15.73:0/2305584451 >> v2:172.21.15.120:3300/0 conn(0x7f92040b5f20 0x7f92040d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:04.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.494+0000 7f9208d8a640 1 -- 172.21.15.73:0/2305584451 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 192457956 0 0) 0x7f92040b7e50 con 0x7f9204162880 2024-04-26T20:51:04.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.494+0000 7f9208d8a640 1 -- 172.21.15.73:0/2305584451 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f91f0003520 con 0x7f9204162880 2024-04-26T20:51:04.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.494+0000 7f9208d8a640 1 -- 172.21.15.73:0/2305584451 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2492906053 0 0) 0x7f91f0003520 con 0x7f9204162880 2024-04-26T20:51:04.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.494+0000 7f9208d8a640 1 -- 172.21.15.73:0/2305584451 >> v2:172.21.15.120:3300/0 conn(0x7f92040b5f20 msgr2=0x7f92040d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:04.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.494+0000 7f9208d8a640 1 --2- 172.21.15.73:0/2305584451 >> v2:172.21.15.120:3300/0 conn(0x7f92040b5f20 0x7f92040d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:04.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.494+0000 7f9208d8a640 1 -- 172.21.15.73:0/2305584451 >> v1:172.21.15.120:6789/0 conn(0x7f92040d3bc0 legacy=0x7f92040b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:04.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.494+0000 7f9208d8a640 1 -- 172.21.15.73:0/2305584451 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92040d07d0 con 0x7f9204162880 2024-04-26T20:51:04.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.494+0000 7f9208d8a640 1 -- 172.21.15.73:0/2305584451 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7f91f4001670 con 0x7f9204162880 2024-04-26T20:51:04.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.494+0000 7f9208d8a640 1 -- 172.21.15.73:0/2305584451 >> v1:172.21.15.73:6789/0 conn(0x7f9204162880 legacy=0x7f92040d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:04.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.494+0000 7f9208d8a640 1 --2- 172.21.15.73:0/2305584451 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91f00039b0 0x7f91f0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:04.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.494+0000 7f9208d8a640 1 --2- 172.21.15.73:0/2305584451 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91f0004800 0x7f91f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:04.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.494+0000 7f9208d8a640 1 --2- 172.21.15.73:0/2305584451 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f91f00051c0 0x7f91f000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:04.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.494+0000 7f9208d8a640 1 -- 172.21.15.73:0/2305584451 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f92040d0580 con 0x7f91f0004800 2024-04-26T20:51:04.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.494+0000 7f9208d8a640 1 -- 172.21.15.73:0/2305584451 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f91f000c060 con 0x7f91f00039b0 2024-04-26T20:51:04.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.494+0000 7f9208d8a640 1 -- 172.21.15.73:0/2305584451 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f91f000c260 con 0x7f91f00051c0 2024-04-26T20:51:04.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.494+0000 7f920958b640 1 --2- 172.21.15.73:0/2305584451 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91f0004800 0x7f91f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:04.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.494+0000 7f9209d8c640 1 --2- 172.21.15.73:0/2305584451 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91f00039b0 0x7f91f0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:04.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.495+0000 7f920a58d640 1 --2- 172.21.15.73:0/2305584451 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f91f00051c0 0x7f91f000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:04.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.495+0000 7f920958b640 1 -- 172.21.15.73:0/2305584451 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f91f00051c0 msgr2=0x7f91f000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:04.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.495+0000 7f920958b640 1 --2- 172.21.15.73:0/2305584451 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f91f00051c0 0x7f91f000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:04.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.495+0000 7f920958b640 1 -- 172.21.15.73:0/2305584451 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91f00039b0 msgr2=0x7f91f0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:04.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.495+0000 7f920958b640 1 --2- 172.21.15.73:0/2305584451 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f91f00039b0 0x7f91f0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:04.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.495+0000 7f920958b640 1 -- 172.21.15.73:0/2305584451 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91f0003520 con 0x7f91f0004800 2024-04-26T20:51:04.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.495+0000 7f920a58d640 1 --2- 172.21.15.73:0/2305584451 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f91f00051c0 0x7f91f000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:04.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.495+0000 7f920a58d640 1 -- 172.21.15.73:0/2305584451 reap_dead start 2024-04-26T20:51:04.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.495+0000 7f920958b640 1 --2- 172.21.15.73:0/2305584451 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91f0004800 0x7f91f0004c80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9200001750 tx=0x7f91f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf38f5c8ec3fb870 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:04.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.495+0000 7f9208d8a640 1 -- 172.21.15.73:0/2305584451 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9200001370 con 0x7f91f0004800 2024-04-26T20:51:04.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.495+0000 7f9208d8a640 1 -- 172.21.15.73:0/2305584451 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92000052d0 con 0x7f91f0004800 2024-04-26T20:51:04.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.495+0000 7f9208d8a640 1 -- 172.21.15.73:0/2305584451 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920002abc0 con 0x7f91f0004800 2024-04-26T20:51:04.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.496+0000 7f920bff0640 1 -- 172.21.15.73:0/2305584451 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91f0004800 msgr2=0x7f91f0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:04.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.496+0000 7f920bff0640 1 --2- 172.21.15.73:0/2305584451 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91f0004800 0x7f91f0004c80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9200001750 tx=0x7f91f0005830 comp rx=0 tx=0).stop 2024-04-26T20:51:04.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.496+0000 7f920bff0640 1 -- 172.21.15.73:0/2305584451 shutdown_connections 2024-04-26T20:51:04.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.496+0000 7f920bff0640 1 --2- 172.21.15.73:0/2305584451 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91f0004800 0x7f91f0004c80 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:04.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.496+0000 7f920bff0640 1 -- 172.21.15.73:0/2305584451 >> 172.21.15.73:0/2305584451 conn(0x7f92040d4f60 msgr2=0x7f92040d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:04.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.496+0000 7f920bff0640 1 -- 172.21.15.73:0/2305584451 shutdown_connections 2024-04-26T20:51:04.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.496+0000 7f920bff0640 1 -- 172.21.15.73:0/2305584451 wait complete. 2024-04-26T20:51:04.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.497+0000 7f920bff0640 1 Processor -- start 2024-04-26T20:51:04.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.497+0000 7f920bff0640 1 -- start start 2024-04-26T20:51:04.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.498+0000 7f920bff0640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91f0004800 0x7f92041f5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:04.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.498+0000 7f920bff0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92040ceec0 0x7f920414eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:04.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.498+0000 7f920bff0640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f920414f060 0x7f920406df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:04.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.498+0000 7f920bff0640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9204162c60 con 0x7f91f0004800 2024-04-26T20:51:04.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.498+0000 7f920bff0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f92040d0d90 con 0x7f92040ceec0 2024-04-26T20:51:04.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.498+0000 7f920bff0640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f9204162de0 con 0x7f920414f060 2024-04-26T20:51:04.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.498+0000 7f9209d8c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91f0004800 0x7f92041f5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:04.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.498+0000 7f9209d8c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91f0004800 0x7f92041f5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42812/0 (socket says 172.21.15.73:42812) 2024-04-26T20:51:04.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.498+0000 7f9209d8c640 1 -- 172.21.15.73:0/3605405652 learned_addr learned my addr 172.21.15.73:0/3605405652 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:04.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.498+0000 7f920a58d640 1 --2- 172.21.15.73:0/3605405652 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f920414f060 0x7f920406df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:04.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.498+0000 7f9209d8c640 1 -- 172.21.15.73:0/3605405652 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f920414f060 msgr2=0x7f920406df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:04.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.498+0000 7f920958b640 1 --2- 172.21.15.73:0/3605405652 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92040ceec0 0x7f920414eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:04.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.498+0000 7f9209d8c640 1 --2- 172.21.15.73:0/3605405652 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f920414f060 0x7f920406df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:04.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.498+0000 7f9209d8c640 1 -- 172.21.15.73:0/3605405652 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92040ceec0 msgr2=0x7f920414eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:04.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.498+0000 7f9209d8c640 1 --2- 172.21.15.73:0/3605405652 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92040ceec0 0x7f920414eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:04.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.498+0000 7f9209d8c640 1 -- 172.21.15.73:0/3605405652 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f920414f5a0 con 0x7f91f0004800 2024-04-26T20:51:04.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.498+0000 7f920958b640 1 --2- 172.21.15.73:0/3605405652 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92040ceec0 0x7f920414eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:04.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.498+0000 7f920a58d640 1 --2- 172.21.15.73:0/3605405652 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f920414f060 0x7f920406df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:04.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.498+0000 7f9209d8c640 1 --2- 172.21.15.73:0/3605405652 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91f0004800 0x7f92041f5780 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f91f40072c0 tx=0x7f91f4007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:04.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.499+0000 7f91fa7fc640 1 -- 172.21.15.73:0/3605405652 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f400f960 con 0x7f91f0004800 2024-04-26T20:51:04.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.499+0000 7f91fa7fc640 1 -- 172.21.15.73:0/3605405652 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91f4018070 con 0x7f91f0004800 2024-04-26T20:51:04.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.499+0000 7f920bff0640 1 -- 172.21.15.73:0/3605405652 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f920414f880 con 0x7f91f0004800 2024-04-26T20:51:04.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.499+0000 7f91fa7fc640 1 -- 172.21.15.73:0/3605405652 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f400fd60 con 0x7f91f0004800 2024-04-26T20:51:04.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.499+0000 7f920bff0640 1 -- 172.21.15.73:0/3605405652 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92041f4c90 con 0x7f91f0004800 2024-04-26T20:51:04.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.500+0000 7f920bff0640 1 -- 172.21.15.73:0/3605405652 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f920414fa50 con 0x7f91f0004800 2024-04-26T20:51:04.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.500+0000 7f91fa7fc640 1 -- 172.21.15.73:0/3605405652 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f91f401d460 con 0x7f91f0004800 2024-04-26T20:51:04.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.500+0000 7f91fa7fc640 1 --2- 172.21.15.73:0/3605405652 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f91e806c920 0x7f91e806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:04.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.501+0000 7f91fa7fc640 1 -- 172.21.15.73:0/3605405652 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f91f408be80 con 0x7f91f0004800 2024-04-26T20:51:04.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.501+0000 7f920958b640 1 --2- 172.21.15.73:0/3605405652 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f91e806c920 0x7f91e806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:04.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.501+0000 7f920958b640 1 --2- 172.21.15.73:0/3605405652 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f91e806c920 0x7f91e806ede0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9200032000 tx=0x7f920002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:04.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.505+0000 7f91fa7fc640 1 -- 172.21.15.73:0/3605405652 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f91f4059f70 con 0x7f91f0004800 2024-04-26T20:51:04.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.687+0000 7f920bff0640 1 -- 172.21.15.73:0/3605405652 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f92040694f0 con 0x7f91f0004800 2024-04-26T20:51:04.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.688+0000 7f91fa7fc640 1 -- 172.21.15.73:0/3605405652 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18514 (secure 0 0 0) 0x7f91f4059870 con 0x7f91f0004800 2024-04-26T20:51:04.689 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:51:04.690 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":25,"fsid":"fbd3f548-b547-4713-856b-09f98f5d1764","created":"2024-04-26T20:50:07.876786+0000","modified":"2024-04-26T20:50:38.657510+0000","last_up_change":"2024-04-26T20:50:19.082741+0000","last_in_change":"2024-04-26T20:50:10.018400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:50:19.993731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:50:29.964420+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:50:30.920082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:50:34.650207+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"72d6f4f3-214a-4392-a02f-c33cbcab3c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6809","nonce":3461705124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6811","nonce":3461705124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6815","nonce":3461705124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6813","nonce":3461705124}]},"public_addr":"172.21.15.73:6809/3461705124","cluster_addr":"172.21.15.73:6811/3461705124","heartbeat_back_addr":"172.21.15.73:6815/3461705124","heartbeat_front_addr":"172.21.15.73:6813/3461705124","state":["exists","up"]},{"osd":1,"uuid":"786bf8ca-6ddc-4b3c-be6c-9152398bd5fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6817","nonce":2827619403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6819","nonce":2827619403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6825","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6827","nonce":2827619403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6821","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6823","nonce":2827619403}]},"public_addr":"172.21.15.73:6817/2827619403","cluster_addr":"172.21.15.73:6819/2827619403","heartbeat_back_addr":"172.21.15.73:6827/2827619403","heartbeat_front_addr":"172.21.15.73:6823/2827619403","state":["exists","up"]},{"osd":2,"uuid":"90963b1a-5437-4ef8-ad8b-6b74c534eeb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6801","nonce":2542681064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6803","nonce":2542681064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6808","nonce":2542681064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6805","nonce":2542681064}]},"public_addr":"172.21.15.73:6801/2542681064","cluster_addr":"172.21.15.73:6803/2542681064","heartbeat_back_addr":"172.21.15.73:6808/2542681064","heartbeat_front_addr":"172.21.15.73:6805/2542681064","state":["exists","up"]},{"osd":3,"uuid":"bd3d2148-6e38-41b4-a453-fe6e3b8b4ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6822","nonce":1604042718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6826","nonce":1604042718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6831","nonce":1604042718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6829","nonce":1604042718}]},"public_addr":"172.21.15.73:6822/1604042718","cluster_addr":"172.21.15.73:6826/1604042718","heartbeat_back_addr":"172.21.15.73:6831/1604042718","heartbeat_front_addr":"172.21.15.73:6829/1604042718","state":["exists","up"]},{"osd":4,"uuid":"e8668dc8-a23f-46d2-8311-cb882da08e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6817","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6819","nonce":3591055561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6824","nonce":3591055561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6831","nonce":3591055561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6828","nonce":3591055561}]},"public_addr":"172.21.15.120:6819/3591055561","cluster_addr":"172.21.15.120:6824/3591055561","heartbeat_back_addr":"172.21.15.120:6831/3591055561","heartbeat_front_addr":"172.21.15.120:6828/3591055561","state":["exists","up"]},{"osd":5,"uuid":"cbac0bdb-96bd-4852-bcce-e564495c3fcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6809","nonce":819450433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6811","nonce":819450433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6815","nonce":819450433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6813","nonce":819450433}]},"public_addr":"172.21.15.120:6809/819450433","cluster_addr":"172.21.15.120:6811/819450433","heartbeat_back_addr":"172.21.15.120:6815/819450433","heartbeat_front_addr":"172.21.15.120:6813/819450433","state":["exists","up"]},{"osd":6,"uuid":"1884b2a8-569f-4272-96a9-f25a09364dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6801","nonce":3946210727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6803","nonce":3946210727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6807","nonce":3946210727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6805","nonce":3946210727}]},"public_addr":"172.21.15.120:6801/3946210727","cluster_addr":"172.21.15.120:6803/3946210727","heartbeat_back_addr":"172.21.15.120:6807/3946210727","heartbeat_front_addr":"172.21.15.120:6805/3946210727","state":["exists","up"]},{"osd":7,"uuid":"c0c3bf6d-a4c5-4f1e-aa2d-330951830666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6818","nonce":2915574674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6821","nonce":2915574674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6827","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6829","nonce":2915574674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6823","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6825","nonce":2915574674}]},"public_addr":"172.21.15.120:6818/2915574674","cluster_addr":"172.21.15.120:6821/2915574674","heartbeat_back_addr":"172.21.15.120:6829/2915574674","heartbeat_front_addr":"172.21.15.120:6825/2915574674","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.218216+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.208710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.112705+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.182789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.164324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.174212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.225396+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.261559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:51:04.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.690+0000 7f920bff0640 1 -- 172.21.15.73:0/3605405652 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f91e806c920 msgr2=0x7f91e806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:04.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.690+0000 7f920bff0640 1 --2- 172.21.15.73:0/3605405652 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f91e806c920 0x7f91e806ede0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9200032000 tx=0x7f920002e000 comp rx=0 tx=0).stop 2024-04-26T20:51:04.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.690+0000 7f920bff0640 1 -- 172.21.15.73:0/3605405652 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91f0004800 msgr2=0x7f92041f5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:04.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.690+0000 7f920bff0640 1 --2- 172.21.15.73:0/3605405652 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91f0004800 0x7f92041f5780 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f91f40072c0 tx=0x7f91f4007780 comp rx=0 tx=0).stop 2024-04-26T20:51:04.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.690+0000 7f920bff0640 1 -- 172.21.15.73:0/3605405652 shutdown_connections 2024-04-26T20:51:04.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.690+0000 7f920bff0640 1 --2- 172.21.15.73:0/3605405652 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f91e806c920 0x7f91e806ede0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:04.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.690+0000 7f920bff0640 1 --2- 172.21.15.73:0/3605405652 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f920414f060 0x7f920406df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:04.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.690+0000 7f920bff0640 1 --2- 172.21.15.73:0/3605405652 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f92040ceec0 0x7f920414eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:04.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.690+0000 7f920bff0640 1 --2- 172.21.15.73:0/3605405652 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91f0004800 0x7f92041f5780 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:04.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.690+0000 7f920bff0640 1 -- 172.21.15.73:0/3605405652 >> 172.21.15.73:0/3605405652 conn(0x7f92040d4f60 msgr2=0x7f92040ba210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:04.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.690+0000 7f920bff0640 1 -- 172.21.15.73:0/3605405652 shutdown_connections 2024-04-26T20:51:04.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.690+0000 7f920bff0640 1 -- 172.21.15.73:0/3605405652 wait complete. 2024-04-26T20:51:04.702 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:51:04.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.799+0000 7fa328146640 1 Processor -- start 2024-04-26T20:51:04.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.799+0000 7fa328146640 1 -- start start 2024-04-26T20:51:04.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.800+0000 7fa328146640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa320162810 0x7fa320070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:04.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.800+0000 7fa328146640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa3200b2340 0x7fa3200cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:04.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.800+0000 7fa328146640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fa3200cd930 0x7fa3200d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:04.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.800+0000 7fa328146640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7fa3200d0750 con 0x7fa320162810 2024-04-26T20:51:04.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.800+0000 7fa328146640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fa3200d16a0 con 0x7fa3200cd930 2024-04-26T20:51:04.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.800+0000 7fa328146640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fa3200d10d0 con 0x7fa3200b2340 2024-04-26T20:51:04.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.800+0000 7fa325ee2640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa320162810 0x7fa320070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:04.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.800+0000 7fa325ee2640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa320162810 0x7fa320070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42824/0 (socket says 172.21.15.73:42824) 2024-04-26T20:51:04.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.800+0000 7fa325ee2640 1 -- 172.21.15.73:0/888859807 learned_addr learned my addr 172.21.15.73:0/888859807 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:04.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.800+0000 7fa3256e1640 1 --2- 172.21.15.73:0/888859807 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa3200b2340 0x7fa3200cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:04.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.800+0000 7fa3266e3640 1 --2- 172.21.15.73:0/888859807 >> v2:172.21.15.120:3300/0 conn(0x7fa3200cd930 0x7fa3200d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:04.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.800+0000 7fa325ee2640 1 -- 172.21.15.73:0/888859807 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa3200b2340 msgr2=0x7fa3200cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:04.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.800+0000 7fa325ee2640 1 --2- 172.21.15.73:0/888859807 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa3200b2340 0x7fa3200cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:04.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.800+0000 7fa325ee2640 1 -- 172.21.15.73:0/888859807 >> v2:172.21.15.120:3300/0 conn(0x7fa3200cd930 msgr2=0x7fa3200d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:04.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.800+0000 7fa325ee2640 1 --2- 172.21.15.73:0/888859807 >> v2:172.21.15.120:3300/0 conn(0x7fa3200cd930 0x7fa3200d1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:04.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.800+0000 7fa325ee2640 1 -- 172.21.15.73:0/888859807 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3200d1370 con 0x7fa320162810 2024-04-26T20:51:04.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.800+0000 7fa3256e1640 1 --2- 172.21.15.73:0/888859807 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa3200b2340 0x7fa3200cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:04.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.800+0000 7fa3266e3640 1 --2- 172.21.15.73:0/888859807 >> v2:172.21.15.120:3300/0 conn(0x7fa3200cd930 0x7fa3200d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:04.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.801+0000 7fa325ee2640 1 --2- 172.21.15.73:0/888859807 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa320162810 0x7fa320070350 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa3200b2250 tx=0x7fa31c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4bdda1fc581e1c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:04.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.801+0000 7fa324ee0640 1 -- 172.21.15.73:0/888859807 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31c037070 con 0x7fa320162810 2024-04-26T20:51:04.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.801+0000 7fa324ee0640 1 -- 172.21.15.73:0/888859807 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa31c032440 con 0x7fa320162810 2024-04-26T20:51:04.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.801+0000 7fa324ee0640 1 -- 172.21.15.73:0/888859807 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31c032810 con 0x7fa320162810 2024-04-26T20:51:04.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.801+0000 7fa328146640 1 -- 172.21.15.73:0/888859807 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa320162810 msgr2=0x7fa320070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:04.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.801+0000 7fa328146640 1 --2- 172.21.15.73:0/888859807 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa320162810 0x7fa320070350 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa3200b2250 tx=0x7fa31c02a1e0 comp rx=0 tx=0).stop 2024-04-26T20:51:04.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.801+0000 7fa328146640 1 -- 172.21.15.73:0/888859807 shutdown_connections 2024-04-26T20:51:04.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.801+0000 7fa328146640 1 --2- 172.21.15.73:0/888859807 >> v2:172.21.15.120:3300/0 conn(0x7fa3200cd930 0x7fa3200d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:04.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.801+0000 7fa328146640 1 --2- 172.21.15.73:0/888859807 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa3200b2340 0x7fa3200cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:04.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.801+0000 7fa328146640 1 --2- 172.21.15.73:0/888859807 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa320162810 0x7fa320070350 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:04.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.801+0000 7fa328146640 1 -- 172.21.15.73:0/888859807 >> 172.21.15.73:0/888859807 conn(0x7fa3200d2da0 msgr2=0x7fa3200d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:04.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.801+0000 7fa328146640 1 -- 172.21.15.73:0/888859807 shutdown_connections 2024-04-26T20:51:04.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.801+0000 7fa328146640 1 -- 172.21.15.73:0/888859807 wait complete. 2024-04-26T20:51:04.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.803+0000 7fa328146640 1 Processor -- start 2024-04-26T20:51:04.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.803+0000 7fa328146640 1 -- start start 2024-04-26T20:51:04.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.803+0000 7fa328146640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa3200b2340 0x7fa320161060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:04.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.803+0000 7fa328146640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa3200cd930 0x7fa32009ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:04.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.803+0000 7fa328146640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa320162810 0x7fa3200c80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:04.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.803+0000 7fa328146640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa320070890 con 0x7fa3200cd930 2024-04-26T20:51:04.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.803+0000 7fa328146640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa3200d0470 con 0x7fa320162810 2024-04-26T20:51:04.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.803+0000 7fa328146640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fa3200d2070 con 0x7fa3200b2340 2024-04-26T20:51:04.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.803+0000 7fa3256e1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa3200cd930 0x7fa32009ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:04.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.803+0000 7fa3256e1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa3200cd930 0x7fa32009ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42838/0 (socket says 172.21.15.73:42838) 2024-04-26T20:51:04.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.803+0000 7fa3256e1640 1 -- 172.21.15.73:0/621345793 learned_addr learned my addr 172.21.15.73:0/621345793 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:04.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.803+0000 7fa325ee2640 1 --2- 172.21.15.73:0/621345793 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa3200b2340 0x7fa320161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:04.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.803+0000 7fa3266e3640 1 --2- 172.21.15.73:0/621345793 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa320162810 0x7fa3200c80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:04.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.804+0000 7fa3256e1640 1 -- 172.21.15.73:0/621345793 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa3200b2340 msgr2=0x7fa320161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:04.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.804+0000 7fa3256e1640 1 --2- 172.21.15.73:0/621345793 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa3200b2340 0x7fa320161060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:04.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.804+0000 7fa3256e1640 1 -- 172.21.15.73:0/621345793 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa320162810 msgr2=0x7fa3200c80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:04.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.804+0000 7fa3256e1640 1 --2- 172.21.15.73:0/621345793 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa320162810 0x7fa3200c80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:04.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.804+0000 7fa3256e1640 1 -- 172.21.15.73:0/621345793 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3200cb040 con 0x7fa3200cd930 2024-04-26T20:51:04.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.804+0000 7fa3266e3640 1 --2- 172.21.15.73:0/621345793 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa320162810 0x7fa3200c80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:04.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.804+0000 7fa3256e1640 1 --2- 172.21.15.73:0/621345793 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa3200cd930 0x7fa32009ee50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa31000c7f0 tx=0x7fa31000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:04.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.804+0000 7fa3167fc640 1 -- 172.21.15.73:0/621345793 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa310015070 con 0x7fa3200cd930 2024-04-26T20:51:04.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.804+0000 7fa3167fc640 1 -- 172.21.15.73:0/621345793 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa310007a30 con 0x7fa3200cd930 2024-04-26T20:51:04.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.804+0000 7fa328146640 1 -- 172.21.15.73:0/621345793 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3200cb320 con 0x7fa3200cd930 2024-04-26T20:51:04.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.804+0000 7fa3167fc640 1 -- 172.21.15.73:0/621345793 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa310007e00 con 0x7fa3200cd930 2024-04-26T20:51:04.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.804+0000 7fa328146640 1 -- 172.21.15.73:0/621345793 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3200cb8e0 con 0x7fa3200cd930 2024-04-26T20:51:04.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.805+0000 7fa328146640 1 -- 172.21.15.73:0/621345793 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3200b3e70 con 0x7fa3200cd930 2024-04-26T20:51:04.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.805+0000 7fa3167fc640 1 -- 172.21.15.73:0/621345793 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa310018070 con 0x7fa3200cd930 2024-04-26T20:51:04.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.806+0000 7fa3167fc640 1 --2- 172.21.15.73:0/621345793 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa30406c8f0 0x7fa30406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:04.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.806+0000 7fa3167fc640 1 -- 172.21.15.73:0/621345793 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa31008bc70 con 0x7fa3200cd930 2024-04-26T20:51:04.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.806+0000 7fa325ee2640 1 --2- 172.21.15.73:0/621345793 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa30406c8f0 0x7fa30406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:04.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.806+0000 7fa325ee2640 1 --2- 172.21.15.73:0/621345793 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa30406c8f0 0x7fa30406edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa31c034000 tx=0x7fa31c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:04.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:04.810+0000 7fa3167fc640 1 -- 172.21.15.73:0/621345793 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa3200b3e70 con 0x7fa3200cd930 2024-04-26T20:51:05.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.024+0000 7fa328146640 1 -- 172.21.15.73:0/621345793 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3200694f0 con 0x7fa3200cd930 2024-04-26T20:51:05.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.024+0000 7fa3167fc640 1 -- 172.21.15.73:0/621345793 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5180 (secure 0 0 0) 0x7fa3100597a0 con 0x7fa3200cd930 2024-04-26T20:51:05.025 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:51:05.026 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/4263167680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6833","nonce":4263167680},{"type":"v1","addr":"172.21.15.120:6835","nonce":4263167680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6834/1614342510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1614342510},{"type":"v1","addr":"172.21.15.120:6834","nonce":1614342510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2146325725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2146325725},{"type":"v1","addr":"172.21.15.73:6835","nonce":2146325725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:50:31.964317+0000","modified":"2024-04-26T20:50:54.049029+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/181202544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":181202544},{"type":"v1","addr":"172.21.15.73:6837","nonce":181202544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4281,"qdb_cluster":[4281]},"id":1}]} 2024-04-26T20:51:05.026 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 8 2024-04-26T20:51:05.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.027+0000 7fa328146640 1 -- 172.21.15.73:0/621345793 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa30406c8f0 msgr2=0x7fa30406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:05.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.027+0000 7fa328146640 1 --2- 172.21.15.73:0/621345793 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa30406c8f0 0x7fa30406edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa31c034000 tx=0x7fa31c02e000 comp rx=0 tx=0).stop 2024-04-26T20:51:05.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.027+0000 7fa328146640 1 -- 172.21.15.73:0/621345793 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa3200cd930 msgr2=0x7fa32009ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:05.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.027+0000 7fa328146640 1 --2- 172.21.15.73:0/621345793 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa3200cd930 0x7fa32009ee50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa31000c7f0 tx=0x7fa31000ccb0 comp rx=0 tx=0).stop 2024-04-26T20:51:05.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.027+0000 7fa328146640 1 -- 172.21.15.73:0/621345793 shutdown_connections 2024-04-26T20:51:05.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.027+0000 7fa328146640 1 --2- 172.21.15.73:0/621345793 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa3200cd930 0x7fa32009ee50 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:05.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.027+0000 7fa328146640 1 --2- 172.21.15.73:0/621345793 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa30406c8f0 0x7fa30406edb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:05.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.027+0000 7fa328146640 1 --2- 172.21.15.73:0/621345793 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa320162810 0x7fa3200c80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:05.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.027+0000 7fa328146640 1 --2- 172.21.15.73:0/621345793 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa3200b2340 0x7fa320161060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:05.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.027+0000 7fa328146640 1 -- 172.21.15.73:0/621345793 >> 172.21.15.73:0/621345793 conn(0x7fa3200d2da0 msgr2=0x7fa320070b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:05.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.027+0000 7fa328146640 1 -- 172.21.15.73:0/621345793 shutdown_connections 2024-04-26T20:51:05.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.027+0000 7fa328146640 1 -- 172.21.15.73:0/621345793 wait complete. 2024-04-26T20:51:05.441 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T20:50:31.964317+0000', 'modified': '2024-04-26T20:50:54.049029+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4281}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4281': {'gid': 4281, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.73:6837/181202544', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6836', 'nonce': 181202544}, {'type': 'v1', 'addr': '172.21.15.73:6837', 'nonce': 181202544}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4281, 'qdb_cluster': [4281]} 2024-04-26T20:51:05.442 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T20:51:05.442 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-26T20:51:05.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.540+0000 7f9662a37640 1 Processor -- start 2024-04-26T20:51:05.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.540+0000 7f9662a37640 1 -- start start 2024-04-26T20:51:05.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.541+0000 7f9662a37640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f965c0b0380 0x7f965c0ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:05.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.541+0000 7f9662a37640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f965c0cfd80 con 0x7f965c0b10b0 2024-04-26T20:51:05.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.541+0000 7f9662a37640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f965c0d0020 con 0x7f965c06c950 2024-04-26T20:51:05.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.541+0000 7f9662a37640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f965c0af8e0 con 0x7f965c0b0380 2024-04-26T20:51:05.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.541+0000 7f965b7fe640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f965c0b10b0 0x7f965c06c240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:42096/0 (socket says 172.21.15.73:42096) 2024-04-26T20:51:05.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.541+0000 7f965b7fe640 1 -- 172.21.15.73:0/1432302913 learned_addr learned my addr 172.21.15.73:0/1432302913 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:05.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.541+0000 7f965bfff640 1 --2- 172.21.15.73:0/1432302913 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f965c0b0380 0x7f965c0ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:05.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.541+0000 7f965affd640 1 -- 172.21.15.73:0/1432302913 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3081579848 0 0) 0x7f965c0cfd80 con 0x7f965c0b10b0 2024-04-26T20:51:05.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.541+0000 7f965affd640 1 -- 172.21.15.73:0/1432302913 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9644003520 con 0x7f965c0b10b0 2024-04-26T20:51:05.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.541+0000 7f965affd640 1 -- 172.21.15.73:0/1432302913 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 266052037 0 0) 0x7f965c0d0020 con 0x7f965c06c950 2024-04-26T20:51:05.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.541+0000 7f965affd640 1 -- 172.21.15.73:0/1432302913 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f965c0cfd80 con 0x7f965c06c950 2024-04-26T20:51:05.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.541+0000 7f965affd640 1 -- 172.21.15.73:0/1432302913 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3854995093 0 0) 0x7f9644003520 con 0x7f965c0b10b0 2024-04-26T20:51:05.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.541+0000 7f965affd640 1 -- 172.21.15.73:0/1432302913 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f965c0b0380 msgr2=0x7f965c0ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:05.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.541+0000 7f965affd640 1 --2- 172.21.15.73:0/1432302913 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f965c0b0380 0x7f965c0ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:05.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.541+0000 7f965affd640 1 -- 172.21.15.73:0/1432302913 >> v1:172.21.15.120:6789/0 conn(0x7f965c06c950 legacy=0x7f965c15e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:05.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.541+0000 7f965affd640 1 -- 172.21.15.73:0/1432302913 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f965c0d0300 con 0x7f965c0b10b0 2024-04-26T20:51:05.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.541+0000 7f965affd640 1 -- 172.21.15.73:0/1432302913 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7f9650001670 con 0x7f965c0b10b0 2024-04-26T20:51:05.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.542+0000 7f965bfff640 1 --2- 172.21.15.73:0/1432302913 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f965c0b0380 0x7f965c0ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:51:05.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.542+0000 7f965affd640 1 -- 172.21.15.73:0/1432302913 >> v1:172.21.15.73:6789/0 conn(0x7f965c0b10b0 legacy=0x7f965c06c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:05.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.542+0000 7f965affd640 1 --2- 172.21.15.73:0/1432302913 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9644003b70 0x7f9644003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:05.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.542+0000 7f965affd640 1 --2- 172.21.15.73:0/1432302913 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f965c0b0380 0x7f9644004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:05.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.542+0000 7f965affd640 1 --2- 172.21.15.73:0/1432302913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9644004d40 0x7f96440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:05.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.542+0000 7f965affd640 1 -- 172.21.15.73:0/1432302913 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f965c0af8e0 con 0x7f9644004d40 2024-04-26T20:51:05.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.542+0000 7f965affd640 1 -- 172.21.15.73:0/1432302913 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f964400bad0 con 0x7f9644003b70 2024-04-26T20:51:05.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.542+0000 7f965affd640 1 -- 172.21.15.73:0/1432302913 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f964400bcd0 con 0x7f965c0b0380 2024-04-26T20:51:05.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.542+0000 7f9660fd4640 1 --2- 172.21.15.73:0/1432302913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9644004d40 0x7f96440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:05.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.542+0000 7f965bfff640 1 --2- 172.21.15.73:0/1432302913 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9644003b70 0x7f9644003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:05.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.542+0000 7f965b7fe640 1 --2- 172.21.15.73:0/1432302913 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f965c0b0380 0x7f9644004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:05.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.542+0000 7f9660fd4640 1 -- 172.21.15.73:0/1432302913 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f965c0b0380 msgr2=0x7f9644004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:05.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.542+0000 7f9660fd4640 1 --2- 172.21.15.73:0/1432302913 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f965c0b0380 0x7f9644004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:05.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.542+0000 7f9660fd4640 1 -- 172.21.15.73:0/1432302913 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9644003b70 msgr2=0x7f9644003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:05.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.542+0000 7f9660fd4640 1 --2- 172.21.15.73:0/1432302913 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9644003b70 0x7f9644003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:05.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.542+0000 7f9660fd4640 1 -- 172.21.15.73:0/1432302913 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9644003520 con 0x7f9644004d40 2024-04-26T20:51:05.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.543+0000 7f9660fd4640 1 --2- 172.21.15.73:0/1432302913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9644004d40 0x7f96440051c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9654001fa0 tx=0x7f9644005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2060ab2ed3c2913 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:05.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.543+0000 7f965affd640 1 -- 172.21.15.73:0/1432302913 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9654001bf0 con 0x7f9644004d40 2024-04-26T20:51:05.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.543+0000 7f965affd640 1 -- 172.21.15.73:0/1432302913 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9654003b70 con 0x7f9644004d40 2024-04-26T20:51:05.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.543+0000 7f965affd640 1 -- 172.21.15.73:0/1432302913 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96540050d0 con 0x7f9644004d40 2024-04-26T20:51:05.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.543+0000 7f9662a37640 1 -- 172.21.15.73:0/1432302913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9644004d40 msgr2=0x7f96440051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:05.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.543+0000 7f9662a37640 1 --2- 172.21.15.73:0/1432302913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9644004d40 0x7f96440051c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9654001fa0 tx=0x7f9644005830 comp rx=0 tx=0).stop 2024-04-26T20:51:05.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.543+0000 7f9660fd4640 1 -- 172.21.15.73:0/1432302913 reap_dead start 2024-04-26T20:51:05.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.543+0000 7f9662a37640 1 -- 172.21.15.73:0/1432302913 shutdown_connections 2024-04-26T20:51:05.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.543+0000 7f9662a37640 1 -- 172.21.15.73:0/1432302913 >> 172.21.15.73:0/1432302913 conn(0x7f965c0d1cf0 msgr2=0x7f965c0d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:05.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.543+0000 7f9662a37640 1 -- 172.21.15.73:0/1432302913 shutdown_connections 2024-04-26T20:51:05.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.543+0000 7f9662a37640 1 -- 172.21.15.73:0/1432302913 wait complete. 2024-04-26T20:51:05.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.545+0000 7f9662a37640 1 Processor -- start 2024-04-26T20:51:05.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.545+0000 7f9662a37640 1 -- start start 2024-04-26T20:51:05.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.545+0000 7f9662a37640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9644004d40 0x7f965c0cb660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:05.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.545+0000 7f9662a37640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f965c14e460 0x7f965c1f3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:05.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.545+0000 7f9662a37640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f965c0c9030 0x7f965c0c7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:05.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.545+0000 7f9662a37640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f965c0b0760 con 0x7f9644004d40 2024-04-26T20:51:05.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.545+0000 7f9662a37640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f965c0d1980 con 0x7f965c0c9030 2024-04-26T20:51:05.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.545+0000 7f9662a37640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f965c0d15c0 con 0x7f965c14e460 2024-04-26T20:51:05.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.545+0000 7f965bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9644004d40 0x7f965c0cb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:05.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.545+0000 7f965bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9644004d40 0x7f965c0cb660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42858/0 (socket says 172.21.15.73:42858) 2024-04-26T20:51:05.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.545+0000 7f965bfff640 1 -- 172.21.15.73:0/3355889478 learned_addr learned my addr 172.21.15.73:0/3355889478 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:05.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.546+0000 7f965b7fe640 1 --2- 172.21.15.73:0/3355889478 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f965c14e460 0x7f965c1f3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:05.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.546+0000 7f9660fd4640 1 --2- 172.21.15.73:0/3355889478 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f965c0c9030 0x7f965c0c7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:05.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.546+0000 7f965bfff640 1 -- 172.21.15.73:0/3355889478 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f965c14e460 msgr2=0x7f965c1f3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:05.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.546+0000 7f965bfff640 1 --2- 172.21.15.73:0/3355889478 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f965c14e460 0x7f965c1f3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:05.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.546+0000 7f965bfff640 1 -- 172.21.15.73:0/3355889478 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f965c0c9030 msgr2=0x7f965c0c7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:05.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.546+0000 7f965bfff640 1 --2- 172.21.15.73:0/3355889478 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f965c0c9030 0x7f965c0c7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:05.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.546+0000 7f965bfff640 1 -- 172.21.15.73:0/3355889478 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f965c1f48b0 con 0x7f9644004d40 2024-04-26T20:51:05.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.546+0000 7f965b7fe640 1 --2- 172.21.15.73:0/3355889478 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f965c14e460 0x7f965c1f3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:05.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.546+0000 7f965bfff640 1 --2- 172.21.15.73:0/3355889478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9644004d40 0x7f965c0cb660 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f964c011a00 tx=0x7f964c011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:05.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.546+0000 7f9658ff9640 1 -- 172.21.15.73:0/3355889478 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964c00faf0 con 0x7f9644004d40 2024-04-26T20:51:05.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.546+0000 7f9658ff9640 1 -- 172.21.15.73:0/3355889478 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f964c00fc90 con 0x7f9644004d40 2024-04-26T20:51:05.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.546+0000 7f9658ff9640 1 -- 172.21.15.73:0/3355889478 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964c004030 con 0x7f9644004d40 2024-04-26T20:51:05.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.546+0000 7f9662a37640 1 -- 172.21.15.73:0/3355889478 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f965c1f4b90 con 0x7f9644004d40 2024-04-26T20:51:05.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.546+0000 7f9662a37640 1 -- 172.21.15.73:0/3355889478 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f965c0cb140 con 0x7f9644004d40 2024-04-26T20:51:05.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.547+0000 7f9658ff9640 1 -- 172.21.15.73:0/3355889478 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f964c012040 con 0x7f9644004d40 2024-04-26T20:51:05.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.548+0000 7f9658ff9640 1 --2- 172.21.15.73:0/3355889478 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f964806c8f0 0x7f964806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:05.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.548+0000 7f9658ff9640 1 -- 172.21.15.73:0/3355889478 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f964c08c2a0 con 0x7f9644004d40 2024-04-26T20:51:05.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.548+0000 7f965b7fe640 1 --2- 172.21.15.73:0/3355889478 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f964806c8f0 0x7f964806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:05.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.548+0000 7f965b7fe640 1 --2- 172.21.15.73:0/3355889478 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f964806c8f0 0x7f964806edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9650001200 tx=0x7f9650009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:05.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.549+0000 7f966582b740 1 Processor -- start 2024-04-26T20:51:05.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.549+0000 7f966582b740 1 -- start start 2024-04-26T20:51:05.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.550+0000 7f966582b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec1531b440 0x55ec1531b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:05.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.550+0000 7f966582b740 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55ec15338ce0 0x55ec15339230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:05.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.550+0000 7f966582b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec152f1120 0x55ec153373a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:05.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.550+0000 7f966582b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ec15255740 con 0x55ec152f1120 2024-04-26T20:51:05.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.550+0000 7f966582b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ec151f13a0 con 0x55ec1531b440 2024-04-26T20:51:05.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.550+0000 7f966582b740 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x55ec15251700 con 0x55ec15338ce0 2024-04-26T20:51:05.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.550+0000 7f9660fd4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec152f1120 0x55ec153373a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:05.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.550+0000 7f9660fd4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec152f1120 0x55ec153373a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42868/0 (socket says 172.21.15.73:42868) 2024-04-26T20:51:05.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.550+0000 7f9660fd4640 1 -- 172.21.15.73:0/1791594883 learned_addr learned my addr 172.21.15.73:0/1791594883 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:05.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.550+0000 7f965b7fe640 1 --2- 172.21.15.73:0/1791594883 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55ec15338ce0 0x55ec15339230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:05.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.550+0000 7f965bfff640 1 --2- 172.21.15.73:0/1791594883 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec1531b440 0x55ec1531b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:05.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.550+0000 7f9660fd4640 1 -- 172.21.15.73:0/1791594883 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55ec15338ce0 msgr2=0x55ec15339230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:05.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.550+0000 7f9660fd4640 1 --2- 172.21.15.73:0/1791594883 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55ec15338ce0 0x55ec15339230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:05.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.550+0000 7f9660fd4640 1 -- 172.21.15.73:0/1791594883 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec1531b440 msgr2=0x55ec1531b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:05.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.550+0000 7f9660fd4640 1 --2- 172.21.15.73:0/1791594883 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec1531b440 0x55ec1531b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:05.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.550+0000 7f9660fd4640 1 -- 172.21.15.73:0/1791594883 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec1520d750 con 0x55ec152f1120 2024-04-26T20:51:05.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.550+0000 7f965b7fe640 1 --2- 172.21.15.73:0/1791594883 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55ec15338ce0 0x55ec15339230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:05.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.550+0000 7f965bfff640 1 --2- 172.21.15.73:0/1791594883 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec1531b440 0x55ec1531b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:05.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.550+0000 7f9660fd4640 1 --2- 172.21.15.73:0/1791594883 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec152f1120 0x55ec153373a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9654005040 tx=0x7f9654037000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8679a3351ddb7bb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:05.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.550+0000 7f9641ffb640 1 -- 172.21.15.73:0/1791594883 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965404b070 con 0x55ec152f1120 2024-04-26T20:51:05.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.550+0000 7f9641ffb640 1 -- 172.21.15.73:0/1791594883 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9654006b50 con 0x55ec152f1120 2024-04-26T20:51:05.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.550+0000 7f9641ffb640 1 -- 172.21.15.73:0/1791594883 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9654040040 con 0x55ec152f1120 2024-04-26T20:51:05.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.551+0000 7f966582b740 1 -- 172.21.15.73:0/1791594883 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec152f1120 msgr2=0x55ec153373a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:05.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.551+0000 7f966582b740 1 --2- 172.21.15.73:0/1791594883 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec152f1120 0x55ec153373a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9654005040 tx=0x7f9654037000 comp rx=0 tx=0).stop 2024-04-26T20:51:05.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.551+0000 7f966582b740 1 -- 172.21.15.73:0/1791594883 shutdown_connections 2024-04-26T20:51:05.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.551+0000 7f966582b740 1 --2- 172.21.15.73:0/1791594883 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec152f1120 0x55ec153373a0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:05.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.551+0000 7f966582b740 1 --2- 172.21.15.73:0/1791594883 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55ec15338ce0 0x55ec15339230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:05.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.551+0000 7f966582b740 1 --2- 172.21.15.73:0/1791594883 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec1531b440 0x55ec1531b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:05.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.551+0000 7f966582b740 1 -- 172.21.15.73:0/1791594883 >> 172.21.15.73:0/1791594883 conn(0x55ec152eda50 msgr2=0x55ec15336250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:05.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.551+0000 7f966582b740 1 -- 172.21.15.73:0/1791594883 shutdown_connections 2024-04-26T20:51:05.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.551+0000 7f966582b740 1 -- 172.21.15.73:0/1791594883 wait complete. 2024-04-26T20:51:05.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.552+0000 7f966582b740 1 -- start start 2024-04-26T20:51:05.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.552+0000 7f966582b740 1 Processor -- start 2024-04-26T20:51:05.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.552+0000 7f966582b740 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55ec1531b440 0x55ec15309cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:05.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.552+0000 7f966582b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec15338ce0 0x55ec1532cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:05.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.552+0000 7f966582b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec1531ec60 0x55ec153371d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:05.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.552+0000 7f966582b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ec152f1ec0 con 0x55ec1531ec60 2024-04-26T20:51:05.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.552+0000 7f966582b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ec151dcfe0 con 0x55ec15338ce0 2024-04-26T20:51:05.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.552+0000 7f966582b740 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x55ec152eb980 con 0x55ec1531b440 2024-04-26T20:51:05.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.553+0000 7f966582b740 10 client.0 _ll_register_callbacks cb 0x55ec151f3150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:51:05.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.553+0000 7f9660fd4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec1531ec60 0x55ec153371d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:05.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.553+0000 7f9660fd4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec1531ec60 0x55ec153371d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42872/0 (socket says 172.21.15.73:42872) 2024-04-26T20:51:05.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.553+0000 7f965bfff640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55ec1531b440 0x55ec15309cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:05.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.553+0000 7f9660fd4640 1 -- 172.21.15.73:0/2220844372 learned_addr learned my addr 172.21.15.73:0/2220844372 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:05.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.553+0000 7f965b7fe640 1 --2- 172.21.15.73:0/2220844372 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec15338ce0 0x55ec1532cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:05.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.553+0000 7f9660fd4640 1 -- 172.21.15.73:0/2220844372 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55ec1531b440 msgr2=0x55ec15309cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:05.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.553+0000 7f9660fd4640 1 --2- 172.21.15.73:0/2220844372 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55ec1531b440 0x55ec15309cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:05.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.553+0000 7f9660fd4640 1 -- 172.21.15.73:0/2220844372 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec15338ce0 msgr2=0x55ec1532cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:05.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.553+0000 7f9660fd4640 1 --2- 172.21.15.73:0/2220844372 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec15338ce0 0x55ec1532cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:05.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.553+0000 7f9660fd4640 1 -- 172.21.15.73:0/2220844372 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec14f672d0 con 0x55ec1531ec60 2024-04-26T20:51:05.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.553+0000 7f9660fd4640 1 --2- 172.21.15.73:0/2220844372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec1531ec60 0x55ec153371d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f96540020c0 tx=0x7f9654035070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61f1482147a9693f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:05.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.553+0000 7f963affd640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-04-26T20:51:05.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.553+0000 7f963affd640 1 -- 172.21.15.73:0/2220844372 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965404b040 con 0x55ec1531ec60 2024-04-26T20:51:05.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.553+0000 7f963affd640 1 -- 172.21.15.73:0/2220844372 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f9654006d00 con 0x55ec1531ec60 2024-04-26T20:51:05.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.553+0000 7f966582b740 1 -- 172.21.15.73:0/2220844372 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9654006b50 con 0x55ec1531ec60 2024-04-26T20:51:05.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.553+0000 7f963affd640 1 -- 172.21.15.73:0/2220844372 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f965405b050 con 0x55ec1531ec60 2024-04-26T20:51:05.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.553+0000 7f966582b740 10 client.4863 fetch_fsmap learned FSMap version 8 2024-04-26T20:51:05.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.553+0000 7f966582b740 1 -- 172.21.15.73:0/2220844372 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55ec150316b0 con 0x55ec1531ec60 2024-04-26T20:51:05.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.554+0000 7f963affd640 1 -- 172.21.15.73:0/2220844372 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9654006b50 con 0x55ec1531ec60 2024-04-26T20:51:05.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.554+0000 7f963affd640 1 -- 172.21.15.73:0/2220844372 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 8) ==== 3502+0+0 (secure 0 0 0) 0x7f96540050d0 con 0x55ec1531ec60 2024-04-26T20:51:05.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.554+0000 7f966582b740 10 client.4863 fetch_fsmap finished waiting for FSMap version 8 2024-04-26T20:51:05.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.554+0000 7f966582b740 10 client.4863 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T20:51:05.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.554+0000 7f966582b740 20 client.4863 populate_metadata read hostname 'smithi073' 2024-04-26T20:51:05.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.554+0000 7f966582b740 1 --2- 172.21.15.73:0/2220844372 >> [v2:172.21.15.73:6836/181202544,v1:172.21.15.73:6837/181202544] conn(0x55ec1530b010 0x55ec1531d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:05.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.554+0000 7f966582b740 4 client.4863 mds_command: new command op to 4281 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:51:05.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.554+0000 7f966582b740 1 -- 172.21.15.73:0/2220844372 --> [v2:172.21.15.73:6836/181202544,v1:172.21.15.73:6837/181202544] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec15263350 con 0x55ec1530b010 2024-04-26T20:51:05.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.554+0000 7f965bfff640 1 --2- 172.21.15.73:0/2220844372 >> [v2:172.21.15.73:6836/181202544,v1:172.21.15.73:6837/181202544] conn(0x55ec1530b010 0x55ec1531d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:05.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.554+0000 7f963affd640 10 client.4863 ms_handle_connect on v2:172.21.15.73:6836/181202544 2024-04-26T20:51:05.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.554+0000 7f965bfff640 1 --2- 172.21.15.73:0/2220844372 >> [v2:172.21.15.73:6836/181202544,v1:172.21.15.73:6837/181202544] conn(0x55ec1530b010 0x55ec1531d3d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=537414e2863f4d2c server_cookie=d5b6ea2860b5d4ea in_seq=0 out_seq=0 2024-04-26T20:51:05.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.555+0000 7f963affd640 1 -- 172.21.15.73:0/2220844372 <== mds.0 v2:172.21.15.73:6836/181202544 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ec15263350 con 0x55ec1530b010 2024-04-26T20:51:05.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.555+0000 7f963affd640 10 client.4863 handle_command_reply: tid=0 2024-04-26T20:51:05.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.555+0000 7f966582b740 1 client.4863 shutdown 2024-04-26T20:51:05.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.555+0000 7f966582b740 1 -- 172.21.15.73:0/2220844372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec1531ec60 msgr2=0x55ec153371d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:05.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.555+0000 7f966582b740 1 --2- 172.21.15.73:0/2220844372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec1531ec60 0x55ec153371d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f96540020c0 tx=0x7f9654035070 comp rx=0 tx=0).stop 2024-04-26T20:51:05.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.555+0000 7f966582b740 1 -- 172.21.15.73:0/2220844372 shutdown_connections 2024-04-26T20:51:05.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.555+0000 7f966582b740 1 --2- 172.21.15.73:0/2220844372 >> [v2:172.21.15.73:6836/181202544,v1:172.21.15.73:6837/181202544] conn(0x55ec1530b010 0x55ec1531d3d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:05.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.555+0000 7f966582b740 1 --2- 172.21.15.73:0/2220844372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec1531ec60 0x55ec153371d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:05.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.555+0000 7f963affd640 0 client.4863 ms_handle_reset on v2:172.21.15.73:6836/181202544 2024-04-26T20:51:05.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.555+0000 7f966582b740 1 --2- 172.21.15.73:0/2220844372 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec15338ce0 0x55ec1532cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:05.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.555+0000 7f966582b740 1 --2- 172.21.15.73:0/2220844372 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55ec1531b440 0x55ec15309cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:05.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.555+0000 7f966582b740 1 -- 172.21.15.73:0/2220844372 >> 172.21.15.73:0/2220844372 conn(0x55ec152f1120 msgr2=0x55ec153397d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:05.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.555+0000 7f966582b740 1 -- 172.21.15.73:0/2220844372 shutdown_connections 2024-04-26T20:51:05.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.555+0000 7f966582b740 1 -- 172.21.15.73:0/2220844372 wait complete. 2024-04-26T20:51:05.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.555+0000 7f966582b740 20 client.4863 trim_cache size 0 max 16384 2024-04-26T20:51:05.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.573+0000 7f966582b740 1 Processor -- start 2024-04-26T20:51:05.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.573+0000 7f966582b740 1 -- start start 2024-04-26T20:51:05.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.574+0000 7f966582b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec1531b440 0x55ec15336f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:05.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.574+0000 7f966582b740 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55ec1531ec60 0x55ec153090d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:05.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.574+0000 7f966582b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec15338ce0 0x55ec152f0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:05.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.574+0000 7f966582b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f965c1fe010 con 0x55ec1531b440 2024-04-26T20:51:05.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.574+0000 7f966582b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f965c15d820 con 0x55ec15338ce0 2024-04-26T20:51:05.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.574+0000 7f966582b740 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f965c1fd9c0 con 0x55ec1531ec60 2024-04-26T20:51:05.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.574+0000 7f965bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec1531b440 0x55ec15336f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:05.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.574+0000 7f965bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec1531b440 0x55ec15336f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42884/0 (socket says 172.21.15.73:42884) 2024-04-26T20:51:05.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.574+0000 7f965bfff640 1 -- 172.21.15.73:0/1538573890 learned_addr learned my addr 172.21.15.73:0/1538573890 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:05.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.574+0000 7f965bfff640 1 -- 172.21.15.73:0/1538573890 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55ec1531ec60 msgr2=0x55ec153090d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:05.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.574+0000 7f965bfff640 1 --2- 172.21.15.73:0/1538573890 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55ec1531ec60 0x55ec153090d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:05.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.574+0000 7f9660fd4640 1 --2- 172.21.15.73:0/1538573890 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec15338ce0 0x55ec152f0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:05.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.574+0000 7f965bfff640 1 -- 172.21.15.73:0/1538573890 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec15338ce0 msgr2=0x55ec152f0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:05.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.574+0000 7f965bfff640 1 --2- 172.21.15.73:0/1538573890 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec15338ce0 0x55ec152f0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:05.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.574+0000 7f965bfff640 1 -- 172.21.15.73:0/1538573890 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec152e70e0 con 0x55ec1531b440 2024-04-26T20:51:05.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.574+0000 7f9660fd4640 1 --2- 172.21.15.73:0/1538573890 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec15338ce0 0x55ec152f0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:05.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.574+0000 7f965bfff640 1 --2- 172.21.15.73:0/1538573890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec1531b440 0x55ec15336f60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f964c059c20 tx=0x7f964c059b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4df6956fb7c0974c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:05.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.574+0000 7f96417fa640 1 -- 172.21.15.73:0/1538573890 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964c022040 con 0x55ec1531b440 2024-04-26T20:51:05.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.575+0000 7f96417fa640 1 -- 172.21.15.73:0/1538573890 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f964c088070 con 0x55ec1531b440 2024-04-26T20:51:05.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.575+0000 7f96417fa640 1 -- 172.21.15.73:0/1538573890 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964c072220 con 0x55ec1531b440 2024-04-26T20:51:05.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.575+0000 7f966582b740 1 -- 172.21.15.73:0/1538573890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec1531b440 msgr2=0x55ec15336f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:05.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.575+0000 7f966582b740 1 --2- 172.21.15.73:0/1538573890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec1531b440 0x55ec15336f60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f964c059c20 tx=0x7f964c059b10 comp rx=0 tx=0).stop 2024-04-26T20:51:05.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.575+0000 7f966582b740 1 -- 172.21.15.73:0/1538573890 shutdown_connections 2024-04-26T20:51:05.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.575+0000 7f966582b740 1 --2- 172.21.15.73:0/1538573890 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec15338ce0 0x55ec152f0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:05.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.575+0000 7f966582b740 1 --2- 172.21.15.73:0/1538573890 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55ec1531ec60 0x55ec153090d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:05.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.575+0000 7f966582b740 1 --2- 172.21.15.73:0/1538573890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec1531b440 0x55ec15336f60 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:05.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.575+0000 7f966582b740 1 -- 172.21.15.73:0/1538573890 >> 172.21.15.73:0/1538573890 conn(0x55ec152f1120 msgr2=0x55ec1532cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:05.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.575+0000 7f966582b740 1 -- 172.21.15.73:0/1538573890 shutdown_connections 2024-04-26T20:51:05.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.575+0000 7f966582b740 1 -- 172.21.15.73:0/1538573890 wait complete. 2024-04-26T20:51:05.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.576+0000 7f966582b740 1 -- start start 2024-04-26T20:51:05.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.576+0000 7f966582b740 1 Processor -- start 2024-04-26T20:51:05.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.577+0000 7f966582b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec1531b440 0x55ec153365f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:05.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.577+0000 7f966582b740 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55ec1531ec60 0x55ec153a8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:05.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.577+0000 7f966582b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec15338ce0 0x55ec152f0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:05.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.577+0000 7f966582b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ec152f1ec0 con 0x55ec1531b440 2024-04-26T20:51:05.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.577+0000 7f966582b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ec151dcfe0 con 0x55ec15338ce0 2024-04-26T20:51:05.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.577+0000 7f966582b740 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x55ec1532e930 con 0x55ec1531ec60 2024-04-26T20:51:05.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.577+0000 7f966582b740 10 client.0 _ll_register_callbacks cb 0x7f9654049070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:51:05.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.577+0000 7f965bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec1531b440 0x55ec153365f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:05.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.577+0000 7f965bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec1531b440 0x55ec153365f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42892/0 (socket says 172.21.15.73:42892) 2024-04-26T20:51:05.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.577+0000 7f965bfff640 1 -- 172.21.15.73:0/731612641 learned_addr learned my addr 172.21.15.73:0/731612641 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:05.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.577+0000 7f965b7fe640 1 --2- 172.21.15.73:0/731612641 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55ec1531ec60 0x55ec153a8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:05.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.577+0000 7f9660fd4640 1 --2- 172.21.15.73:0/731612641 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec15338ce0 0x55ec152f0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:05.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.577+0000 7f965bfff640 1 -- 172.21.15.73:0/731612641 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55ec1531ec60 msgr2=0x55ec153a8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:05.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.577+0000 7f965bfff640 1 --2- 172.21.15.73:0/731612641 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55ec1531ec60 0x55ec153a8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:05.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.577+0000 7f965bfff640 1 -- 172.21.15.73:0/731612641 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec15338ce0 msgr2=0x55ec152f0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:05.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.577+0000 7f965bfff640 1 --2- 172.21.15.73:0/731612641 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec15338ce0 0x55ec152f0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:05.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.577+0000 7f965bfff640 1 -- 172.21.15.73:0/731612641 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec153a88b0 con 0x55ec1531b440 2024-04-26T20:51:05.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.577+0000 7f965b7fe640 1 --2- 172.21.15.73:0/731612641 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55ec1531ec60 0x55ec153a8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:05.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.577+0000 7f965bfff640 1 --2- 172.21.15.73:0/731612641 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec1531b440 0x55ec153365f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f964c059990 tx=0x7f964c006df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fa2852ccf20e81c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:05.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.577+0000 7f963affd640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-04-26T20:51:05.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.577+0000 7f963affd640 1 -- 172.21.15.73:0/731612641 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964c078070 con 0x55ec1531b440 2024-04-26T20:51:05.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.577+0000 7f963affd640 1 -- 172.21.15.73:0/731612641 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f964c079260 con 0x55ec1531b440 2024-04-26T20:51:05.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.577+0000 7f966582b740 1 -- 172.21.15.73:0/731612641 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f964c088070 con 0x55ec1531b440 2024-04-26T20:51:05.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.577+0000 7f963affd640 1 -- 172.21.15.73:0/731612641 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f964c022070 con 0x55ec1531b440 2024-04-26T20:51:05.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.577+0000 7f966582b740 10 client.4869 fetch_fsmap learned FSMap version 8 2024-04-26T20:51:05.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.577+0000 7f966582b740 1 -- 172.21.15.73:0/731612641 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55ec15330670 con 0x55ec1531b440 2024-04-26T20:51:05.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.578+0000 7f963affd640 1 -- 172.21.15.73:0/731612641 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f964c088070 con 0x55ec1531b440 2024-04-26T20:51:05.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.578+0000 7f963affd640 1 -- 172.21.15.73:0/731612641 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 8) ==== 3502+0+0 (secure 0 0 0) 0x7f964c056300 con 0x55ec1531b440 2024-04-26T20:51:05.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.578+0000 7f966582b740 10 client.4869 fetch_fsmap finished waiting for FSMap version 8 2024-04-26T20:51:05.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.578+0000 7f966582b740 10 client.4869 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T20:51:05.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.578+0000 7f966582b740 20 client.4869 populate_metadata read hostname 'smithi073' 2024-04-26T20:51:05.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.578+0000 7f966582b740 1 --2- 172.21.15.73:0/731612641 >> [v2:172.21.15.73:6836/181202544,v1:172.21.15.73:6837/181202544] conn(0x55ec1532bbe0 0x55ec1532bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:05.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.578+0000 7f966582b740 4 client.4869 mds_command: new command op to 4281 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T20:51:05.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.578+0000 7f966582b740 1 -- 172.21.15.73:0/731612641 --> [v2:172.21.15.73:6836/181202544,v1:172.21.15.73:6837/181202544] -- command(tid 0: {"prefix": "status"}) -- 0x55ec1525d1c0 con 0x55ec1532bbe0 2024-04-26T20:51:05.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.578+0000 7f965b7fe640 1 --2- 172.21.15.73:0/731612641 >> [v2:172.21.15.73:6836/181202544,v1:172.21.15.73:6837/181202544] conn(0x55ec1532bbe0 0x55ec1532bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:05.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.578+0000 7f963affd640 10 client.4869 ms_handle_connect on v2:172.21.15.73:6836/181202544 2024-04-26T20:51:05.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.578+0000 7f965b7fe640 1 --2- 172.21.15.73:0/731612641 >> [v2:172.21.15.73:6836/181202544,v1:172.21.15.73:6837/181202544] conn(0x55ec1532bbe0 0x55ec1532bfc0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5881dcb7097cd348 server_cookie=65656e4cb6c4b7dd in_seq=0 out_seq=0 2024-04-26T20:51:05.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.578+0000 7f963affd640 1 -- 172.21.15.73:0/731612641 <== mds.0 v2:172.21.15.73:6836/181202544 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x55ec1525d1c0 con 0x55ec1532bbe0 2024-04-26T20:51:05.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.578+0000 7f963affd640 10 client.4869 handle_command_reply: tid=0 2024-04-26T20:51:05.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.578+0000 7f966582b740 1 client.4869 shutdown 2024-04-26T20:51:05.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.578+0000 7f966582b740 1 -- 172.21.15.73:0/731612641 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec1531b440 msgr2=0x55ec153365f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:05.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.578+0000 7f966582b740 1 --2- 172.21.15.73:0/731612641 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec1531b440 0x55ec153365f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f964c059990 tx=0x7f964c006df0 comp rx=0 tx=0).stop 2024-04-26T20:51:05.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.579+0000 7f966582b740 1 -- 172.21.15.73:0/731612641 shutdown_connections 2024-04-26T20:51:05.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.579+0000 7f966582b740 1 --2- 172.21.15.73:0/731612641 >> [v2:172.21.15.73:6836/181202544,v1:172.21.15.73:6837/181202544] conn(0x55ec1532bbe0 0x55ec1532bfc0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:05.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.579+0000 7f966582b740 1 --2- 172.21.15.73:0/731612641 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ec15338ce0 0x55ec152f0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:05.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.579+0000 7f966582b740 1 --2- 172.21.15.73:0/731612641 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55ec1531ec60 0x55ec153a8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:05.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.579+0000 7f963affd640 0 client.4869 ms_handle_reset on v2:172.21.15.73:6836/181202544 2024-04-26T20:51:05.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.579+0000 7f966582b740 1 --2- 172.21.15.73:0/731612641 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec1531b440 0x55ec153365f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:05.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.579+0000 7f966582b740 1 -- 172.21.15.73:0/731612641 >> 172.21.15.73:0/731612641 conn(0x55ec152f1120 msgr2=0x55ec15319e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:05.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.579+0000 7f966582b740 1 -- 172.21.15.73:0/731612641 shutdown_connections 2024-04-26T20:51:05.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.579+0000 7f966582b740 1 -- 172.21.15.73:0/731612641 wait complete. 2024-04-26T20:51:05.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.579+0000 7f966582b740 20 client.4869 trim_cache size 0 max 16384 2024-04-26T20:51:05.591 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-26T20:51:05.591 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_fsid": "fbd3f548-b547-4713-856b-09f98f5d1764", 2024-04-26T20:51:05.592 INFO:teuthology.orchestra.run.smithi073.stdout: "whoami": 0, 2024-04-26T20:51:05.592 INFO:teuthology.orchestra.run.smithi073.stdout: "id": 4281, 2024-04-26T20:51:05.592 INFO:teuthology.orchestra.run.smithi073.stdout: "want_state": "up:active", 2024-04-26T20:51:05.592 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "up:active", 2024-04-26T20:51:05.592 INFO:teuthology.orchestra.run.smithi073.stdout: "fs_name": "cephfs", 2024-04-26T20:51:05.592 INFO:teuthology.orchestra.run.smithi073.stdout: "rank_uptime": 32.947619625999998, 2024-04-26T20:51:05.592 INFO:teuthology.orchestra.run.smithi073.stdout: "mdsmap_epoch": 8, 2024-04-26T20:51:05.592 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch": 25, 2024-04-26T20:51:05.592 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch_barrier": 25, 2024-04-26T20:51:05.592 INFO:teuthology.orchestra.run.smithi073.stdout: "uptime": 40.018069420000003 2024-04-26T20:51:05.592 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-26T20:51:05.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.579+0000 7f9662a37640 1 -- 172.21.15.73:0/3355889478 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f964806c8f0 msgr2=0x7f964806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:05.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.579+0000 7f9662a37640 1 --2- 172.21.15.73:0/3355889478 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f964806c8f0 0x7f964806edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9650001200 tx=0x7f9650009040 comp rx=0 tx=0).stop 2024-04-26T20:51:05.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.579+0000 7f9662a37640 1 -- 172.21.15.73:0/3355889478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9644004d40 msgr2=0x7f965c0cb660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:05.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.579+0000 7f9662a37640 1 --2- 172.21.15.73:0/3355889478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9644004d40 0x7f965c0cb660 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f964c011a00 tx=0x7f964c011ec0 comp rx=0 tx=0).stop 2024-04-26T20:51:05.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.579+0000 7f9662a37640 1 -- 172.21.15.73:0/3355889478 shutdown_connections 2024-04-26T20:51:05.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.579+0000 7f9662a37640 1 --2- 172.21.15.73:0/3355889478 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f964806c8f0 0x7f964806edb0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:05.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.579+0000 7f9662a37640 1 --2- 172.21.15.73:0/3355889478 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f965c0c9030 0x7f965c0c7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:05.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.579+0000 7f9662a37640 1 --2- 172.21.15.73:0/3355889478 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f965c14e460 0x7f965c1f3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:05.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.579+0000 7f9662a37640 1 --2- 172.21.15.73:0/3355889478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9644004d40 0x7f965c0cb660 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:05.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.579+0000 7f9662a37640 1 -- 172.21.15.73:0/3355889478 >> 172.21.15.73:0/3355889478 conn(0x7f965c0d1cf0 msgr2=0x7f965c0cc000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:05.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.579+0000 7f9662a37640 1 -- 172.21.15.73:0/3355889478 shutdown_connections 2024-04-26T20:51:05.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:05.579+0000 7f9662a37640 1 -- 172.21.15.73:0/3355889478 wait complete. 2024-04-26T20:51:05.596 INFO:teuthology.run_tasks:Running task sequential... 2024-04-26T20:51:05.610 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-26T20:51:05.610 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-26T20:51:05.610 INFO:teuthology.task.print:Enabling mgr modules 2024-04-26T20:51:05.610 INFO:teuthology.run_tasks:Running task kclient... 2024-04-26T20:51:05.634 INFO:tasks.kclient:Mounting kernel clients... 2024-04-26T20:51:05.634 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-26T20:51:05.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:05.635 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list 2024-04-26T20:51:05.650 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-26T20:51:05.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:05.653 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T20:51:05.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:05.713 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link delete ceph-brx 2024-04-26T20:51:05.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:05.808 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list 2024-04-26T20:51:05.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:05.823 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link delete ceph-brx 2024-04-26T20:51:05.889 INFO:teuthology.orchestra.run.smithi073.stderr:Cannot find device "ceph-brx" 2024-04-26T20:51:05.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:51:05.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:05.891 DEBUG:teuthology.orchestra.run.smithi120:> ip netns list 2024-04-26T20:51:05.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:05.909 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip link delete ceph-brx 2024-04-26T20:51:05.979 INFO:teuthology.orchestra.run.smithi120.stderr:Cannot find device "ceph-brx" 2024-04-26T20:51:05.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:51:05.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:05.981 DEBUG:teuthology.orchestra.run.smithi120:> ip netns list 2024-04-26T20:51:06.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:06.038 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip link delete ceph-brx 2024-04-26T20:51:06.105 INFO:teuthology.orchestra.run.smithi120.stderr:Cannot find device "ceph-brx" 2024-04-26T20:51:06.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:51:06.106 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-04-26T20:51:06.106 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T20:51:06.107 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T20:51:06.107 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T20:51:06.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.206+0000 7f706d39d640 1 Processor -- start 2024-04-26T20:51:06.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.206+0000 7f706d39d640 1 -- start start 2024-04-26T20:51:06.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.207+0000 7f706d39d640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f70680b3860 0x7f706816a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:06.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.207+0000 7f706d39d640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70680d0ba0 con 0x7f70680d06f0 2024-04-26T20:51:06.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.207+0000 7f706d39d640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70680d2590 con 0x7f706815f3a0 2024-04-26T20:51:06.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.207+0000 7f706d39d640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f70680d0e20 con 0x7f70680b3860 2024-04-26T20:51:06.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.207+0000 7f70677fe640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f70680b3860 0x7f706816a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:06.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.207+0000 7f70667fc640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f70680d06f0 0x7f70680d1e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:42110/0 (socket says 172.21.15.73:42110) 2024-04-26T20:51:06.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.207+0000 7f70667fc640 1 -- 172.21.15.73:0/785108444 learned_addr learned my addr 172.21.15.73:0/785108444 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:06.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.207+0000 7f70677fe640 1 -- 172.21.15.73:0/785108444 >> v1:172.21.15.120:6789/0 conn(0x7f706815f3a0 legacy=0x7f70680d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:06.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.207+0000 7f70677fe640 1 -- 172.21.15.73:0/785108444 >> v1:172.21.15.73:6789/0 conn(0x7f70680d06f0 legacy=0x7f70680d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:06.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.207+0000 7f70677fe640 1 -- 172.21.15.73:0/785108444 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70680b1d70 con 0x7f70680b3860 2024-04-26T20:51:06.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.208+0000 7f70677fe640 1 --2- 172.21.15.73:0/785108444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70680b3860 0x7f706816a000 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7060009d90 tx=0x7f706002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e9d608861e62f33 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:06.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.208+0000 7f7065ffb640 1 -- 172.21.15.73:0/785108444 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7060007740 con 0x7f70680b3860 2024-04-26T20:51:06.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.208+0000 7f7065ffb640 1 -- 172.21.15.73:0/785108444 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70600078e0 con 0x7f70680b3860 2024-04-26T20:51:06.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.208+0000 7f7065ffb640 1 -- 172.21.15.73:0/785108444 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7060007cb0 con 0x7f70680b3860 2024-04-26T20:51:06.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.208+0000 7f706d39d640 1 -- 172.21.15.73:0/785108444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70680b3860 msgr2=0x7f706816a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:06.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.208+0000 7f706d39d640 1 --2- 172.21.15.73:0/785108444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70680b3860 0x7f706816a000 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7060009d90 tx=0x7f706002a660 comp rx=0 tx=0).stop 2024-04-26T20:51:06.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.208+0000 7f706d39d640 1 -- 172.21.15.73:0/785108444 shutdown_connections 2024-04-26T20:51:06.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.208+0000 7f706d39d640 1 --2- 172.21.15.73:0/785108444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70680b3860 0x7f706816a000 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:06.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.208+0000 7f706d39d640 1 -- 172.21.15.73:0/785108444 >> 172.21.15.73:0/785108444 conn(0x7f70680d4f90 msgr2=0x7f70680d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:06.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.209+0000 7f706d39d640 1 -- 172.21.15.73:0/785108444 shutdown_connections 2024-04-26T20:51:06.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.209+0000 7f706d39d640 1 -- 172.21.15.73:0/785108444 wait complete. 2024-04-26T20:51:06.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.210+0000 7f706d39d640 1 Processor -- start 2024-04-26T20:51:06.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.210+0000 7f706d39d640 1 -- start start 2024-04-26T20:51:06.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.211+0000 7f706d39d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f70680b3860 0x7f70680cf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:06.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.211+0000 7f706d39d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f70680d06f0 0x7f706809ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:06.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.211+0000 7f706d39d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f706815f3a0 0x7f70681504e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:06.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.211+0000 7f706d39d640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f706815f780 con 0x7f706815f3a0 2024-04-26T20:51:06.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.211+0000 7f706d39d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f70680b2330 con 0x7f70680d06f0 2024-04-26T20:51:06.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.211+0000 7f706d39d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f706815f900 con 0x7f70680b3860 2024-04-26T20:51:06.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.211+0000 7f70677fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f706815f3a0 0x7f70681504e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:06.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.211+0000 7f70677fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f706815f3a0 0x7f70681504e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42900/0 (socket says 172.21.15.73:42900) 2024-04-26T20:51:06.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.211+0000 7f70677fe640 1 -- 172.21.15.73:0/2117876940 learned_addr learned my addr 172.21.15.73:0/2117876940 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:06.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.211+0000 7f70677fe640 1 -- 172.21.15.73:0/2117876940 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f70680b3860 msgr2=0x7f70680cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:06.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.211+0000 7f7066ffd640 1 --2- 172.21.15.73:0/2117876940 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f70680b3860 0x7f70680cf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:06.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.211+0000 7f70667fc640 1 --2- 172.21.15.73:0/2117876940 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f70680d06f0 0x7f706809ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:06.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.211+0000 7f70677fe640 1 --2- 172.21.15.73:0/2117876940 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f70680b3860 0x7f70680cf0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:06.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.211+0000 7f70677fe640 1 -- 172.21.15.73:0/2117876940 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f70680d06f0 msgr2=0x7f706809ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:06.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.211+0000 7f70677fe640 1 --2- 172.21.15.73:0/2117876940 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f70680d06f0 0x7f706809ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:06.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.211+0000 7f70677fe640 1 -- 172.21.15.73:0/2117876940 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7068150a20 con 0x7f706815f3a0 2024-04-26T20:51:06.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.211+0000 7f70677fe640 1 --2- 172.21.15.73:0/2117876940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f706815f3a0 0x7f70681504e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f706002ac50 tx=0x7f7060004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:06.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.211+0000 7f704f7fe640 1 -- 172.21.15.73:0/2117876940 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7060040070 con 0x7f706815f3a0 2024-04-26T20:51:06.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.211+0000 7f704f7fe640 1 -- 172.21.15.73:0/2117876940 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7060031c90 con 0x7f706815f3a0 2024-04-26T20:51:06.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.211+0000 7f706d39d640 1 -- 172.21.15.73:0/2117876940 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7068150f50 con 0x7f706815f3a0 2024-04-26T20:51:06.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.211+0000 7f704f7fe640 1 -- 172.21.15.73:0/2117876940 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7060045680 con 0x7f706815f3a0 2024-04-26T20:51:06.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.211+0000 7f706d39d640 1 -- 172.21.15.73:0/2117876940 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7068151510 con 0x7f706815f3a0 2024-04-26T20:51:06.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.212+0000 7f706d39d640 1 -- 172.21.15.73:0/2117876940 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70680694f0 con 0x7f706815f3a0 2024-04-26T20:51:06.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.213+0000 7f704f7fe640 1 -- 172.21.15.73:0/2117876940 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7060039040 con 0x7f706815f3a0 2024-04-26T20:51:06.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.213+0000 7f704f7fe640 1 --2- 172.21.15.73:0/2117876940 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f705406c8f0 0x7f705406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:06.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.213+0000 7f704f7fe640 1 -- 172.21.15.73:0/2117876940 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f70600ad810 con 0x7f706815f3a0 2024-04-26T20:51:06.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.213+0000 7f7066ffd640 1 --2- 172.21.15.73:0/2117876940 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f705406c8f0 0x7f705406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:06.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.214+0000 7f7066ffd640 1 --2- 172.21.15.73:0/2117876940 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f705406c8f0 0x7f705406edb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7058001500 tx=0x7f7058009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:06.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.217+0000 7f704f7fe640 1 -- 172.21.15.73:0/2117876940 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7068150a20 con 0x7f706815f3a0 2024-04-26T20:51:06.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.431+0000 7f706d39d640 1 -- 172.21.15.73:0/2117876940 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f70680b6860 con 0x7f706815f3a0 2024-04-26T20:51:06.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.432+0000 7f704f7fe640 1 -- 172.21.15.73:0/2117876940 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+88 (secure 0 0 0) 0x7f70600abe20 con 0x7f706815f3a0 2024-04-26T20:51:06.433 INFO:teuthology.orchestra.run.smithi073.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-26T20:51:06.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.433+0000 7f706d39d640 1 -- 172.21.15.73:0/2117876940 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f705406c8f0 msgr2=0x7f705406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:06.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.433+0000 7f706d39d640 1 --2- 172.21.15.73:0/2117876940 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f705406c8f0 0x7f705406edb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7058001500 tx=0x7f7058009040 comp rx=0 tx=0).stop 2024-04-26T20:51:06.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.433+0000 7f706d39d640 1 -- 172.21.15.73:0/2117876940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f706815f3a0 msgr2=0x7f70681504e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:06.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.433+0000 7f706d39d640 1 --2- 172.21.15.73:0/2117876940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f706815f3a0 0x7f70681504e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f706002ac50 tx=0x7f7060004a40 comp rx=0 tx=0).stop 2024-04-26T20:51:06.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.433+0000 7f706d39d640 1 -- 172.21.15.73:0/2117876940 shutdown_connections 2024-04-26T20:51:06.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.433+0000 7f706d39d640 1 --2- 172.21.15.73:0/2117876940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f706815f3a0 0x7f70681504e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:06.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.433+0000 7f706d39d640 1 --2- 172.21.15.73:0/2117876940 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f705406c8f0 0x7f705406edb0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:06.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.433+0000 7f706d39d640 1 --2- 172.21.15.73:0/2117876940 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f70680d06f0 0x7f706809ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:06.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.433+0000 7f706d39d640 1 --2- 172.21.15.73:0/2117876940 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f70680b3860 0x7f70680cf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:06.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.433+0000 7f706d39d640 1 -- 172.21.15.73:0/2117876940 >> 172.21.15.73:0/2117876940 conn(0x7f70680d4f90 msgr2=0x7f706806cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:06.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.434+0000 7f706d39d640 1 -- 172.21.15.73:0/2117876940 shutdown_connections 2024-04-26T20:51:06.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06.434+0000 7f706d39d640 1 -- 172.21.15.73:0/2117876940 wait complete. 2024-04-26T20:51:06.445 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T20:51:06.446 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T20:51:06.446 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi073.front.sepia.ceph.com 2024-04-26T20:51:06.446 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T20:51:06.446 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T20:51:06.446 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T20:51:06.446 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T20:51:06.446 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T20:51:06.446 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T20:51:06.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:06.446 DEBUG:teuthology.orchestra.run.smithi073:> ip addr 2024-04-26T20:51:06.461 INFO:teuthology.orchestra.run.smithi073.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:51:06.461 INFO:teuthology.orchestra.run.smithi073.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:51:06.461 INFO:teuthology.orchestra.run.smithi073.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:51:06.462 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:51:06.462 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 ::1/128 scope host 2024-04-26T20:51:06.462 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:51:06.462 INFO:teuthology.orchestra.run.smithi073.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:51:06.462 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:ea brd ff:ff:ff:ff:ff:ff 2024-04-26T20:51:06.462 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f0 2024-04-26T20:51:06.462 INFO:teuthology.orchestra.run.smithi073.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:51:06.462 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:eb brd ff:ff:ff:ff:ff:ff 2024-04-26T20:51:06.462 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f1 2024-04-26T20:51:06.462 INFO:teuthology.orchestra.run.smithi073.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:51:06.462 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7c brd ff:ff:ff:ff:ff:ff 2024-04-26T20:51:06.462 INFO:teuthology.orchestra.run.smithi073.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T20:51:06.462 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7d brd ff:ff:ff:ff:ff:ff 2024-04-26T20:51:06.463 INFO:teuthology.orchestra.run.smithi073.stdout: inet 172.21.15.73/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-26T20:51:06.463 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft 42721sec preferred_lft 42721sec 2024-04-26T20:51:06.463 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::b99b:8e91:ffc2:71f/64 scope link noprefixroute 2024-04-26T20:51:06.463 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:51:06.463 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T20:51:06.463 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:51:06.463 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2024-04-26T20:51:06.463 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link add name ceph-brx type bridge 2024-04-26T20:51:06.463 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip addr flush dev ceph-brx 2024-04-26T20:51:06.464 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link set ceph-brx up 2024-04-26T20:51:06.464 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T20:51:06.464 DEBUG:teuthology.orchestra.run.smithi073:> ') 2024-04-26T20:51:06.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:51:06.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:51:06.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:06.596 DEBUG:teuthology.orchestra.run.smithi073:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T20:51:06.618 INFO:teuthology.orchestra.run.smithi073.stdout:1 2024-04-26T20:51:06.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:06.620 DEBUG:teuthology.orchestra.run.smithi073:> route 2024-04-26T20:51:06.676 INFO:teuthology.orchestra.run.smithi073.stdout:Kernel IP routing table 2024-04-26T20:51:06.676 INFO:teuthology.orchestra.run.smithi073.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T20:51:06.676 INFO:teuthology.orchestra.run.smithi073.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T20:51:06.676 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T20:51:06.677 INFO:teuthology.orchestra.run.smithi073.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T20:51:06.677 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:51:06.677 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2024-04-26T20:51:06.678 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T20:51:06.678 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T20:51:06.678 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T20:51:06.678 DEBUG:teuthology.orchestra.run.smithi073:> ') 2024-04-26T20:51:06.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:51:06.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:51:06.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:06.802 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list 2024-04-26T20:51:06.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:06.817 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list-id 2024-04-26T20:51:06.872 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:51:06.872 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2024-04-26T20:51:06.872 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:51:06.872 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-26T20:51:06.872 DEBUG:teuthology.orchestra.run.smithi073:> ') 2024-04-26T20:51:06.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:51:06.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:51:06.987 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-26T20:51:06.987 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:51:06.987 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2024-04-26T20:51:06.987 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-26T20:51:06.987 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T20:51:06.987 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-26T20:51:06.987 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-26T20:51:06.987 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-26T20:51:06.987 DEBUG:teuthology.orchestra.run.smithi073:> ') 2024-04-26T20:51:07.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:51:07.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:51:07.095 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:51:07.095 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2024-04-26T20:51:07.095 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link set brx.0 up 2024-04-26T20:51:07.095 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T20:51:07.095 DEBUG:teuthology.orchestra.run.smithi073:> ') 2024-04-26T20:51:07.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:51:07.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:51:07.168 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:51:07.168 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T20:51:07.182 INFO:teuthology.orchestra.run.smithi073.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T20:51:07.183 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:51:07.183 DEBUG:teuthology.orchestra.run.smithi073:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T20:51:07.237 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T20:51:07.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:07.238 DEBUG:teuthology.orchestra.run.smithi073:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T20:51:07.320 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T20:51:07.320 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T20:51:07.320 INFO:teuthology.orchestra.run.smithi073.stdout:invalid new device string format 2024-04-26T20:51:07.420 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T20:51:07.420 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T20:51:07.420 INFO:teuthology.orchestra.run.smithi073.stdout:invalid new device string format 2024-04-26T20:51:07.420 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: resolved to: "172.21.15.73:3300,172.21.15.120:3300,172.21.15.120:3301" 2024-04-26T20:51:07.420 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T20:51:07.424 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T20:51:07.424 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T20:51:07.425 INFO:teuthology.orchestra.run.smithi073.stdout:invalid new device string format 2024-04-26T20:51:07.425 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: resolved to: "172.21.15.73:3300,172.21.15.120:3300,172.21.15.120:3301" 2024-04-26T20:51:07.425 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T20:51:07.425 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: resolved to: "172.21.15.73:6789,172.21.15.120:6789,172.21.15.120:6790" 2024-04-26T20:51:07.425 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.73:6789,172.21.15.120:6789,172.21.15.120:6790:/ 2024-04-26T20:51:07.425 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=fbd3f548-b547-4713-856b-09f98f5d1764" will pass to kernel 2024-04-26T20:51:07.425 INFO:teuthology.orchestra.run.smithi073.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T20:51:07.426 INFO:teuthology.orchestra.run.smithi073.stdout: You just mounted a file system that supports labels which does not 2024-04-26T20:51:07.426 INFO:teuthology.orchestra.run.smithi073.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T20:51:07.426 INFO:teuthology.orchestra.run.smithi073.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T20:51:07.426 INFO:teuthology.orchestra.run.smithi073.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T20:51:07.428 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T20:51:07.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:07.428 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T20:51:07.491 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T20:51:07.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T20:51:07.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:51:07.586 INFO:teuthology.orchestra.run.smithi073.stdout:client4684 2024-04-26T20:51:07.587 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-26T20:51:07.587 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:51:07.602 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T20:51:07.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T20:51:07.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:51:07.697 INFO:teuthology.orchestra.run.smithi073.stdout:client4684 2024-04-26T20:51:07.698 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T20:51:07.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T20:51:07.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:51:07.753 INFO:teuthology.orchestra.run.smithi073.stdout:client4684 2024-04-26T20:51:07.753 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/fbd3f548-b547-4713-856b-09f98f5d1764.client4684/status') 2024-04-26T20:51:07.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:51:07.810 INFO:teuthology.orchestra.run.smithi073.stdout:instance: client.4684 (3)172.21.15.73:0/1664500238 2024-04-26T20:51:07.810 INFO:teuthology.orchestra.run.smithi073.stdout:blocklisted: false 2024-04-26T20:51:07.810 INFO:teuthology.orchestra.run.smithi073.stderr:0+1 records in 2024-04-26T20:51:07.810 INFO:teuthology.orchestra.run.smithi073.stderr:0+1 records out 2024-04-26T20:51:07.810 INFO:teuthology.orchestra.run.smithi073.stderr:70 bytes copied, 2.1573e-05 s, 3.2 MB/s 2024-04-26T20:51:07.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:51:07.817 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-04-26T20:51:07.817 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T20:51:07.817 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-26T20:51:07.817 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T20:51:07.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.912+0000 7f3185d86640 1 Processor -- start 2024-04-26T20:51:07.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.912+0000 7f3185d86640 1 -- start start 2024-04-26T20:51:07.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.913+0000 7f3185d86640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f318015ab60 0x7f318006c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:07.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.913+0000 7f3185d86640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f31801617f0 0x7f318006c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:07.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.913+0000 7f3185d86640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f318015af40 con 0x7f318015d5f0 2024-04-26T20:51:07.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.913+0000 7f3185d86640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f318015e320 con 0x7f31801617f0 2024-04-26T20:51:07.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.913+0000 7f3185d86640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f3180160d10 con 0x7f318015ab60 2024-04-26T20:51:07.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.913+0000 7f317f7fe640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f318015ab60 0x7f318006c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:07.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.913+0000 7f317effd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f31801617f0 0x7f318006c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:07.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.913+0000 7f317f7fe640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f318015ab60 0x7f318006c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.73:43950/0 (socket says 172.21.15.73:43950) 2024-04-26T20:51:07.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.913+0000 7f317effd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f31801617f0 0x7f318006c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.73:54750/0 (socket says 172.21.15.73:54750) 2024-04-26T20:51:07.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.913+0000 7f317f7fe640 1 -- 172.21.15.73:0/4276462119 learned_addr learned my addr 172.21.15.73:0/4276462119 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:07.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.914+0000 7f317effd640 1 -- 172.21.15.73:0/4276462119 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f318015ab60 msgr2=0x7f318006c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:07.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.914+0000 7f317effd640 1 --2- 172.21.15.73:0/4276462119 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f318015ab60 0x7f318006c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:07.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.914+0000 7f317effd640 1 -- 172.21.15.73:0/4276462119 >> v1:172.21.15.120:6789/0 conn(0x7f318015d5f0 legacy=0x7f318015daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:07.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.914+0000 7f317effd640 1 -- 172.21.15.73:0/4276462119 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3180160ff0 con 0x7f31801617f0 2024-04-26T20:51:07.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.914+0000 7f317f7fe640 1 --2- 172.21.15.73:0/4276462119 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f318015ab60 0x7f318006c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:51:07.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.914+0000 7f317effd640 1 --2- 172.21.15.73:0/4276462119 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f31801617f0 0x7f318006c780 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f31740097a0 tx=0x7f317402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca467af4894e0826 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:07.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.915+0000 7f317e7fc640 1 -- 172.21.15.73:0/4276462119 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3174037070 con 0x7f31801617f0 2024-04-26T20:51:07.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.915+0000 7f317e7fc640 1 -- 172.21.15.73:0/4276462119 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3174032440 con 0x7f31801617f0 2024-04-26T20:51:07.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.915+0000 7f317e7fc640 1 -- 172.21.15.73:0/4276462119 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3174032810 con 0x7f31801617f0 2024-04-26T20:51:07.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.915+0000 7f3185d86640 1 -- 172.21.15.73:0/4276462119 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f31801617f0 msgr2=0x7f318006c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:07.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.915+0000 7f3185d86640 1 --2- 172.21.15.73:0/4276462119 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f31801617f0 0x7f318006c780 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f31740097a0 tx=0x7f317402a180 comp rx=0 tx=0).stop 2024-04-26T20:51:07.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.915+0000 7f3185d86640 1 -- 172.21.15.73:0/4276462119 shutdown_connections 2024-04-26T20:51:07.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.915+0000 7f3185d86640 1 --2- 172.21.15.73:0/4276462119 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f31801617f0 0x7f318006c780 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:07.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.915+0000 7f3185d86640 1 --2- 172.21.15.73:0/4276462119 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f318015ab60 0x7f318006c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:07.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.915+0000 7f3185d86640 1 -- 172.21.15.73:0/4276462119 >> 172.21.15.73:0/4276462119 conn(0x7f31800d1cf0 msgr2=0x7f31800d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:07.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.915+0000 7f3185d86640 1 -- 172.21.15.73:0/4276462119 shutdown_connections 2024-04-26T20:51:07.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.915+0000 7f3185d86640 1 -- 172.21.15.73:0/4276462119 wait complete. 2024-04-26T20:51:07.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.916+0000 7f3185d86640 1 Processor -- start 2024-04-26T20:51:07.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.916+0000 7f3185d86640 1 -- start start 2024-04-26T20:51:07.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.917+0000 7f3185d86640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f318015ab60 0x7f318015c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:07.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.917+0000 7f3185d86640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f318015d5f0 0x7f31801592d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:07.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.917+0000 7f3185d86640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f31801617f0 0x7f31800caf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:07.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.917+0000 7f3185d86640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f31800cc4c0 con 0x7f318015d5f0 2024-04-26T20:51:07.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.917+0000 7f3185d86640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f31800cc340 con 0x7f31801617f0 2024-04-26T20:51:07.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.917+0000 7f3185d86640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f31800cc640 con 0x7f318015ab60 2024-04-26T20:51:07.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.917+0000 7f317effd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f318015d5f0 0x7f31801592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:07.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.917+0000 7f317effd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f318015d5f0 0x7f31801592d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42902/0 (socket says 172.21.15.73:42902) 2024-04-26T20:51:07.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.917+0000 7f317effd640 1 -- 172.21.15.73:0/508968552 learned_addr learned my addr 172.21.15.73:0/508968552 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:07.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.917+0000 7f317f7fe640 1 --2- 172.21.15.73:0/508968552 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f318015ab60 0x7f318015c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:07.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.917+0000 7f317ffff640 1 --2- 172.21.15.73:0/508968552 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f31801617f0 0x7f31800caf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:07.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.918+0000 7f317effd640 1 -- 172.21.15.73:0/508968552 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f318015ab60 msgr2=0x7f318015c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:07.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.918+0000 7f317effd640 1 --2- 172.21.15.73:0/508968552 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f318015ab60 0x7f318015c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:07.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.918+0000 7f317effd640 1 -- 172.21.15.73:0/508968552 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f31801617f0 msgr2=0x7f31800caf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:07.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.918+0000 7f317effd640 1 --2- 172.21.15.73:0/508968552 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f31801617f0 0x7f31800caf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:07.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.918+0000 7f317effd640 1 -- 172.21.15.73:0/508968552 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31801f3030 con 0x7f318015d5f0 2024-04-26T20:51:07.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.918+0000 7f317ffff640 1 --2- 172.21.15.73:0/508968552 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f31801617f0 0x7f31800caf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:07.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.918+0000 7f317effd640 1 --2- 172.21.15.73:0/508968552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f318015d5f0 0x7f31801592d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f31740098c0 tx=0x7f31740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:07.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.918+0000 7f3167fff640 1 -- 172.21.15.73:0/508968552 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3174037040 con 0x7f318015d5f0 2024-04-26T20:51:07.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.918+0000 7f3185d86640 1 -- 172.21.15.73:0/508968552 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31800b4df0 con 0x7f318015d5f0 2024-04-26T20:51:07.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.918+0000 7f3167fff640 1 -- 172.21.15.73:0/508968552 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3174042070 con 0x7f318015d5f0 2024-04-26T20:51:07.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.918+0000 7f3167fff640 1 -- 172.21.15.73:0/508968552 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3174030940 con 0x7f318015d5f0 2024-04-26T20:51:07.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.918+0000 7f3185d86640 1 -- 172.21.15.73:0/508968552 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31800b53b0 con 0x7f318015d5f0 2024-04-26T20:51:07.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.919+0000 7f3185d86640 1 -- 172.21.15.73:0/508968552 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31800694f0 con 0x7f318015d5f0 2024-04-26T20:51:07.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.919+0000 7f3167fff640 1 -- 172.21.15.73:0/508968552 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3174030ae0 con 0x7f318015d5f0 2024-04-26T20:51:07.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.920+0000 7f3167fff640 1 --2- 172.21.15.73:0/508968552 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f316c06c8f0 0x7f316c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:07.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.920+0000 7f3167fff640 1 -- 172.21.15.73:0/508968552 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f31740ac8a0 con 0x7f318015d5f0 2024-04-26T20:51:07.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.920+0000 7f317f7fe640 1 --2- 172.21.15.73:0/508968552 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f316c06c8f0 0x7f316c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:07.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.920+0000 7f317f7fe640 1 --2- 172.21.15.73:0/508968552 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f316c06c8f0 0x7f316c06edb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f31800cd020 tx=0x7f3170008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:07.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:07.924+0000 7f3167fff640 1 -- 172.21.15.73:0/508968552 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3174045360 con 0x7f318015d5f0 2024-04-26T20:51:08.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:08.138+0000 7f3185d86640 1 -- 172.21.15.73:0/508968552 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f31800d29e0 con 0x7f318015d5f0 2024-04-26T20:51:08.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:08.138+0000 7f3167fff640 1 -- 172.21.15.73:0/508968552 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+88 (secure 0 0 0) 0x7f317407a910 con 0x7f318015d5f0 2024-04-26T20:51:08.140 INFO:teuthology.orchestra.run.smithi073.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-26T20:51:08.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:08.140+0000 7f3185d86640 1 -- 172.21.15.73:0/508968552 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f316c06c8f0 msgr2=0x7f316c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:08.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:08.140+0000 7f3185d86640 1 --2- 172.21.15.73:0/508968552 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f316c06c8f0 0x7f316c06edb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f31800cd020 tx=0x7f3170008040 comp rx=0 tx=0).stop 2024-04-26T20:51:08.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:08.140+0000 7f3185d86640 1 -- 172.21.15.73:0/508968552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f318015d5f0 msgr2=0x7f31801592d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:08.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:08.140+0000 7f3185d86640 1 --2- 172.21.15.73:0/508968552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f318015d5f0 0x7f31801592d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f31740098c0 tx=0x7f31740046c0 comp rx=0 tx=0).stop 2024-04-26T20:51:08.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:08.140+0000 7f3185d86640 1 -- 172.21.15.73:0/508968552 shutdown_connections 2024-04-26T20:51:08.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:08.140+0000 7f3185d86640 1 --2- 172.21.15.73:0/508968552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f318015d5f0 0x7f31801592d0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:08.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:08.140+0000 7f3185d86640 1 --2- 172.21.15.73:0/508968552 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f316c06c8f0 0x7f316c06edb0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:08.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:08.140+0000 7f3185d86640 1 --2- 172.21.15.73:0/508968552 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f31801617f0 0x7f31800caf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:08.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:08.140+0000 7f3185d86640 1 --2- 172.21.15.73:0/508968552 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f318015ab60 0x7f318015c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:08.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:08.140+0000 7f3185d86640 1 -- 172.21.15.73:0/508968552 >> 172.21.15.73:0/508968552 conn(0x7f31800d1cf0 msgr2=0x7f318006ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:08.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:08.140+0000 7f3185d86640 1 -- 172.21.15.73:0/508968552 shutdown_connections 2024-04-26T20:51:08.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:08.140+0000 7f3185d86640 1 -- 172.21.15.73:0/508968552 wait complete. 2024-04-26T20:51:08.152 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T20:51:08.152 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T20:51:08.152 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi073.front.sepia.ceph.com 2024-04-26T20:51:08.152 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-26T20:51:08.152 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-26T20:51:08.152 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T20:51:08.152 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T20:51:08.153 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T20:51:08.153 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-26T20:51:08.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:08.153 DEBUG:teuthology.orchestra.run.smithi073:> ip addr 2024-04-26T20:51:08.168 INFO:teuthology.orchestra.run.smithi073.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:51:08.168 INFO:teuthology.orchestra.run.smithi073.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:51:08.168 INFO:teuthology.orchestra.run.smithi073.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:51:08.168 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:51:08.168 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 ::1/128 scope host 2024-04-26T20:51:08.168 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:51:08.168 INFO:teuthology.orchestra.run.smithi073.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:51:08.168 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:ea brd ff:ff:ff:ff:ff:ff 2024-04-26T20:51:08.168 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f0 2024-04-26T20:51:08.168 INFO:teuthology.orchestra.run.smithi073.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:51:08.168 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:eb brd ff:ff:ff:ff:ff:ff 2024-04-26T20:51:08.169 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f1 2024-04-26T20:51:08.169 INFO:teuthology.orchestra.run.smithi073.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:51:08.169 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7c brd ff:ff:ff:ff:ff:ff 2024-04-26T20:51:08.169 INFO:teuthology.orchestra.run.smithi073.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T20:51:08.169 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7d brd ff:ff:ff:ff:ff:ff 2024-04-26T20:51:08.169 INFO:teuthology.orchestra.run.smithi073.stdout: inet 172.21.15.73/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-26T20:51:08.169 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft 42719sec preferred_lft 42719sec 2024-04-26T20:51:08.169 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::b99b:8e91:ffc2:71f/64 scope link noprefixroute 2024-04-26T20:51:08.169 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:51:08.169 INFO:teuthology.orchestra.run.smithi073.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T20:51:08.169 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 66:7f:fa:7c:30:18 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:51:08.169 INFO:teuthology.orchestra.run.smithi073.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T20:51:08.169 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:51:08.169 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::1cd0:bfff:fec2:3f98/64 scope link tentative 2024-04-26T20:51:08.169 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:51:08.170 INFO:teuthology.orchestra.run.smithi073.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T20:51:08.170 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 66:7f:fa:7c:30:18 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:51:08.170 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::647f:faff:fe7c:3018/64 scope link tentative 2024-04-26T20:51:08.170 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:51:08.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:08.170 DEBUG:teuthology.orchestra.run.smithi073:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T20:51:08.229 INFO:teuthology.orchestra.run.smithi073.stdout:1 2024-04-26T20:51:08.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:08.231 DEBUG:teuthology.orchestra.run.smithi073:> route 2024-04-26T20:51:08.287 INFO:teuthology.orchestra.run.smithi073.stdout:Kernel IP routing table 2024-04-26T20:51:08.287 INFO:teuthology.orchestra.run.smithi073.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T20:51:08.287 INFO:teuthology.orchestra.run.smithi073.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T20:51:08.287 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T20:51:08.287 INFO:teuthology.orchestra.run.smithi073.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T20:51:08.288 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:51:08.288 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2024-04-26T20:51:08.288 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T20:51:08.288 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T20:51:08.288 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T20:51:08.288 DEBUG:teuthology.orchestra.run.smithi073:> ') 2024-04-26T20:51:08.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:51:08.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:51:08.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:08.411 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list 2024-04-26T20:51:08.425 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T20:51:08.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:08.426 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list-id 2024-04-26T20:51:08.480 INFO:teuthology.orchestra.run.smithi073.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-26T20:51:08.480 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:51:08.480 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2024-04-26T20:51:08.480 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:51:08.481 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-26T20:51:08.481 DEBUG:teuthology.orchestra.run.smithi073:> ') 2024-04-26T20:51:08.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:51:08.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:51:08.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:08.595 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-26T20:51:08.618 INFO:teuthology.orchestra.run.smithi073.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:51:08.618 INFO:teuthology.orchestra.run.smithi073.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:51:08.618 INFO:teuthology.orchestra.run.smithi073.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:51:08.618 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:51:08.619 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 ::1/128 scope host 2024-04-26T20:51:08.619 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:51:08.619 INFO:teuthology.orchestra.run.smithi073.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T20:51:08.619 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 9e:01:6e:e3:55:c0 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-26T20:51:08.619 INFO:teuthology.orchestra.run.smithi073.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-26T20:51:08.619 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:51:08.619 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::9c01:6eff:fee3:55c0/64 scope link tentative 2024-04-26T20:51:08.619 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:51:08.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:08.620 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-26T20:51:08.685 INFO:teuthology.orchestra.run.smithi073.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:51:08.685 INFO:teuthology.orchestra.run.smithi073.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:51:08.685 INFO:teuthology.orchestra.run.smithi073.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:51:08.685 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:51:08.685 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 ::1/128 scope host 2024-04-26T20:51:08.685 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:51:08.685 INFO:teuthology.orchestra.run.smithi073.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T20:51:08.685 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 9e:01:6e:e3:55:c0 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-26T20:51:08.685 INFO:teuthology.orchestra.run.smithi073.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-26T20:51:08.685 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:51:08.685 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::9c01:6eff:fee3:55c0/64 scope link tentative 2024-04-26T20:51:08.686 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:51:08.687 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-26T20:51:08.687 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:51:08.687 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2024-04-26T20:51:08.687 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-26T20:51:08.687 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-26T20:51:08.687 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-26T20:51:08.687 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-26T20:51:08.687 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-26T20:51:08.687 DEBUG:teuthology.orchestra.run.smithi073:> ') 2024-04-26T20:51:08.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:51:08.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:51:08.835 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:51:08.835 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2024-04-26T20:51:08.835 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link set brx.1 up 2024-04-26T20:51:08.835 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link set dev brx.1 master ceph-brx 2024-04-26T20:51:08.835 DEBUG:teuthology.orchestra.run.smithi073:> ') 2024-04-26T20:51:08.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:51:08.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:51:08.909 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:51:08.909 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-26T20:51:08.924 INFO:teuthology.orchestra.run.smithi073.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-26T20:51:08.924 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:51:08.924 DEBUG:teuthology.orchestra.run.smithi073:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-26T20:51:08.979 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T20:51:08.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:08.979 DEBUG:teuthology.orchestra.run.smithi073:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-26T20:51:09.060 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-26T20:51:09.061 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-26T20:51:09.061 INFO:teuthology.orchestra.run.smithi073.stdout:invalid new device string format 2024-04-26T20:51:09.069 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-26T20:51:09.069 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-26T20:51:09.069 INFO:teuthology.orchestra.run.smithi073.stdout:invalid new device string format 2024-04-26T20:51:09.069 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: resolved to: "172.21.15.73:3300,172.21.15.120:3300,172.21.15.120:3301" 2024-04-26T20:51:09.069 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T20:51:09.074 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-26T20:51:09.074 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-26T20:51:09.074 INFO:teuthology.orchestra.run.smithi073.stdout:invalid new device string format 2024-04-26T20:51:09.074 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: resolved to: "172.21.15.73:3300,172.21.15.120:3300,172.21.15.120:3301" 2024-04-26T20:51:09.075 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T20:51:09.075 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: resolved to: "172.21.15.73:6789,172.21.15.120:6789,172.21.15.120:6790" 2024-04-26T20:51:09.075 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.73:6789,172.21.15.120:6789,172.21.15.120:6790:/ 2024-04-26T20:51:09.075 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=fbd3f548-b547-4713-856b-09f98f5d1764" will pass to kernel 2024-04-26T20:51:09.075 INFO:teuthology.orchestra.run.smithi073.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-04-26T20:51:09.075 INFO:teuthology.orchestra.run.smithi073.stdout: You just mounted a file system that supports labels which does not 2024-04-26T20:51:09.075 INFO:teuthology.orchestra.run.smithi073.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T20:51:09.075 INFO:teuthology.orchestra.run.smithi073.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T20:51:09.075 INFO:teuthology.orchestra.run.smithi073.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T20:51:09.076 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T20:51:09.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:09.077 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-26T20:51:09.141 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T20:51:09.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T20:51:09.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:51:09.236 INFO:teuthology.orchestra.run.smithi073.stdout:client4693 2024-04-26T20:51:09.236 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-26T20:51:09.237 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:51:09.252 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T20:51:09.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T20:51:09.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:51:09.347 INFO:teuthology.orchestra.run.smithi073.stdout:client4693 2024-04-26T20:51:09.347 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T20:51:09.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T20:51:09.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:51:09.402 INFO:teuthology.orchestra.run.smithi073.stdout:client4693 2024-04-26T20:51:09.403 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/fbd3f548-b547-4713-856b-09f98f5d1764.client4693/status') 2024-04-26T20:51:09.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:51:09.459 INFO:teuthology.orchestra.run.smithi073.stdout:instance: client.4693 (3)172.21.15.73:0/1901573581 2024-04-26T20:51:09.460 INFO:teuthology.orchestra.run.smithi073.stdout:blocklisted: false 2024-04-26T20:51:09.460 INFO:teuthology.orchestra.run.smithi073.stderr:0+1 records in 2024-04-26T20:51:09.460 INFO:teuthology.orchestra.run.smithi073.stderr:0+1 records out 2024-04-26T20:51:09.460 INFO:teuthology.orchestra.run.smithi073.stderr:70 bytes copied, 2.2287e-05 s, 3.1 MB/s 2024-04-26T20:51:09.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:51:09.467 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-04-26T20:51:09.467 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T20:51:09.467 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-26T20:51:09.467 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T20:51:09.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.563+0000 7f20c0493640 1 Processor -- start 2024-04-26T20:51:09.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.563+0000 7f20c0493640 1 -- start start 2024-04-26T20:51:09.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.564+0000 7f20c0493640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f20b8071af0 0x7f20b8167bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:09.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.564+0000 7f20c0493640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20b80ce840 con 0x7f20b80d1380 2024-04-26T20:51:09.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.564+0000 7f20c0493640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20b80ceb00 con 0x7f20b815d1e0 2024-04-26T20:51:09.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.564+0000 7f20c0493640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f20b80cede0 con 0x7f20b8071af0 2024-04-26T20:51:09.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.564+0000 7f20bea30640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f20b8071af0 0x7f20b8167bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:09.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.564+0000 7f20bda2e640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f20b80d1380 0x7f20b80d17a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40132/0 (socket says 172.21.15.73:40132) 2024-04-26T20:51:09.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.564+0000 7f20bda2e640 1 -- 172.21.15.73:0/3326322843 learned_addr learned my addr 172.21.15.73:0/3326322843 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:09.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.565+0000 7f20bea30640 1 -- 172.21.15.73:0/3326322843 >> v1:172.21.15.120:6789/0 conn(0x7f20b815d1e0 legacy=0x7f20b80d09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:09.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.565+0000 7f20bea30640 1 -- 172.21.15.73:0/3326322843 >> v1:172.21.15.73:6789/0 conn(0x7f20b80d1380 legacy=0x7f20b80d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:09.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.565+0000 7f20bea30640 1 -- 172.21.15.73:0/3326322843 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20b80702e0 con 0x7f20b8071af0 2024-04-26T20:51:09.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.565+0000 7f20bea30640 1 --2- 172.21.15.73:0/3326322843 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f20b8071af0 0x7f20b8167bc0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f20b40097a0 tx=0x7f20b402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1cea09d21ed87f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:09.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.565+0000 7f20bd22d640 1 -- 172.21.15.73:0/3326322843 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b4037070 con 0x7f20b8071af0 2024-04-26T20:51:09.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.565+0000 7f20bd22d640 1 -- 172.21.15.73:0/3326322843 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20b4032440 con 0x7f20b8071af0 2024-04-26T20:51:09.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.565+0000 7f20bd22d640 1 -- 172.21.15.73:0/3326322843 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b4032810 con 0x7f20b8071af0 2024-04-26T20:51:09.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.566+0000 7f20c0493640 1 -- 172.21.15.73:0/3326322843 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f20b8071af0 msgr2=0x7f20b8167bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:09.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.566+0000 7f20c0493640 1 --2- 172.21.15.73:0/3326322843 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f20b8071af0 0x7f20b8167bc0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f20b40097a0 tx=0x7f20b402a1e0 comp rx=0 tx=0).stop 2024-04-26T20:51:09.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.566+0000 7f20c0493640 1 -- 172.21.15.73:0/3326322843 shutdown_connections 2024-04-26T20:51:09.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.566+0000 7f20c0493640 1 --2- 172.21.15.73:0/3326322843 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f20b8071af0 0x7f20b8167bc0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:09.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.566+0000 7f20c0493640 1 -- 172.21.15.73:0/3326322843 >> 172.21.15.73:0/3326322843 conn(0x7f20b80d2da0 msgr2=0x7f20b80d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:09.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.566+0000 7f20c0493640 1 -- 172.21.15.73:0/3326322843 shutdown_connections 2024-04-26T20:51:09.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.566+0000 7f20c0493640 1 -- 172.21.15.73:0/3326322843 wait complete. 2024-04-26T20:51:09.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.567+0000 7f20c0493640 1 Processor -- start 2024-04-26T20:51:09.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.567+0000 7f20c0493640 1 -- start start 2024-04-26T20:51:09.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.568+0000 7f20c0493640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f20b8071af0 0x7f20b815b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:09.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.568+0000 7f20c0493640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f20b80d1380 0x7f20b80c8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:09.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.568+0000 7f20c0493640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f20b815d1e0 0x7f20b806e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:09.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.568+0000 7f20c0493640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f20b80ce4c0 con 0x7f20b8071af0 2024-04-26T20:51:09.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.568+0000 7f20c0493640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f20b8070650 con 0x7f20b80d1380 2024-04-26T20:51:09.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.568+0000 7f20c0493640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f20b8168100 con 0x7f20b815d1e0 2024-04-26T20:51:09.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.568+0000 7f20be22f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f20b8071af0 0x7f20b815b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:09.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.568+0000 7f20be22f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f20b8071af0 0x7f20b815b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51750/0 (socket says 172.21.15.73:51750) 2024-04-26T20:51:09.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.568+0000 7f20be22f640 1 -- 172.21.15.73:0/521619407 learned_addr learned my addr 172.21.15.73:0/521619407 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:09.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.568+0000 7f20be22f640 1 -- 172.21.15.73:0/521619407 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f20b815d1e0 msgr2=0x7f20b806e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:09.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.568+0000 7f20be22f640 1 --2- 172.21.15.73:0/521619407 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f20b815d1e0 0x7f20b806e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:09.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.568+0000 7f20be22f640 1 -- 172.21.15.73:0/521619407 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f20b80d1380 msgr2=0x7f20b80c8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:09.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.568+0000 7f20bda2e640 1 --2- 172.21.15.73:0/521619407 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f20b80d1380 0x7f20b80c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:09.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.568+0000 7f20be22f640 1 --2- 172.21.15.73:0/521619407 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f20b80d1380 0x7f20b80c8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:09.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.568+0000 7f20be22f640 1 -- 172.21.15.73:0/521619407 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20b80c73f0 con 0x7f20b8071af0 2024-04-26T20:51:09.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.568+0000 7f20be22f640 1 --2- 172.21.15.73:0/521619407 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f20b8071af0 0x7f20b815b780 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f20b000ba50 tx=0x7f20b000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:09.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.568+0000 7f20aeffd640 1 -- 172.21.15.73:0/521619407 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b0019070 con 0x7f20b8071af0 2024-04-26T20:51:09.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.568+0000 7f20aeffd640 1 -- 172.21.15.73:0/521619407 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20b000cc20 con 0x7f20b8071af0 2024-04-26T20:51:09.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.569+0000 7f20aeffd640 1 -- 172.21.15.73:0/521619407 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b0014680 con 0x7f20b8071af0 2024-04-26T20:51:09.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.569+0000 7f20c0493640 1 -- 172.21.15.73:0/521619407 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20b815bcc0 con 0x7f20b8071af0 2024-04-26T20:51:09.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.569+0000 7f20c0493640 1 -- 172.21.15.73:0/521619407 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20b80c7970 con 0x7f20b8071af0 2024-04-26T20:51:09.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.570+0000 7f20c0493640 1 -- 172.21.15.73:0/521619407 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20b80694f0 con 0x7f20b8071af0 2024-04-26T20:51:09.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.570+0000 7f20aeffd640 1 -- 172.21.15.73:0/521619407 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f20b001c070 con 0x7f20b8071af0 2024-04-26T20:51:09.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.570+0000 7f20aeffd640 1 --2- 172.21.15.73:0/521619407 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f209c06c8f0 0x7f209c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:09.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.570+0000 7f20aeffd640 1 -- 172.21.15.73:0/521619407 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f20b008bb20 con 0x7f20b8071af0 2024-04-26T20:51:09.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.570+0000 7f20bda2e640 1 --2- 172.21.15.73:0/521619407 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f209c06c8f0 0x7f209c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:09.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.571+0000 7f20bda2e640 1 --2- 172.21.15.73:0/521619407 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f209c06c8f0 0x7f209c06edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f20b80a0a80 tx=0x7f20a8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:09.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.575+0000 7f20aeffd640 1 -- 172.21.15.73:0/521619407 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f20b0059650 con 0x7f20b8071af0 2024-04-26T20:51:09.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.786+0000 7f20c0493640 1 -- 172.21.15.73:0/521619407 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f20b80cf170 con 0x7f20b8071af0 2024-04-26T20:51:09.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.786+0000 7f20aeffd640 1 -- 172.21.15.73:0/521619407 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+88 (secure 0 0 0) 0x7f20b005d950 con 0x7f20b8071af0 2024-04-26T20:51:09.787 INFO:teuthology.orchestra.run.smithi073.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-26T20:51:09.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.788+0000 7f20c0493640 1 -- 172.21.15.73:0/521619407 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f209c06c8f0 msgr2=0x7f209c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:09.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.788+0000 7f20c0493640 1 --2- 172.21.15.73:0/521619407 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f209c06c8f0 0x7f209c06edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f20b80a0a80 tx=0x7f20a8009040 comp rx=0 tx=0).stop 2024-04-26T20:51:09.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.788+0000 7f20c0493640 1 -- 172.21.15.73:0/521619407 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f20b8071af0 msgr2=0x7f20b815b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:09.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.788+0000 7f20c0493640 1 --2- 172.21.15.73:0/521619407 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f20b8071af0 0x7f20b815b780 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f20b000ba50 tx=0x7f20b000bf10 comp rx=0 tx=0).stop 2024-04-26T20:51:09.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.788+0000 7f20c0493640 1 -- 172.21.15.73:0/521619407 shutdown_connections 2024-04-26T20:51:09.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.788+0000 7f20c0493640 1 --2- 172.21.15.73:0/521619407 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f209c06c8f0 0x7f209c06edb0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:09.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.788+0000 7f20c0493640 1 --2- 172.21.15.73:0/521619407 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f20b815d1e0 0x7f20b806e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:09.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.788+0000 7f20c0493640 1 --2- 172.21.15.73:0/521619407 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f20b80d1380 0x7f20b80c8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:09.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.788+0000 7f20c0493640 1 --2- 172.21.15.73:0/521619407 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f20b8071af0 0x7f20b815b780 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:09.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.788+0000 7f20c0493640 1 -- 172.21.15.73:0/521619407 >> 172.21.15.73:0/521619407 conn(0x7f20b80d2da0 msgr2=0x7f20b80b04f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:09.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.789+0000 7f20c0493640 1 -- 172.21.15.73:0/521619407 shutdown_connections 2024-04-26T20:51:09.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:09.789+0000 7f20c0493640 1 -- 172.21.15.73:0/521619407 wait complete. 2024-04-26T20:51:09.800 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T20:51:09.800 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T20:51:09.800 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi120.front.sepia.ceph.com 2024-04-26T20:51:09.800 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-26T20:51:09.800 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-26T20:51:09.800 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T20:51:09.800 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T20:51:09.800 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T20:51:09.801 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-26T20:51:09.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:09.801 DEBUG:teuthology.orchestra.run.smithi120:> ip addr 2024-04-26T20:51:09.819 INFO:teuthology.orchestra.run.smithi120.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:51:09.819 INFO:teuthology.orchestra.run.smithi120.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:51:09.819 INFO:teuthology.orchestra.run.smithi120.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:51:09.819 INFO:teuthology.orchestra.run.smithi120.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:51:09.819 INFO:teuthology.orchestra.run.smithi120.stdout: inet6 ::1/128 scope host 2024-04-26T20:51:09.819 INFO:teuthology.orchestra.run.smithi120.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:51:09.819 INFO:teuthology.orchestra.run.smithi120.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:51:09.819 INFO:teuthology.orchestra.run.smithi120.stdout: link/ether 0c:c4:7a:d8:a1:26 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:51:09.819 INFO:teuthology.orchestra.run.smithi120.stdout: altname enp7s0f0 2024-04-26T20:51:09.820 INFO:teuthology.orchestra.run.smithi120.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:51:09.820 INFO:teuthology.orchestra.run.smithi120.stdout: link/ether 0c:c4:7a:d8:a1:27 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:51:09.820 INFO:teuthology.orchestra.run.smithi120.stdout: altname enp7s0f1 2024-04-26T20:51:09.820 INFO:teuthology.orchestra.run.smithi120.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:51:09.820 INFO:teuthology.orchestra.run.smithi120.stdout: link/ether 0c:c4:7a:8f:47:8e brd ff:ff:ff:ff:ff:ff 2024-04-26T20:51:09.820 INFO:teuthology.orchestra.run.smithi120.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T20:51:09.820 INFO:teuthology.orchestra.run.smithi120.stdout: link/ether 0c:c4:7a:8f:47:8f brd ff:ff:ff:ff:ff:ff 2024-04-26T20:51:09.820 INFO:teuthology.orchestra.run.smithi120.stdout: inet 172.21.15.120/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-26T20:51:09.820 INFO:teuthology.orchestra.run.smithi120.stdout: valid_lft 42723sec preferred_lft 42723sec 2024-04-26T20:51:09.820 INFO:teuthology.orchestra.run.smithi120.stdout: inet6 fe80::d331:3fa:9337:1d9f/64 scope link noprefixroute 2024-04-26T20:51:09.820 INFO:teuthology.orchestra.run.smithi120.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:51:09.820 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T20:51:09.821 DEBUG:teuthology.orchestra.run.smithi120:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:51:09.821 DEBUG:teuthology.orchestra.run.smithi120:> set -e 2024-04-26T20:51:09.821 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip link add name ceph-brx type bridge 2024-04-26T20:51:09.821 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip addr flush dev ceph-brx 2024-04-26T20:51:09.821 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip link set ceph-brx up 2024-04-26T20:51:09.821 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T20:51:09.821 DEBUG:teuthology.orchestra.run.smithi120:> ') 2024-04-26T20:51:09.914 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:51:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:51:10.004 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:51:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:51:10.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:10.011 DEBUG:teuthology.orchestra.run.smithi120:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T20:51:10.035 INFO:teuthology.orchestra.run.smithi120.stdout:1 2024-04-26T20:51:10.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:10.037 DEBUG:teuthology.orchestra.run.smithi120:> route 2024-04-26T20:51:10.126 INFO:teuthology.orchestra.run.smithi120.stdout:Kernel IP routing table 2024-04-26T20:51:10.126 INFO:teuthology.orchestra.run.smithi120.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T20:51:10.127 INFO:teuthology.orchestra.run.smithi120.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T20:51:10.127 INFO:teuthology.orchestra.run.smithi120.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T20:51:10.127 INFO:teuthology.orchestra.run.smithi120.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T20:51:10.127 DEBUG:teuthology.orchestra.run.smithi120:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:51:10.127 DEBUG:teuthology.orchestra.run.smithi120:> set -e 2024-04-26T20:51:10.127 DEBUG:teuthology.orchestra.run.smithi120:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T20:51:10.127 DEBUG:teuthology.orchestra.run.smithi120:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T20:51:10.127 DEBUG:teuthology.orchestra.run.smithi120:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T20:51:10.127 DEBUG:teuthology.orchestra.run.smithi120:> ') 2024-04-26T20:51:10.175 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:51:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:51:10.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:51:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:51:10.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:10.352 DEBUG:teuthology.orchestra.run.smithi120:> ip netns list 2024-04-26T20:51:10.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:10.368 DEBUG:teuthology.orchestra.run.smithi120:> ip netns list-id 2024-04-26T20:51:10.423 DEBUG:teuthology.orchestra.run.smithi120:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:51:10.423 DEBUG:teuthology.orchestra.run.smithi120:> set -e 2024-04-26T20:51:10.423 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T20:51:10.423 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-26T20:51:10.423 DEBUG:teuthology.orchestra.run.smithi120:> ') 2024-04-26T20:51:10.513 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:51:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:51:10.535 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:51:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:51:10.542 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-26T20:51:10.542 DEBUG:teuthology.orchestra.run.smithi120:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:51:10.542 DEBUG:teuthology.orchestra.run.smithi120:> set -e 2024-04-26T20:51:10.542 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-26T20:51:10.542 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T20:51:10.542 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-26T20:51:10.542 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-26T20:51:10.542 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-26T20:51:10.542 DEBUG:teuthology.orchestra.run.smithi120:> ') 2024-04-26T20:51:10.591 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:51:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:51:10.657 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:51:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:51:10.663 DEBUG:teuthology.orchestra.run.smithi120:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:51:10.663 DEBUG:teuthology.orchestra.run.smithi120:> set -e 2024-04-26T20:51:10.663 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip link set brx.0 up 2024-04-26T20:51:10.663 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T20:51:10.663 DEBUG:teuthology.orchestra.run.smithi120:> ') 2024-04-26T20:51:10.711 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:51:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:51:10.731 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:51:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:51:10.737 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:51:10.738 DEBUG:teuthology.orchestra.run.smithi120:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-26T20:51:10.753 INFO:teuthology.orchestra.run.smithi120.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-26T20:51:10.753 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:51:10.753 DEBUG:teuthology.orchestra.run.smithi120:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-26T20:51:10.807 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T20:51:10.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:10.808 DEBUG:teuthology.orchestra.run.smithi120:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T20:51:10.898 INFO:teuthology.orchestra.run.smithi120.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T20:51:10.898 INFO:teuthology.orchestra.run.smithi120.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-26T20:51:10.899 INFO:teuthology.orchestra.run.smithi120.stdout:invalid new device string format 2024-04-26T20:51:10.997 INFO:teuthology.orchestra.run.smithi120.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T20:51:10.998 INFO:teuthology.orchestra.run.smithi120.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-26T20:51:10.998 INFO:teuthology.orchestra.run.smithi120.stdout:invalid new device string format 2024-04-26T20:51:10.998 INFO:teuthology.orchestra.run.smithi120.stdout:mount.ceph: resolved to: "172.21.15.73:3300,172.21.15.120:3300,172.21.15.120:3301" 2024-04-26T20:51:10.998 INFO:teuthology.orchestra.run.smithi120.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T20:51:11.007 INFO:teuthology.orchestra.run.smithi120.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T20:51:11.007 INFO:teuthology.orchestra.run.smithi120.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-26T20:51:11.007 INFO:teuthology.orchestra.run.smithi120.stdout:invalid new device string format 2024-04-26T20:51:11.007 INFO:teuthology.orchestra.run.smithi120.stdout:mount.ceph: resolved to: "172.21.15.73:3300,172.21.15.120:3300,172.21.15.120:3301" 2024-04-26T20:51:11.007 INFO:teuthology.orchestra.run.smithi120.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T20:51:11.007 INFO:teuthology.orchestra.run.smithi120.stdout:mount.ceph: resolved to: "172.21.15.73:6789,172.21.15.120:6789,172.21.15.120:6790" 2024-04-26T20:51:11.007 INFO:teuthology.orchestra.run.smithi120.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.73:6789,172.21.15.120:6789,172.21.15.120:6790:/ 2024-04-26T20:51:11.008 INFO:teuthology.orchestra.run.smithi120.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=fbd3f548-b547-4713-856b-09f98f5d1764" will pass to kernel 2024-04-26T20:51:11.008 INFO:teuthology.orchestra.run.smithi120.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-26T20:51:11.008 INFO:teuthology.orchestra.run.smithi120.stdout: You just mounted a file system that supports labels which does not 2024-04-26T20:51:11.008 INFO:teuthology.orchestra.run.smithi120.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T20:51:11.008 INFO:teuthology.orchestra.run.smithi120.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T20:51:11.008 INFO:teuthology.orchestra.run.smithi120.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T20:51:11.010 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T20:51:11.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:11.010 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-26T20:51:11.034 DEBUG:teuthology.orchestra.run.smithi120:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T20:51:11.122 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:51:11 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T20:51:11.123 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:51:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:51:11.129 INFO:teuthology.orchestra.run.smithi120.stdout:client4565 2024-04-26T20:51:11.129 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-26T20:51:11.129 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:51:11.144 DEBUG:teuthology.orchestra.run.smithi120:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T20:51:11.232 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:51:11 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T20:51:11.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:51:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:51:11.240 INFO:teuthology.orchestra.run.smithi120.stdout:client4565 2024-04-26T20:51:11.241 DEBUG:teuthology.orchestra.run.smithi120:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T20:51:11.289 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:51:11 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T20:51:11.290 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:51:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:51:11.296 INFO:teuthology.orchestra.run.smithi120.stdout:client4565 2024-04-26T20:51:11.297 DEBUG:teuthology.orchestra.run.smithi120:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/fbd3f548-b547-4713-856b-09f98f5d1764.client4565/status') 2024-04-26T20:51:11.344 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:51:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:51:11.353 INFO:teuthology.orchestra.run.smithi120.stdout:instance: client.4565 (3)192.168.0.1:0/1183576766 2024-04-26T20:51:11.353 INFO:teuthology.orchestra.run.smithi120.stdout:blocklisted: false 2024-04-26T20:51:11.354 INFO:teuthology.orchestra.run.smithi120.stderr:0+1 records in 2024-04-26T20:51:11.354 INFO:teuthology.orchestra.run.smithi120.stderr:0+1 records out 2024-04-26T20:51:11.354 INFO:teuthology.orchestra.run.smithi120.stderr:69 bytes copied, 2.2358e-05 s, 3.1 MB/s 2024-04-26T20:51:11.354 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:51:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:51:11.361 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-04-26T20:51:11.361 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T20:51:11.361 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-26T20:51:11.361 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T20:51:11.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.466+0000 7fe900324640 1 Processor -- start 2024-04-26T20:51:11.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.466+0000 7fe900324640 1 -- start start 2024-04-26T20:51:11.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.467+0000 7fe900324640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fe8f80d1b30 0x7fe8f80d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:11.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.467+0000 7fe900324640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8f80d10d0 con 0x7fe8f8162810 2024-04-26T20:51:11.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.467+0000 7fe900324640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8f80d1370 con 0x7fe8f80b2340 2024-04-26T20:51:11.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.467+0000 7fe900324640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fe8f80d0750 con 0x7fe8f80d1b30 2024-04-26T20:51:11.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.467+0000 7fe8fe0c0640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fe8f8162810 0x7fe8f8070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40146/0 (socket says 172.21.15.73:40146) 2024-04-26T20:51:11.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.467+0000 7fe8fe0c0640 1 -- 172.21.15.73:0/1922549276 learned_addr learned my addr 172.21.15.73:0/1922549276 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:11.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.468+0000 7fe8fe8c1640 1 --2- 172.21.15.73:0/1922549276 >> v2:172.21.15.120:3300/0 conn(0x7fe8f80d1b30 0x7fe8f80d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:11.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.468+0000 7fe8fd0be640 1 -- 172.21.15.73:0/1922549276 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1519257679 0 0) 0x7fe8f80d10d0 con 0x7fe8f8162810 2024-04-26T20:51:11.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.468+0000 7fe8fd0be640 1 -- 172.21.15.73:0/1922549276 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe8e4003520 con 0x7fe8f8162810 2024-04-26T20:51:11.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.468+0000 7fe8fd0be640 1 -- 172.21.15.73:0/1922549276 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3590570280 0 0) 0x7fe8e4003520 con 0x7fe8f8162810 2024-04-26T20:51:11.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.468+0000 7fe8fd0be640 1 -- 172.21.15.73:0/1922549276 >> v2:172.21.15.120:3300/0 conn(0x7fe8f80d1b30 msgr2=0x7fe8f80d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:11.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.468+0000 7fe8fd0be640 1 --2- 172.21.15.73:0/1922549276 >> v2:172.21.15.120:3300/0 conn(0x7fe8f80d1b30 0x7fe8f80d1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:11.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.468+0000 7fe8fd0be640 1 -- 172.21.15.73:0/1922549276 >> v1:172.21.15.120:6789/0 conn(0x7fe8f80b2340 legacy=0x7fe8f80cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:11.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.468+0000 7fe8fd0be640 1 -- 172.21.15.73:0/1922549276 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8f80d01b0 con 0x7fe8f8162810 2024-04-26T20:51:11.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.468+0000 7fe8fd0be640 1 -- 172.21.15.73:0/1922549276 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7fe8f40019f0 con 0x7fe8f8162810 2024-04-26T20:51:11.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.468+0000 7fe8fd0be640 1 -- 172.21.15.73:0/1922549276 >> v1:172.21.15.73:6789/0 conn(0x7fe8f8162810 legacy=0x7fe8f8070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:11.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.468+0000 7fe8fd0be640 1 --2- 172.21.15.73:0/1922549276 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8e40039b0 0x7fe8e4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:11.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.468+0000 7fe8fd0be640 1 --2- 172.21.15.73:0/1922549276 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe8e4004800 0x7fe8e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:11.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.468+0000 7fe8fd0be640 1 --2- 172.21.15.73:0/1922549276 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe8e40051c0 0x7fe8e400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:11.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.468+0000 7fe8fd0be640 1 -- 172.21.15.73:0/1922549276 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f80d0750 con 0x7fe8e40039b0 2024-04-26T20:51:11.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.468+0000 7fe8fd0be640 1 -- 172.21.15.73:0/1922549276 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e400c060 con 0x7fe8e4004800 2024-04-26T20:51:11.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.468+0000 7fe8fd0be640 1 -- 172.21.15.73:0/1922549276 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fe8e400c260 con 0x7fe8e40051c0 2024-04-26T20:51:11.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.468+0000 7fe8fe0c0640 1 --2- 172.21.15.73:0/1922549276 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8e40039b0 0x7fe8e4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:11.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.468+0000 7fe8fd8bf640 1 --2- 172.21.15.73:0/1922549276 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe8e4004800 0x7fe8e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:11.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.468+0000 7fe8fe8c1640 1 --2- 172.21.15.73:0/1922549276 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe8e40051c0 0x7fe8e400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:11.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.468+0000 7fe8fe0c0640 1 -- 172.21.15.73:0/1922549276 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe8e40051c0 msgr2=0x7fe8e400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:11.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.468+0000 7fe8fe0c0640 1 --2- 172.21.15.73:0/1922549276 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe8e40051c0 0x7fe8e400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:11.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.468+0000 7fe8fe0c0640 1 -- 172.21.15.73:0/1922549276 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe8e4004800 msgr2=0x7fe8e4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:11.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.468+0000 7fe8fe0c0640 1 --2- 172.21.15.73:0/1922549276 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe8e4004800 0x7fe8e4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:11.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.468+0000 7fe8fe0c0640 1 -- 172.21.15.73:0/1922549276 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8e4003520 con 0x7fe8e40039b0 2024-04-26T20:51:11.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.468+0000 7fe8fd8bf640 1 --2- 172.21.15.73:0/1922549276 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe8e4004800 0x7fe8e4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:11.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.468+0000 7fe8fe8c1640 1 -- 172.21.15.73:0/1922549276 reap_dead start 2024-04-26T20:51:11.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.469+0000 7fe8fe0c0640 1 --2- 172.21.15.73:0/1922549276 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8e40039b0 0x7fe8e4003dd0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe8f400a860 tx=0x7fe8e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6324ad67ed57c45b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:11.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.469+0000 7fe8fd0be640 1 -- 172.21.15.73:0/1922549276 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f400a480 con 0x7fe8e40039b0 2024-04-26T20:51:11.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.469+0000 7fe8fd0be640 1 -- 172.21.15.73:0/1922549276 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8f402f070 con 0x7fe8e40039b0 2024-04-26T20:51:11.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.469+0000 7fe8fd0be640 1 -- 172.21.15.73:0/1922549276 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f402bcd0 con 0x7fe8e40039b0 2024-04-26T20:51:11.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.470+0000 7fe900324640 1 -- 172.21.15.73:0/1922549276 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8e40039b0 msgr2=0x7fe8e4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:11.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.470+0000 7fe900324640 1 --2- 172.21.15.73:0/1922549276 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8e40039b0 0x7fe8e4003dd0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe8f400a860 tx=0x7fe8e4005830 comp rx=0 tx=0).stop 2024-04-26T20:51:11.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.470+0000 7fe900324640 1 -- 172.21.15.73:0/1922549276 shutdown_connections 2024-04-26T20:51:11.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.470+0000 7fe900324640 1 --2- 172.21.15.73:0/1922549276 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8e40039b0 0x7fe8e4003dd0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:11.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.470+0000 7fe900324640 1 -- 172.21.15.73:0/1922549276 >> 172.21.15.73:0/1922549276 conn(0x7fe8f80d2da0 msgr2=0x7fe8f80d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:11.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.470+0000 7fe900324640 1 -- 172.21.15.73:0/1922549276 shutdown_connections 2024-04-26T20:51:11.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.470+0000 7fe900324640 1 -- 172.21.15.73:0/1922549276 wait complete. 2024-04-26T20:51:11.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.471+0000 7fe900324640 1 Processor -- start 2024-04-26T20:51:11.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.471+0000 7fe900324640 1 -- start start 2024-04-26T20:51:11.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.472+0000 7fe900324640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe8e40039b0 0x7fe8f8154400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:11.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.472+0000 7fe900324640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe8f8153e00 0x7fe8f80c8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:11.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.472+0000 7fe900324640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8f806e700 0x7fe8f806eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:11.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.472+0000 7fe900324640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f80d1fc0 con 0x7fe8f806e700 2024-04-26T20:51:11.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.472+0000 7fe900324640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f80d16a0 con 0x7fe8e40039b0 2024-04-26T20:51:11.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.472+0000 7fe900324640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fe8f80d2140 con 0x7fe8f8153e00 2024-04-26T20:51:11.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.472+0000 7fe8fe8c1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8f806e700 0x7fe8f806eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:11.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.472+0000 7fe8fe0c0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe8e40039b0 0x7fe8f8154400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:11.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.472+0000 7fe8fe8c1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8f806e700 0x7fe8f806eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51762/0 (socket says 172.21.15.73:51762) 2024-04-26T20:51:11.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.472+0000 7fe8fe8c1640 1 -- 172.21.15.73:0/1275983092 learned_addr learned my addr 172.21.15.73:0/1275983092 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:11.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.472+0000 7fe8fd8bf640 1 --2- 172.21.15.73:0/1275983092 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe8f8153e00 0x7fe8f80c8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:11.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.472+0000 7fe8fe8c1640 1 -- 172.21.15.73:0/1275983092 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe8f8153e00 msgr2=0x7fe8f80c8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:11.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.472+0000 7fe8fe8c1640 1 --2- 172.21.15.73:0/1275983092 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe8f8153e00 0x7fe8f80c8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:11.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.472+0000 7fe8fe8c1640 1 -- 172.21.15.73:0/1275983092 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe8e40039b0 msgr2=0x7fe8f8154400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:11.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.472+0000 7fe8fe8c1640 1 --2- 172.21.15.73:0/1275983092 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe8e40039b0 0x7fe8f8154400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:11.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.472+0000 7fe8fe8c1640 1 -- 172.21.15.73:0/1275983092 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8f80c68b0 con 0x7fe8f806e700 2024-04-26T20:51:11.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.472+0000 7fe8fe8c1640 1 --2- 172.21.15.73:0/1275983092 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8f806e700 0x7fe8f806eb80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe8f80cf3c0 tx=0x7fe8f0011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:11.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.473+0000 7fe8eeffd640 1 -- 172.21.15.73:0/1275983092 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f0019020 con 0x7fe8f806e700 2024-04-26T20:51:11.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.473+0000 7fe8eeffd640 1 -- 172.21.15.73:0/1275983092 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8f0003610 con 0x7fe8f806e700 2024-04-26T20:51:11.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.473+0000 7fe8eeffd640 1 -- 172.21.15.73:0/1275983092 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f0003a40 con 0x7fe8f806e700 2024-04-26T20:51:11.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.473+0000 7fe900324640 1 -- 172.21.15.73:0/1275983092 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8f80c6b90 con 0x7fe8f806e700 2024-04-26T20:51:11.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.473+0000 7fe900324640 1 -- 172.21.15.73:0/1275983092 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8f80c7150 con 0x7fe8f806e700 2024-04-26T20:51:11.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.474+0000 7fe900324640 1 -- 172.21.15.73:0/1275983092 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8f806f0c0 con 0x7fe8f806e700 2024-04-26T20:51:11.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.474+0000 7fe8eeffd640 1 -- 172.21.15.73:0/1275983092 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe8f001c020 con 0x7fe8f806e700 2024-04-26T20:51:11.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.475+0000 7fe8eeffd640 1 --2- 172.21.15.73:0/1275983092 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe8dc06c920 0x7fe8dc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:11.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.475+0000 7fe8fe0c0640 1 --2- 172.21.15.73:0/1275983092 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe8dc06c920 0x7fe8dc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:11.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.475+0000 7fe8eeffd640 1 -- 172.21.15.73:0/1275983092 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe8f008be00 con 0x7fe8f806e700 2024-04-26T20:51:11.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.475+0000 7fe8fe0c0640 1 --2- 172.21.15.73:0/1275983092 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe8dc06c920 0x7fe8dc06ede0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4033000 tx=0x7fe8f4009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:11.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.479+0000 7fe8eeffd640 1 -- 172.21.15.73:0/1275983092 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe8f0023030 con 0x7fe8f806e700 2024-04-26T20:51:11.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.694+0000 7fe900324640 1 -- 172.21.15.73:0/1275983092 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe8f80694f0 con 0x7fe8f806e700 2024-04-26T20:51:11.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.695+0000 7fe8eeffd640 1 -- 172.21.15.73:0/1275983092 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+88 (secure 0 0 0) 0x7fe8f0059930 con 0x7fe8f806e700 2024-04-26T20:51:11.696 INFO:teuthology.orchestra.run.smithi073.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-26T20:51:11.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.697+0000 7fe900324640 1 -- 172.21.15.73:0/1275983092 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe8dc06c920 msgr2=0x7fe8dc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:11.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.697+0000 7fe900324640 1 --2- 172.21.15.73:0/1275983092 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe8dc06c920 0x7fe8dc06ede0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4033000 tx=0x7fe8f4009030 comp rx=0 tx=0).stop 2024-04-26T20:51:11.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.697+0000 7fe900324640 1 -- 172.21.15.73:0/1275983092 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8f806e700 msgr2=0x7fe8f806eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:11.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.697+0000 7fe900324640 1 --2- 172.21.15.73:0/1275983092 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8f806e700 0x7fe8f806eb80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe8f80cf3c0 tx=0x7fe8f0011920 comp rx=0 tx=0).stop 2024-04-26T20:51:11.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.697+0000 7fe900324640 1 -- 172.21.15.73:0/1275983092 shutdown_connections 2024-04-26T20:51:11.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.697+0000 7fe900324640 1 --2- 172.21.15.73:0/1275983092 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8f806e700 0x7fe8f806eb80 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:11.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.697+0000 7fe900324640 1 --2- 172.21.15.73:0/1275983092 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe8dc06c920 0x7fe8dc06ede0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:11.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.697+0000 7fe900324640 1 --2- 172.21.15.73:0/1275983092 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe8f8153e00 0x7fe8f80c8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:11.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.697+0000 7fe900324640 1 --2- 172.21.15.73:0/1275983092 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe8e40039b0 0x7fe8f8154400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:11.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.697+0000 7fe900324640 1 -- 172.21.15.73:0/1275983092 >> 172.21.15.73:0/1275983092 conn(0x7fe8f80d2da0 msgr2=0x7fe8f80cf410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:11.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.697+0000 7fe900324640 1 -- 172.21.15.73:0/1275983092 shutdown_connections 2024-04-26T20:51:11.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:11.697+0000 7fe900324640 1 -- 172.21.15.73:0/1275983092 wait complete. 2024-04-26T20:51:11.708 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T20:51:11.708 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T20:51:11.708 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi120.front.sepia.ceph.com 2024-04-26T20:51:11.709 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-26T20:51:11.709 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-26T20:51:11.709 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T20:51:11.709 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T20:51:11.709 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T20:51:11.709 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-26T20:51:11.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:11.709 DEBUG:teuthology.orchestra.run.smithi120:> ip addr 2024-04-26T20:51:11.725 INFO:teuthology.orchestra.run.smithi120.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:51:11.725 INFO:teuthology.orchestra.run.smithi120.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:51:11.725 INFO:teuthology.orchestra.run.smithi120.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:51:11.725 INFO:teuthology.orchestra.run.smithi120.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:51:11.725 INFO:teuthology.orchestra.run.smithi120.stdout: inet6 ::1/128 scope host 2024-04-26T20:51:11.725 INFO:teuthology.orchestra.run.smithi120.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:51:11.725 INFO:teuthology.orchestra.run.smithi120.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:51:11.725 INFO:teuthology.orchestra.run.smithi120.stdout: link/ether 0c:c4:7a:d8:a1:26 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:51:11.725 INFO:teuthology.orchestra.run.smithi120.stdout: altname enp7s0f0 2024-04-26T20:51:11.725 INFO:teuthology.orchestra.run.smithi120.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:51:11.725 INFO:teuthology.orchestra.run.smithi120.stdout: link/ether 0c:c4:7a:d8:a1:27 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:51:11.725 INFO:teuthology.orchestra.run.smithi120.stdout: altname enp7s0f1 2024-04-26T20:51:11.725 INFO:teuthology.orchestra.run.smithi120.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:51:11.726 INFO:teuthology.orchestra.run.smithi120.stdout: link/ether 0c:c4:7a:8f:47:8e brd ff:ff:ff:ff:ff:ff 2024-04-26T20:51:11.726 INFO:teuthology.orchestra.run.smithi120.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T20:51:11.726 INFO:teuthology.orchestra.run.smithi120.stdout: link/ether 0c:c4:7a:8f:47:8f brd ff:ff:ff:ff:ff:ff 2024-04-26T20:51:11.726 INFO:teuthology.orchestra.run.smithi120.stdout: inet 172.21.15.120/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-26T20:51:11.726 INFO:teuthology.orchestra.run.smithi120.stdout: valid_lft 42721sec preferred_lft 42721sec 2024-04-26T20:51:11.726 INFO:teuthology.orchestra.run.smithi120.stdout: inet6 fe80::d331:3fa:9337:1d9f/64 scope link noprefixroute 2024-04-26T20:51:11.726 INFO:teuthology.orchestra.run.smithi120.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:51:11.726 INFO:teuthology.orchestra.run.smithi120.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T20:51:11.726 INFO:teuthology.orchestra.run.smithi120.stdout: link/ether 46:9f:a8:81:73:ed brd ff:ff:ff:ff:ff:ff 2024-04-26T20:51:11.726 INFO:teuthology.orchestra.run.smithi120.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T20:51:11.726 INFO:teuthology.orchestra.run.smithi120.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:51:11.726 INFO:teuthology.orchestra.run.smithi120.stdout: inet6 fe80::6ccc:1ff:feec:ae38/64 scope link tentative 2024-04-26T20:51:11.726 INFO:teuthology.orchestra.run.smithi120.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:51:11.726 INFO:teuthology.orchestra.run.smithi120.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T20:51:11.727 INFO:teuthology.orchestra.run.smithi120.stdout: link/ether 46:9f:a8:81:73:ed brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T20:51:11.727 INFO:teuthology.orchestra.run.smithi120.stdout: inet6 fe80::449f:a8ff:fe81:73ed/64 scope link tentative 2024-04-26T20:51:11.727 INFO:teuthology.orchestra.run.smithi120.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:51:11.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:11.727 DEBUG:teuthology.orchestra.run.smithi120:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T20:51:11.789 INFO:teuthology.orchestra.run.smithi120.stdout:1 2024-04-26T20:51:11.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:11.790 DEBUG:teuthology.orchestra.run.smithi120:> route 2024-04-26T20:51:11.845 INFO:teuthology.orchestra.run.smithi120.stdout:Kernel IP routing table 2024-04-26T20:51:11.846 INFO:teuthology.orchestra.run.smithi120.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T20:51:11.846 INFO:teuthology.orchestra.run.smithi120.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T20:51:11.846 INFO:teuthology.orchestra.run.smithi120.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T20:51:11.846 INFO:teuthology.orchestra.run.smithi120.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T20:51:11.846 DEBUG:teuthology.orchestra.run.smithi120:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:51:11.846 DEBUG:teuthology.orchestra.run.smithi120:> set -e 2024-04-26T20:51:11.846 DEBUG:teuthology.orchestra.run.smithi120:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T20:51:11.846 DEBUG:teuthology.orchestra.run.smithi120:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T20:51:11.847 DEBUG:teuthology.orchestra.run.smithi120:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T20:51:11.847 DEBUG:teuthology.orchestra.run.smithi120:> ') 2024-04-26T20:51:11.934 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:51:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:51:11.964 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:51:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:51:11.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:11.970 DEBUG:teuthology.orchestra.run.smithi120:> ip netns list 2024-04-26T20:51:11.985 INFO:teuthology.orchestra.run.smithi120.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-26T20:51:11.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:11.985 DEBUG:teuthology.orchestra.run.smithi120:> ip netns list-id 2024-04-26T20:51:12.040 INFO:teuthology.orchestra.run.smithi120.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-26T20:51:12.040 DEBUG:teuthology.orchestra.run.smithi120:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:51:12.040 DEBUG:teuthology.orchestra.run.smithi120:> set -e 2024-04-26T20:51:12.040 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T20:51:12.040 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-26T20:51:12.040 DEBUG:teuthology.orchestra.run.smithi120:> ') 2024-04-26T20:51:12.127 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:51:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:51:12.148 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:51:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:51:12.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:12.154 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-26T20:51:12.177 INFO:teuthology.orchestra.run.smithi120.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:51:12.177 INFO:teuthology.orchestra.run.smithi120.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:51:12.177 INFO:teuthology.orchestra.run.smithi120.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:51:12.177 INFO:teuthology.orchestra.run.smithi120.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:51:12.177 INFO:teuthology.orchestra.run.smithi120.stdout: inet6 ::1/128 scope host 2024-04-26T20:51:12.177 INFO:teuthology.orchestra.run.smithi120.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:51:12.177 INFO:teuthology.orchestra.run.smithi120.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T20:51:12.177 INFO:teuthology.orchestra.run.smithi120.stdout: link/ether f6:8f:66:d1:ec:62 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-26T20:51:12.177 INFO:teuthology.orchestra.run.smithi120.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-26T20:51:12.177 INFO:teuthology.orchestra.run.smithi120.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:51:12.177 INFO:teuthology.orchestra.run.smithi120.stdout: inet6 fe80::f48f:66ff:fed1:ec62/64 scope link 2024-04-26T20:51:12.178 INFO:teuthology.orchestra.run.smithi120.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:51:12.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:12.179 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-26T20:51:12.242 INFO:teuthology.orchestra.run.smithi120.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:51:12.242 INFO:teuthology.orchestra.run.smithi120.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:51:12.242 INFO:teuthology.orchestra.run.smithi120.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:51:12.242 INFO:teuthology.orchestra.run.smithi120.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:51:12.242 INFO:teuthology.orchestra.run.smithi120.stdout: inet6 ::1/128 scope host 2024-04-26T20:51:12.242 INFO:teuthology.orchestra.run.smithi120.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:51:12.242 INFO:teuthology.orchestra.run.smithi120.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T20:51:12.242 INFO:teuthology.orchestra.run.smithi120.stdout: link/ether f6:8f:66:d1:ec:62 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-26T20:51:12.242 INFO:teuthology.orchestra.run.smithi120.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-26T20:51:12.242 INFO:teuthology.orchestra.run.smithi120.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:51:12.243 INFO:teuthology.orchestra.run.smithi120.stdout: inet6 fe80::f48f:66ff:fed1:ec62/64 scope link 2024-04-26T20:51:12.243 INFO:teuthology.orchestra.run.smithi120.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:51:12.244 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-26T20:51:12.244 DEBUG:teuthology.orchestra.run.smithi120:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:51:12.244 DEBUG:teuthology.orchestra.run.smithi120:> set -e 2024-04-26T20:51:12.244 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-26T20:51:12.244 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-26T20:51:12.244 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-26T20:51:12.244 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-26T20:51:12.244 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-26T20:51:12.244 DEBUG:teuthology.orchestra.run.smithi120:> ') 2024-04-26T20:51:12.331 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:51:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:51:12.386 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:51:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:51:12.392 DEBUG:teuthology.orchestra.run.smithi120:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:51:12.392 DEBUG:teuthology.orchestra.run.smithi120:> set -e 2024-04-26T20:51:12.392 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip link set brx.1 up 2024-04-26T20:51:12.392 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip link set dev brx.1 master ceph-brx 2024-04-26T20:51:12.392 DEBUG:teuthology.orchestra.run.smithi120:> ') 2024-04-26T20:51:12.439 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:51:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:51:12.460 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:51:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:51:12.466 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:51:12.466 DEBUG:teuthology.orchestra.run.smithi120:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-26T20:51:12.485 INFO:teuthology.orchestra.run.smithi120.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-26T20:51:12.485 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:51:12.485 DEBUG:teuthology.orchestra.run.smithi120:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-26T20:51:12.539 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T20:51:12.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:12.540 DEBUG:teuthology.orchestra.run.smithi120:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-26T20:51:12.620 INFO:teuthology.orchestra.run.smithi120.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-26T20:51:12.620 INFO:teuthology.orchestra.run.smithi120.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-26T20:51:12.621 INFO:teuthology.orchestra.run.smithi120.stdout:invalid new device string format 2024-04-26T20:51:12.629 INFO:teuthology.orchestra.run.smithi120.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-26T20:51:12.629 INFO:teuthology.orchestra.run.smithi120.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-26T20:51:12.629 INFO:teuthology.orchestra.run.smithi120.stdout:invalid new device string format 2024-04-26T20:51:12.629 INFO:teuthology.orchestra.run.smithi120.stdout:mount.ceph: resolved to: "172.21.15.73:3300,172.21.15.120:3300,172.21.15.120:3301" 2024-04-26T20:51:12.629 INFO:teuthology.orchestra.run.smithi120.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T20:51:12.648 INFO:teuthology.orchestra.run.smithi120.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-26T20:51:12.648 INFO:teuthology.orchestra.run.smithi120.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-26T20:51:12.648 INFO:teuthology.orchestra.run.smithi120.stdout:invalid new device string format 2024-04-26T20:51:12.648 INFO:teuthology.orchestra.run.smithi120.stdout:mount.ceph: resolved to: "172.21.15.73:3300,172.21.15.120:3300,172.21.15.120:3301" 2024-04-26T20:51:12.648 INFO:teuthology.orchestra.run.smithi120.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T20:51:12.648 INFO:teuthology.orchestra.run.smithi120.stdout:mount.ceph: resolved to: "172.21.15.73:6789,172.21.15.120:6789,172.21.15.120:6790" 2024-04-26T20:51:12.648 INFO:teuthology.orchestra.run.smithi120.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.73:6789,172.21.15.120:6789,172.21.15.120:6790:/ 2024-04-26T20:51:12.648 INFO:teuthology.orchestra.run.smithi120.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=fbd3f548-b547-4713-856b-09f98f5d1764" will pass to kernel 2024-04-26T20:51:12.649 INFO:teuthology.orchestra.run.smithi120.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-04-26T20:51:12.649 INFO:teuthology.orchestra.run.smithi120.stdout: You just mounted a file system that supports labels which does not 2024-04-26T20:51:12.649 INFO:teuthology.orchestra.run.smithi120.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T20:51:12.649 INFO:teuthology.orchestra.run.smithi120.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T20:51:12.649 INFO:teuthology.orchestra.run.smithi120.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T20:51:12.650 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T20:51:12.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:12.651 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-26T20:51:12.715 DEBUG:teuthology.orchestra.run.smithi120:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T20:51:12.807 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:51:12 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T20:51:12.810 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:51:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:51:12.815 INFO:teuthology.orchestra.run.smithi120.stdout:client4571 2024-04-26T20:51:12.815 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-26T20:51:12.815 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:51:12.830 DEBUG:teuthology.orchestra.run.smithi120:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T20:51:12.918 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:51:12 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T20:51:12.919 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:51:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:51:12.925 INFO:teuthology.orchestra.run.smithi120.stdout:client4571 2024-04-26T20:51:12.925 DEBUG:teuthology.orchestra.run.smithi120:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T20:51:12.972 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:51:12 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T20:51:12.974 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:51:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:51:12.979 INFO:teuthology.orchestra.run.smithi120.stdout:client4571 2024-04-26T20:51:12.980 DEBUG:teuthology.orchestra.run.smithi120:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/fbd3f548-b547-4713-856b-09f98f5d1764.client4571/status') 2024-04-26T20:51:13.026 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:51:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:51:13.036 INFO:teuthology.orchestra.run.smithi120.stdout:instance: client.4571 (3)192.168.0.2:0/3023075931 2024-04-26T20:51:13.036 INFO:teuthology.orchestra.run.smithi120.stdout:blocklisted: false 2024-04-26T20:51:13.036 INFO:teuthology.orchestra.run.smithi120.stderr:0+1 records in 2024-04-26T20:51:13.036 INFO:teuthology.orchestra.run.smithi120.stderr:0+1 records out 2024-04-26T20:51:13.036 INFO:teuthology.orchestra.run.smithi120.stderr:69 bytes copied, 2.1698e-05 s, 3.2 MB/s 2024-04-26T20:51:13.037 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-26T20:51:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:51:13.043 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-26T20:51:13.061 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:51:13.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.162+0000 7f1d6906d640 1 Processor -- start 2024-04-26T20:51:13.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.162+0000 7f1d6906d640 1 -- start start 2024-04-26T20:51:13.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.163+0000 7f1d6906d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f1d641608f0 0x7f1d640d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:13.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.163+0000 7f1d6906d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1d640d0a90 0x7f1d640cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:13.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.163+0000 7f1d6906d640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d640cde20 con 0x7f1d640cfdd0 2024-04-26T20:51:13.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.163+0000 7f1d6906d640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f1d640cea30 con 0x7f1d641608f0 2024-04-26T20:51:13.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.163+0000 7f1d6906d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f1d640ce100 con 0x7f1d640d0a90 2024-04-26T20:51:13.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.163+0000 7f1d6359e640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f1d640cfdd0 0x7f1d64160e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40156/0 (socket says 172.21.15.73:40156) 2024-04-26T20:51:13.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.163+0000 7f1d6359e640 1 -- 172.21.15.73:0/1437271332 learned_addr learned my addr 172.21.15.73:0/1437271332 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:13.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.163+0000 7f1d6259c640 1 --2- 172.21.15.73:0/1437271332 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1d640d0a90 0x7f1d640cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:13.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.163+0000 7f1d62d9d640 1 --2- 172.21.15.73:0/1437271332 >> v2:172.21.15.120:3300/0 conn(0x7f1d641608f0 0x7f1d640d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:13.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.163+0000 7f1d61d9b640 1 -- 172.21.15.73:0/1437271332 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1132121510 0 0) 0x7f1d640cde20 con 0x7f1d640cfdd0 2024-04-26T20:51:13.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.164+0000 7f1d61d9b640 1 -- 172.21.15.73:0/1437271332 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d4c003520 con 0x7f1d640cfdd0 2024-04-26T20:51:13.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.164+0000 7f1d61d9b640 1 -- 172.21.15.73:0/1437271332 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1631581863 0 0) 0x7f1d4c003520 con 0x7f1d640cfdd0 2024-04-26T20:51:13.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.164+0000 7f1d61d9b640 1 -- 172.21.15.73:0/1437271332 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1d640d0a90 msgr2=0x7f1d640cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:13.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.164+0000 7f1d61d9b640 1 --2- 172.21.15.73:0/1437271332 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1d640d0a90 0x7f1d640cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:13.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.164+0000 7f1d61d9b640 1 -- 172.21.15.73:0/1437271332 >> v2:172.21.15.120:3300/0 conn(0x7f1d641608f0 msgr2=0x7f1d640d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:13.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.164+0000 7f1d61d9b640 1 --2- 172.21.15.73:0/1437271332 >> v2:172.21.15.120:3300/0 conn(0x7f1d641608f0 0x7f1d640d1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:13.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.164+0000 7f1d61d9b640 1 -- 172.21.15.73:0/1437271332 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d640ce3e0 con 0x7f1d640cfdd0 2024-04-26T20:51:13.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.164+0000 7f1d61d9b640 1 -- 172.21.15.73:0/1437271332 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7f1d5c001670 con 0x7f1d640cfdd0 2024-04-26T20:51:13.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.164+0000 7f1d62d9d640 1 --2- 172.21.15.73:0/1437271332 >> v2:172.21.15.120:3300/0 conn(0x7f1d641608f0 0x7f1d640d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:13.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.164+0000 7f1d61d9b640 1 -- 172.21.15.73:0/1437271332 >> v1:172.21.15.73:6789/0 conn(0x7f1d640cfdd0 legacy=0x7f1d64160e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:13.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.164+0000 7f1d61d9b640 1 --2- 172.21.15.73:0/1437271332 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d4c0039b0 0x7f1d4c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:13.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.164+0000 7f1d61d9b640 1 --2- 172.21.15.73:0/1437271332 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d4c004800 0x7f1d4c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:13.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.164+0000 7f1d61d9b640 1 --2- 172.21.15.73:0/1437271332 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1d640d0a90 0x7f1d4c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:13.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.164+0000 7f1d61d9b640 1 -- 172.21.15.73:0/1437271332 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1d640cea30 con 0x7f1d4c0039b0 2024-04-26T20:51:13.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.164+0000 7f1d61d9b640 1 -- 172.21.15.73:0/1437271332 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1d640ce100 con 0x7f1d4c004800 2024-04-26T20:51:13.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.164+0000 7f1d61d9b640 1 -- 172.21.15.73:0/1437271332 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f1d4c00b9c0 con 0x7f1d640d0a90 2024-04-26T20:51:13.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.164+0000 7f1d62d9d640 1 --2- 172.21.15.73:0/1437271332 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d4c0039b0 0x7f1d4c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:13.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.164+0000 7f1d6259c640 1 --2- 172.21.15.73:0/1437271332 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d4c004800 0x7f1d4c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:13.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.164+0000 7f1d6359e640 1 --2- 172.21.15.73:0/1437271332 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1d640d0a90 0x7f1d4c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:13.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.164+0000 7f1d62d9d640 1 -- 172.21.15.73:0/1437271332 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1d640d0a90 msgr2=0x7f1d4c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:13.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.164+0000 7f1d62d9d640 1 --2- 172.21.15.73:0/1437271332 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1d640d0a90 0x7f1d4c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:13.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.164+0000 7f1d62d9d640 1 -- 172.21.15.73:0/1437271332 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d4c004800 msgr2=0x7f1d4c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:13.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.164+0000 7f1d62d9d640 1 --2- 172.21.15.73:0/1437271332 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d4c004800 0x7f1d4c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:13.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.164+0000 7f1d62d9d640 1 -- 172.21.15.73:0/1437271332 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d4c003520 con 0x7f1d4c0039b0 2024-04-26T20:51:13.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.165+0000 7f1d62d9d640 1 --2- 172.21.15.73:0/1437271332 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d4c0039b0 0x7f1d4c003dd0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1d540073e0 tx=0x7f1d4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c471281fb0411198 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:13.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.165+0000 7f1d61d9b640 1 -- 172.21.15.73:0/1437271332 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d54009c80 con 0x7f1d4c0039b0 2024-04-26T20:51:13.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.165+0000 7f1d61d9b640 1 -- 172.21.15.73:0/1437271332 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d5402c680 con 0x7f1d4c0039b0 2024-04-26T20:51:13.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.165+0000 7f1d61d9b640 1 -- 172.21.15.73:0/1437271332 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5402ca30 con 0x7f1d4c0039b0 2024-04-26T20:51:13.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.165+0000 7f1d6906d640 1 -- 172.21.15.73:0/1437271332 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d4c0039b0 msgr2=0x7f1d4c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:13.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.165+0000 7f1d6906d640 1 --2- 172.21.15.73:0/1437271332 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d4c0039b0 0x7f1d4c003dd0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1d540073e0 tx=0x7f1d4c005830 comp rx=0 tx=0).stop 2024-04-26T20:51:13.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.166+0000 7f1d6359e640 1 -- 172.21.15.73:0/1437271332 reap_dead start 2024-04-26T20:51:13.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.166+0000 7f1d6906d640 1 -- 172.21.15.73:0/1437271332 shutdown_connections 2024-04-26T20:51:13.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.166+0000 7f1d6906d640 1 -- 172.21.15.73:0/1437271332 >> 172.21.15.73:0/1437271332 conn(0x7f1d640d2ec0 msgr2=0x7f1d640d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:13.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.166+0000 7f1d6906d640 1 -- 172.21.15.73:0/1437271332 shutdown_connections 2024-04-26T20:51:13.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.166+0000 7f1d6906d640 1 -- 172.21.15.73:0/1437271332 wait complete. 2024-04-26T20:51:13.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.167+0000 7f1d6906d640 1 Processor -- start 2024-04-26T20:51:13.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.167+0000 7f1d6906d640 1 -- start start 2024-04-26T20:51:13.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.168+0000 7f1d6906d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d4c0039b0 0x7f1d640b1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:13.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.168+0000 7f1d6906d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1d640b62b0 0x7f1d640c6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:13.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.168+0000 7f1d6906d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d640b7b30 0x7f1d640b7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:13.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.168+0000 7f1d6906d640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1d640b3310 con 0x7f1d640b7b30 2024-04-26T20:51:13.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.168+0000 7f1d6906d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1d640d0ef0 con 0x7f1d4c0039b0 2024-04-26T20:51:13.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.168+0000 7f1d6906d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f1d640ceec0 con 0x7f1d640b62b0 2024-04-26T20:51:13.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.168+0000 7f1d6359e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d640b7b30 0x7f1d640b7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:13.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.168+0000 7f1d6359e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d640b7b30 0x7f1d640b7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51780/0 (socket says 172.21.15.73:51780) 2024-04-26T20:51:13.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.168+0000 7f1d6359e640 1 -- 172.21.15.73:0/781496447 learned_addr learned my addr 172.21.15.73:0/781496447 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:13.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.168+0000 7f1d62d9d640 1 --2- 172.21.15.73:0/781496447 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d4c0039b0 0x7f1d640b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:13.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.168+0000 7f1d6259c640 1 --2- 172.21.15.73:0/781496447 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1d640b62b0 0x7f1d640c6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:13.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.168+0000 7f1d6359e640 1 -- 172.21.15.73:0/781496447 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1d640b62b0 msgr2=0x7f1d640c6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:13.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.168+0000 7f1d6359e640 1 --2- 172.21.15.73:0/781496447 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1d640b62b0 0x7f1d640c6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:13.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.168+0000 7f1d6359e640 1 -- 172.21.15.73:0/781496447 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d4c0039b0 msgr2=0x7f1d640b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:13.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.168+0000 7f1d6359e640 1 --2- 172.21.15.73:0/781496447 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d4c0039b0 0x7f1d640b1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:13.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.168+0000 7f1d6359e640 1 -- 172.21.15.73:0/781496447 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d640b6a90 con 0x7f1d640b7b30 2024-04-26T20:51:13.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.168+0000 7f1d6359e640 1 --2- 172.21.15.73:0/781496447 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d640b7b30 0x7f1d640b7fb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1d6406fda0 tx=0x7f1d5c007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:13.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.169+0000 7f1d4b7fe640 1 -- 172.21.15.73:0/781496447 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c01a070 con 0x7f1d640b7b30 2024-04-26T20:51:13.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.169+0000 7f1d4b7fe640 1 -- 172.21.15.73:0/781496447 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d5c00f700 con 0x7f1d640b7b30 2024-04-26T20:51:13.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.169+0000 7f1d6906d640 1 -- 172.21.15.73:0/781496447 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d640b6d70 con 0x7f1d640b7b30 2024-04-26T20:51:13.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.169+0000 7f1d4b7fe640 1 -- 172.21.15.73:0/781496447 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c00fab0 con 0x7f1d640b7b30 2024-04-26T20:51:13.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.169+0000 7f1d6906d640 1 -- 172.21.15.73:0/781496447 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d640b7330 con 0x7f1d640b7b30 2024-04-26T20:51:13.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.170+0000 7f1d6906d640 1 -- 172.21.15.73:0/781496447 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d640694f0 con 0x7f1d640b7b30 2024-04-26T20:51:13.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.170+0000 7f1d4b7fe640 1 -- 172.21.15.73:0/781496447 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1d5c01d070 con 0x7f1d640b7b30 2024-04-26T20:51:13.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.170+0000 7f1d4b7fe640 1 --2- 172.21.15.73:0/781496447 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1d5006c8f0 0x7f1d5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:13.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.171+0000 7f1d4b7fe640 1 -- 172.21.15.73:0/781496447 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f1d5c08b640 con 0x7f1d640b7b30 2024-04-26T20:51:13.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.171+0000 7f1d62d9d640 1 --2- 172.21.15.73:0/781496447 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1d5006c8f0 0x7f1d5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:13.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.171+0000 7f1d62d9d640 1 --2- 172.21.15.73:0/781496447 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1d5006c8f0 0x7f1d5006edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1d54035000 tx=0x7f1d5400b030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:13.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.175+0000 7f1d4b7fe640 1 -- 172.21.15.73:0/781496447 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1d5c059010 con 0x7f1d640b7b30 2024-04-26T20:51:13.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.352+0000 7f1d6906d640 1 -- 172.21.15.73:0/781496447 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1d640c6600 con 0x7f1d640b7b30 2024-04-26T20:51:13.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.353+0000 7f1d4b7fe640 1 -- 172.21.15.73:0/781496447 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18902 (secure 0 0 0) 0x7f1d5c05d310 con 0x7f1d640b7b30 2024-04-26T20:51:13.354 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:51:13.354 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":26,"fsid":"fbd3f548-b547-4713-856b-09f98f5d1764","created":"2024-04-26T20:50:07.876786+0000","modified":"2024-04-26T20:51:12.779991+0000","last_up_change":"2024-04-26T20:50:19.082741+0000","last_in_change":"2024-04-26T20:50:10.018400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:50:19.993731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:50:29.964420+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:50:30.920082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:50:34.650207+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"72d6f4f3-214a-4392-a02f-c33cbcab3c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6809","nonce":3461705124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6811","nonce":3461705124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6815","nonce":3461705124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6813","nonce":3461705124}]},"public_addr":"172.21.15.73:6809/3461705124","cluster_addr":"172.21.15.73:6811/3461705124","heartbeat_back_addr":"172.21.15.73:6815/3461705124","heartbeat_front_addr":"172.21.15.73:6813/3461705124","state":["exists","up"]},{"osd":1,"uuid":"786bf8ca-6ddc-4b3c-be6c-9152398bd5fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6817","nonce":2827619403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6819","nonce":2827619403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6825","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6827","nonce":2827619403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6821","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6823","nonce":2827619403}]},"public_addr":"172.21.15.73:6817/2827619403","cluster_addr":"172.21.15.73:6819/2827619403","heartbeat_back_addr":"172.21.15.73:6827/2827619403","heartbeat_front_addr":"172.21.15.73:6823/2827619403","state":["exists","up"]},{"osd":2,"uuid":"90963b1a-5437-4ef8-ad8b-6b74c534eeb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6801","nonce":2542681064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6803","nonce":2542681064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6808","nonce":2542681064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6805","nonce":2542681064}]},"public_addr":"172.21.15.73:6801/2542681064","cluster_addr":"172.21.15.73:6803/2542681064","heartbeat_back_addr":"172.21.15.73:6808/2542681064","heartbeat_front_addr":"172.21.15.73:6805/2542681064","state":["exists","up"]},{"osd":3,"uuid":"bd3d2148-6e38-41b4-a453-fe6e3b8b4ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6822","nonce":1604042718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6826","nonce":1604042718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6831","nonce":1604042718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6829","nonce":1604042718}]},"public_addr":"172.21.15.73:6822/1604042718","cluster_addr":"172.21.15.73:6826/1604042718","heartbeat_back_addr":"172.21.15.73:6831/1604042718","heartbeat_front_addr":"172.21.15.73:6829/1604042718","state":["exists","up"]},{"osd":4,"uuid":"e8668dc8-a23f-46d2-8311-cb882da08e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6817","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6819","nonce":3591055561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6824","nonce":3591055561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6831","nonce":3591055561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6828","nonce":3591055561}]},"public_addr":"172.21.15.120:6819/3591055561","cluster_addr":"172.21.15.120:6824/3591055561","heartbeat_back_addr":"172.21.15.120:6831/3591055561","heartbeat_front_addr":"172.21.15.120:6828/3591055561","state":["exists","up"]},{"osd":5,"uuid":"cbac0bdb-96bd-4852-bcce-e564495c3fcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6809","nonce":819450433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6811","nonce":819450433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6815","nonce":819450433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6813","nonce":819450433}]},"public_addr":"172.21.15.120:6809/819450433","cluster_addr":"172.21.15.120:6811/819450433","heartbeat_back_addr":"172.21.15.120:6815/819450433","heartbeat_front_addr":"172.21.15.120:6813/819450433","state":["exists","up"]},{"osd":6,"uuid":"1884b2a8-569f-4272-96a9-f25a09364dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6801","nonce":3946210727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6803","nonce":3946210727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6807","nonce":3946210727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6805","nonce":3946210727}]},"public_addr":"172.21.15.120:6801/3946210727","cluster_addr":"172.21.15.120:6803/3946210727","heartbeat_back_addr":"172.21.15.120:6807/3946210727","heartbeat_front_addr":"172.21.15.120:6805/3946210727","state":["exists","up"]},{"osd":7,"uuid":"c0c3bf6d-a4c5-4f1e-aa2d-330951830666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6818","nonce":2915574674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6821","nonce":2915574674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6827","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6829","nonce":2915574674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6823","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6825","nonce":2915574674}]},"public_addr":"172.21.15.120:6818/2915574674","cluster_addr":"172.21.15.120:6821/2915574674","heartbeat_back_addr":"172.21.15.120:6829/2915574674","heartbeat_front_addr":"172.21.15.120:6825/2915574674","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.218216+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.208710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.112705+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.182789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.164324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.174212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.225396+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.261559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:51:13.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.355+0000 7f1d6906d640 1 -- 172.21.15.73:0/781496447 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1d5006c8f0 msgr2=0x7f1d5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:13.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.355+0000 7f1d6906d640 1 --2- 172.21.15.73:0/781496447 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1d5006c8f0 0x7f1d5006edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1d54035000 tx=0x7f1d5400b030 comp rx=0 tx=0).stop 2024-04-26T20:51:13.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.355+0000 7f1d6906d640 1 -- 172.21.15.73:0/781496447 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d640b7b30 msgr2=0x7f1d640b7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:13.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.355+0000 7f1d6906d640 1 --2- 172.21.15.73:0/781496447 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d640b7b30 0x7f1d640b7fb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1d6406fda0 tx=0x7f1d5c007430 comp rx=0 tx=0).stop 2024-04-26T20:51:13.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.355+0000 7f1d6906d640 1 -- 172.21.15.73:0/781496447 shutdown_connections 2024-04-26T20:51:13.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.355+0000 7f1d6906d640 1 --2- 172.21.15.73:0/781496447 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d640b7b30 0x7f1d640b7fb0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:13.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.355+0000 7f1d6906d640 1 --2- 172.21.15.73:0/781496447 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1d5006c8f0 0x7f1d5006edb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:13.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.355+0000 7f1d6906d640 1 --2- 172.21.15.73:0/781496447 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1d640b62b0 0x7f1d640c6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:13.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.355+0000 7f1d6906d640 1 --2- 172.21.15.73:0/781496447 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d4c0039b0 0x7f1d640b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:13.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.355+0000 7f1d6906d640 1 -- 172.21.15.73:0/781496447 >> 172.21.15.73:0/781496447 conn(0x7f1d640d2ec0 msgr2=0x7f1d6415d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:13.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.355+0000 7f1d6906d640 1 -- 172.21.15.73:0/781496447 shutdown_connections 2024-04-26T20:51:13.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.355+0000 7f1d6906d640 1 -- 172.21.15.73:0/781496447 wait complete. 2024-04-26T20:51:13.367 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:50:19.993731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:50:29.964420+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:50:30.920082+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T20:50:34.650207+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T20:51:13.368 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:51:13.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.466+0000 7f25b227e640 1 Processor -- start 2024-04-26T20:51:13.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.466+0000 7f25b227e640 1 -- start start 2024-04-26T20:51:13.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.467+0000 7f25b227e640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f25ac0cfde0 0x7f25ac0d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:13.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.467+0000 7f25b227e640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f25ac070350 0x7f25ac070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:13.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.467+0000 7f25b227e640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25ac0d2390 con 0x7f25ac071090 2024-04-26T20:51:13.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.467+0000 7f25b227e640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f25ac0afab0 con 0x7f25ac0cfde0 2024-04-26T20:51:13.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.467+0000 7f25b227e640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f25ac06dd20 con 0x7f25ac070350 2024-04-26T20:51:13.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.467+0000 7f25abfff640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f25ac0cfde0 0x7f25ac0d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:13.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.467+0000 7f25b081b640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f25ac071090 0x7f25ac15e9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40160/0 (socket says 172.21.15.73:40160) 2024-04-26T20:51:13.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.467+0000 7f25abfff640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f25ac0cfde0 0x7f25ac0d1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51788/0 (socket says 172.21.15.73:51788) 2024-04-26T20:51:13.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.467+0000 7f25b081b640 1 -- 172.21.15.73:0/4081495265 learned_addr learned my addr 172.21.15.73:0/4081495265 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:13.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.467+0000 7f25ab7fe640 1 --2- 172.21.15.73:0/4081495265 >> v2:172.21.15.120:3300/0 conn(0x7f25ac070350 0x7f25ac070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:13.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.467+0000 7f25abfff640 1 -- 172.21.15.73:0/4081495265 >> v2:172.21.15.120:3300/0 conn(0x7f25ac070350 msgr2=0x7f25ac070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:13.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.467+0000 7f25abfff640 1 --2- 172.21.15.73:0/4081495265 >> v2:172.21.15.120:3300/0 conn(0x7f25ac070350 0x7f25ac070b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:13.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.467+0000 7f25abfff640 1 -- 172.21.15.73:0/4081495265 >> v1:172.21.15.73:6789/0 conn(0x7f25ac071090 legacy=0x7f25ac15e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:13.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.467+0000 7f25aaffd640 1 -- 172.21.15.73:0/4081495265 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2660837063 0 0) 0x7f25ac0d2390 con 0x7f25ac071090 2024-04-26T20:51:13.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.467+0000 7f25abfff640 1 -- 172.21.15.73:0/4081495265 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25ac06d3f0 con 0x7f25ac0cfde0 2024-04-26T20:51:13.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.468+0000 7f25abfff640 1 --2- 172.21.15.73:0/4081495265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25ac0cfde0 0x7f25ac0d1b60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f25ac06cb80 tx=0x7f259c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=583cc265990b82f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:13.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.468+0000 7f25aaffd640 1 -- 172.21.15.73:0/4081495265 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f259c037070 con 0x7f25ac0cfde0 2024-04-26T20:51:13.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.468+0000 7f25aaffd640 1 -- 172.21.15.73:0/4081495265 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f259c032440 con 0x7f25ac0cfde0 2024-04-26T20:51:13.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.468+0000 7f25aaffd640 1 -- 172.21.15.73:0/4081495265 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f259c02ae60 con 0x7f25ac0cfde0 2024-04-26T20:51:13.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.468+0000 7f25b227e640 1 -- 172.21.15.73:0/4081495265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25ac0cfde0 msgr2=0x7f25ac0d1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:13.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.468+0000 7f25b227e640 1 --2- 172.21.15.73:0/4081495265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25ac0cfde0 0x7f25ac0d1b60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f25ac06cb80 tx=0x7f259c02a110 comp rx=0 tx=0).stop 2024-04-26T20:51:13.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.468+0000 7f25b227e640 1 -- 172.21.15.73:0/4081495265 shutdown_connections 2024-04-26T20:51:13.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.468+0000 7f25b227e640 1 --2- 172.21.15.73:0/4081495265 >> v2:172.21.15.120:3300/0 conn(0x7f25ac070350 0x7f25ac070b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:13.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.468+0000 7f25b227e640 1 --2- 172.21.15.73:0/4081495265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25ac0cfde0 0x7f25ac0d1b60 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:13.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.468+0000 7f25b227e640 1 -- 172.21.15.73:0/4081495265 >> 172.21.15.73:0/4081495265 conn(0x7f25ac0d2d80 msgr2=0x7f25ac0d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:13.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.469+0000 7f25b227e640 1 -- 172.21.15.73:0/4081495265 shutdown_connections 2024-04-26T20:51:13.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.469+0000 7f25b227e640 1 -- 172.21.15.73:0/4081495265 wait complete. 2024-04-26T20:51:13.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.470+0000 7f25b227e640 1 Processor -- start 2024-04-26T20:51:13.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.470+0000 7f25b227e640 1 -- start start 2024-04-26T20:51:13.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.470+0000 7f25b227e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25ac070350 0x7f25ac159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:13.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.470+0000 7f25b227e640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f25ac071090 0x7f25ac1e7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:13.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.470+0000 7f25b227e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f25ac0cfde0 0x7f25ac1e7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:13.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.471+0000 7f25b227e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f25ac06d9b0 con 0x7f25ac070350 2024-04-26T20:51:13.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.471+0000 7f25b227e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f25ac0714c0 con 0x7f25ac0cfde0 2024-04-26T20:51:13.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.471+0000 7f25b227e640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f25ac0d20a0 con 0x7f25ac071090 2024-04-26T20:51:13.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.471+0000 7f25abfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25ac070350 0x7f25ac159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:13.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.471+0000 7f25abfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25ac070350 0x7f25ac159ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51800/0 (socket says 172.21.15.73:51800) 2024-04-26T20:51:13.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.471+0000 7f25abfff640 1 -- 172.21.15.73:0/3180448106 learned_addr learned my addr 172.21.15.73:0/3180448106 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:13.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.471+0000 7f25b081b640 1 --2- 172.21.15.73:0/3180448106 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f25ac0cfde0 0x7f25ac1e7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:13.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.471+0000 7f25ab7fe640 1 --2- 172.21.15.73:0/3180448106 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f25ac071090 0x7f25ac1e7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:13.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.471+0000 7f25abfff640 1 -- 172.21.15.73:0/3180448106 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f25ac071090 msgr2=0x7f25ac1e7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:13.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.471+0000 7f25abfff640 1 --2- 172.21.15.73:0/3180448106 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f25ac071090 0x7f25ac1e7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:13.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.471+0000 7f25abfff640 1 -- 172.21.15.73:0/3180448106 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f25ac0cfde0 msgr2=0x7f25ac1e7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:13.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.471+0000 7f25abfff640 1 --2- 172.21.15.73:0/3180448106 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f25ac0cfde0 0x7f25ac1e7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:13.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.471+0000 7f25abfff640 1 -- 172.21.15.73:0/3180448106 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25ac15c6b0 con 0x7f25ac070350 2024-04-26T20:51:13.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.471+0000 7f25abfff640 1 --2- 172.21.15.73:0/3180448106 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25ac070350 0x7f25ac159ed0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f259c009890 tx=0x7f259c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:13.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.471+0000 7f25a8ff9640 1 -- 172.21.15.73:0/3180448106 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f259c037040 con 0x7f25ac070350 2024-04-26T20:51:13.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.471+0000 7f25a8ff9640 1 -- 172.21.15.73:0/3180448106 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f259c042070 con 0x7f25ac070350 2024-04-26T20:51:13.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.471+0000 7f25a8ff9640 1 -- 172.21.15.73:0/3180448106 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f259c030960 con 0x7f25ac070350 2024-04-26T20:51:13.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.471+0000 7f25b227e640 1 -- 172.21.15.73:0/3180448106 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25ac15c990 con 0x7f25ac070350 2024-04-26T20:51:13.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.471+0000 7f25b227e640 1 -- 172.21.15.73:0/3180448106 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25ac15cf50 con 0x7f25ac070350 2024-04-26T20:51:13.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.472+0000 7f25b227e640 1 -- 172.21.15.73:0/3180448106 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25ac15d150 con 0x7f25ac070350 2024-04-26T20:51:13.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.473+0000 7f25a8ff9640 1 -- 172.21.15.73:0/3180448106 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f259c030b00 con 0x7f25ac070350 2024-04-26T20:51:13.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.473+0000 7f25a8ff9640 1 --2- 172.21.15.73:0/3180448106 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f259806c8f0 0x7f259806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:13.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.473+0000 7f25a8ff9640 1 -- 172.21.15.73:0/3180448106 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f259c0ac7d0 con 0x7f25ac070350 2024-04-26T20:51:13.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.473+0000 7f25ab7fe640 1 --2- 172.21.15.73:0/3180448106 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f259806c8f0 0x7f259806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:13.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.474+0000 7f25ab7fe640 1 --2- 172.21.15.73:0/3180448106 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f259806c8f0 0x7f259806edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f25ac15d960 tx=0x7f25a0009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:13.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.477+0000 7f25a8ff9640 1 -- 172.21.15.73:0/3180448106 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f25ac15d150 con 0x7f25ac070350 2024-04-26T20:51:13.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.653+0000 7f25b227e640 1 -- 172.21.15.73:0/3180448106 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f25ac0694f0 con 0x7f25ac070350 2024-04-26T20:51:13.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.654+0000 7f25a8ff9640 1 -- 172.21.15.73:0/3180448106 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f259c07a120 con 0x7f25ac070350 2024-04-26T20:51:13.655 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-26T20:51:13.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.655+0000 7f25b227e640 1 -- 172.21.15.73:0/3180448106 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f259806c8f0 msgr2=0x7f259806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:13.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.655+0000 7f25b227e640 1 --2- 172.21.15.73:0/3180448106 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f259806c8f0 0x7f259806edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f25ac15d960 tx=0x7f25a0009190 comp rx=0 tx=0).stop 2024-04-26T20:51:13.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.655+0000 7f25b227e640 1 -- 172.21.15.73:0/3180448106 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25ac070350 msgr2=0x7f25ac159ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:13.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.655+0000 7f25b227e640 1 --2- 172.21.15.73:0/3180448106 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25ac070350 0x7f25ac159ed0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f259c009890 tx=0x7f259c0046c0 comp rx=0 tx=0).stop 2024-04-26T20:51:13.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.655+0000 7f25b227e640 1 -- 172.21.15.73:0/3180448106 shutdown_connections 2024-04-26T20:51:13.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.655+0000 7f25b227e640 1 --2- 172.21.15.73:0/3180448106 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f259806c8f0 0x7f259806edb0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:13.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.655+0000 7f25b227e640 1 --2- 172.21.15.73:0/3180448106 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f25ac0cfde0 0x7f25ac1e7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:13.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.655+0000 7f25b227e640 1 --2- 172.21.15.73:0/3180448106 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f25ac071090 0x7f25ac1e7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:13.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.655+0000 7f25b227e640 1 --2- 172.21.15.73:0/3180448106 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25ac070350 0x7f25ac159ed0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:13.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.656+0000 7f25b227e640 1 -- 172.21.15.73:0/3180448106 >> 172.21.15.73:0/3180448106 conn(0x7f25ac0d2d80 msgr2=0x7f25ac0b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:13.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.656+0000 7f25b227e640 1 -- 172.21.15.73:0/3180448106 shutdown_connections 2024-04-26T20:51:13.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.656+0000 7f25b227e640 1 -- 172.21.15.73:0/3180448106 wait complete. 2024-04-26T20:51:13.667 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:51:13.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.765+0000 7f64fe3bf640 1 Processor -- start 2024-04-26T20:51:13.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.766+0000 7f64fe3bf640 1 -- start start 2024-04-26T20:51:13.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.766+0000 7f64fe3bf640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f64f8070930 0x7f64f8160ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:13.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.766+0000 7f64fe3bf640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64f80b0990 con 0x7f64f815d100 2024-04-26T20:51:13.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.766+0000 7f64fe3bf640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64f80b0c70 con 0x7f64f806c950 2024-04-26T20:51:13.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.766+0000 7f64fe3bf640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f64f80b0f50 con 0x7f64f8070930 2024-04-26T20:51:13.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.766+0000 7f64fc95c640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f64f8070930 0x7f64f8160ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:13.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.766+0000 7f64f7fff640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f64f815d100 0x7f64f806c240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40170/0 (socket says 172.21.15.73:40170) 2024-04-26T20:51:13.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.767+0000 7f64f7fff640 1 -- 172.21.15.73:0/233056618 learned_addr learned my addr 172.21.15.73:0/233056618 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:13.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.767+0000 7f64fc95c640 1 -- 172.21.15.73:0/233056618 >> v1:172.21.15.120:6789/0 conn(0x7f64f806c950 legacy=0x7f64f8073160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:13.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.767+0000 7f64fc95c640 1 -- 172.21.15.73:0/233056618 >> v1:172.21.15.73:6789/0 conn(0x7f64f815d100 legacy=0x7f64f806c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:13.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.767+0000 7f64fc95c640 1 -- 172.21.15.73:0/233056618 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64f80b1230 con 0x7f64f8070930 2024-04-26T20:51:13.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.767+0000 7f64f6ffd640 1 -- 172.21.15.73:0/233056618 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 372487875 0 0) 0x7f64f80b0990 con 0x7f64f815d100 2024-04-26T20:51:13.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.767+0000 7f64fc95c640 1 --2- 172.21.15.73:0/233056618 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64f8070930 0x7f64f8160ba0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f64f0009d90 tx=0x7f64f002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56d089e5ced9ee3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:13.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.768+0000 7f64f6ffd640 1 -- 172.21.15.73:0/233056618 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f0007740 con 0x7f64f8070930 2024-04-26T20:51:13.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.768+0000 7f64f6ffd640 1 -- 172.21.15.73:0/233056618 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64f00078e0 con 0x7f64f8070930 2024-04-26T20:51:13.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.768+0000 7f64f6ffd640 1 -- 172.21.15.73:0/233056618 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f0007cb0 con 0x7f64f8070930 2024-04-26T20:51:13.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.768+0000 7f64fe3bf640 1 -- 172.21.15.73:0/233056618 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64f8070930 msgr2=0x7f64f8160ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:13.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.768+0000 7f64fe3bf640 1 --2- 172.21.15.73:0/233056618 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64f8070930 0x7f64f8160ba0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f64f0009d90 tx=0x7f64f002a6a0 comp rx=0 tx=0).stop 2024-04-26T20:51:13.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.768+0000 7f64fe3bf640 1 -- 172.21.15.73:0/233056618 shutdown_connections 2024-04-26T20:51:13.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.768+0000 7f64fe3bf640 1 --2- 172.21.15.73:0/233056618 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64f8070930 0x7f64f8160ba0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:13.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.768+0000 7f64fe3bf640 1 -- 172.21.15.73:0/233056618 >> 172.21.15.73:0/233056618 conn(0x7f64f80b39b0 msgr2=0x7f64f80b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:13.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.768+0000 7f64fe3bf640 1 -- 172.21.15.73:0/233056618 shutdown_connections 2024-04-26T20:51:13.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.768+0000 7f64fe3bf640 1 -- 172.21.15.73:0/233056618 wait complete. 2024-04-26T20:51:13.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.769+0000 7f64fe3bf640 1 Processor -- start 2024-04-26T20:51:13.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.769+0000 7f64fe3bf640 1 -- start start 2024-04-26T20:51:13.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.770+0000 7f64fe3bf640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64f806c950 0x7f64f814b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:13.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.770+0000 7f64fe3bf640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f64f8070930 0x7f64f8073160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:13.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.770+0000 7f64fe3bf640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64f815d100 0x7f64f80736a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:13.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.770+0000 7f64fe3bf640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f64f80aef10 con 0x7f64f806c950 2024-04-26T20:51:13.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.770+0000 7f64fe3bf640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f64f80aed90 con 0x7f64f815d100 2024-04-26T20:51:13.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.770+0000 7f64fe3bf640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f64f80af090 con 0x7f64f8070930 2024-04-26T20:51:13.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.770+0000 7f64f7fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64f806c950 0x7f64f814b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:13.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.770+0000 7f64f7fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64f806c950 0x7f64f814b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51816/0 (socket says 172.21.15.73:51816) 2024-04-26T20:51:13.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.770+0000 7f64f7fff640 1 -- 172.21.15.73:0/3434889246 learned_addr learned my addr 172.21.15.73:0/3434889246 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:13.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.771+0000 7f64f7fff640 1 -- 172.21.15.73:0/3434889246 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f64f8070930 msgr2=0x7f64f8073160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:13.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.771+0000 7f64f7fff640 1 --2- 172.21.15.73:0/3434889246 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f64f8070930 0x7f64f8073160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:13.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.771+0000 7f64f7fff640 1 -- 172.21.15.73:0/3434889246 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64f815d100 msgr2=0x7f64f80736a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:13.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.771+0000 7f64fc95c640 1 --2- 172.21.15.73:0/3434889246 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64f815d100 0x7f64f80736a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:13.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.771+0000 7f64f7fff640 1 --2- 172.21.15.73:0/3434889246 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64f815d100 0x7f64f80736a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:13.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.771+0000 7f64f7fff640 1 -- 172.21.15.73:0/3434889246 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64f81f8da0 con 0x7f64f806c950 2024-04-26T20:51:13.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.771+0000 7f64f7fff640 1 --2- 172.21.15.73:0/3434889246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64f806c950 0x7f64f814b3a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f64e800c500 tx=0x7f64e800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:13.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.771+0000 7f64f4ff9640 1 -- 172.21.15.73:0/3434889246 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e8018070 con 0x7f64f806c950 2024-04-26T20:51:13.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.771+0000 7f64f4ff9640 1 -- 172.21.15.73:0/3434889246 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64e8007560 con 0x7f64f806c950 2024-04-26T20:51:13.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.771+0000 7f64fe3bf640 1 -- 172.21.15.73:0/3434889246 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64f81f9080 con 0x7f64f806c950 2024-04-26T20:51:13.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.771+0000 7f64f4ff9640 1 -- 172.21.15.73:0/3434889246 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e8004a20 con 0x7f64f806c950 2024-04-26T20:51:13.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.771+0000 7f64fe3bf640 1 -- 172.21.15.73:0/3434889246 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64f815a830 con 0x7f64f806c950 2024-04-26T20:51:13.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.772+0000 7f64fe3bf640 1 -- 172.21.15.73:0/3434889246 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64f80694f0 con 0x7f64f806c950 2024-04-26T20:51:13.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.772+0000 7f64f4ff9640 1 -- 172.21.15.73:0/3434889246 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f64e801b070 con 0x7f64f806c950 2024-04-26T20:51:13.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.773+0000 7f64f4ff9640 1 --2- 172.21.15.73:0/3434889246 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f64e406c8f0 0x7f64e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:13.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.773+0000 7f64f4ff9640 1 -- 172.21.15.73:0/3434889246 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f64e808b260 con 0x7f64f806c950 2024-04-26T20:51:13.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.773+0000 7f64f77fe640 1 --2- 172.21.15.73:0/3434889246 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f64e406c8f0 0x7f64e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:13.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.773+0000 7f64f77fe640 1 --2- 172.21.15.73:0/3434889246 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f64e406c8f0 0x7f64e406edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f64ec006fd0 tx=0x7f64ec006bc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:13.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.777+0000 7f64f4ff9640 1 -- 172.21.15.73:0/3434889246 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f64e8059320 con 0x7f64f806c950 2024-04-26T20:51:13.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.956+0000 7f64fe3bf640 1 -- 172.21.15.73:0/3434889246 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f64f806cd70 con 0x7f64f806c950 2024-04-26T20:51:13.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.956+0000 7f64f4ff9640 1 -- 172.21.15.73:0/3434889246 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f64e8058c20 con 0x7f64f806c950 2024-04-26T20:51:13.957 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-26T20:51:13.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.958+0000 7f64fe3bf640 1 -- 172.21.15.73:0/3434889246 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f64e406c8f0 msgr2=0x7f64e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:13.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.958+0000 7f64fe3bf640 1 --2- 172.21.15.73:0/3434889246 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f64e406c8f0 0x7f64e406edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f64ec006fd0 tx=0x7f64ec006bc0 comp rx=0 tx=0).stop 2024-04-26T20:51:13.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.958+0000 7f64fe3bf640 1 -- 172.21.15.73:0/3434889246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64f806c950 msgr2=0x7f64f814b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:13.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.958+0000 7f64fe3bf640 1 --2- 172.21.15.73:0/3434889246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64f806c950 0x7f64f814b3a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f64e800c500 tx=0x7f64e800c9c0 comp rx=0 tx=0).stop 2024-04-26T20:51:13.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.958+0000 7f64fe3bf640 1 -- 172.21.15.73:0/3434889246 shutdown_connections 2024-04-26T20:51:13.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.958+0000 7f64fe3bf640 1 --2- 172.21.15.73:0/3434889246 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f64e406c8f0 0x7f64e406edb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:13.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.958+0000 7f64fe3bf640 1 --2- 172.21.15.73:0/3434889246 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f64f815d100 0x7f64f80736a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:13.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.958+0000 7f64fe3bf640 1 --2- 172.21.15.73:0/3434889246 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f64f8070930 0x7f64f8073160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:13.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.958+0000 7f64fe3bf640 1 --2- 172.21.15.73:0/3434889246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64f806c950 0x7f64f814b3a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:13.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.958+0000 7f64fe3bf640 1 -- 172.21.15.73:0/3434889246 >> 172.21.15.73:0/3434889246 conn(0x7f64f80b39b0 msgr2=0x7f64f815d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:13.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.958+0000 7f64fe3bf640 1 -- 172.21.15.73:0/3434889246 shutdown_connections 2024-04-26T20:51:13.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:13.958+0000 7f64fe3bf640 1 -- 172.21.15.73:0/3434889246 wait complete. 2024-04-26T20:51:13.969 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:51:14.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.068+0000 7f7345333640 1 Processor -- start 2024-04-26T20:51:14.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.068+0000 7f7345333640 1 -- start start 2024-04-26T20:51:14.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.069+0000 7f7345333640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f73400d1a00 0x7f73400d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:14.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.069+0000 7f7345333640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f73400d2320 0x7f73401625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:14.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.069+0000 7f7345333640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f73400b40d0 0x7f7340162b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:14.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.069+0000 7f7345333640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f7340163070 con 0x7f73400b40d0 2024-04-26T20:51:14.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.069+0000 7f7345333640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f73400d0a10 con 0x7f73400d1a00 2024-04-26T20:51:14.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.069+0000 7f7345333640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f73400d0cf0 con 0x7f73400d2320 2024-04-26T20:51:14.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.069+0000 7f733f7fe640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f73400b40d0 0x7f7340162b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:14.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.069+0000 7f733f7fe640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f73400b40d0 0x7f7340162b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51818/0 (socket says 172.21.15.73:51818) 2024-04-26T20:51:14.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.069+0000 7f733f7fe640 1 -- 172.21.15.73:0/1317681950 learned_addr learned my addr 172.21.15.73:0/1317681950 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:14.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.069+0000 7f733e7fc640 1 --2- 172.21.15.73:0/1317681950 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f73400d2320 0x7f73401625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:14.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.069+0000 7f733effd640 1 --2- 172.21.15.73:0/1317681950 >> v2:172.21.15.120:3300/0 conn(0x7f73400d1a00 0x7f73400d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:14.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.069+0000 7f733f7fe640 1 -- 172.21.15.73:0/1317681950 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f73400d2320 msgr2=0x7f73401625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:14.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.069+0000 7f733f7fe640 1 --2- 172.21.15.73:0/1317681950 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f73400d2320 0x7f73401625f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:14.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.069+0000 7f733f7fe640 1 -- 172.21.15.73:0/1317681950 >> v2:172.21.15.120:3300/0 conn(0x7f73400d1a00 msgr2=0x7f73400d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:14.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.069+0000 7f733f7fe640 1 --2- 172.21.15.73:0/1317681950 >> v2:172.21.15.120:3300/0 conn(0x7f73400d1a00 0x7f73400d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:14.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.070+0000 7f733f7fe640 1 -- 172.21.15.73:0/1317681950 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73400d0fd0 con 0x7f73400b40d0 2024-04-26T20:51:14.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.070+0000 7f733e7fc640 1 --2- 172.21.15.73:0/1317681950 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f73400d2320 0x7f73401625f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:14.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.070+0000 7f733effd640 1 --2- 172.21.15.73:0/1317681950 >> v2:172.21.15.120:3300/0 conn(0x7f73400d1a00 0x7f73400d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:14.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.070+0000 7f733f7fe640 1 --2- 172.21.15.73:0/1317681950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73400b40d0 0x7f7340162b30 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f734006c650 tx=0x7f733802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af35b4b2a1005402 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:14.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.070+0000 7f733dffb640 1 -- 172.21.15.73:0/1317681950 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7338037070 con 0x7f73400b40d0 2024-04-26T20:51:14.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.070+0000 7f733dffb640 1 -- 172.21.15.73:0/1317681950 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7338032440 con 0x7f73400b40d0 2024-04-26T20:51:14.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.070+0000 7f733dffb640 1 -- 172.21.15.73:0/1317681950 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7338032810 con 0x7f73400b40d0 2024-04-26T20:51:14.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.071+0000 7f7345333640 1 -- 172.21.15.73:0/1317681950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73400b40d0 msgr2=0x7f7340162b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:14.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.071+0000 7f7345333640 1 --2- 172.21.15.73:0/1317681950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73400b40d0 0x7f7340162b30 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f734006c650 tx=0x7f733802a140 comp rx=0 tx=0).stop 2024-04-26T20:51:14.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.071+0000 7f7345333640 1 -- 172.21.15.73:0/1317681950 shutdown_connections 2024-04-26T20:51:14.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.071+0000 7f7345333640 1 --2- 172.21.15.73:0/1317681950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73400b40d0 0x7f7340162b30 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:14.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.071+0000 7f7345333640 1 --2- 172.21.15.73:0/1317681950 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f73400d2320 0x7f73401625f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:14.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.071+0000 7f7345333640 1 --2- 172.21.15.73:0/1317681950 >> v2:172.21.15.120:3300/0 conn(0x7f73400d1a00 0x7f73400d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:14.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.071+0000 7f7345333640 1 -- 172.21.15.73:0/1317681950 >> 172.21.15.73:0/1317681950 conn(0x7f73400d4fd0 msgr2=0x7f73400d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:14.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.071+0000 7f7345333640 1 -- 172.21.15.73:0/1317681950 shutdown_connections 2024-04-26T20:51:14.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.071+0000 7f7345333640 1 -- 172.21.15.73:0/1317681950 wait complete. 2024-04-26T20:51:14.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.072+0000 7f7345333640 1 Processor -- start 2024-04-26T20:51:14.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.072+0000 7f7345333640 1 -- start start 2024-04-26T20:51:14.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.073+0000 7f7345333640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f73400b40d0 0x7f7340151eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:14.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.073+0000 7f7345333640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73400d1a00 0x7f73400cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:14.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.073+0000 7f7345333640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73400d2320 0x7f73400ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:14.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.073+0000 7f7345333640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f73400d3050 con 0x7f73400d2320 2024-04-26T20:51:14.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.073+0000 7f7345333640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f73400d2ed0 con 0x7f73400d1a00 2024-04-26T20:51:14.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.073+0000 7f7345333640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f73400d31d0 con 0x7f73400b40d0 2024-04-26T20:51:14.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.073+0000 7f733f7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73400d2320 0x7f73400ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:14.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.073+0000 7f733f7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73400d2320 0x7f73400ca6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51832/0 (socket says 172.21.15.73:51832) 2024-04-26T20:51:14.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.073+0000 7f733f7fe640 1 -- 172.21.15.73:0/1818269544 learned_addr learned my addr 172.21.15.73:0/1818269544 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:14.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.073+0000 7f733effd640 1 --2- 172.21.15.73:0/1818269544 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f73400b40d0 0x7f7340151eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:14.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.073+0000 7f733e7fc640 1 --2- 172.21.15.73:0/1818269544 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73400d1a00 0x7f73400cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:14.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.073+0000 7f733f7fe640 1 -- 172.21.15.73:0/1818269544 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f73400b40d0 msgr2=0x7f7340151eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:14.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.073+0000 7f733f7fe640 1 --2- 172.21.15.73:0/1818269544 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f73400b40d0 0x7f7340151eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:14.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.073+0000 7f733f7fe640 1 -- 172.21.15.73:0/1818269544 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73400d1a00 msgr2=0x7f73400cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:14.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.073+0000 7f733f7fe640 1 --2- 172.21.15.73:0/1818269544 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73400d1a00 0x7f73400cedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:14.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.073+0000 7f733f7fe640 1 -- 172.21.15.73:0/1818269544 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73401524d0 con 0x7f73400d2320 2024-04-26T20:51:14.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.074+0000 7f733f7fe640 1 --2- 172.21.15.73:0/1818269544 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73400d2320 0x7f73400ca6e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f733802a180 tx=0x7f73380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:14.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.074+0000 7f73277fe640 1 -- 172.21.15.73:0/1818269544 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7338037040 con 0x7f73400d2320 2024-04-26T20:51:14.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.074+0000 7f73277fe640 1 -- 172.21.15.73:0/1818269544 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7338042070 con 0x7f73400d2320 2024-04-26T20:51:14.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.074+0000 7f73277fe640 1 -- 172.21.15.73:0/1818269544 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7338030bd0 con 0x7f73400d2320 2024-04-26T20:51:14.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.074+0000 7f7345333640 1 -- 172.21.15.73:0/1818269544 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73401527b0 con 0x7f73400d2320 2024-04-26T20:51:14.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.074+0000 7f7345333640 1 -- 172.21.15.73:0/1818269544 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7340152d70 con 0x7f73400d2320 2024-04-26T20:51:14.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.075+0000 7f7345333640 1 -- 172.21.15.73:0/1818269544 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73400694f0 con 0x7f73400d2320 2024-04-26T20:51:14.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.075+0000 7f73277fe640 1 -- 172.21.15.73:0/1818269544 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7338047460 con 0x7f73400d2320 2024-04-26T20:51:14.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.076+0000 7f73277fe640 1 --2- 172.21.15.73:0/1818269544 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f732c06c920 0x7f732c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:14.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.076+0000 7f733effd640 1 --2- 172.21.15.73:0/1818269544 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f732c06c920 0x7f732c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:14.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.076+0000 7f73277fe640 1 -- 172.21.15.73:0/1818269544 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f73380acba0 con 0x7f73400d2320 2024-04-26T20:51:14.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.076+0000 7f733effd640 1 --2- 172.21.15.73:0/1818269544 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f732c06c920 0x7f732c06ede0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7330007130 tx=0x7f73300079c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:14.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.080+0000 7f73277fe640 1 -- 172.21.15.73:0/1818269544 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f733807abd0 con 0x7f73400d2320 2024-04-26T20:51:14.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.256+0000 7f7345333640 1 -- 172.21.15.73:0/1818269544 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f73400cbe60 con 0x7f73400d2320 2024-04-26T20:51:14.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.256+0000 7f73277fe640 1 -- 172.21.15.73:0/1818269544 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f733807a4d0 con 0x7f73400d2320 2024-04-26T20:51:14.258 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-26T20:51:14.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.258+0000 7f7345333640 1 -- 172.21.15.73:0/1818269544 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f732c06c920 msgr2=0x7f732c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:14.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.258+0000 7f7345333640 1 --2- 172.21.15.73:0/1818269544 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f732c06c920 0x7f732c06ede0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7330007130 tx=0x7f73300079c0 comp rx=0 tx=0).stop 2024-04-26T20:51:14.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.258+0000 7f7345333640 1 -- 172.21.15.73:0/1818269544 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73400d2320 msgr2=0x7f73400ca6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:14.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.258+0000 7f7345333640 1 --2- 172.21.15.73:0/1818269544 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73400d2320 0x7f73400ca6e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f733802a180 tx=0x7f73380046c0 comp rx=0 tx=0).stop 2024-04-26T20:51:14.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.258+0000 7f7345333640 1 -- 172.21.15.73:0/1818269544 shutdown_connections 2024-04-26T20:51:14.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.258+0000 7f7345333640 1 --2- 172.21.15.73:0/1818269544 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73400d2320 0x7f73400ca6e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:14.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.258+0000 7f7345333640 1 --2- 172.21.15.73:0/1818269544 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f732c06c920 0x7f732c06ede0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:14.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.258+0000 7f7345333640 1 --2- 172.21.15.73:0/1818269544 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73400d1a00 0x7f73400cedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:14.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.258+0000 7f7345333640 1 --2- 172.21.15.73:0/1818269544 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f73400b40d0 0x7f7340151eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:14.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.258+0000 7f7345333640 1 -- 172.21.15.73:0/1818269544 >> 172.21.15.73:0/1818269544 conn(0x7f73400d4fd0 msgr2=0x7f734006dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:14.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.258+0000 7f7345333640 1 -- 172.21.15.73:0/1818269544 shutdown_connections 2024-04-26T20:51:14.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.258+0000 7f7345333640 1 -- 172.21.15.73:0/1818269544 wait complete. 2024-04-26T20:51:14.270 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T20:51:14.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.371+0000 7fc5b4d89640 1 Processor -- start 2024-04-26T20:51:14.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.371+0000 7fc5b4d89640 1 -- start start 2024-04-26T20:51:14.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.372+0000 7fc5b4d89640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fc5b015b600 0x7fc5b0071f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:14.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.372+0000 7fc5b4d89640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc5b00cf5e0 0x7fc5b00ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:14.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.372+0000 7fc5b4d89640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc5b00cbbd0 0x7fc5b00cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:14.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.372+0000 7fc5b4d89640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7fc5b00cb9b0 con 0x7fc5b00cbbd0 2024-04-26T20:51:14.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.372+0000 7fc5b4d89640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fc5b0072450 con 0x7fc5b015b600 2024-04-26T20:51:14.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.372+0000 7fc5b4d89640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fc5b015b9e0 con 0x7fc5b00cf5e0 2024-04-26T20:51:14.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.372+0000 7fc5aed9d640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc5b00cbbd0 0x7fc5b00cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:14.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.372+0000 7fc5aed9d640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc5b00cbbd0 0x7fc5b00cef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51838/0 (socket says 172.21.15.73:51838) 2024-04-26T20:51:14.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.372+0000 7fc5aed9d640 1 -- 172.21.15.73:0/1627053935 learned_addr learned my addr 172.21.15.73:0/1627053935 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:14.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.372+0000 7fc5a5d9b640 1 --2- 172.21.15.73:0/1627053935 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc5b00cf5e0 0x7fc5b00ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:14.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.372+0000 7fc5ae59c640 1 --2- 172.21.15.73:0/1627053935 >> v2:172.21.15.120:3300/0 conn(0x7fc5b015b600 0x7fc5b0071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:14.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.372+0000 7fc5aed9d640 1 -- 172.21.15.73:0/1627053935 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc5b00cf5e0 msgr2=0x7fc5b00ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:14.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.372+0000 7fc5aed9d640 1 --2- 172.21.15.73:0/1627053935 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc5b00cf5e0 0x7fc5b00ce9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:14.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.372+0000 7fc5aed9d640 1 -- 172.21.15.73:0/1627053935 >> v2:172.21.15.120:3300/0 conn(0x7fc5b015b600 msgr2=0x7fc5b0071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:14.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.372+0000 7fc5aed9d640 1 --2- 172.21.15.73:0/1627053935 >> v2:172.21.15.120:3300/0 conn(0x7fc5b015b600 0x7fc5b0071f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:14.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.372+0000 7fc5aed9d640 1 -- 172.21.15.73:0/1627053935 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5b00d0740 con 0x7fc5b00cbbd0 2024-04-26T20:51:14.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.372+0000 7fc5a5d9b640 1 --2- 172.21.15.73:0/1627053935 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc5b00cf5e0 0x7fc5b00ce9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:14.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.373+0000 7fc5aed9d640 1 --2- 172.21.15.73:0/1627053935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5b00cbbd0 0x7fc5b00cef10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0071ad0 tx=0x7fc5a802b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c61d9f4eed18d58b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:14.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.373+0000 7fc5add9b640 1 -- 172.21.15.73:0/1627053935 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a8037070 con 0x7fc5b00cbbd0 2024-04-26T20:51:14.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.373+0000 7fc5add9b640 1 -- 172.21.15.73:0/1627053935 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5a8032600 con 0x7fc5b00cbbd0 2024-04-26T20:51:14.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.373+0000 7fc5add9b640 1 -- 172.21.15.73:0/1627053935 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a80329d0 con 0x7fc5b00cbbd0 2024-04-26T20:51:14.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.374+0000 7fc5b4d89640 1 -- 172.21.15.73:0/1627053935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5b00cbbd0 msgr2=0x7fc5b00cef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:14.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.374+0000 7fc5b4d89640 1 --2- 172.21.15.73:0/1627053935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5b00cbbd0 0x7fc5b00cef10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc5b0071ad0 tx=0x7fc5a802b7b0 comp rx=0 tx=0).stop 2024-04-26T20:51:14.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.374+0000 7fc5b4d89640 1 -- 172.21.15.73:0/1627053935 shutdown_connections 2024-04-26T20:51:14.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.374+0000 7fc5b4d89640 1 --2- 172.21.15.73:0/1627053935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5b00cbbd0 0x7fc5b00cef10 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:14.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.374+0000 7fc5b4d89640 1 --2- 172.21.15.73:0/1627053935 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc5b00cf5e0 0x7fc5b00ce9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:14.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.374+0000 7fc5b4d89640 1 --2- 172.21.15.73:0/1627053935 >> v2:172.21.15.120:3300/0 conn(0x7fc5b015b600 0x7fc5b0071f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:14.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.374+0000 7fc5b4d89640 1 -- 172.21.15.73:0/1627053935 >> 172.21.15.73:0/1627053935 conn(0x7fc5b00d0be0 msgr2=0x7fc5b00d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:14.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.374+0000 7fc5b4d89640 1 -- 172.21.15.73:0/1627053935 shutdown_connections 2024-04-26T20:51:14.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.374+0000 7fc5b4d89640 1 -- 172.21.15.73:0/1627053935 wait complete. 2024-04-26T20:51:14.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.375+0000 7fc5b4d89640 1 Processor -- start 2024-04-26T20:51:14.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.376+0000 7fc5b4d89640 1 -- start start 2024-04-26T20:51:14.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.376+0000 7fc5b4d89640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc5b00cbbd0 0x7fc5b00c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:14.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.376+0000 7fc5b4d89640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5b00cf5e0 0x7fc5b01562f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:14.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.376+0000 7fc5b4d89640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5b015b600 0x7fc5b00b5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:14.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.376+0000 7fc5b4d89640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b00d02f0 con 0x7fc5b00cf5e0 2024-04-26T20:51:14.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.376+0000 7fc5b4d89640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b00cb640 con 0x7fc5b015b600 2024-04-26T20:51:14.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.376+0000 7fc5b4d89640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fc5b00d0470 con 0x7fc5b00cbbd0 2024-04-26T20:51:14.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.376+0000 7fc5a5d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5b00cf5e0 0x7fc5b01562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:14.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.376+0000 7fc5a5d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5b00cf5e0 0x7fc5b01562f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51854/0 (socket says 172.21.15.73:51854) 2024-04-26T20:51:14.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.376+0000 7fc5a5d9b640 1 -- 172.21.15.73:0/2734092450 learned_addr learned my addr 172.21.15.73:0/2734092450 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:14.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.376+0000 7fc5ae59c640 1 --2- 172.21.15.73:0/2734092450 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc5b00cbbd0 0x7fc5b00c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:14.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.376+0000 7fc5aed9d640 1 --2- 172.21.15.73:0/2734092450 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5b015b600 0x7fc5b00b5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:14.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.377+0000 7fc5a5d9b640 1 -- 172.21.15.73:0/2734092450 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc5b00cbbd0 msgr2=0x7fc5b00c8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:14.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.377+0000 7fc5a5d9b640 1 --2- 172.21.15.73:0/2734092450 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc5b00cbbd0 0x7fc5b00c8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:14.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.377+0000 7fc5a5d9b640 1 -- 172.21.15.73:0/2734092450 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5b015b600 msgr2=0x7fc5b00b5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:14.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.377+0000 7fc5a5d9b640 1 --2- 172.21.15.73:0/2734092450 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5b015b600 0x7fc5b00b5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:14.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.377+0000 7fc5a5d9b640 1 -- 172.21.15.73:0/2734092450 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5b01e4f80 con 0x7fc5b00cf5e0 2024-04-26T20:51:14.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.377+0000 7fc5a5d9b640 1 --2- 172.21.15.73:0/2734092450 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5b00cf5e0 0x7fc5b01562f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc59800e810 tx=0x7fc59800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:14.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.377+0000 7fc5a77fe640 1 -- 172.21.15.73:0/2734092450 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59800f040 con 0x7fc5b00cf5e0 2024-04-26T20:51:14.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.377+0000 7fc5a77fe640 1 -- 172.21.15.73:0/2734092450 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc598004030 con 0x7fc5b00cf5e0 2024-04-26T20:51:14.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.377+0000 7fc5b4d89640 1 -- 172.21.15.73:0/2734092450 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5b01e5260 con 0x7fc5b00cf5e0 2024-04-26T20:51:14.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.377+0000 7fc5a77fe640 1 -- 172.21.15.73:0/2734092450 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc598004400 con 0x7fc5b00cf5e0 2024-04-26T20:51:14.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.377+0000 7fc5b4d89640 1 -- 172.21.15.73:0/2734092450 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5b01e5820 con 0x7fc5b00cf5e0 2024-04-26T20:51:14.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.378+0000 7fc5b4d89640 1 -- 172.21.15.73:0/2734092450 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5b00694f0 con 0x7fc5b00cf5e0 2024-04-26T20:51:14.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.379+0000 7fc5a77fe640 1 -- 172.21.15.73:0/2734092450 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc59800c6d0 con 0x7fc5b00cf5e0 2024-04-26T20:51:14.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.379+0000 7fc5a77fe640 1 --2- 172.21.15.73:0/2734092450 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc59006c920 0x7fc59006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:14.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.379+0000 7fc5a77fe640 1 -- 172.21.15.73:0/2734092450 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fc59808bac0 con 0x7fc5b00cf5e0 2024-04-26T20:51:14.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.379+0000 7fc5ae59c640 1 --2- 172.21.15.73:0/2734092450 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc59006c920 0x7fc59006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:14.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.379+0000 7fc5ae59c640 1 --2- 172.21.15.73:0/2734092450 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc59006c920 0x7fc59006ede0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0009aa0 tx=0x7fc5a0009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:14.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.383+0000 7fc5a77fe640 1 -- 172.21.15.73:0/2734092450 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc598017020 con 0x7fc5b00cf5e0 2024-04-26T20:51:14.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.558+0000 7fc5b4d89640 1 -- 172.21.15.73:0/2734092450 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fc5b00ce7b0 con 0x7fc5b00cf5e0 2024-04-26T20:51:14.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.558+0000 7fc5a77fe640 1 -- 172.21.15.73:0/2734092450 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7fc598059530 con 0x7fc5b00cf5e0 2024-04-26T20:51:14.560 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-26T20:51:14.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.560+0000 7fc5b4d89640 1 -- 172.21.15.73:0/2734092450 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc59006c920 msgr2=0x7fc59006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:14.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.560+0000 7fc5b4d89640 1 --2- 172.21.15.73:0/2734092450 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc59006c920 0x7fc59006ede0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0009aa0 tx=0x7fc5a0009190 comp rx=0 tx=0).stop 2024-04-26T20:51:14.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.560+0000 7fc5b4d89640 1 -- 172.21.15.73:0/2734092450 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5b00cf5e0 msgr2=0x7fc5b01562f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:14.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.560+0000 7fc5b4d89640 1 --2- 172.21.15.73:0/2734092450 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5b00cf5e0 0x7fc5b01562f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc59800e810 tx=0x7fc59800ecd0 comp rx=0 tx=0).stop 2024-04-26T20:51:14.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.560+0000 7fc5b4d89640 1 -- 172.21.15.73:0/2734092450 shutdown_connections 2024-04-26T20:51:14.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.560+0000 7fc5b4d89640 1 --2- 172.21.15.73:0/2734092450 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5b00cf5e0 0x7fc5b01562f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:14.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.560+0000 7fc5b4d89640 1 --2- 172.21.15.73:0/2734092450 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc59006c920 0x7fc59006ede0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:14.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.560+0000 7fc5b4d89640 1 --2- 172.21.15.73:0/2734092450 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc5b015b600 0x7fc5b00b5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:14.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.560+0000 7fc5b4d89640 1 --2- 172.21.15.73:0/2734092450 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc5b00cbbd0 0x7fc5b00c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:14.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.560+0000 7fc5b4d89640 1 -- 172.21.15.73:0/2734092450 >> 172.21.15.73:0/2734092450 conn(0x7fc5b00d0be0 msgr2=0x7fc5b015a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:14.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.560+0000 7fc5b4d89640 1 -- 172.21.15.73:0/2734092450 shutdown_connections 2024-04-26T20:51:14.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.560+0000 7fc5b4d89640 1 -- 172.21.15.73:0/2734092450 wait complete. 2024-04-26T20:51:14.573 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:51:14.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.668+0000 7f8113faf640 1 Processor -- start 2024-04-26T20:51:14.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.669+0000 7f8113faf640 1 -- start start 2024-04-26T20:51:14.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.669+0000 7f8113faf640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f810c162880 0x7f810c0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:14.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.669+0000 7f8113faf640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f810c0b5f20 0x7f810c0d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:14.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.669+0000 7f8113faf640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f810c0b7e50 con 0x7f810c0d39f0 2024-04-26T20:51:14.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.669+0000 7f8113faf640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f810c0d0580 con 0x7f810c0b5f20 2024-04-26T20:51:14.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.669+0000 7f8113faf640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f810c0d1800 con 0x7f810c162880 2024-04-26T20:51:14.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.670+0000 7f811154a640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f810c0b5f20 0x7f810c0d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:14.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.670+0000 7f8111d4b640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f810c162880 0x7f810c0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:14.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.670+0000 7f811154a640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f810c0b5f20 0x7f810c0d34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.73:47378/0 (socket says 172.21.15.73:47378) 2024-04-26T20:51:14.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.670+0000 7f8111d4b640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f810c162880 0x7f810c0d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.73:35954/0 (socket says 172.21.15.73:35954) 2024-04-26T20:51:14.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.670+0000 7f811154a640 1 -- 172.21.15.73:0/4135871724 learned_addr learned my addr 172.21.15.73:0/4135871724 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:14.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.670+0000 7f811254c640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f810c0d39f0 0x7f810c0b69e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.73:45698/0 (socket says 172.21.15.73:45698) 2024-04-26T20:51:14.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.670+0000 7f811154a640 1 -- 172.21.15.73:0/4135871724 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f810c162880 msgr2=0x7f810c0d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:14.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.670+0000 7f811154a640 1 --2- 172.21.15.73:0/4135871724 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f810c162880 0x7f810c0d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:14.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.670+0000 7f811154a640 1 -- 172.21.15.73:0/4135871724 >> v1:172.21.15.120:6789/0 conn(0x7f810c0d39f0 legacy=0x7f810c0b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:14.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.670+0000 7f811154a640 1 -- 172.21.15.73:0/4135871724 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f810c0d07d0 con 0x7f810c0b5f20 2024-04-26T20:51:14.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.670+0000 7f8111d4b640 1 --2- 172.21.15.73:0/4135871724 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f810c162880 0x7f810c0d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:51:14.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.671+0000 7f811154a640 1 --2- 172.21.15.73:0/4135871724 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f810c0b5f20 0x7f810c0d34b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8108009870 tx=0x7f810802a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f5b5b8c104326ff server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:14.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.671+0000 7f8110d49640 1 -- 172.21.15.73:0/4135871724 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8108037070 con 0x7f810c0b5f20 2024-04-26T20:51:14.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.671+0000 7f8110d49640 1 -- 172.21.15.73:0/4135871724 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8108032440 con 0x7f810c0b5f20 2024-04-26T20:51:14.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.671+0000 7f8110d49640 1 -- 172.21.15.73:0/4135871724 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8108032810 con 0x7f810c0b5f20 2024-04-26T20:51:14.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.672+0000 7f8113faf640 1 -- 172.21.15.73:0/4135871724 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f810c0b5f20 msgr2=0x7f810c0d34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:14.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.672+0000 7f8113faf640 1 --2- 172.21.15.73:0/4135871724 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f810c0b5f20 0x7f810c0d34b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8108009870 tx=0x7f810802a270 comp rx=0 tx=0).stop 2024-04-26T20:51:14.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.672+0000 7f8113faf640 1 -- 172.21.15.73:0/4135871724 shutdown_connections 2024-04-26T20:51:14.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.672+0000 7f8113faf640 1 --2- 172.21.15.73:0/4135871724 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f810c0b5f20 0x7f810c0d34b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:14.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.672+0000 7f8113faf640 1 --2- 172.21.15.73:0/4135871724 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f810c162880 0x7f810c0d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:14.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.672+0000 7f8113faf640 1 -- 172.21.15.73:0/4135871724 >> 172.21.15.73:0/4135871724 conn(0x7f810c0d4f60 msgr2=0x7f810c0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:14.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.672+0000 7f8113faf640 1 -- 172.21.15.73:0/4135871724 shutdown_connections 2024-04-26T20:51:14.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.672+0000 7f8113faf640 1 -- 172.21.15.73:0/4135871724 wait complete. 2024-04-26T20:51:14.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.673+0000 7f8113faf640 1 Processor -- start 2024-04-26T20:51:14.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.673+0000 7f8113faf640 1 -- start start 2024-04-26T20:51:14.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.673+0000 7f8113faf640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f810c0b5f20 0x7f810c156c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:14.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.673+0000 7f8113faf640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f810c0d39f0 0x7f810c0cdb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:14.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.673+0000 7f8113faf640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f810c162880 0x7f810c15f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:14.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.673+0000 7f8113faf640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f810c162c60 con 0x7f810c0b5f20 2024-04-26T20:51:14.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.673+0000 7f8113faf640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f810c0d0d90 con 0x7f810c162880 2024-04-26T20:51:14.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.673+0000 7f8113faf640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f810c162de0 con 0x7f810c0d39f0 2024-04-26T20:51:14.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.674+0000 7f8111d4b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f810c0b5f20 0x7f810c156c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:14.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.674+0000 7f8111d4b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f810c0b5f20 0x7f810c156c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51858/0 (socket says 172.21.15.73:51858) 2024-04-26T20:51:14.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.674+0000 7f8111d4b640 1 -- 172.21.15.73:0/2799312742 learned_addr learned my addr 172.21.15.73:0/2799312742 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:14.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.674+0000 7f811154a640 1 --2- 172.21.15.73:0/2799312742 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f810c0d39f0 0x7f810c0cdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:14.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.674+0000 7f811254c640 1 --2- 172.21.15.73:0/2799312742 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f810c162880 0x7f810c15f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:14.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.674+0000 7f8111d4b640 1 -- 172.21.15.73:0/2799312742 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f810c0d39f0 msgr2=0x7f810c0cdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:14.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.674+0000 7f8111d4b640 1 --2- 172.21.15.73:0/2799312742 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f810c0d39f0 0x7f810c0cdb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:14.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.674+0000 7f8111d4b640 1 -- 172.21.15.73:0/2799312742 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f810c162880 msgr2=0x7f810c15f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:14.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.674+0000 7f8111d4b640 1 --2- 172.21.15.73:0/2799312742 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f810c162880 0x7f810c15f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:14.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.674+0000 7f8111d4b640 1 -- 172.21.15.73:0/2799312742 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f810c0ce540 con 0x7f810c0b5f20 2024-04-26T20:51:14.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.674+0000 7f8111d4b640 1 --2- 172.21.15.73:0/2799312742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f810c0b5f20 0x7f810c156c80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f80fc00ec10 tx=0x7f80fc00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:14.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.674+0000 7f81027fc640 1 -- 172.21.15.73:0/2799312742 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80fc017070 con 0x7f810c0b5f20 2024-04-26T20:51:14.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.674+0000 7f81027fc640 1 -- 172.21.15.73:0/2799312742 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80fc00cc30 con 0x7f810c0b5f20 2024-04-26T20:51:14.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.674+0000 7f81027fc640 1 -- 172.21.15.73:0/2799312742 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80fc013470 con 0x7f810c0b5f20 2024-04-26T20:51:14.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.674+0000 7f8113faf640 1 -- 172.21.15.73:0/2799312742 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f810c0ce820 con 0x7f810c0b5f20 2024-04-26T20:51:14.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.674+0000 7f8113faf640 1 -- 172.21.15.73:0/2799312742 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f810c0cede0 con 0x7f810c0b5f20 2024-04-26T20:51:14.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.675+0000 7f8113faf640 1 -- 172.21.15.73:0/2799312742 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f810c0d3430 con 0x7f810c0b5f20 2024-04-26T20:51:14.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.676+0000 7f81027fc640 1 -- 172.21.15.73:0/2799312742 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f80fc01a070 con 0x7f810c0b5f20 2024-04-26T20:51:14.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.676+0000 7f81027fc640 1 --2- 172.21.15.73:0/2799312742 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f80f006c8f0 0x7f80f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:14.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.676+0000 7f81027fc640 1 -- 172.21.15.73:0/2799312742 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f80fc08c0c0 con 0x7f810c0b5f20 2024-04-26T20:51:14.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.676+0000 7f811154a640 1 --2- 172.21.15.73:0/2799312742 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f80f006c8f0 0x7f80f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:14.679 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.677+0000 7f811154a640 1 --2- 172.21.15.73:0/2799312742 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f80f006c8f0 0x7f80f006edb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8108034000 tx=0x7f810802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:14.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.680+0000 7f81027fc640 1 -- 172.21.15.73:0/2799312742 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f80fc059b30 con 0x7f810c0b5f20 2024-04-26T20:51:14.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.859+0000 7f8113faf640 1 -- 172.21.15.73:0/2799312742 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f810c0694f0 con 0x7f810c0b5f20 2024-04-26T20:51:14.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.861+0000 7f81027fc640 1 -- 172.21.15.73:0/2799312742 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18902 (secure 0 0 0) 0x7f80fc05de30 con 0x7f810c0b5f20 2024-04-26T20:51:14.862 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:51:14.862 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":27,"fsid":"fbd3f548-b547-4713-856b-09f98f5d1764","created":"2024-04-26T20:50:07.876786+0000","modified":"2024-04-26T20:51:13.783133+0000","last_up_change":"2024-04-26T20:50:19.082741+0000","last_in_change":"2024-04-26T20:50:10.018400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:50:19.993731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:50:29.964420+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:50:30.920082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:50:34.650207+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"72d6f4f3-214a-4392-a02f-c33cbcab3c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6809","nonce":3461705124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6811","nonce":3461705124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6815","nonce":3461705124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6813","nonce":3461705124}]},"public_addr":"172.21.15.73:6809/3461705124","cluster_addr":"172.21.15.73:6811/3461705124","heartbeat_back_addr":"172.21.15.73:6815/3461705124","heartbeat_front_addr":"172.21.15.73:6813/3461705124","state":["exists","up"]},{"osd":1,"uuid":"786bf8ca-6ddc-4b3c-be6c-9152398bd5fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6817","nonce":2827619403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6819","nonce":2827619403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6825","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6827","nonce":2827619403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6821","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6823","nonce":2827619403}]},"public_addr":"172.21.15.73:6817/2827619403","cluster_addr":"172.21.15.73:6819/2827619403","heartbeat_back_addr":"172.21.15.73:6827/2827619403","heartbeat_front_addr":"172.21.15.73:6823/2827619403","state":["exists","up"]},{"osd":2,"uuid":"90963b1a-5437-4ef8-ad8b-6b74c534eeb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6801","nonce":2542681064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6803","nonce":2542681064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6808","nonce":2542681064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6805","nonce":2542681064}]},"public_addr":"172.21.15.73:6801/2542681064","cluster_addr":"172.21.15.73:6803/2542681064","heartbeat_back_addr":"172.21.15.73:6808/2542681064","heartbeat_front_addr":"172.21.15.73:6805/2542681064","state":["exists","up"]},{"osd":3,"uuid":"bd3d2148-6e38-41b4-a453-fe6e3b8b4ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6822","nonce":1604042718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6826","nonce":1604042718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6831","nonce":1604042718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6829","nonce":1604042718}]},"public_addr":"172.21.15.73:6822/1604042718","cluster_addr":"172.21.15.73:6826/1604042718","heartbeat_back_addr":"172.21.15.73:6831/1604042718","heartbeat_front_addr":"172.21.15.73:6829/1604042718","state":["exists","up"]},{"osd":4,"uuid":"e8668dc8-a23f-46d2-8311-cb882da08e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6817","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6819","nonce":3591055561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6824","nonce":3591055561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6831","nonce":3591055561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6828","nonce":3591055561}]},"public_addr":"172.21.15.120:6819/3591055561","cluster_addr":"172.21.15.120:6824/3591055561","heartbeat_back_addr":"172.21.15.120:6831/3591055561","heartbeat_front_addr":"172.21.15.120:6828/3591055561","state":["exists","up"]},{"osd":5,"uuid":"cbac0bdb-96bd-4852-bcce-e564495c3fcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6809","nonce":819450433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6811","nonce":819450433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6815","nonce":819450433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6813","nonce":819450433}]},"public_addr":"172.21.15.120:6809/819450433","cluster_addr":"172.21.15.120:6811/819450433","heartbeat_back_addr":"172.21.15.120:6815/819450433","heartbeat_front_addr":"172.21.15.120:6813/819450433","state":["exists","up"]},{"osd":6,"uuid":"1884b2a8-569f-4272-96a9-f25a09364dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6801","nonce":3946210727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6803","nonce":3946210727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6807","nonce":3946210727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6805","nonce":3946210727}]},"public_addr":"172.21.15.120:6801/3946210727","cluster_addr":"172.21.15.120:6803/3946210727","heartbeat_back_addr":"172.21.15.120:6807/3946210727","heartbeat_front_addr":"172.21.15.120:6805/3946210727","state":["exists","up"]},{"osd":7,"uuid":"c0c3bf6d-a4c5-4f1e-aa2d-330951830666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6818","nonce":2915574674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6821","nonce":2915574674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6827","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6829","nonce":2915574674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6823","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6825","nonce":2915574674}]},"public_addr":"172.21.15.120:6818/2915574674","cluster_addr":"172.21.15.120:6821/2915574674","heartbeat_back_addr":"172.21.15.120:6829/2915574674","heartbeat_front_addr":"172.21.15.120:6825/2915574674","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.218216+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.208710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.112705+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.182789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.164324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.174212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.225396+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.261559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:51:14.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.862+0000 7f8113faf640 1 -- 172.21.15.73:0/2799312742 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f80f006c8f0 msgr2=0x7f80f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:14.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.862+0000 7f8113faf640 1 --2- 172.21.15.73:0/2799312742 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f80f006c8f0 0x7f80f006edb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8108034000 tx=0x7f810802e000 comp rx=0 tx=0).stop 2024-04-26T20:51:14.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.862+0000 7f8113faf640 1 -- 172.21.15.73:0/2799312742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f810c0b5f20 msgr2=0x7f810c156c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:14.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.862+0000 7f8113faf640 1 --2- 172.21.15.73:0/2799312742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f810c0b5f20 0x7f810c156c80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f80fc00ec10 tx=0x7f80fc00c520 comp rx=0 tx=0).stop 2024-04-26T20:51:14.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.863+0000 7f8113faf640 1 -- 172.21.15.73:0/2799312742 shutdown_connections 2024-04-26T20:51:14.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.863+0000 7f8113faf640 1 --2- 172.21.15.73:0/2799312742 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f80f006c8f0 0x7f80f006edb0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:14.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.863+0000 7f8113faf640 1 --2- 172.21.15.73:0/2799312742 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f810c162880 0x7f810c15f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:14.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.863+0000 7f8113faf640 1 --2- 172.21.15.73:0/2799312742 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f810c0d39f0 0x7f810c0cdb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:14.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.863+0000 7f8113faf640 1 --2- 172.21.15.73:0/2799312742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f810c0b5f20 0x7f810c156c80 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:14.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.863+0000 7f8113faf640 1 -- 172.21.15.73:0/2799312742 >> 172.21.15.73:0/2799312742 conn(0x7f810c0d4f60 msgr2=0x7f810c0d5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:14.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.863+0000 7f8113faf640 1 -- 172.21.15.73:0/2799312742 shutdown_connections 2024-04-26T20:51:14.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.863+0000 7f8113faf640 1 -- 172.21.15.73:0/2799312742 wait complete. 2024-04-26T20:51:14.875 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:50:19.993731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:50:29.964420+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:50:30.920082+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T20:50:34.650207+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T20:51:14.875 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:51:14.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.973+0000 7f9c97493640 1 Processor -- start 2024-04-26T20:51:14.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.974+0000 7f9c97493640 1 -- start start 2024-04-26T20:51:14.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.974+0000 7f9c97493640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f9c90160360 0x7f9c9006c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:14.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.974+0000 7f9c97493640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c90076290 con 0x7f9c9006c780 2024-04-26T20:51:14.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.974+0000 7f9c97493640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c90145500 con 0x7f9c900b4ae0 2024-04-26T20:51:14.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.974+0000 7f9c97493640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f9c9015fed0 con 0x7f9c90160360 2024-04-26T20:51:14.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.974+0000 7f9c95a30640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f9c9006c780 0x7f9c90142900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40186/0 (socket says 172.21.15.73:40186) 2024-04-26T20:51:14.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.974+0000 7f9c95a30640 1 -- 172.21.15.73:0/1141213272 learned_addr learned my addr 172.21.15.73:0/1141213272 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:14.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.975+0000 7f9c8ffff640 1 -- 172.21.15.73:0/1141213272 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 935652326 0 0) 0x7f9c90076290 con 0x7f9c9006c780 2024-04-26T20:51:14.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.975+0000 7f9c8ffff640 1 -- 172.21.15.73:0/1141213272 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c78003520 con 0x7f9c9006c780 2024-04-26T20:51:14.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.975+0000 7f9c94a2e640 1 --2- 172.21.15.73:0/1141213272 >> v2:172.21.15.120:3300/0 conn(0x7f9c90160360 0x7f9c9006c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:14.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.975+0000 7f9c8ffff640 1 -- 172.21.15.73:0/1141213272 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1778905474 0 0) 0x7f9c78003520 con 0x7f9c9006c780 2024-04-26T20:51:14.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.975+0000 7f9c8ffff640 1 -- 172.21.15.73:0/1141213272 >> v2:172.21.15.120:3300/0 conn(0x7f9c90160360 msgr2=0x7f9c9006c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:14.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.975+0000 7f9c8ffff640 1 --2- 172.21.15.73:0/1141213272 >> v2:172.21.15.120:3300/0 conn(0x7f9c90160360 0x7f9c9006c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:14.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.975+0000 7f9c8ffff640 1 -- 172.21.15.73:0/1141213272 >> v1:172.21.15.120:6789/0 conn(0x7f9c900b4ae0 legacy=0x7f9c90144990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:14.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.975+0000 7f9c8ffff640 1 -- 172.21.15.73:0/1141213272 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c901457e0 con 0x7f9c9006c780 2024-04-26T20:51:14.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.975+0000 7f9c8ffff640 1 -- 172.21.15.73:0/1141213272 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7f9c800019f0 con 0x7f9c9006c780 2024-04-26T20:51:14.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.975+0000 7f9c8ffff640 1 -- 172.21.15.73:0/1141213272 >> v1:172.21.15.73:6789/0 conn(0x7f9c9006c780 legacy=0x7f9c90142900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:14.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.975+0000 7f9c8ffff640 1 --2- 172.21.15.73:0/1141213272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c780039b0 0x7f9c78003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:14.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.975+0000 7f9c8ffff640 1 --2- 172.21.15.73:0/1141213272 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c78004800 0x7f9c78004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:14.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.975+0000 7f9c8ffff640 1 --2- 172.21.15.73:0/1141213272 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9c780051c0 0x7f9c7800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:14.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.975+0000 7f9c8ffff640 1 -- 172.21.15.73:0/1141213272 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9c9015fed0 con 0x7f9c780039b0 2024-04-26T20:51:14.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.975+0000 7f9c8ffff640 1 -- 172.21.15.73:0/1141213272 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9c7800c060 con 0x7f9c78004800 2024-04-26T20:51:14.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.975+0000 7f9c8ffff640 1 -- 172.21.15.73:0/1141213272 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f9c7800c260 con 0x7f9c780051c0 2024-04-26T20:51:14.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.975+0000 7f9c9522f640 1 --2- 172.21.15.73:0/1141213272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c780039b0 0x7f9c78003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:14.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.975+0000 7f9c94a2e640 1 --2- 172.21.15.73:0/1141213272 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c78004800 0x7f9c78004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:14.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.975+0000 7f9c95a30640 1 --2- 172.21.15.73:0/1141213272 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9c780051c0 0x7f9c7800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:14.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.975+0000 7f9c9522f640 1 -- 172.21.15.73:0/1141213272 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9c780051c0 msgr2=0x7f9c7800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:14.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.975+0000 7f9c9522f640 1 --2- 172.21.15.73:0/1141213272 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9c780051c0 0x7f9c7800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:14.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.975+0000 7f9c9522f640 1 -- 172.21.15.73:0/1141213272 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c78004800 msgr2=0x7f9c78004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:14.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.975+0000 7f9c9522f640 1 --2- 172.21.15.73:0/1141213272 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c78004800 0x7f9c78004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:14.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.975+0000 7f9c9522f640 1 -- 172.21.15.73:0/1141213272 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c78003520 con 0x7f9c780039b0 2024-04-26T20:51:14.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.975+0000 7f9c95a30640 1 --2- 172.21.15.73:0/1141213272 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9c780051c0 0x7f9c7800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:14.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.975+0000 7f9c94a2e640 1 --2- 172.21.15.73:0/1141213272 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c78004800 0x7f9c78004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:14.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.975+0000 7f9c95a30640 1 -- 172.21.15.73:0/1141213272 reap_dead start 2024-04-26T20:51:14.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.976+0000 7f9c9522f640 1 --2- 172.21.15.73:0/1141213272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c780039b0 0x7f9c78003dd0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9c88001750 tx=0x7f9c78005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9628bb39bf973256 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:14.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.976+0000 7f9c8ffff640 1 -- 172.21.15.73:0/1141213272 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c88001370 con 0x7f9c780039b0 2024-04-26T20:51:14.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.976+0000 7f9c8ffff640 1 -- 172.21.15.73:0/1141213272 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c880052d0 con 0x7f9c780039b0 2024-04-26T20:51:14.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.976+0000 7f9c8ffff640 1 -- 172.21.15.73:0/1141213272 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8802abc0 con 0x7f9c780039b0 2024-04-26T20:51:14.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.977+0000 7f9c97493640 1 -- 172.21.15.73:0/1141213272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c780039b0 msgr2=0x7f9c78003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:14.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.977+0000 7f9c97493640 1 --2- 172.21.15.73:0/1141213272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c780039b0 0x7f9c78003dd0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9c88001750 tx=0x7f9c78005830 comp rx=0 tx=0).stop 2024-04-26T20:51:14.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.977+0000 7f9c97493640 1 -- 172.21.15.73:0/1141213272 shutdown_connections 2024-04-26T20:51:14.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.977+0000 7f9c97493640 1 --2- 172.21.15.73:0/1141213272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c780039b0 0x7f9c78003dd0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:14.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.977+0000 7f9c97493640 1 -- 172.21.15.73:0/1141213272 >> 172.21.15.73:0/1141213272 conn(0x7f9c900b3110 msgr2=0x7f9c900b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:14.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.977+0000 7f9c97493640 1 -- 172.21.15.73:0/1141213272 shutdown_connections 2024-04-26T20:51:14.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.977+0000 7f9c97493640 1 -- 172.21.15.73:0/1141213272 wait complete. 2024-04-26T20:51:14.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.978+0000 7f9c97493640 1 Processor -- start 2024-04-26T20:51:14.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.978+0000 7f9c97493640 1 -- start start 2024-04-26T20:51:14.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.979+0000 7f9c97493640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c780039b0 0x7f9c9006faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:14.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.979+0000 7f9c97493640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c9013c770 0x7f9c9013d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:14.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.979+0000 7f9c97493640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9c9013d590 0x7f9c9013fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:14.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.979+0000 7f9c97493640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9c9015fb60 con 0x7f9c780039b0 2024-04-26T20:51:14.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.979+0000 7f9c97493640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9c9006cbb0 con 0x7f9c9013c770 2024-04-26T20:51:14.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.979+0000 7f9c97493640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f9c900b4ec0 con 0x7f9c9013d590 2024-04-26T20:51:14.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.979+0000 7f9c9522f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c780039b0 0x7f9c9006faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:14.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.979+0000 7f9c9522f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c780039b0 0x7f9c9006faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51864/0 (socket says 172.21.15.73:51864) 2024-04-26T20:51:14.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.979+0000 7f9c9522f640 1 -- 172.21.15.73:0/1083048127 learned_addr learned my addr 172.21.15.73:0/1083048127 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:14.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.979+0000 7f9c94a2e640 1 --2- 172.21.15.73:0/1083048127 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c9013c770 0x7f9c9013d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:14.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.979+0000 7f9c95a30640 1 --2- 172.21.15.73:0/1083048127 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9c9013d590 0x7f9c9013fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:14.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.979+0000 7f9c9522f640 1 -- 172.21.15.73:0/1083048127 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9c9013d590 msgr2=0x7f9c9013fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:14.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.979+0000 7f9c9522f640 1 --2- 172.21.15.73:0/1083048127 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9c9013d590 0x7f9c9013fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:14.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.979+0000 7f9c9522f640 1 -- 172.21.15.73:0/1083048127 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c9013c770 msgr2=0x7f9c9013d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:14.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.979+0000 7f9c9522f640 1 --2- 172.21.15.73:0/1083048127 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c9013c770 0x7f9c9013d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:14.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.979+0000 7f9c9522f640 1 -- 172.21.15.73:0/1083048127 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c9006ffe0 con 0x7f9c780039b0 2024-04-26T20:51:14.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.979+0000 7f9c95a30640 1 --2- 172.21.15.73:0/1083048127 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9c9013d590 0x7f9c9013fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:14.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.979+0000 7f9c94a2e640 1 --2- 172.21.15.73:0/1083048127 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c9013c770 0x7f9c9013d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:14.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.979+0000 7f9c9522f640 1 --2- 172.21.15.73:0/1083048127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c780039b0 0x7f9c9006faa0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9c88001870 tx=0x7f9c88001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:14.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.979+0000 7f9c8dffb640 1 -- 172.21.15.73:0/1083048127 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8803f020 con 0x7f9c780039b0 2024-04-26T20:51:14.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.979+0000 7f9c8dffb640 1 -- 172.21.15.73:0/1083048127 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c88005480 con 0x7f9c780039b0 2024-04-26T20:51:14.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.979+0000 7f9c8dffb640 1 -- 172.21.15.73:0/1083048127 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c88037040 con 0x7f9c780039b0 2024-04-26T20:51:14.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.979+0000 7f9c97493640 1 -- 172.21.15.73:0/1083048127 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c9006ea00 con 0x7f9c780039b0 2024-04-26T20:51:14.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.980+0000 7f9c97493640 1 -- 172.21.15.73:0/1083048127 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c9006efc0 con 0x7f9c780039b0 2024-04-26T20:51:14.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.981+0000 7f9c97493640 1 -- 172.21.15.73:0/1083048127 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c900d0900 con 0x7f9c780039b0 2024-04-26T20:51:14.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.981+0000 7f9c8dffb640 1 -- 172.21.15.73:0/1083048127 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9c88042020 con 0x7f9c780039b0 2024-04-26T20:51:14.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.981+0000 7f9c8dffb640 1 --2- 172.21.15.73:0/1083048127 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f9c7c06c8f0 0x7f9c7c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:14.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.981+0000 7f9c8dffb640 1 -- 172.21.15.73:0/1083048127 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f9c880abcd0 con 0x7f9c780039b0 2024-04-26T20:51:14.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.981+0000 7f9c94a2e640 1 --2- 172.21.15.73:0/1083048127 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f9c7c06c8f0 0x7f9c7c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:14.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.982+0000 7f9c94a2e640 1 --2- 172.21.15.73:0/1083048127 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f9c7c06c8f0 0x7f9c7c06edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9c901f87f0 tx=0x7f9c84008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:14.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:14.986+0000 7f9c8dffb640 1 -- 172.21.15.73:0/1083048127 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9c900d0900 con 0x7f9c780039b0 2024-04-26T20:51:15.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.161+0000 7f9c97493640 1 -- 172.21.15.73:0/1083048127 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9c900d1dd0 con 0x7f9c780039b0 2024-04-26T20:51:15.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.161+0000 7f9c8dffb640 1 -- 172.21.15.73:0/1083048127 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f9c88049020 con 0x7f9c780039b0 2024-04-26T20:51:15.162 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-26T20:51:15.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.163+0000 7f9c97493640 1 -- 172.21.15.73:0/1083048127 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f9c7c06c8f0 msgr2=0x7f9c7c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:15.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.163+0000 7f9c97493640 1 --2- 172.21.15.73:0/1083048127 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f9c7c06c8f0 0x7f9c7c06edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9c901f87f0 tx=0x7f9c84008040 comp rx=0 tx=0).stop 2024-04-26T20:51:15.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.163+0000 7f9c97493640 1 -- 172.21.15.73:0/1083048127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c780039b0 msgr2=0x7f9c9006faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:15.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.163+0000 7f9c97493640 1 --2- 172.21.15.73:0/1083048127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c780039b0 0x7f9c9006faa0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9c88001870 tx=0x7f9c88001120 comp rx=0 tx=0).stop 2024-04-26T20:51:15.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.163+0000 7f9c97493640 1 -- 172.21.15.73:0/1083048127 shutdown_connections 2024-04-26T20:51:15.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.163+0000 7f9c97493640 1 --2- 172.21.15.73:0/1083048127 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f9c7c06c8f0 0x7f9c7c06edb0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:15.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.163+0000 7f9c97493640 1 --2- 172.21.15.73:0/1083048127 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9c9013d590 0x7f9c9013fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:15.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.163+0000 7f9c97493640 1 --2- 172.21.15.73:0/1083048127 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c9013c770 0x7f9c9013d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:15.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.163+0000 7f9c97493640 1 --2- 172.21.15.73:0/1083048127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c780039b0 0x7f9c9006faa0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:15.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.163+0000 7f9c97493640 1 -- 172.21.15.73:0/1083048127 >> 172.21.15.73:0/1083048127 conn(0x7f9c900b3110 msgr2=0x7f9c90162010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:15.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.163+0000 7f9c97493640 1 -- 172.21.15.73:0/1083048127 shutdown_connections 2024-04-26T20:51:15.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.163+0000 7f9c97493640 1 -- 172.21.15.73:0/1083048127 wait complete. 2024-04-26T20:51:15.175 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:51:15.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.275+0000 7f1f92ccf640 1 Processor -- start 2024-04-26T20:51:15.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.275+0000 7f1f92ccf640 1 -- start start 2024-04-26T20:51:15.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.275+0000 7f1f92ccf640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f1f8c0b0380 0x7f1f8c0ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:15.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.275+0000 7f1f92ccf640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1f8c06c950 0x7f1f8c15e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:15.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.275+0000 7f1f92ccf640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f8c0cfd80 con 0x7f1f8c0b10b0 2024-04-26T20:51:15.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.275+0000 7f1f92ccf640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f1f8c0af8e0 con 0x7f1f8c06c950 2024-04-26T20:51:15.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.275+0000 7f1f92ccf640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f1f8c0d0020 con 0x7f1f8c0b0380 2024-04-26T20:51:15.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.276+0000 7f1f9126c640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1f8c06c950 0x7f1f8c15e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:15.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.276+0000 7f1f9126c640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1f8c06c950 0x7f1f8c15e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51876/0 (socket says 172.21.15.73:51876) 2024-04-26T20:51:15.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.276+0000 7f1f9126c640 1 -- 172.21.15.73:0/1642461141 learned_addr learned my addr 172.21.15.73:0/1642461141 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:15.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.276+0000 7f1f9126c640 1 -- 172.21.15.73:0/1642461141 >> v2:172.21.15.120:3300/0 conn(0x7f1f8c0b0380 msgr2=0x7f1f8c0ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:15.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.276+0000 7f1f9126c640 1 --2- 172.21.15.73:0/1642461141 >> v2:172.21.15.120:3300/0 conn(0x7f1f8c0b0380 0x7f1f8c0ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:15.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.276+0000 7f1f9126c640 1 -- 172.21.15.73:0/1642461141 >> v1:172.21.15.120:6789/0 conn(0x7f1f8c0b10b0 legacy=0x7f1f8c06c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:15.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.276+0000 7f1f9126c640 1 -- 172.21.15.73:0/1642461141 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f8c0d0300 con 0x7f1f8c06c950 2024-04-26T20:51:15.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.277+0000 7f1f9126c640 1 --2- 172.21.15.73:0/1642461141 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f8c06c950 0x7f1f8c15e970 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c072f00 tx=0x7f1f8402a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d661c3bb7f6ef317 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:15.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.277+0000 7f1f8b7fe640 1 -- 172.21.15.73:0/1642461141 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f84036070 con 0x7f1f8c06c950 2024-04-26T20:51:15.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.277+0000 7f1f8b7fe640 1 -- 172.21.15.73:0/1642461141 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f84031600 con 0x7f1f8c06c950 2024-04-26T20:51:15.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.277+0000 7f1f8b7fe640 1 -- 172.21.15.73:0/1642461141 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f840319d0 con 0x7f1f8c06c950 2024-04-26T20:51:15.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.277+0000 7f1f92ccf640 1 -- 172.21.15.73:0/1642461141 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f8c06c950 msgr2=0x7f1f8c15e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:15.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.277+0000 7f1f92ccf640 1 --2- 172.21.15.73:0/1642461141 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f8c06c950 0x7f1f8c15e970 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c072f00 tx=0x7f1f8402a640 comp rx=0 tx=0).stop 2024-04-26T20:51:15.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.277+0000 7f1f92ccf640 1 -- 172.21.15.73:0/1642461141 shutdown_connections 2024-04-26T20:51:15.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.277+0000 7f1f92ccf640 1 --2- 172.21.15.73:0/1642461141 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f8c06c950 0x7f1f8c15e970 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:15.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.277+0000 7f1f92ccf640 1 --2- 172.21.15.73:0/1642461141 >> v2:172.21.15.120:3300/0 conn(0x7f1f8c0b0380 0x7f1f8c0ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:15.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.277+0000 7f1f92ccf640 1 -- 172.21.15.73:0/1642461141 >> 172.21.15.73:0/1642461141 conn(0x7f1f8c0d1cf0 msgr2=0x7f1f8c0d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:15.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.277+0000 7f1f92ccf640 1 -- 172.21.15.73:0/1642461141 shutdown_connections 2024-04-26T20:51:15.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.277+0000 7f1f92ccf640 1 -- 172.21.15.73:0/1642461141 wait complete. 2024-04-26T20:51:15.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.278+0000 7f1f92ccf640 1 Processor -- start 2024-04-26T20:51:15.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.278+0000 7f1f92ccf640 1 -- start start 2024-04-26T20:51:15.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.279+0000 7f1f92ccf640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f8c06c950 0x7f1f8c162500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:15.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.279+0000 7f1f92ccf640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f8c0b0380 0x7f1f8c06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:15.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.279+0000 7f1f92ccf640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1f8c0b10b0 0x7f1f8c0701d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:15.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.279+0000 7f1f92ccf640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c0b0760 con 0x7f1f8c06c950 2024-04-26T20:51:15.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.279+0000 7f1f92ccf640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c0d1980 con 0x7f1f8c0b0380 2024-04-26T20:51:15.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.279+0000 7f1f92ccf640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f1f8c0d15c0 con 0x7f1f8c0b10b0 2024-04-26T20:51:15.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.279+0000 7f1f90a6b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f8c06c950 0x7f1f8c162500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:15.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.279+0000 7f1f90a6b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f8c06c950 0x7f1f8c162500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51884/0 (socket says 172.21.15.73:51884) 2024-04-26T20:51:15.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.279+0000 7f1f90a6b640 1 -- 172.21.15.73:0/2896718462 learned_addr learned my addr 172.21.15.73:0/2896718462 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:15.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.280+0000 7f1f8bfff640 1 --2- 172.21.15.73:0/2896718462 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f8c0b0380 0x7f1f8c06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:15.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.280+0000 7f1f90a6b640 1 -- 172.21.15.73:0/2896718462 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1f8c0b10b0 msgr2=0x7f1f8c0701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:15.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.280+0000 7f1f90a6b640 1 --2- 172.21.15.73:0/2896718462 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1f8c0b10b0 0x7f1f8c0701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:15.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.280+0000 7f1f90a6b640 1 -- 172.21.15.73:0/2896718462 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f8c0b0380 msgr2=0x7f1f8c06fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:15.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.280+0000 7f1f90a6b640 1 --2- 172.21.15.73:0/2896718462 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f8c0b0380 0x7f1f8c06fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:15.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.280+0000 7f1f90a6b640 1 -- 172.21.15.73:0/2896718462 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f8c1f6c40 con 0x7f1f8c06c950 2024-04-26T20:51:15.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.280+0000 7f1f90a6b640 1 --2- 172.21.15.73:0/2896718462 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f8c06c950 0x7f1f8c162500 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c00b4a0 tx=0x7f1f7c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:15.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.280+0000 7f1f897fa640 1 -- 172.21.15.73:0/2896718462 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c014070 con 0x7f1f8c06c950 2024-04-26T20:51:15.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.280+0000 7f1f897fa640 1 -- 172.21.15.73:0/2896718462 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f7c010610 con 0x7f1f8c06c950 2024-04-26T20:51:15.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.280+0000 7f1f92ccf640 1 -- 172.21.15.73:0/2896718462 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f8c1f6f20 con 0x7f1f8c06c950 2024-04-26T20:51:15.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.280+0000 7f1f897fa640 1 -- 172.21.15.73:0/2896718462 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c010a00 con 0x7f1f8c06c950 2024-04-26T20:51:15.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.280+0000 7f1f92ccf640 1 -- 172.21.15.73:0/2896718462 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f8c1f74e0 con 0x7f1f8c06c950 2024-04-26T20:51:15.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.281+0000 7f1f92ccf640 1 -- 172.21.15.73:0/2896718462 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f8c0694f0 con 0x7f1f8c06c950 2024-04-26T20:51:15.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.282+0000 7f1f897fa640 1 -- 172.21.15.73:0/2896718462 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1f7c017070 con 0x7f1f8c06c950 2024-04-26T20:51:15.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.282+0000 7f1f897fa640 1 --2- 172.21.15.73:0/2896718462 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1f6c06c8f0 0x7f1f6c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:15.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.282+0000 7f1f8bfff640 1 --2- 172.21.15.73:0/2896718462 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1f6c06c8f0 0x7f1f6c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:15.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.282+0000 7f1f897fa640 1 -- 172.21.15.73:0/2896718462 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f1f7c08b950 con 0x7f1f8c06c950 2024-04-26T20:51:15.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.282+0000 7f1f8bfff640 1 --2- 172.21.15.73:0/2896718462 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1f6c06c8f0 0x7f1f6c06edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c0cee80 tx=0x7f1f74008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:15.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.286+0000 7f1f897fa640 1 -- 172.21.15.73:0/2896718462 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1f7c059a10 con 0x7f1f8c06c950 2024-04-26T20:51:15.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.461+0000 7f1f92ccf640 1 -- 172.21.15.73:0/2896718462 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1f8c06df70 con 0x7f1f8c06c950 2024-04-26T20:51:15.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.461+0000 7f1f897fa640 1 -- 172.21.15.73:0/2896718462 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f1f7c059310 con 0x7f1f8c06c950 2024-04-26T20:51:15.463 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-26T20:51:15.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.463+0000 7f1f92ccf640 1 -- 172.21.15.73:0/2896718462 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1f6c06c8f0 msgr2=0x7f1f6c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:15.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.463+0000 7f1f92ccf640 1 --2- 172.21.15.73:0/2896718462 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1f6c06c8f0 0x7f1f6c06edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c0cee80 tx=0x7f1f74008040 comp rx=0 tx=0).stop 2024-04-26T20:51:15.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.463+0000 7f1f92ccf640 1 -- 172.21.15.73:0/2896718462 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f8c06c950 msgr2=0x7f1f8c162500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:15.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.463+0000 7f1f92ccf640 1 --2- 172.21.15.73:0/2896718462 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f8c06c950 0x7f1f8c162500 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c00b4a0 tx=0x7f1f7c00b960 comp rx=0 tx=0).stop 2024-04-26T20:51:15.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.463+0000 7f1f92ccf640 1 -- 172.21.15.73:0/2896718462 shutdown_connections 2024-04-26T20:51:15.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.463+0000 7f1f92ccf640 1 --2- 172.21.15.73:0/2896718462 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1f6c06c8f0 0x7f1f6c06edb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:15.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.463+0000 7f1f92ccf640 1 --2- 172.21.15.73:0/2896718462 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1f8c0b10b0 0x7f1f8c0701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:15.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.463+0000 7f1f92ccf640 1 --2- 172.21.15.73:0/2896718462 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f8c0b0380 0x7f1f8c06fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:15.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.463+0000 7f1f92ccf640 1 --2- 172.21.15.73:0/2896718462 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f8c06c950 0x7f1f8c162500 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:15.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.463+0000 7f1f92ccf640 1 -- 172.21.15.73:0/2896718462 >> 172.21.15.73:0/2896718462 conn(0x7f1f8c0d1cf0 msgr2=0x7f1f8c0b6900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:15.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.464+0000 7f1f92ccf640 1 -- 172.21.15.73:0/2896718462 shutdown_connections 2024-04-26T20:51:15.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.464+0000 7f1f92ccf640 1 -- 172.21.15.73:0/2896718462 wait complete. 2024-04-26T20:51:15.475 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:51:15.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.573+0000 7fd357ec4640 1 Processor -- start 2024-04-26T20:51:15.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.574+0000 7fd357ec4640 1 -- start start 2024-04-26T20:51:15.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.574+0000 7fd357ec4640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fd35015f770 0x7fd3500b1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:15.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.574+0000 7fd357ec4640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd3500d1920 0x7fd3500cdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:15.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.574+0000 7fd357ec4640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd35006e580 con 0x7fd3500d0d60 2024-04-26T20:51:15.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.574+0000 7fd357ec4640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7fd35006f3e0 con 0x7fd3500d1920 2024-04-26T20:51:15.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.574+0000 7fd357ec4640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fd3500cd6e0 con 0x7fd35015f770 2024-04-26T20:51:15.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.575+0000 7fd356461640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd3500d1920 0x7fd3500cdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:15.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.575+0000 7fd355c60640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fd35015f770 0x7fd3500b1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:15.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.575+0000 7fd356461640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd3500d1920 0x7fd3500cdb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51886/0 (socket says 172.21.15.73:51886) 2024-04-26T20:51:15.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.575+0000 7fd356461640 1 -- 172.21.15.73:0/4121533397 learned_addr learned my addr 172.21.15.73:0/4121533397 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:15.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.575+0000 7fd356461640 1 -- 172.21.15.73:0/4121533397 >> v2:172.21.15.120:3300/0 conn(0x7fd35015f770 msgr2=0x7fd3500b1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:15.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.575+0000 7fd356461640 1 --2- 172.21.15.73:0/4121533397 >> v2:172.21.15.120:3300/0 conn(0x7fd35015f770 0x7fd3500b1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:15.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.575+0000 7fd356461640 1 -- 172.21.15.73:0/4121533397 >> v1:172.21.15.120:6789/0 conn(0x7fd3500d0d60 legacy=0x7fd3500d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:15.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.575+0000 7fd356461640 1 -- 172.21.15.73:0/4121533397 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd35006e860 con 0x7fd3500d1920 2024-04-26T20:51:15.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.576+0000 7fd356461640 1 --2- 172.21.15.73:0/4121533397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3500d1920 0x7fd3500cdb70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd350073030 tx=0x7fd34802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bebc6dcf8ec5a3d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:15.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.576+0000 7fd354c5e640 1 -- 172.21.15.73:0/4121533397 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd348036070 con 0x7fd3500d1920 2024-04-26T20:51:15.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.576+0000 7fd354c5e640 1 -- 172.21.15.73:0/4121533397 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd348031600 con 0x7fd3500d1920 2024-04-26T20:51:15.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.576+0000 7fd354c5e640 1 -- 172.21.15.73:0/4121533397 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3480319d0 con 0x7fd3500d1920 2024-04-26T20:51:15.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.576+0000 7fd357ec4640 1 -- 172.21.15.73:0/4121533397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3500d1920 msgr2=0x7fd3500cdb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:15.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.576+0000 7fd357ec4640 1 --2- 172.21.15.73:0/4121533397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3500d1920 0x7fd3500cdb70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd350073030 tx=0x7fd34802a730 comp rx=0 tx=0).stop 2024-04-26T20:51:15.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.576+0000 7fd357ec4640 1 -- 172.21.15.73:0/4121533397 shutdown_connections 2024-04-26T20:51:15.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.576+0000 7fd357ec4640 1 --2- 172.21.15.73:0/4121533397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3500d1920 0x7fd3500cdb70 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:15.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.576+0000 7fd357ec4640 1 --2- 172.21.15.73:0/4121533397 >> v2:172.21.15.120:3300/0 conn(0x7fd35015f770 0x7fd3500b1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:15.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.576+0000 7fd357ec4640 1 -- 172.21.15.73:0/4121533397 >> 172.21.15.73:0/4121533397 conn(0x7fd3500d2e60 msgr2=0x7fd3500d3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:15.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.576+0000 7fd357ec4640 1 -- 172.21.15.73:0/4121533397 shutdown_connections 2024-04-26T20:51:15.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.576+0000 7fd357ec4640 1 -- 172.21.15.73:0/4121533397 wait complete. 2024-04-26T20:51:15.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.578+0000 7fd357ec4640 1 Processor -- start 2024-04-26T20:51:15.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.578+0000 7fd357ec4640 1 -- start start 2024-04-26T20:51:15.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.578+0000 7fd357ec4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd3500d0d60 0x7fd3500b7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:15.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.578+0000 7fd357ec4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3500d1920 0x7fd35014f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:15.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.578+0000 7fd357ec4640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd35015f770 0x7fd3500cc610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:15.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.578+0000 7fd357ec4640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd3500ce230 con 0x7fd3500d1920 2024-04-26T20:51:15.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.578+0000 7fd357ec4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd3500ce0b0 con 0x7fd3500d0d60 2024-04-26T20:51:15.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.578+0000 7fd357ec4640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fd3500ce3b0 con 0x7fd35015f770 2024-04-26T20:51:15.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.578+0000 7fd35545f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3500d1920 0x7fd35014f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:15.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.578+0000 7fd35545f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3500d1920 0x7fd35014f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51898/0 (socket says 172.21.15.73:51898) 2024-04-26T20:51:15.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.578+0000 7fd35545f640 1 -- 172.21.15.73:0/2289467218 learned_addr learned my addr 172.21.15.73:0/2289467218 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:15.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.578+0000 7fd355c60640 1 --2- 172.21.15.73:0/2289467218 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd3500d0d60 0x7fd3500b7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:15.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.579+0000 7fd356461640 1 --2- 172.21.15.73:0/2289467218 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd35015f770 0x7fd3500cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:15.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.579+0000 7fd35545f640 1 -- 172.21.15.73:0/2289467218 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd35015f770 msgr2=0x7fd3500cc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:15.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.579+0000 7fd35545f640 1 --2- 172.21.15.73:0/2289467218 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd35015f770 0x7fd3500cc610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:15.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.579+0000 7fd35545f640 1 -- 172.21.15.73:0/2289467218 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd3500d0d60 msgr2=0x7fd3500b7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:15.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.579+0000 7fd35545f640 1 --2- 172.21.15.73:0/2289467218 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd3500d0d60 0x7fd3500b7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:15.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.579+0000 7fd35545f640 1 -- 172.21.15.73:0/2289467218 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3500b69d0 con 0x7fd3500d1920 2024-04-26T20:51:15.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.579+0000 7fd355c60640 1 --2- 172.21.15.73:0/2289467218 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd3500d0d60 0x7fd3500b7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:15.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.579+0000 7fd35545f640 1 --2- 172.21.15.73:0/2289467218 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3500d1920 0x7fd35014f230 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd34000c450 tx=0x7fd34000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:15.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.579+0000 7fd3467fc640 1 -- 172.21.15.73:0/2289467218 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3400052d0 con 0x7fd3500d1920 2024-04-26T20:51:15.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.579+0000 7fd3467fc640 1 -- 172.21.15.73:0/2289467218 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd340005470 con 0x7fd3500d1920 2024-04-26T20:51:15.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.579+0000 7fd357ec4640 1 -- 172.21.15.73:0/2289467218 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3500b8250 con 0x7fd3500d1920 2024-04-26T20:51:15.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.579+0000 7fd3467fc640 1 -- 172.21.15.73:0/2289467218 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd340012870 con 0x7fd3500d1920 2024-04-26T20:51:15.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.579+0000 7fd357ec4640 1 -- 172.21.15.73:0/2289467218 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3500b7230 con 0x7fd3500d1920 2024-04-26T20:51:15.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.580+0000 7fd357ec4640 1 -- 172.21.15.73:0/2289467218 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3500694f0 con 0x7fd3500d1920 2024-04-26T20:51:15.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.580+0000 7fd3467fc640 1 -- 172.21.15.73:0/2289467218 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd340012a10 con 0x7fd3500d1920 2024-04-26T20:51:15.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.581+0000 7fd3467fc640 1 --2- 172.21.15.73:0/2289467218 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd33406c8f0 0x7fd33406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:15.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.581+0000 7fd355c60640 1 --2- 172.21.15.73:0/2289467218 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd33406c8f0 0x7fd33406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:15.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.581+0000 7fd3467fc640 1 -- 172.21.15.73:0/2289467218 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fd34008bb50 con 0x7fd3500d1920 2024-04-26T20:51:15.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.581+0000 7fd355c60640 1 --2- 172.21.15.73:0/2289467218 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd33406c8f0 0x7fd33406edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd34c009af0 tx=0x7fd34c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:15.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.585+0000 7fd3467fc640 1 -- 172.21.15.73:0/2289467218 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd340059b80 con 0x7fd3500d1920 2024-04-26T20:51:15.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.763+0000 7fd357ec4640 1 -- 172.21.15.73:0/2289467218 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd35006fe50 con 0x7fd3500d1920 2024-04-26T20:51:15.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.763+0000 7fd3467fc640 1 -- 172.21.15.73:0/2289467218 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7fd340059480 con 0x7fd3500d1920 2024-04-26T20:51:15.764 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-26T20:51:15.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.765+0000 7fd357ec4640 1 -- 172.21.15.73:0/2289467218 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd33406c8f0 msgr2=0x7fd33406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:15.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.765+0000 7fd357ec4640 1 --2- 172.21.15.73:0/2289467218 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd33406c8f0 0x7fd33406edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd34c009af0 tx=0x7fd34c008040 comp rx=0 tx=0).stop 2024-04-26T20:51:15.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.765+0000 7fd357ec4640 1 -- 172.21.15.73:0/2289467218 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3500d1920 msgr2=0x7fd35014f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:15.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.765+0000 7fd357ec4640 1 --2- 172.21.15.73:0/2289467218 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3500d1920 0x7fd35014f230 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd34000c450 tx=0x7fd34000c910 comp rx=0 tx=0).stop 2024-04-26T20:51:15.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.765+0000 7fd357ec4640 1 -- 172.21.15.73:0/2289467218 shutdown_connections 2024-04-26T20:51:15.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.765+0000 7fd357ec4640 1 --2- 172.21.15.73:0/2289467218 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3500d1920 0x7fd35014f230 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:15.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.765+0000 7fd357ec4640 1 --2- 172.21.15.73:0/2289467218 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd33406c8f0 0x7fd33406edb0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:15.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.765+0000 7fd357ec4640 1 --2- 172.21.15.73:0/2289467218 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd35015f770 0x7fd3500cc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:15.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.765+0000 7fd357ec4640 1 --2- 172.21.15.73:0/2289467218 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd3500d0d60 0x7fd3500b7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:15.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.765+0000 7fd357ec4640 1 -- 172.21.15.73:0/2289467218 >> 172.21.15.73:0/2289467218 conn(0x7fd3500d2e60 msgr2=0x7fd3500d11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:15.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.765+0000 7fd357ec4640 1 -- 172.21.15.73:0/2289467218 shutdown_connections 2024-04-26T20:51:15.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.765+0000 7fd357ec4640 1 -- 172.21.15.73:0/2289467218 wait complete. 2024-04-26T20:51:15.777 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T20:51:15.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.875+0000 7fc6b102a640 1 Processor -- start 2024-04-26T20:51:15.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.875+0000 7fc6b102a640 1 -- start start 2024-04-26T20:51:15.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.876+0000 7fc6b102a640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc6ac071260 0x7fc6ac15e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:15.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.876+0000 7fc6b102a640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6ac0d2390 con 0x7fc6ac0cfde0 2024-04-26T20:51:15.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.876+0000 7fc6b102a640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6ac06d3f0 con 0x7fc6ac070350 2024-04-26T20:51:15.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.876+0000 7fc6b102a640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fc6ac0afab0 con 0x7fc6ac071260 2024-04-26T20:51:15.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.876+0000 7fc6aad9d640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fc6ac0cfde0 0x7fc6ac0d1b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40196/0 (socket says 172.21.15.73:40196) 2024-04-26T20:51:15.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.876+0000 7fc6aad9d640 1 -- 172.21.15.73:0/1870169019 learned_addr learned my addr 172.21.15.73:0/1870169019 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:15.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.876+0000 7fc6a9d9b640 1 -- 172.21.15.73:0/1870169019 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3094981277 0 0) 0x7fc6ac0d2390 con 0x7fc6ac0cfde0 2024-04-26T20:51:15.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.876+0000 7fc6a9d9b640 1 -- 172.21.15.73:0/1870169019 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc694003520 con 0x7fc6ac0cfde0 2024-04-26T20:51:15.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.876+0000 7fc6ab59e640 1 --2- 172.21.15.73:0/1870169019 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc6ac071260 0x7fc6ac15e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:15.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.877+0000 7fc6a9d9b640 1 -- 172.21.15.73:0/1870169019 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 468748954 0 0) 0x7fc694003520 con 0x7fc6ac0cfde0 2024-04-26T20:51:15.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.877+0000 7fc6a9d9b640 1 -- 172.21.15.73:0/1870169019 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc6ac071260 msgr2=0x7fc6ac15e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:15.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.877+0000 7fc6a9d9b640 1 --2- 172.21.15.73:0/1870169019 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc6ac071260 0x7fc6ac15e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:15.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.877+0000 7fc6a9d9b640 1 -- 172.21.15.73:0/1870169019 >> v1:172.21.15.120:6789/0 conn(0x7fc6ac070350 legacy=0x7fc6ac070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:15.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.877+0000 7fc6a9d9b640 1 -- 172.21.15.73:0/1870169019 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6ac06d6d0 con 0x7fc6ac0cfde0 2024-04-26T20:51:15.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.877+0000 7fc6a9d9b640 1 -- 172.21.15.73:0/1870169019 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7fc6a0001670 con 0x7fc6ac0cfde0 2024-04-26T20:51:15.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.877+0000 7fc6a9d9b640 1 -- 172.21.15.73:0/1870169019 >> v1:172.21.15.73:6789/0 conn(0x7fc6ac0cfde0 legacy=0x7fc6ac0d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:15.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.877+0000 7fc6a9d9b640 1 --2- 172.21.15.73:0/1870169019 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc6940039e0 0x7fc694003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:15.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.877+0000 7fc6a9d9b640 1 --2- 172.21.15.73:0/1870169019 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc694004800 0x7fc694004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:15.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.877+0000 7fc6a9d9b640 1 --2- 172.21.15.73:0/1870169019 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc6ac071260 0x7fc694005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:15.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.877+0000 7fc6a9d9b640 1 -- 172.21.15.73:0/1870169019 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc69400baf0 con 0x7fc694004800 2024-04-26T20:51:15.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.877+0000 7fc6a9d9b640 1 -- 172.21.15.73:0/1870169019 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc69400bca0 con 0x7fc6940039e0 2024-04-26T20:51:15.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.877+0000 7fc6a9d9b640 1 -- 172.21.15.73:0/1870169019 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fc69400bea0 con 0x7fc6ac071260 2024-04-26T20:51:15.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.877+0000 7fc6aa59c640 1 --2- 172.21.15.73:0/1870169019 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc694004800 0x7fc694004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:15.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.877+0000 7fc6aad9d640 1 --2- 172.21.15.73:0/1870169019 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc6940039e0 0x7fc694003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:15.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.877+0000 7fc6ab59e640 1 --2- 172.21.15.73:0/1870169019 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc6ac071260 0x7fc694005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:15.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.877+0000 7fc6aa59c640 1 -- 172.21.15.73:0/1870169019 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc6ac071260 msgr2=0x7fc694005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:15.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.877+0000 7fc6aa59c640 1 --2- 172.21.15.73:0/1870169019 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc6ac071260 0x7fc694005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:15.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.877+0000 7fc6aa59c640 1 -- 172.21.15.73:0/1870169019 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc6940039e0 msgr2=0x7fc694003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:15.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.877+0000 7fc6aa59c640 1 --2- 172.21.15.73:0/1870169019 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc6940039e0 0x7fc694003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:15.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.877+0000 7fc6aa59c640 1 -- 172.21.15.73:0/1870169019 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc694003520 con 0x7fc694004800 2024-04-26T20:51:15.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.877+0000 7fc6ab59e640 1 --2- 172.21.15.73:0/1870169019 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc6ac071260 0x7fc694005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:15.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.878+0000 7fc6aa59c640 1 --2- 172.21.15.73:0/1870169019 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc694004800 0x7fc694004c80 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc69c001750 tx=0x7fc694005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b83405665dcfcd44 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:15.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.878+0000 7fc6a9d9b640 1 -- 172.21.15.73:0/1870169019 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc69c001370 con 0x7fc694004800 2024-04-26T20:51:15.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.878+0000 7fc6a9d9b640 1 -- 172.21.15.73:0/1870169019 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc69c0052d0 con 0x7fc694004800 2024-04-26T20:51:15.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.878+0000 7fc6a9d9b640 1 -- 172.21.15.73:0/1870169019 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc69c02abe0 con 0x7fc694004800 2024-04-26T20:51:15.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.878+0000 7fc6b102a640 1 -- 172.21.15.73:0/1870169019 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc694004800 msgr2=0x7fc694004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:15.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.878+0000 7fc6b102a640 1 --2- 172.21.15.73:0/1870169019 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc694004800 0x7fc694004c80 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc69c001750 tx=0x7fc694005830 comp rx=0 tx=0).stop 2024-04-26T20:51:15.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.878+0000 7fc6ab59e640 1 -- 172.21.15.73:0/1870169019 reap_dead start 2024-04-26T20:51:15.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.878+0000 7fc6b102a640 1 -- 172.21.15.73:0/1870169019 shutdown_connections 2024-04-26T20:51:15.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.878+0000 7fc6b102a640 1 -- 172.21.15.73:0/1870169019 >> 172.21.15.73:0/1870169019 conn(0x7fc6ac0d2d80 msgr2=0x7fc6ac0d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:15.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.878+0000 7fc6b102a640 1 -- 172.21.15.73:0/1870169019 shutdown_connections 2024-04-26T20:51:15.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.878+0000 7fc6b102a640 1 -- 172.21.15.73:0/1870169019 wait complete. 2024-04-26T20:51:15.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.880+0000 7fc6b102a640 1 Processor -- start 2024-04-26T20:51:15.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.880+0000 7fc6b102a640 1 -- start start 2024-04-26T20:51:15.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.881+0000 7fc6b102a640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc694004800 0x7fc6ac15aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:15.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.881+0000 7fc6b102a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6ac1f5650 0x7fc6ac1f8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:15.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.881+0000 7fc6b102a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc6ac1f8680 0x7fc6ac14b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:15.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.881+0000 7fc6b102a640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ac06f1c0 con 0x7fc6ac1f5650 2024-04-26T20:51:15.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.881+0000 7fc6b102a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ac06dd20 con 0x7fc6ac1f8680 2024-04-26T20:51:15.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.881+0000 7fc6b102a640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fc6ac06f340 con 0x7fc694004800 2024-04-26T20:51:15.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.881+0000 7fc6aa59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6ac1f5650 0x7fc6ac1f8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:15.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.881+0000 7fc6aa59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6ac1f5650 0x7fc6ac1f8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51924/0 (socket says 172.21.15.73:51924) 2024-04-26T20:51:15.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.881+0000 7fc6aa59c640 1 -- 172.21.15.73:0/1919524504 learned_addr learned my addr 172.21.15.73:0/1919524504 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:15.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.881+0000 7fc6aad9d640 1 --2- 172.21.15.73:0/1919524504 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc694004800 0x7fc6ac15aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:15.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.881+0000 7fc6ab59e640 1 --2- 172.21.15.73:0/1919524504 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc6ac1f8680 0x7fc6ac14b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:15.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.881+0000 7fc6aa59c640 1 -- 172.21.15.73:0/1919524504 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc694004800 msgr2=0x7fc6ac15aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:15.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.881+0000 7fc6aa59c640 1 --2- 172.21.15.73:0/1919524504 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc694004800 0x7fc6ac15aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:15.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.881+0000 7fc6aa59c640 1 -- 172.21.15.73:0/1919524504 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc6ac1f8680 msgr2=0x7fc6ac14b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:15.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.881+0000 7fc6aa59c640 1 --2- 172.21.15.73:0/1919524504 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc6ac1f8680 0x7fc6ac14b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:15.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.881+0000 7fc6aa59c640 1 -- 172.21.15.73:0/1919524504 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6ac1f5ad0 con 0x7fc6ac1f5650 2024-04-26T20:51:15.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.881+0000 7fc6ab59e640 1 --2- 172.21.15.73:0/1919524504 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc6ac1f8680 0x7fc6ac14b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:15.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.881+0000 7fc6aa59c640 1 --2- 172.21.15.73:0/1919524504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6ac1f5650 0x7fc6ac1f8140 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc69c001870 tx=0x7fc69c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:15.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.881+0000 7fc6937fe640 1 -- 172.21.15.73:0/1919524504 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc69c040020 con 0x7fc6ac1f5650 2024-04-26T20:51:15.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.881+0000 7fc6937fe640 1 -- 172.21.15.73:0/1919524504 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc69c007680 con 0x7fc6ac1f5650 2024-04-26T20:51:15.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.881+0000 7fc6b102a640 1 -- 172.21.15.73:0/1919524504 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6ac15b140 con 0x7fc6ac1f5650 2024-04-26T20:51:15.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.881+0000 7fc6937fe640 1 -- 172.21.15.73:0/1919524504 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc69c007a30 con 0x7fc6ac1f5650 2024-04-26T20:51:15.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.882+0000 7fc6b102a640 1 -- 172.21.15.73:0/1919524504 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6ac15b700 con 0x7fc6ac1f5650 2024-04-26T20:51:15.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.883+0000 7fc6b102a640 1 -- 172.21.15.73:0/1919524504 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6ac0694f0 con 0x7fc6ac1f5650 2024-04-26T20:51:15.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.883+0000 7fc6937fe640 1 -- 172.21.15.73:0/1919524504 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc69c043020 con 0x7fc6ac1f5650 2024-04-26T20:51:15.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.883+0000 7fc6937fe640 1 --2- 172.21.15.73:0/1919524504 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc69806c8f0 0x7fc69806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:15.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.883+0000 7fc6937fe640 1 -- 172.21.15.73:0/1919524504 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fc69c0abeb0 con 0x7fc6ac1f5650 2024-04-26T20:51:15.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.883+0000 7fc6aad9d640 1 --2- 172.21.15.73:0/1919524504 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc69806c8f0 0x7fc69806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:15.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.884+0000 7fc6aad9d640 1 --2- 172.21.15.73:0/1919524504 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc69806c8f0 0x7fc69806edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0001200 tx=0x7fc6a0007040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:15.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:15.888+0000 7fc6937fe640 1 -- 172.21.15.73:0/1919524504 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc69c04a030 con 0x7fc6ac1f5650 2024-04-26T20:51:16.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.064+0000 7fc6b102a640 1 -- 172.21.15.73:0/1919524504 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fc6ac1f8a60 con 0x7fc6ac1f5650 2024-04-26T20:51:16.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.064+0000 7fc6937fe640 1 -- 172.21.15.73:0/1919524504 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7fc69c079ee0 con 0x7fc6ac1f5650 2024-04-26T20:51:16.066 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-26T20:51:16.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.066+0000 7fc6b102a640 1 -- 172.21.15.73:0/1919524504 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc69806c8f0 msgr2=0x7fc69806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:16.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.066+0000 7fc6b102a640 1 --2- 172.21.15.73:0/1919524504 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc69806c8f0 0x7fc69806edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0001200 tx=0x7fc6a0007040 comp rx=0 tx=0).stop 2024-04-26T20:51:16.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.066+0000 7fc6b102a640 1 -- 172.21.15.73:0/1919524504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6ac1f5650 msgr2=0x7fc6ac1f8140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:16.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.066+0000 7fc6b102a640 1 --2- 172.21.15.73:0/1919524504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6ac1f5650 0x7fc6ac1f8140 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc69c001870 tx=0x7fc69c001120 comp rx=0 tx=0).stop 2024-04-26T20:51:16.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.066+0000 7fc6b102a640 1 -- 172.21.15.73:0/1919524504 shutdown_connections 2024-04-26T20:51:16.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.066+0000 7fc6b102a640 1 --2- 172.21.15.73:0/1919524504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6ac1f5650 0x7fc6ac1f8140 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:16.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.066+0000 7fc6b102a640 1 --2- 172.21.15.73:0/1919524504 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc69806c8f0 0x7fc69806edb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:16.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.066+0000 7fc6b102a640 1 --2- 172.21.15.73:0/1919524504 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc6ac1f8680 0x7fc6ac14b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:16.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.066+0000 7fc6b102a640 1 --2- 172.21.15.73:0/1919524504 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc694004800 0x7fc6ac15aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:16.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.066+0000 7fc6b102a640 1 -- 172.21.15.73:0/1919524504 >> 172.21.15.73:0/1919524504 conn(0x7fc6ac0d2d80 msgr2=0x7fc6ac0c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:16.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.066+0000 7fc6b102a640 1 -- 172.21.15.73:0/1919524504 shutdown_connections 2024-04-26T20:51:16.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.066+0000 7fc6b102a640 1 -- 172.21.15.73:0/1919524504 wait complete. 2024-04-26T20:51:16.079 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:51:16.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.175+0000 7fbce55f0640 1 Processor -- start 2024-04-26T20:51:16.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.175+0000 7fbce55f0640 1 -- start start 2024-04-26T20:51:16.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.176+0000 7fbce55f0640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbce00d06f0 0x7fbce00d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:16.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.176+0000 7fbce55f0640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbce00b2850 0x7fbce00b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:16.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.176+0000 7fbce55f0640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbce00d0b70 con 0x7fbce015f3a0 2024-04-26T20:51:16.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.176+0000 7fbce55f0640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7fbce00d0e20 con 0x7fbce00b2850 2024-04-26T20:51:16.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.176+0000 7fbce55f0640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fbce00b1d70 con 0x7fbce00d06f0 2024-04-26T20:51:16.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.176+0000 7fbcdf7fe640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbce00b2850 0x7fbce00b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:16.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.176+0000 7fbcdf7fe640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbce00b2850 0x7fbce00b2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51932/0 (socket says 172.21.15.73:51932) 2024-04-26T20:51:16.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.176+0000 7fbcdf7fe640 1 -- 172.21.15.73:0/673303464 learned_addr learned my addr 172.21.15.73:0/673303464 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:16.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.176+0000 7fbcdf7fe640 1 -- 172.21.15.73:0/673303464 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbce00d06f0 msgr2=0x7fbce00d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:16.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.176+0000 7fbcdf7fe640 1 --2- 172.21.15.73:0/673303464 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbce00d06f0 0x7fbce00d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:16.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.176+0000 7fbcdf7fe640 1 -- 172.21.15.73:0/673303464 >> v1:172.21.15.120:6789/0 conn(0x7fbce015f3a0 legacy=0x7fbce00d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:16.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.176+0000 7fbcdf7fe640 1 -- 172.21.15.73:0/673303464 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbce00b2050 con 0x7fbce00b2850 2024-04-26T20:51:16.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.177+0000 7fbcdf7fe640 1 --2- 172.21.15.73:0/673303464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbce00b2850 0x7fbce00b2cc0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbcd8009e50 tx=0x7fbcd802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a039c4a5e07b68f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:16.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.177+0000 7fbcddffb640 1 -- 172.21.15.73:0/673303464 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd8036070 con 0x7fbce00b2850 2024-04-26T20:51:16.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.177+0000 7fbcddffb640 1 -- 172.21.15.73:0/673303464 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcd8031600 con 0x7fbce00b2850 2024-04-26T20:51:16.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.177+0000 7fbcddffb640 1 -- 172.21.15.73:0/673303464 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd80319d0 con 0x7fbce00b2850 2024-04-26T20:51:16.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.177+0000 7fbce55f0640 1 -- 172.21.15.73:0/673303464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbce00b2850 msgr2=0x7fbce00b2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:16.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.177+0000 7fbce55f0640 1 --2- 172.21.15.73:0/673303464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbce00b2850 0x7fbce00b2cc0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbcd8009e50 tx=0x7fbcd802a6c0 comp rx=0 tx=0).stop 2024-04-26T20:51:16.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.177+0000 7fbce55f0640 1 -- 172.21.15.73:0/673303464 shutdown_connections 2024-04-26T20:51:16.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.177+0000 7fbce55f0640 1 --2- 172.21.15.73:0/673303464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbce00b2850 0x7fbce00b2cc0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:16.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.177+0000 7fbce55f0640 1 --2- 172.21.15.73:0/673303464 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbce00d06f0 0x7fbce00d1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:16.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.177+0000 7fbce55f0640 1 -- 172.21.15.73:0/673303464 >> 172.21.15.73:0/673303464 conn(0x7fbce00d4f90 msgr2=0x7fbce00d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:16.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.177+0000 7fbce55f0640 1 -- 172.21.15.73:0/673303464 shutdown_connections 2024-04-26T20:51:16.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.177+0000 7fbce55f0640 1 -- 172.21.15.73:0/673303464 wait complete. 2024-04-26T20:51:16.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.179+0000 7fbce55f0640 1 Processor -- start 2024-04-26T20:51:16.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.179+0000 7fbce55f0640 1 -- start start 2024-04-26T20:51:16.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.179+0000 7fbce55f0640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbce00b2850 0x7fbce01553a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:16.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.179+0000 7fbce55f0640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbce00d06f0 0x7fbce00cb200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:16.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.179+0000 7fbce55f0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbce015f3a0 0x7fbce00cde00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:16.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.179+0000 7fbce55f0640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbce015f900 con 0x7fbce00d06f0 2024-04-26T20:51:16.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.179+0000 7fbce55f0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbce015f780 con 0x7fbce015f3a0 2024-04-26T20:51:16.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.179+0000 7fbce55f0640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fbce00d23c0 con 0x7fbce00b2850 2024-04-26T20:51:16.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.179+0000 7fbcde7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbce00d06f0 0x7fbce00cb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:16.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.179+0000 7fbcde7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbce00d06f0 0x7fbce00cb200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51938/0 (socket says 172.21.15.73:51938) 2024-04-26T20:51:16.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.179+0000 7fbcde7fc640 1 -- 172.21.15.73:0/2351809493 learned_addr learned my addr 172.21.15.73:0/2351809493 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:16.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.180+0000 7fbcdf7fe640 1 --2- 172.21.15.73:0/2351809493 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbce015f3a0 0x7fbce00cde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:16.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.180+0000 7fbcdeffd640 1 --2- 172.21.15.73:0/2351809493 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbce00b2850 0x7fbce01553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:16.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.180+0000 7fbcde7fc640 1 -- 172.21.15.73:0/2351809493 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbce00b2850 msgr2=0x7fbce01553a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:16.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.180+0000 7fbcde7fc640 1 --2- 172.21.15.73:0/2351809493 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbce00b2850 0x7fbce01553a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:16.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.180+0000 7fbcde7fc640 1 -- 172.21.15.73:0/2351809493 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbce015f3a0 msgr2=0x7fbce00cde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:16.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.180+0000 7fbcde7fc640 1 --2- 172.21.15.73:0/2351809493 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbce015f3a0 0x7fbce00cde00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:16.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.180+0000 7fbcde7fc640 1 -- 172.21.15.73:0/2351809493 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbce00cd3b0 con 0x7fbce00d06f0 2024-04-26T20:51:16.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.180+0000 7fbcdeffd640 1 --2- 172.21.15.73:0/2351809493 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbce00b2850 0x7fbce01553a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:16.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.180+0000 7fbcde7fc640 1 --2- 172.21.15.73:0/2351809493 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbce00d06f0 0x7fbce00cb200 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbcd400b440 tx=0x7fbcd400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:16.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.180+0000 7fbcc77fe640 1 -- 172.21.15.73:0/2351809493 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd4015020 con 0x7fbce00d06f0 2024-04-26T20:51:16.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.180+0000 7fbcc77fe640 1 -- 172.21.15.73:0/2351809493 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcd4011890 con 0x7fbce00d06f0 2024-04-26T20:51:16.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.180+0000 7fbce55f0640 1 -- 172.21.15.73:0/2351809493 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbce00c97b0 con 0x7fbce00d06f0 2024-04-26T20:51:16.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.180+0000 7fbcc77fe640 1 -- 172.21.15.73:0/2351809493 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd4011c80 con 0x7fbce00d06f0 2024-04-26T20:51:16.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.180+0000 7fbce55f0640 1 -- 172.21.15.73:0/2351809493 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbce00c9d70 con 0x7fbce00d06f0 2024-04-26T20:51:16.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.181+0000 7fbce55f0640 1 -- 172.21.15.73:0/2351809493 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbce00694f0 con 0x7fbce00d06f0 2024-04-26T20:51:16.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.182+0000 7fbcc77fe640 1 -- 172.21.15.73:0/2351809493 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbcd4018020 con 0x7fbce00d06f0 2024-04-26T20:51:16.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.182+0000 7fbcc77fe640 1 --2- 172.21.15.73:0/2351809493 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fbccc06c8f0 0x7fbccc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:16.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.182+0000 7fbcc77fe640 1 -- 172.21.15.73:0/2351809493 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fbcd408bf90 con 0x7fbce00d06f0 2024-04-26T20:51:16.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.182+0000 7fbcdeffd640 1 --2- 172.21.15.73:0/2351809493 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fbccc06c8f0 0x7fbccc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:16.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.182+0000 7fbcdeffd640 1 --2- 172.21.15.73:0/2351809493 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fbccc06c8f0 0x7fbccc06edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0003eb0 tx=0x7fbcd0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:16.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.186+0000 7fbcc77fe640 1 -- 172.21.15.73:0/2351809493 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbcd401f020 con 0x7fbce00d06f0 2024-04-26T20:51:16.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.364+0000 7fbce55f0640 1 -- 172.21.15.73:0/2351809493 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbce008e760 con 0x7fbce00d06f0 2024-04-26T20:51:16.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.365+0000 7fbcc77fe640 1 -- 172.21.15.73:0/2351809493 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18902 (secure 0 0 0) 0x7fbcd400d040 con 0x7fbce00d06f0 2024-04-26T20:51:16.366 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:51:16.366 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":27,"fsid":"fbd3f548-b547-4713-856b-09f98f5d1764","created":"2024-04-26T20:50:07.876786+0000","modified":"2024-04-26T20:51:13.783133+0000","last_up_change":"2024-04-26T20:50:19.082741+0000","last_in_change":"2024-04-26T20:50:10.018400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:50:19.993731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:50:29.964420+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:50:30.920082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:50:34.650207+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"72d6f4f3-214a-4392-a02f-c33cbcab3c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6809","nonce":3461705124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6811","nonce":3461705124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6815","nonce":3461705124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6813","nonce":3461705124}]},"public_addr":"172.21.15.73:6809/3461705124","cluster_addr":"172.21.15.73:6811/3461705124","heartbeat_back_addr":"172.21.15.73:6815/3461705124","heartbeat_front_addr":"172.21.15.73:6813/3461705124","state":["exists","up"]},{"osd":1,"uuid":"786bf8ca-6ddc-4b3c-be6c-9152398bd5fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6817","nonce":2827619403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6819","nonce":2827619403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6825","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6827","nonce":2827619403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6821","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6823","nonce":2827619403}]},"public_addr":"172.21.15.73:6817/2827619403","cluster_addr":"172.21.15.73:6819/2827619403","heartbeat_back_addr":"172.21.15.73:6827/2827619403","heartbeat_front_addr":"172.21.15.73:6823/2827619403","state":["exists","up"]},{"osd":2,"uuid":"90963b1a-5437-4ef8-ad8b-6b74c534eeb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6801","nonce":2542681064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6803","nonce":2542681064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6808","nonce":2542681064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6805","nonce":2542681064}]},"public_addr":"172.21.15.73:6801/2542681064","cluster_addr":"172.21.15.73:6803/2542681064","heartbeat_back_addr":"172.21.15.73:6808/2542681064","heartbeat_front_addr":"172.21.15.73:6805/2542681064","state":["exists","up"]},{"osd":3,"uuid":"bd3d2148-6e38-41b4-a453-fe6e3b8b4ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6822","nonce":1604042718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6826","nonce":1604042718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6831","nonce":1604042718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6829","nonce":1604042718}]},"public_addr":"172.21.15.73:6822/1604042718","cluster_addr":"172.21.15.73:6826/1604042718","heartbeat_back_addr":"172.21.15.73:6831/1604042718","heartbeat_front_addr":"172.21.15.73:6829/1604042718","state":["exists","up"]},{"osd":4,"uuid":"e8668dc8-a23f-46d2-8311-cb882da08e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6817","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6819","nonce":3591055561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6824","nonce":3591055561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6831","nonce":3591055561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6828","nonce":3591055561}]},"public_addr":"172.21.15.120:6819/3591055561","cluster_addr":"172.21.15.120:6824/3591055561","heartbeat_back_addr":"172.21.15.120:6831/3591055561","heartbeat_front_addr":"172.21.15.120:6828/3591055561","state":["exists","up"]},{"osd":5,"uuid":"cbac0bdb-96bd-4852-bcce-e564495c3fcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6809","nonce":819450433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6811","nonce":819450433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6815","nonce":819450433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6813","nonce":819450433}]},"public_addr":"172.21.15.120:6809/819450433","cluster_addr":"172.21.15.120:6811/819450433","heartbeat_back_addr":"172.21.15.120:6815/819450433","heartbeat_front_addr":"172.21.15.120:6813/819450433","state":["exists","up"]},{"osd":6,"uuid":"1884b2a8-569f-4272-96a9-f25a09364dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6801","nonce":3946210727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6803","nonce":3946210727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6807","nonce":3946210727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6805","nonce":3946210727}]},"public_addr":"172.21.15.120:6801/3946210727","cluster_addr":"172.21.15.120:6803/3946210727","heartbeat_back_addr":"172.21.15.120:6807/3946210727","heartbeat_front_addr":"172.21.15.120:6805/3946210727","state":["exists","up"]},{"osd":7,"uuid":"c0c3bf6d-a4c5-4f1e-aa2d-330951830666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6818","nonce":2915574674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6821","nonce":2915574674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6827","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6829","nonce":2915574674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6823","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6825","nonce":2915574674}]},"public_addr":"172.21.15.120:6818/2915574674","cluster_addr":"172.21.15.120:6821/2915574674","heartbeat_back_addr":"172.21.15.120:6829/2915574674","heartbeat_front_addr":"172.21.15.120:6825/2915574674","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.218216+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.208710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.112705+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.182789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.164324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.174212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.225396+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.261559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:51:16.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.367+0000 7fbce55f0640 1 -- 172.21.15.73:0/2351809493 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fbccc06c8f0 msgr2=0x7fbccc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:16.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.367+0000 7fbce55f0640 1 --2- 172.21.15.73:0/2351809493 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fbccc06c8f0 0x7fbccc06edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0003eb0 tx=0x7fbcd0009040 comp rx=0 tx=0).stop 2024-04-26T20:51:16.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.367+0000 7fbce55f0640 1 -- 172.21.15.73:0/2351809493 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbce00d06f0 msgr2=0x7fbce00cb200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:16.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.367+0000 7fbce55f0640 1 --2- 172.21.15.73:0/2351809493 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbce00d06f0 0x7fbce00cb200 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbcd400b440 tx=0x7fbcd400b900 comp rx=0 tx=0).stop 2024-04-26T20:51:16.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.367+0000 7fbce55f0640 1 -- 172.21.15.73:0/2351809493 shutdown_connections 2024-04-26T20:51:16.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.367+0000 7fbce55f0640 1 --2- 172.21.15.73:0/2351809493 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbce00d06f0 0x7fbce00cb200 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:16.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.367+0000 7fbce55f0640 1 --2- 172.21.15.73:0/2351809493 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fbccc06c8f0 0x7fbccc06edb0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:16.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.367+0000 7fbce55f0640 1 --2- 172.21.15.73:0/2351809493 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbce015f3a0 0x7fbce00cde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:16.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.367+0000 7fbce55f0640 1 --2- 172.21.15.73:0/2351809493 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbce00b2850 0x7fbce01553a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:16.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.367+0000 7fbce55f0640 1 -- 172.21.15.73:0/2351809493 >> 172.21.15.73:0/2351809493 conn(0x7fbce00d4f90 msgr2=0x7fbce006d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:16.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.367+0000 7fbce55f0640 1 -- 172.21.15.73:0/2351809493 shutdown_connections 2024-04-26T20:51:16.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.367+0000 7fbce55f0640 1 -- 172.21.15.73:0/2351809493 wait complete. 2024-04-26T20:51:16.379 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:50:19.993731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:50:29.964420+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:50:30.920082+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T20:50:34.650207+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T20:51:16.380 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:51:16.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.479+0000 7f1394293640 1 Processor -- start 2024-04-26T20:51:16.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.479+0000 7f1394293640 1 -- start start 2024-04-26T20:51:16.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.480+0000 7f1394293640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f138c070350 0x7f138c070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:16.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.480+0000 7f1394293640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f138c071090 0x7f138c15e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:16.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.480+0000 7f1394293640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f138c0d2390 con 0x7f138c0cfde0 2024-04-26T20:51:16.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.480+0000 7f1394293640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f138c071500 con 0x7f138c071090 2024-04-26T20:51:16.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.480+0000 7f1394293640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f138c0afab0 con 0x7f138c070350 2024-04-26T20:51:16.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.480+0000 7f139182e640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f138c070350 0x7f138c070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:16.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.480+0000 7f1392830640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f138c071090 0x7f138c15e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:16.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.480+0000 7f139182e640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f138c070350 0x7f138c070b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.73:36034/0 (socket says 172.21.15.73:36034) 2024-04-26T20:51:16.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.480+0000 7f139182e640 1 -- 172.21.15.73:0/597559409 learned_addr learned my addr 172.21.15.73:0/597559409 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:16.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.480+0000 7f139202f640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f138c0cfde0 0x7f138c0d1b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.73:45772/0 (socket says 172.21.15.73:45772) 2024-04-26T20:51:16.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.480+0000 7f139182e640 1 -- 172.21.15.73:0/597559409 >> v2:172.21.15.120:3300/0 conn(0x7f138c071090 msgr2=0x7f138c15e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:16.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.480+0000 7f139182e640 1 --2- 172.21.15.73:0/597559409 >> v2:172.21.15.120:3300/0 conn(0x7f138c071090 0x7f138c15e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:16.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.480+0000 7f139182e640 1 -- 172.21.15.73:0/597559409 >> v1:172.21.15.120:6789/0 conn(0x7f138c0cfde0 legacy=0x7f138c0d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:16.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.480+0000 7f139182e640 1 -- 172.21.15.73:0/597559409 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f138c06d3f0 con 0x7f138c070350 2024-04-26T20:51:16.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.481+0000 7f139182e640 1 --2- 172.21.15.73:0/597559409 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f138c070350 0x7f138c070b50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f137c009870 tx=0x7f137c02a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b824e0d299b77e6c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:16.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.481+0000 7f139102d640 1 -- 172.21.15.73:0/597559409 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137c037070 con 0x7f138c070350 2024-04-26T20:51:16.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.481+0000 7f139102d640 1 -- 172.21.15.73:0/597559409 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f137c032440 con 0x7f138c070350 2024-04-26T20:51:16.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.481+0000 7f139102d640 1 -- 172.21.15.73:0/597559409 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137c032810 con 0x7f138c070350 2024-04-26T20:51:16.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.482+0000 7f1394293640 1 -- 172.21.15.73:0/597559409 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f138c070350 msgr2=0x7f138c070b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:16.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.482+0000 7f1394293640 1 --2- 172.21.15.73:0/597559409 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f138c070350 0x7f138c070b50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f137c009870 tx=0x7f137c02a1c0 comp rx=0 tx=0).stop 2024-04-26T20:51:16.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.482+0000 7f1394293640 1 -- 172.21.15.73:0/597559409 shutdown_connections 2024-04-26T20:51:16.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.482+0000 7f1394293640 1 --2- 172.21.15.73:0/597559409 >> v2:172.21.15.120:3300/0 conn(0x7f138c071090 0x7f138c15e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:16.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.482+0000 7f1394293640 1 --2- 172.21.15.73:0/597559409 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f138c070350 0x7f138c070b50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:16.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.482+0000 7f1394293640 1 -- 172.21.15.73:0/597559409 >> 172.21.15.73:0/597559409 conn(0x7f138c0d2d80 msgr2=0x7f138c0d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:16.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.482+0000 7f1394293640 1 -- 172.21.15.73:0/597559409 shutdown_connections 2024-04-26T20:51:16.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.482+0000 7f1394293640 1 -- 172.21.15.73:0/597559409 wait complete. 2024-04-26T20:51:16.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.483+0000 7f1394293640 1 Processor -- start 2024-04-26T20:51:16.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.483+0000 7f1394293640 1 -- start start 2024-04-26T20:51:16.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.484+0000 7f1394293640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f138c070350 0x7f138c14a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:16.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.484+0000 7f1394293640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f138c071090 0x7f138c160d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:16.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.484+0000 7f1394293640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f138c0cfde0 0x7f138c1f8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:16.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.484+0000 7f1394293640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f138c06d9b0 con 0x7f138c071090 2024-04-26T20:51:16.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.484+0000 7f1394293640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f138c06dd20 con 0x7f138c070350 2024-04-26T20:51:16.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.484+0000 7f1394293640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f138c15ef30 con 0x7f138c0cfde0 2024-04-26T20:51:16.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.484+0000 7f139182e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f138c071090 0x7f138c160d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:16.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.484+0000 7f139182e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f138c071090 0x7f138c160d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51950/0 (socket says 172.21.15.73:51950) 2024-04-26T20:51:16.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.484+0000 7f139182e640 1 -- 172.21.15.73:0/2174355799 learned_addr learned my addr 172.21.15.73:0/2174355799 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:16.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.484+0000 7f139202f640 1 --2- 172.21.15.73:0/2174355799 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f138c070350 0x7f138c14a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:16.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.484+0000 7f1392830640 1 --2- 172.21.15.73:0/2174355799 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f138c0cfde0 0x7f138c1f8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:16.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.484+0000 7f139182e640 1 -- 172.21.15.73:0/2174355799 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f138c0cfde0 msgr2=0x7f138c1f8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:16.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.484+0000 7f139182e640 1 --2- 172.21.15.73:0/2174355799 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f138c0cfde0 0x7f138c1f8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:16.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.484+0000 7f139182e640 1 -- 172.21.15.73:0/2174355799 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f138c070350 msgr2=0x7f138c14a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:16.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.484+0000 7f139182e640 1 --2- 172.21.15.73:0/2174355799 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f138c070350 0x7f138c14a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:16.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.484+0000 7f139182e640 1 -- 172.21.15.73:0/2174355799 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f138c1f8cd0 con 0x7f138c071090 2024-04-26T20:51:16.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.484+0000 7f139182e640 1 --2- 172.21.15.73:0/2174355799 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f138c071090 0x7f138c160d70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f137c02a200 tx=0x7f137c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:16.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.485+0000 7f1382ffd640 1 -- 172.21.15.73:0/2174355799 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137c037040 con 0x7f138c071090 2024-04-26T20:51:16.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.485+0000 7f1382ffd640 1 -- 172.21.15.73:0/2174355799 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f137c042070 con 0x7f138c071090 2024-04-26T20:51:16.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.485+0000 7f1394293640 1 -- 172.21.15.73:0/2174355799 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f138c1f8fb0 con 0x7f138c071090 2024-04-26T20:51:16.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.485+0000 7f1382ffd640 1 -- 172.21.15.73:0/2174355799 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137c030860 con 0x7f138c071090 2024-04-26T20:51:16.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.485+0000 7f1394293640 1 -- 172.21.15.73:0/2174355799 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f138c160850 con 0x7f138c071090 2024-04-26T20:51:16.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.486+0000 7f1394293640 1 -- 172.21.15.73:0/2174355799 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f138c160360 con 0x7f138c071090 2024-04-26T20:51:16.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.486+0000 7f1382ffd640 1 -- 172.21.15.73:0/2174355799 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f137c02ad10 con 0x7f138c071090 2024-04-26T20:51:16.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.486+0000 7f1382ffd640 1 --2- 172.21.15.73:0/2174355799 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f137006c8f0 0x7f137006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:16.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.486+0000 7f1382ffd640 1 -- 172.21.15.73:0/2174355799 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f137c0ac5a0 con 0x7f138c071090 2024-04-26T20:51:16.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.486+0000 7f139202f640 1 --2- 172.21.15.73:0/2174355799 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f137006c8f0 0x7f137006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:16.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.487+0000 7f139202f640 1 --2- 172.21.15.73:0/2174355799 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f137006c8f0 0x7f137006edb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1388000f60 tx=0x7f1388007ce0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:16.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.491+0000 7f1382ffd640 1 -- 172.21.15.73:0/2174355799 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f137c07a550 con 0x7f138c071090 2024-04-26T20:51:16.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.668+0000 7f1394293640 1 -- 172.21.15.73:0/2174355799 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f138c0694f0 con 0x7f138c071090 2024-04-26T20:51:16.669 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.668+0000 7f1382ffd640 1 -- 172.21.15.73:0/2174355799 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f137c079e50 con 0x7f138c071090 2024-04-26T20:51:16.669 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-26T20:51:16.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.670+0000 7f1394293640 1 -- 172.21.15.73:0/2174355799 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f137006c8f0 msgr2=0x7f137006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:16.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.670+0000 7f1394293640 1 --2- 172.21.15.73:0/2174355799 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f137006c8f0 0x7f137006edb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1388000f60 tx=0x7f1388007ce0 comp rx=0 tx=0).stop 2024-04-26T20:51:16.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.670+0000 7f1394293640 1 -- 172.21.15.73:0/2174355799 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f138c071090 msgr2=0x7f138c160d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:16.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.670+0000 7f1394293640 1 --2- 172.21.15.73:0/2174355799 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f138c071090 0x7f138c160d70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f137c02a200 tx=0x7f137c0046c0 comp rx=0 tx=0).stop 2024-04-26T20:51:16.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.670+0000 7f1394293640 1 -- 172.21.15.73:0/2174355799 shutdown_connections 2024-04-26T20:51:16.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.670+0000 7f1394293640 1 --2- 172.21.15.73:0/2174355799 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f138c071090 0x7f138c160d70 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:16.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.670+0000 7f1394293640 1 --2- 172.21.15.73:0/2174355799 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f137006c8f0 0x7f137006edb0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:16.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.670+0000 7f1394293640 1 --2- 172.21.15.73:0/2174355799 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f138c0cfde0 0x7f138c1f8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:16.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.670+0000 7f1394293640 1 --2- 172.21.15.73:0/2174355799 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f138c070350 0x7f138c14a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:16.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.670+0000 7f1394293640 1 -- 172.21.15.73:0/2174355799 >> 172.21.15.73:0/2174355799 conn(0x7f138c0d2d80 msgr2=0x7f138c0afc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:16.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.670+0000 7f1394293640 1 -- 172.21.15.73:0/2174355799 shutdown_connections 2024-04-26T20:51:16.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.670+0000 7f1394293640 1 -- 172.21.15.73:0/2174355799 wait complete. 2024-04-26T20:51:16.683 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:51:16.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.781+0000 7fa757db4640 1 Processor -- start 2024-04-26T20:51:16.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.781+0000 7fa757db4640 1 -- start start 2024-04-26T20:51:16.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.782+0000 7fa757db4640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa75015f2d0 0x7fa7501604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:16.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.782+0000 7fa757db4640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fa750072700 0x7fa75006f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:16.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.782+0000 7fa757db4640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7500700e0 con 0x7fa75006fcd0 2024-04-26T20:51:16.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.782+0000 7fa757db4640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fa750074010 con 0x7fa750072700 2024-04-26T20:51:16.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.782+0000 7fa757db4640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fa750072f70 con 0x7fa75015f2d0 2024-04-26T20:51:16.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.782+0000 7fa756351640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fa75006fcd0 0x7fa7500cbb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40206/0 (socket says 172.21.15.73:40206) 2024-04-26T20:51:16.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.782+0000 7fa756351640 1 -- 172.21.15.73:0/1401170808 learned_addr learned my addr 172.21.15.73:0/1401170808 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:16.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.782+0000 7fa755b50640 1 --2- 172.21.15.73:0/1401170808 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa75015f2d0 0x7fa7501604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:16.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.782+0000 7fa75534f640 1 --2- 172.21.15.73:0/1401170808 >> v2:172.21.15.120:3300/0 conn(0x7fa750072700 0x7fa75006f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:16.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.782+0000 7fa754b4e640 1 -- 172.21.15.73:0/1401170808 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2890278469 0 0) 0x7fa748001490 con 0x7fa75006fcd0 2024-04-26T20:51:16.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.782+0000 7fa754b4e640 1 -- 172.21.15.73:0/1401170808 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa73c003520 con 0x7fa75006fcd0 2024-04-26T20:51:16.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.782+0000 7fa755b50640 1 -- 172.21.15.73:0/1401170808 >> v2:172.21.15.120:3300/0 conn(0x7fa750072700 msgr2=0x7fa75006f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:16.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.782+0000 7fa755b50640 1 --2- 172.21.15.73:0/1401170808 >> v2:172.21.15.120:3300/0 conn(0x7fa750072700 0x7fa75006f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:16.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.782+0000 7fa755b50640 1 -- 172.21.15.73:0/1401170808 >> v1:172.21.15.73:6789/0 conn(0x7fa75006fcd0 legacy=0x7fa7500cbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:16.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.782+0000 7fa755b50640 1 -- 172.21.15.73:0/1401170808 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa750073250 con 0x7fa75015f2d0 2024-04-26T20:51:16.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.783+0000 7fa755b50640 1 --2- 172.21.15.73:0/1401170808 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa75015f2d0 0x7fa7501604e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa74c0097a0 tx=0x7fa74c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bcca0f19f836ea8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:16.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.783+0000 7fa754b4e640 1 -- 172.21.15.73:0/1401170808 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74c037070 con 0x7fa75015f2d0 2024-04-26T20:51:16.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.783+0000 7fa754b4e640 1 -- 172.21.15.73:0/1401170808 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa74c032440 con 0x7fa75015f2d0 2024-04-26T20:51:16.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.783+0000 7fa754b4e640 1 -- 172.21.15.73:0/1401170808 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74c0327f0 con 0x7fa75015f2d0 2024-04-26T20:51:16.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.784+0000 7fa757db4640 1 -- 172.21.15.73:0/1401170808 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa75015f2d0 msgr2=0x7fa7501604e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:16.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.784+0000 7fa757db4640 1 --2- 172.21.15.73:0/1401170808 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa75015f2d0 0x7fa7501604e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa74c0097a0 tx=0x7fa74c02a180 comp rx=0 tx=0).stop 2024-04-26T20:51:16.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.784+0000 7fa757db4640 1 -- 172.21.15.73:0/1401170808 shutdown_connections 2024-04-26T20:51:16.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.784+0000 7fa757db4640 1 --2- 172.21.15.73:0/1401170808 >> v2:172.21.15.120:3300/0 conn(0x7fa750072700 0x7fa75006f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:16.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.784+0000 7fa757db4640 1 --2- 172.21.15.73:0/1401170808 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa75015f2d0 0x7fa7501604e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:16.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.784+0000 7fa757db4640 1 -- 172.21.15.73:0/1401170808 >> 172.21.15.73:0/1401170808 conn(0x7fa7500d0be0 msgr2=0x7fa7500d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:16.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.784+0000 7fa757db4640 1 -- 172.21.15.73:0/1401170808 shutdown_connections 2024-04-26T20:51:16.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.784+0000 7fa757db4640 1 -- 172.21.15.73:0/1401170808 wait complete. 2024-04-26T20:51:16.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.785+0000 7fa757db4640 1 Processor -- start 2024-04-26T20:51:16.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.785+0000 7fa757db4640 1 -- start start 2024-04-26T20:51:16.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.786+0000 7fa757db4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa75006fcd0 0x7fa7500c86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:16.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.786+0000 7fa757db4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa750072700 0x7fa7501f3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:16.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.786+0000 7fa757db4640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa75015f2d0 0x7fa7501e8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:16.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.786+0000 7fa757db4640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa750073af0 con 0x7fa75006fcd0 2024-04-26T20:51:16.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.786+0000 7fa757db4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa750073810 con 0x7fa750072700 2024-04-26T20:51:16.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.786+0000 7fa757db4640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fa75015f6b0 con 0x7fa75015f2d0 2024-04-26T20:51:16.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.786+0000 7fa755b50640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa75006fcd0 0x7fa7500c86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:16.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.786+0000 7fa755b50640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa75006fcd0 0x7fa7500c86c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51954/0 (socket says 172.21.15.73:51954) 2024-04-26T20:51:16.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.786+0000 7fa755b50640 1 -- 172.21.15.73:0/2830178153 learned_addr learned my addr 172.21.15.73:0/2830178153 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:16.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.786+0000 7fa756351640 1 --2- 172.21.15.73:0/2830178153 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa75015f2d0 0x7fa7501e8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:16.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.786+0000 7fa75534f640 1 --2- 172.21.15.73:0/2830178153 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa750072700 0x7fa7501f3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:16.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.786+0000 7fa755b50640 1 -- 172.21.15.73:0/2830178153 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa75015f2d0 msgr2=0x7fa7501e8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:16.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.786+0000 7fa755b50640 1 --2- 172.21.15.73:0/2830178153 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa75015f2d0 0x7fa7501e8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:16.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.786+0000 7fa755b50640 1 -- 172.21.15.73:0/2830178153 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa750072700 msgr2=0x7fa7501f3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:16.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.786+0000 7fa755b50640 1 --2- 172.21.15.73:0/2830178153 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa750072700 0x7fa7501f3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:16.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.786+0000 7fa755b50640 1 -- 172.21.15.73:0/2830178153 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7500c9270 con 0x7fa75006fcd0 2024-04-26T20:51:16.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.786+0000 7fa756351640 1 --2- 172.21.15.73:0/2830178153 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa75015f2d0 0x7fa7501e8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:16.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.786+0000 7fa75534f640 1 --2- 172.21.15.73:0/2830178153 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa750072700 0x7fa7501f3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:16.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.786+0000 7fa755b50640 1 --2- 172.21.15.73:0/2830178153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa75006fcd0 0x7fa7500c86c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa74c035040 tx=0x7fa74c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:16.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.787+0000 7fa7467fc640 1 -- 172.21.15.73:0/2830178153 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74c037040 con 0x7fa75006fcd0 2024-04-26T20:51:16.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.787+0000 7fa7467fc640 1 -- 172.21.15.73:0/2830178153 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa74c0044c0 con 0x7fa75006fcd0 2024-04-26T20:51:16.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.787+0000 7fa757db4640 1 -- 172.21.15.73:0/2830178153 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7500c9550 con 0x7fa75006fcd0 2024-04-26T20:51:16.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.787+0000 7fa7467fc640 1 -- 172.21.15.73:0/2830178153 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74c0325f0 con 0x7fa75006fcd0 2024-04-26T20:51:16.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.787+0000 7fa757db4640 1 -- 172.21.15.73:0/2830178153 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7500c9b10 con 0x7fa75006fcd0 2024-04-26T20:51:16.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.788+0000 7fa757db4640 1 -- 172.21.15.73:0/2830178153 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa750072b60 con 0x7fa75006fcd0 2024-04-26T20:51:16.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.788+0000 7fa7467fc640 1 -- 172.21.15.73:0/2830178153 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa74c044020 con 0x7fa75006fcd0 2024-04-26T20:51:16.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.788+0000 7fa7467fc640 1 --2- 172.21.15.73:0/2830178153 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa73406c8f0 0x7fa73406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:16.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.788+0000 7fa7467fc640 1 -- 172.21.15.73:0/2830178153 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fa74c0ac120 con 0x7fa75006fcd0 2024-04-26T20:51:16.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.789+0000 7fa75534f640 1 --2- 172.21.15.73:0/2830178153 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa73406c8f0 0x7fa73406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:16.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.789+0000 7fa75534f640 1 --2- 172.21.15.73:0/2830178153 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa73406c8f0 0x7fa73406edb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa740009ac0 tx=0x7fa740008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:16.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.793+0000 7fa7467fc640 1 -- 172.21.15.73:0/2830178153 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa750072b60 con 0x7fa75006fcd0 2024-04-26T20:51:16.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.971+0000 7fa757db4640 1 -- 172.21.15.73:0/2830178153 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa750155b80 con 0x7fa75006fcd0 2024-04-26T20:51:16.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.972+0000 7fa7467fc640 1 -- 172.21.15.73:0/2830178153 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7fa74c07a1e0 con 0x7fa75006fcd0 2024-04-26T20:51:16.973 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-26T20:51:16.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.974+0000 7fa757db4640 1 -- 172.21.15.73:0/2830178153 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa73406c8f0 msgr2=0x7fa73406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:16.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.974+0000 7fa757db4640 1 --2- 172.21.15.73:0/2830178153 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa73406c8f0 0x7fa73406edb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa740009ac0 tx=0x7fa740008040 comp rx=0 tx=0).stop 2024-04-26T20:51:16.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.974+0000 7fa757db4640 1 -- 172.21.15.73:0/2830178153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa75006fcd0 msgr2=0x7fa7500c86c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:16.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.974+0000 7fa757db4640 1 --2- 172.21.15.73:0/2830178153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa75006fcd0 0x7fa7500c86c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa74c035040 tx=0x7fa74c0046c0 comp rx=0 tx=0).stop 2024-04-26T20:51:16.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.974+0000 7fa757db4640 1 -- 172.21.15.73:0/2830178153 shutdown_connections 2024-04-26T20:51:16.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.974+0000 7fa757db4640 1 --2- 172.21.15.73:0/2830178153 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa73406c8f0 0x7fa73406edb0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:16.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.974+0000 7fa757db4640 1 --2- 172.21.15.73:0/2830178153 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa75015f2d0 0x7fa7501e8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:16.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.974+0000 7fa757db4640 1 --2- 172.21.15.73:0/2830178153 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa750072700 0x7fa7501f3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:16.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.974+0000 7fa757db4640 1 --2- 172.21.15.73:0/2830178153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa75006fcd0 0x7fa7500c86c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:16.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.974+0000 7fa757db4640 1 -- 172.21.15.73:0/2830178153 >> 172.21.15.73:0/2830178153 conn(0x7fa7500d0be0 msgr2=0x7fa7500cdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:16.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.974+0000 7fa757db4640 1 -- 172.21.15.73:0/2830178153 shutdown_connections 2024-04-26T20:51:16.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:16.974+0000 7fa757db4640 1 -- 172.21.15.73:0/2830178153 wait complete. 2024-04-26T20:51:16.986 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:51:17.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.084+0000 7f2a9bb71640 1 Processor -- start 2024-04-26T20:51:17.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.084+0000 7f2a9bb71640 1 -- start start 2024-04-26T20:51:17.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.085+0000 7f2a9bb71640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2a94070350 0x7f2a94070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:17.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.085+0000 7f2a9bb71640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f2a94071090 0x7f2a9415e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:17.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.085+0000 7f2a9bb71640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a940d2390 con 0x7f2a940cfde0 2024-04-26T20:51:17.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.085+0000 7f2a9bb71640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f2a940afab0 con 0x7f2a94070350 2024-04-26T20:51:17.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.085+0000 7f2a9bb71640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f2a9406dd20 con 0x7f2a94071090 2024-04-26T20:51:17.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.085+0000 7f2a9910c640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2a94070350 0x7f2a94070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:17.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.085+0000 7f2a9910c640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2a94070350 0x7f2a94070b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51960/0 (socket says 172.21.15.73:51960) 2024-04-26T20:51:17.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.085+0000 7f2a9910c640 1 -- 172.21.15.73:0/2422706707 learned_addr learned my addr 172.21.15.73:0/2422706707 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:17.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.085+0000 7f2a9990d640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f2a940cfde0 0x7f2a940d1b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40218/0 (socket says 172.21.15.73:40218) 2024-04-26T20:51:17.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.085+0000 7f2a9a10e640 1 --2- 172.21.15.73:0/2422706707 >> v2:172.21.15.120:3300/0 conn(0x7f2a94071090 0x7f2a9415e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:17.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.085+0000 7f2a9910c640 1 -- 172.21.15.73:0/2422706707 >> v2:172.21.15.120:3300/0 conn(0x7f2a94071090 msgr2=0x7f2a9415e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:17.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.085+0000 7f2a9910c640 1 --2- 172.21.15.73:0/2422706707 >> v2:172.21.15.120:3300/0 conn(0x7f2a94071090 0x7f2a9415e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:17.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.085+0000 7f2a9910c640 1 -- 172.21.15.73:0/2422706707 >> v1:172.21.15.73:6789/0 conn(0x7f2a940cfde0 legacy=0x7f2a940d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:17.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.085+0000 7f2a9910c640 1 -- 172.21.15.73:0/2422706707 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a9406d3f0 con 0x7f2a94070350 2024-04-26T20:51:17.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.085+0000 7f2a9890b640 1 -- 172.21.15.73:0/2422706707 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3197825133 0 0) 0x7f2a940d2390 con 0x7f2a940cfde0 2024-04-26T20:51:17.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.086+0000 7f2a9910c640 1 --2- 172.21.15.73:0/2422706707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a94070350 0x7f2a94070b50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2a940c6860 tx=0x7f2a8402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78057c508034e420 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:17.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.086+0000 7f2a9890b640 1 -- 172.21.15.73:0/2422706707 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a84037070 con 0x7f2a94070350 2024-04-26T20:51:17.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.086+0000 7f2a9890b640 1 -- 172.21.15.73:0/2422706707 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a84032440 con 0x7f2a94070350 2024-04-26T20:51:17.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.086+0000 7f2a9890b640 1 -- 172.21.15.73:0/2422706707 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a84032810 con 0x7f2a94070350 2024-04-26T20:51:17.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.086+0000 7f2a9bb71640 1 -- 172.21.15.73:0/2422706707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a94070350 msgr2=0x7f2a94070b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:17.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.086+0000 7f2a9bb71640 1 --2- 172.21.15.73:0/2422706707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a94070350 0x7f2a94070b50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2a940c6860 tx=0x7f2a8402a1e0 comp rx=0 tx=0).stop 2024-04-26T20:51:17.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.086+0000 7f2a9bb71640 1 -- 172.21.15.73:0/2422706707 shutdown_connections 2024-04-26T20:51:17.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.086+0000 7f2a9bb71640 1 --2- 172.21.15.73:0/2422706707 >> v2:172.21.15.120:3300/0 conn(0x7f2a94071090 0x7f2a9415e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:17.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.086+0000 7f2a9bb71640 1 --2- 172.21.15.73:0/2422706707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a94070350 0x7f2a94070b50 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:17.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.086+0000 7f2a9bb71640 1 -- 172.21.15.73:0/2422706707 >> 172.21.15.73:0/2422706707 conn(0x7f2a940d2d80 msgr2=0x7f2a940d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:17.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.087+0000 7f2a9bb71640 1 -- 172.21.15.73:0/2422706707 shutdown_connections 2024-04-26T20:51:17.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.087+0000 7f2a9bb71640 1 -- 172.21.15.73:0/2422706707 wait complete. 2024-04-26T20:51:17.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.088+0000 7f2a9bb71640 1 Processor -- start 2024-04-26T20:51:17.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.088+0000 7f2a9bb71640 1 -- start start 2024-04-26T20:51:17.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.088+0000 7f2a9bb71640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2a94070350 0x7f2a9414ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:17.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.088+0000 7f2a9bb71640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a94071090 0x7f2a9414b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:17.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.088+0000 7f2a9bb71640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a940cfde0 0x7f2a940a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:17.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.088+0000 7f2a9bb71640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9406d9b0 con 0x7f2a940cfde0 2024-04-26T20:51:17.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.088+0000 7f2a9bb71640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2a940714c0 con 0x7f2a94071090 2024-04-26T20:51:17.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.088+0000 7f2a9bb71640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f2a9415ef30 con 0x7f2a94070350 2024-04-26T20:51:17.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.088+0000 7f2a9a10e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a940cfde0 0x7f2a940a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:17.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.089+0000 7f2a9a10e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a940cfde0 0x7f2a940a01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51974/0 (socket says 172.21.15.73:51974) 2024-04-26T20:51:17.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.089+0000 7f2a9a10e640 1 -- 172.21.15.73:0/3275787115 learned_addr learned my addr 172.21.15.73:0/3275787115 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:17.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.089+0000 7f2a9990d640 1 --2- 172.21.15.73:0/3275787115 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2a94070350 0x7f2a9414ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:17.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.089+0000 7f2a9910c640 1 --2- 172.21.15.73:0/3275787115 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a94071090 0x7f2a9414b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:17.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.089+0000 7f2a9a10e640 1 -- 172.21.15.73:0/3275787115 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2a94070350 msgr2=0x7f2a9414ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:17.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.089+0000 7f2a9a10e640 1 --2- 172.21.15.73:0/3275787115 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2a94070350 0x7f2a9414ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:17.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.089+0000 7f2a9a10e640 1 -- 172.21.15.73:0/3275787115 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a94071090 msgr2=0x7f2a9414b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:17.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.089+0000 7f2a9a10e640 1 --2- 172.21.15.73:0/3275787115 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a94071090 0x7f2a9414b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:17.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.089+0000 7f2a9a10e640 1 -- 172.21.15.73:0/3275787115 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a9414f140 con 0x7f2a940cfde0 2024-04-26T20:51:17.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.089+0000 7f2a9a10e640 1 --2- 172.21.15.73:0/3275787115 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a940cfde0 0x7f2a940a01a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c007e40 tx=0x7f2a8c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:17.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.089+0000 7f2a8a7fc640 1 -- 172.21.15.73:0/3275787115 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c014620 con 0x7f2a940cfde0 2024-04-26T20:51:17.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.089+0000 7f2a8a7fc640 1 -- 172.21.15.73:0/3275787115 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a8c019070 con 0x7f2a940cfde0 2024-04-26T20:51:17.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.089+0000 7f2a8a7fc640 1 -- 172.21.15.73:0/3275787115 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c014a20 con 0x7f2a940cfde0 2024-04-26T20:51:17.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.089+0000 7f2a9bb71640 1 -- 172.21.15.73:0/3275787115 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a9409f100 con 0x7f2a940cfde0 2024-04-26T20:51:17.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.089+0000 7f2a9bb71640 1 -- 172.21.15.73:0/3275787115 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a9409f6c0 con 0x7f2a940cfde0 2024-04-26T20:51:17.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.090+0000 7f2a9bb71640 1 -- 172.21.15.73:0/3275787115 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a940694f0 con 0x7f2a940cfde0 2024-04-26T20:51:17.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.091+0000 7f2a8a7fc640 1 -- 172.21.15.73:0/3275787115 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2a8c014bc0 con 0x7f2a940cfde0 2024-04-26T20:51:17.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.091+0000 7f2a8a7fc640 1 --2- 172.21.15.73:0/3275787115 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f2a7806c8f0 0x7f2a7806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:17.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.091+0000 7f2a8a7fc640 1 -- 172.21.15.73:0/3275787115 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f2a8c08c7a0 con 0x7f2a940cfde0 2024-04-26T20:51:17.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.091+0000 7f2a9990d640 1 --2- 172.21.15.73:0/3275787115 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f2a7806c8f0 0x7f2a7806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:17.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.091+0000 7f2a9990d640 1 --2- 172.21.15.73:0/3275787115 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f2a7806c8f0 0x7f2a7806edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2a90000f60 tx=0x7f2a90007c00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:17.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.095+0000 7f2a8a7fc640 1 -- 172.21.15.73:0/3275787115 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2a9414f140 con 0x7f2a940cfde0 2024-04-26T20:51:17.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.271+0000 7f2a9bb71640 1 -- 172.21.15.73:0/3275787115 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2a940d0890 con 0x7f2a940cfde0 2024-04-26T20:51:17.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.271+0000 7f2a8a7fc640 1 -- 172.21.15.73:0/3275787115 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f2a8c05a860 con 0x7f2a940cfde0 2024-04-26T20:51:17.273 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-26T20:51:17.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.273+0000 7f2a9bb71640 1 -- 172.21.15.73:0/3275787115 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f2a7806c8f0 msgr2=0x7f2a7806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:17.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.273+0000 7f2a9bb71640 1 --2- 172.21.15.73:0/3275787115 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f2a7806c8f0 0x7f2a7806edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2a90000f60 tx=0x7f2a90007c00 comp rx=0 tx=0).stop 2024-04-26T20:51:17.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.273+0000 7f2a9bb71640 1 -- 172.21.15.73:0/3275787115 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a940cfde0 msgr2=0x7f2a940a01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:17.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.273+0000 7f2a9bb71640 1 --2- 172.21.15.73:0/3275787115 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a940cfde0 0x7f2a940a01a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c007e40 tx=0x7f2a8c00e4b0 comp rx=0 tx=0).stop 2024-04-26T20:51:17.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.273+0000 7f2a9bb71640 1 -- 172.21.15.73:0/3275787115 shutdown_connections 2024-04-26T20:51:17.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.273+0000 7f2a9bb71640 1 --2- 172.21.15.73:0/3275787115 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a940cfde0 0x7f2a940a01a0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:17.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.273+0000 7f2a9bb71640 1 --2- 172.21.15.73:0/3275787115 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f2a7806c8f0 0x7f2a7806edb0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:17.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.273+0000 7f2a9bb71640 1 --2- 172.21.15.73:0/3275787115 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a94071090 0x7f2a9414b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:17.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.273+0000 7f2a9bb71640 1 --2- 172.21.15.73:0/3275787115 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2a94070350 0x7f2a9414ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:17.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.273+0000 7f2a9bb71640 1 -- 172.21.15.73:0/3275787115 >> 172.21.15.73:0/3275787115 conn(0x7f2a940d2d80 msgr2=0x7f2a9406e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:17.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.273+0000 7f2a9bb71640 1 -- 172.21.15.73:0/3275787115 shutdown_connections 2024-04-26T20:51:17.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.273+0000 7f2a9bb71640 1 -- 172.21.15.73:0/3275787115 wait complete. 2024-04-26T20:51:17.285 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T20:51:17.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.384+0000 7f965bc47640 1 Processor -- start 2024-04-26T20:51:17.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.384+0000 7f965bc47640 1 -- start start 2024-04-26T20:51:17.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.385+0000 7f965bc47640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f965415f170 0x7f96540d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:17.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.385+0000 7f965bc47640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f96540d3bc0 0x7f96540d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:17.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.385+0000 7f965bc47640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96540d1040 con 0x7f965415f550 2024-04-26T20:51:17.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.385+0000 7f965bc47640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f96540c9430 con 0x7f96540d3bc0 2024-04-26T20:51:17.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.385+0000 7f965bc47640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f96540c9680 con 0x7f965415f170 2024-04-26T20:51:17.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.385+0000 7f965a1e4640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f96540d3bc0 0x7f96540d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:17.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.385+0000 7f96591e2640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f965415f550 0x7f96540d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40234/0 (socket says 172.21.15.73:40234) 2024-04-26T20:51:17.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.385+0000 7f96591e2640 1 -- 172.21.15.73:0/4160595525 learned_addr learned my addr 172.21.15.73:0/4160595525 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:17.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.385+0000 7f965a1e4640 1 -- 172.21.15.73:0/4160595525 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f965415f170 msgr2=0x7f96540d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:17.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.385+0000 7f96599e3640 1 --2- 172.21.15.73:0/4160595525 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f965415f170 0x7f96540d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:17.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.386+0000 7f965a1e4640 1 --2- 172.21.15.73:0/4160595525 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f965415f170 0x7f96540d5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:17.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.386+0000 7f965a1e4640 1 -- 172.21.15.73:0/4160595525 >> v1:172.21.15.73:6789/0 conn(0x7f965415f550 legacy=0x7f96540d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:17.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.386+0000 7f965a1e4640 1 -- 172.21.15.73:0/4160595525 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96540c9960 con 0x7f96540d3bc0 2024-04-26T20:51:17.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.386+0000 7f96599e3640 1 --2- 172.21.15.73:0/4160595525 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f965415f170 0x7f96540d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:17.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.386+0000 7f965a1e4640 1 --2- 172.21.15.73:0/4160595525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96540d3bc0 0x7f96540d2cb0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f964c0097a0 tx=0x7f964c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71275d7e5a3a7e61 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:17.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.386+0000 7f96589e1640 1 -- 172.21.15.73:0/4160595525 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964c037070 con 0x7f96540d3bc0 2024-04-26T20:51:17.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.386+0000 7f96589e1640 1 -- 172.21.15.73:0/4160595525 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f964c032440 con 0x7f96540d3bc0 2024-04-26T20:51:17.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.386+0000 7f96589e1640 1 -- 172.21.15.73:0/4160595525 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964c032810 con 0x7f96540d3bc0 2024-04-26T20:51:17.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.387+0000 7f965bc47640 1 -- 172.21.15.73:0/4160595525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96540d3bc0 msgr2=0x7f96540d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:17.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.387+0000 7f965bc47640 1 --2- 172.21.15.73:0/4160595525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96540d3bc0 0x7f96540d2cb0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f964c0097a0 tx=0x7f964c02a180 comp rx=0 tx=0).stop 2024-04-26T20:51:17.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.387+0000 7f965bc47640 1 -- 172.21.15.73:0/4160595525 shutdown_connections 2024-04-26T20:51:17.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.387+0000 7f965bc47640 1 --2- 172.21.15.73:0/4160595525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96540d3bc0 0x7f96540d2cb0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:17.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.387+0000 7f965bc47640 1 --2- 172.21.15.73:0/4160595525 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f965415f170 0x7f96540d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:17.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.387+0000 7f965bc47640 1 -- 172.21.15.73:0/4160595525 >> 172.21.15.73:0/4160595525 conn(0x7f96540d4f60 msgr2=0x7f96540d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:17.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.387+0000 7f965bc47640 1 -- 172.21.15.73:0/4160595525 shutdown_connections 2024-04-26T20:51:17.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.387+0000 7f965bc47640 1 -- 172.21.15.73:0/4160595525 wait complete. 2024-04-26T20:51:17.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.388+0000 7f965bc47640 1 Processor -- start 2024-04-26T20:51:17.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.388+0000 7f965bc47640 1 -- start start 2024-04-26T20:51:17.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.389+0000 7f965bc47640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96540d3bc0 0x7f96540a0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:17.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.389+0000 7f965bc47640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f965415f170 0x7f96540a27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:17.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.389+0000 7f965bc47640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96540a1ce0 0x7f9654154460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:17.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.389+0000 7f965bc47640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f96540ca5c0 con 0x7f96540d3bc0 2024-04-26T20:51:17.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.389+0000 7f965bc47640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f96540ca440 con 0x7f96540a1ce0 2024-04-26T20:51:17.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.389+0000 7f965bc47640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f96540ca740 con 0x7f965415f170 2024-04-26T20:51:17.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.389+0000 7f96599e3640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96540d3bc0 0x7f96540a0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:17.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.389+0000 7f96599e3640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96540d3bc0 0x7f96540a0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51994/0 (socket says 172.21.15.73:51994) 2024-04-26T20:51:17.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.389+0000 7f96599e3640 1 -- 172.21.15.73:0/1248860411 learned_addr learned my addr 172.21.15.73:0/1248860411 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:17.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.389+0000 7f96599e3640 1 -- 172.21.15.73:0/1248860411 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f965415f170 msgr2=0x7f96540a27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:17.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.389+0000 7f96599e3640 1 --2- 172.21.15.73:0/1248860411 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f965415f170 0x7f96540a27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:17.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.389+0000 7f96599e3640 1 -- 172.21.15.73:0/1248860411 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96540a1ce0 msgr2=0x7f9654154460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:17.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.389+0000 7f965a1e4640 1 --2- 172.21.15.73:0/1248860411 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96540a1ce0 0x7f9654154460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:17.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.389+0000 7f96599e3640 1 --2- 172.21.15.73:0/1248860411 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96540a1ce0 0x7f9654154460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:17.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.389+0000 7f96599e3640 1 -- 172.21.15.73:0/1248860411 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96540a0c40 con 0x7f96540d3bc0 2024-04-26T20:51:17.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.389+0000 7f965a1e4640 1 --2- 172.21.15.73:0/1248860411 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96540a1ce0 0x7f9654154460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:17.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.389+0000 7f96599e3640 1 --2- 172.21.15.73:0/1248860411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96540d3bc0 0x7f96540a0530 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f965000cf90 tx=0x7f965000a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:17.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.390+0000 7f964a7fc640 1 -- 172.21.15.73:0/1248860411 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965001a070 con 0x7f96540d3bc0 2024-04-26T20:51:17.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.390+0000 7f964a7fc640 1 -- 172.21.15.73:0/1248860411 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f965000d840 con 0x7f96540d3bc0 2024-04-26T20:51:17.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.390+0000 7f964a7fc640 1 -- 172.21.15.73:0/1248860411 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f965000dc10 con 0x7f96540d3bc0 2024-04-26T20:51:17.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.390+0000 7f965bc47640 1 -- 172.21.15.73:0/1248860411 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96540a0f20 con 0x7f96540d3bc0 2024-04-26T20:51:17.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.390+0000 7f965bc47640 1 -- 172.21.15.73:0/1248860411 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96540a14e0 con 0x7f96540d3bc0 2024-04-26T20:51:17.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.391+0000 7f965bc47640 1 -- 172.21.15.73:0/1248860411 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96540694f0 con 0x7f96540d3bc0 2024-04-26T20:51:17.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.391+0000 7f964a7fc640 1 -- 172.21.15.73:0/1248860411 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f965001d070 con 0x7f96540d3bc0 2024-04-26T20:51:17.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.391+0000 7f964a7fc640 1 --2- 172.21.15.73:0/1248860411 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f963806c920 0x7f963806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:17.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.391+0000 7f964a7fc640 1 -- 172.21.15.73:0/1248860411 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f965008c720 con 0x7f96540d3bc0 2024-04-26T20:51:17.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.391+0000 7f96591e2640 1 --2- 172.21.15.73:0/1248860411 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f963806c920 0x7f963806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:17.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.392+0000 7f96591e2640 1 --2- 172.21.15.73:0/1248860411 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f963806c920 0x7f963806ede0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9644000f60 tx=0x7f9644007a80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:17.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.396+0000 7f964a7fc640 1 -- 172.21.15.73:0/1248860411 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f965005a100 con 0x7f96540d3bc0 2024-04-26T20:51:17.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.570+0000 7f965bc47640 1 -- 172.21.15.73:0/1248860411 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f96540d2d40 con 0x7f96540d3bc0 2024-04-26T20:51:17.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.570+0000 7f964a7fc640 1 -- 172.21.15.73:0/1248860411 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f965005e400 con 0x7f96540d3bc0 2024-04-26T20:51:17.571 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-26T20:51:17.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.572+0000 7f965bc47640 1 -- 172.21.15.73:0/1248860411 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f963806c920 msgr2=0x7f963806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:17.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.572+0000 7f965bc47640 1 --2- 172.21.15.73:0/1248860411 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f963806c920 0x7f963806ede0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9644000f60 tx=0x7f9644007a80 comp rx=0 tx=0).stop 2024-04-26T20:51:17.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.572+0000 7f965bc47640 1 -- 172.21.15.73:0/1248860411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96540d3bc0 msgr2=0x7f96540a0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:17.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.572+0000 7f965bc47640 1 --2- 172.21.15.73:0/1248860411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96540d3bc0 0x7f96540a0530 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f965000cf90 tx=0x7f965000a7d0 comp rx=0 tx=0).stop 2024-04-26T20:51:17.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.572+0000 7f965bc47640 1 -- 172.21.15.73:0/1248860411 shutdown_connections 2024-04-26T20:51:17.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.572+0000 7f965bc47640 1 --2- 172.21.15.73:0/1248860411 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f963806c920 0x7f963806ede0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:17.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.572+0000 7f965bc47640 1 --2- 172.21.15.73:0/1248860411 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96540a1ce0 0x7f9654154460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:17.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.572+0000 7f965bc47640 1 --2- 172.21.15.73:0/1248860411 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f965415f170 0x7f96540a27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:17.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.572+0000 7f965bc47640 1 --2- 172.21.15.73:0/1248860411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96540d3bc0 0x7f96540a0530 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:17.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.572+0000 7f965bc47640 1 -- 172.21.15.73:0/1248860411 >> 172.21.15.73:0/1248860411 conn(0x7f96540d4f60 msgr2=0x7f96540cdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:17.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.572+0000 7f965bc47640 1 -- 172.21.15.73:0/1248860411 shutdown_connections 2024-04-26T20:51:17.574 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.572+0000 7f965bc47640 1 -- 172.21.15.73:0/1248860411 wait complete. 2024-04-26T20:51:17.584 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:51:17.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.681+0000 7f878d247640 1 Processor -- start 2024-04-26T20:51:17.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.681+0000 7f878d247640 1 -- start start 2024-04-26T20:51:17.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.682+0000 7f878d247640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f87880b2340 0x7f87880cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:17.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.682+0000 7f878d247640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f87880cd930 0x7f87880d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:17.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.682+0000 7f878d247640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87880d10d0 con 0x7f8788162810 2024-04-26T20:51:17.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.682+0000 7f878d247640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f87880d0750 con 0x7f87880cd930 2024-04-26T20:51:17.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.682+0000 7f878d247640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f87880d16a0 con 0x7f87880b2340 2024-04-26T20:51:17.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.682+0000 7f878759e640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f87880cd930 0x7f87880d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:17.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.682+0000 7f878659c640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f87880b2340 0x7f87880cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:17.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.682+0000 7f8786d9d640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f8788162810 0x7f8788070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.73:45776/0 (socket says 172.21.15.73:45776) 2024-04-26T20:51:17.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.682+0000 7f8786d9d640 1 -- 172.21.15.73:0/3015478011 learned_addr learned my addr 172.21.15.73:0/3015478011 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:17.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.682+0000 7f878659c640 1 -- 172.21.15.73:0/3015478011 >> v2:172.21.15.120:3300/0 conn(0x7f87880cd930 msgr2=0x7f87880d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:17.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.682+0000 7f878659c640 1 --2- 172.21.15.73:0/3015478011 >> v2:172.21.15.120:3300/0 conn(0x7f87880cd930 0x7f87880d1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:17.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.682+0000 7f878659c640 1 -- 172.21.15.73:0/3015478011 >> v1:172.21.15.120:6789/0 conn(0x7f8788162810 legacy=0x7f8788070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:17.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.682+0000 7f878659c640 1 -- 172.21.15.73:0/3015478011 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87880d1370 con 0x7f87880b2340 2024-04-26T20:51:17.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.682+0000 7f878759e640 1 --2- 172.21.15.73:0/3015478011 >> v2:172.21.15.120:3300/0 conn(0x7f87880cd930 0x7f87880d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:51:17.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.683+0000 7f878659c640 1 --2- 172.21.15.73:0/3015478011 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f87880b2340 0x7f87880cd3f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f877c0097a0 tx=0x7f877c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9ba1f95edfab5a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:17.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.683+0000 7f8785d9b640 1 -- 172.21.15.73:0/3015478011 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877c037070 con 0x7f87880b2340 2024-04-26T20:51:17.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.683+0000 7f8785d9b640 1 -- 172.21.15.73:0/3015478011 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f877c032440 con 0x7f87880b2340 2024-04-26T20:51:17.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.683+0000 7f8785d9b640 1 -- 172.21.15.73:0/3015478011 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877c032810 con 0x7f87880b2340 2024-04-26T20:51:17.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.684+0000 7f878d247640 1 -- 172.21.15.73:0/3015478011 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f87880b2340 msgr2=0x7f87880cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:17.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.684+0000 7f878d247640 1 --2- 172.21.15.73:0/3015478011 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f87880b2340 0x7f87880cd3f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f877c0097a0 tx=0x7f877c02a180 comp rx=0 tx=0).stop 2024-04-26T20:51:17.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.684+0000 7f878d247640 1 -- 172.21.15.73:0/3015478011 shutdown_connections 2024-04-26T20:51:17.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.684+0000 7f878d247640 1 --2- 172.21.15.73:0/3015478011 >> v2:172.21.15.120:3300/0 conn(0x7f87880cd930 0x7f87880d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:17.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.684+0000 7f878d247640 1 --2- 172.21.15.73:0/3015478011 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f87880b2340 0x7f87880cd3f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:17.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.684+0000 7f878d247640 1 -- 172.21.15.73:0/3015478011 >> 172.21.15.73:0/3015478011 conn(0x7f87880d2da0 msgr2=0x7f87880d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:17.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.684+0000 7f878d247640 1 -- 172.21.15.73:0/3015478011 shutdown_connections 2024-04-26T20:51:17.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.684+0000 7f878d247640 1 -- 172.21.15.73:0/3015478011 wait complete. 2024-04-26T20:51:17.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.685+0000 7f878d247640 1 Processor -- start 2024-04-26T20:51:17.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.686+0000 7f878d247640 1 -- start start 2024-04-26T20:51:17.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.686+0000 7f878d247640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f87880b2340 0x7f8788150f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:17.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.686+0000 7f878d247640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87880cd930 0x7f87880c7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:17.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.686+0000 7f878d247640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8788162810 0x7f878815f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:17.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.686+0000 7f878d247640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f87880d2070 con 0x7f8788162810 2024-04-26T20:51:17.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.686+0000 7f878d247640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f87880d0470 con 0x7f87880cd930 2024-04-26T20:51:17.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.686+0000 7f878d247640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f87880d21f0 con 0x7f87880b2340 2024-04-26T20:51:17.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.686+0000 7f878759e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8788162810 0x7f878815f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:17.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.686+0000 7f878759e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8788162810 0x7f878815f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52000/0 (socket says 172.21.15.73:52000) 2024-04-26T20:51:17.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.686+0000 7f878759e640 1 -- 172.21.15.73:0/61955713 learned_addr learned my addr 172.21.15.73:0/61955713 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:17.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.687+0000 7f878659c640 1 --2- 172.21.15.73:0/61955713 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87880cd930 0x7f87880c7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:17.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.687+0000 7f8786d9d640 1 --2- 172.21.15.73:0/61955713 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f87880b2340 0x7f8788150f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:17.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.687+0000 7f878759e640 1 -- 172.21.15.73:0/61955713 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f87880b2340 msgr2=0x7f8788150f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:17.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.687+0000 7f878759e640 1 --2- 172.21.15.73:0/61955713 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f87880b2340 0x7f8788150f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:17.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.687+0000 7f878759e640 1 -- 172.21.15.73:0/61955713 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87880cd930 msgr2=0x7f87880c7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:17.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.687+0000 7f878759e640 1 --2- 172.21.15.73:0/61955713 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87880cd930 0x7f87880c7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:17.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.687+0000 7f878759e640 1 -- 172.21.15.73:0/61955713 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87881f9270 con 0x7f8788162810 2024-04-26T20:51:17.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.687+0000 7f878659c640 1 --2- 172.21.15.73:0/61955713 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87880cd930 0x7f87880c7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:17.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.687+0000 7f878759e640 1 --2- 172.21.15.73:0/61955713 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8788162810 0x7f878815f860 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8780009ea0 tx=0x7f8780009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:17.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.687+0000 7f876f7fe640 1 -- 172.21.15.73:0/61955713 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8780018030 con 0x7f8788162810 2024-04-26T20:51:17.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.687+0000 7f876f7fe640 1 -- 172.21.15.73:0/61955713 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f878000cde0 con 0x7f8788162810 2024-04-26T20:51:17.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.687+0000 7f878d247640 1 -- 172.21.15.73:0/61955713 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f878815eaa0 con 0x7f8788162810 2024-04-26T20:51:17.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.687+0000 7f876f7fe640 1 -- 172.21.15.73:0/61955713 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8780003bb0 con 0x7f8788162810 2024-04-26T20:51:17.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.687+0000 7f878d247640 1 -- 172.21.15.73:0/61955713 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f878815f060 con 0x7f8788162810 2024-04-26T20:51:17.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.688+0000 7f878d247640 1 -- 172.21.15.73:0/61955713 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87880694f0 con 0x7f8788162810 2024-04-26T20:51:17.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.689+0000 7f876f7fe640 1 -- 172.21.15.73:0/61955713 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f878001b070 con 0x7f8788162810 2024-04-26T20:51:17.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.689+0000 7f876f7fe640 1 --2- 172.21.15.73:0/61955713 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f877406c8f0 0x7f877406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:17.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.689+0000 7f8786d9d640 1 --2- 172.21.15.73:0/61955713 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f877406c8f0 0x7f877406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:17.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.689+0000 7f876f7fe640 1 -- 172.21.15.73:0/61955713 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f878008c9b0 con 0x7f8788162810 2024-04-26T20:51:17.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.689+0000 7f8786d9d640 1 --2- 172.21.15.73:0/61955713 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f877406c8f0 0x7f877406edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8778000f60 tx=0x7f8778009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:17.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.694+0000 7f876f7fe640 1 -- 172.21.15.73:0/61955713 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8780010040 con 0x7f8788162810 2024-04-26T20:51:17.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.871+0000 7f878d247640 1 -- 172.21.15.73:0/61955713 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f878806e490 con 0x7f8788162810 2024-04-26T20:51:17.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.872+0000 7f876f7fe640 1 -- 172.21.15.73:0/61955713 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18902 (secure 0 0 0) 0x7f878005a9e0 con 0x7f8788162810 2024-04-26T20:51:17.873 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:51:17.873 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":27,"fsid":"fbd3f548-b547-4713-856b-09f98f5d1764","created":"2024-04-26T20:50:07.876786+0000","modified":"2024-04-26T20:51:13.783133+0000","last_up_change":"2024-04-26T20:50:19.082741+0000","last_in_change":"2024-04-26T20:50:10.018400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:50:19.993731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:50:29.964420+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:50:30.920082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:50:34.650207+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"72d6f4f3-214a-4392-a02f-c33cbcab3c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6809","nonce":3461705124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6811","nonce":3461705124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6815","nonce":3461705124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6813","nonce":3461705124}]},"public_addr":"172.21.15.73:6809/3461705124","cluster_addr":"172.21.15.73:6811/3461705124","heartbeat_back_addr":"172.21.15.73:6815/3461705124","heartbeat_front_addr":"172.21.15.73:6813/3461705124","state":["exists","up"]},{"osd":1,"uuid":"786bf8ca-6ddc-4b3c-be6c-9152398bd5fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6817","nonce":2827619403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6819","nonce":2827619403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6825","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6827","nonce":2827619403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6821","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6823","nonce":2827619403}]},"public_addr":"172.21.15.73:6817/2827619403","cluster_addr":"172.21.15.73:6819/2827619403","heartbeat_back_addr":"172.21.15.73:6827/2827619403","heartbeat_front_addr":"172.21.15.73:6823/2827619403","state":["exists","up"]},{"osd":2,"uuid":"90963b1a-5437-4ef8-ad8b-6b74c534eeb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6801","nonce":2542681064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6803","nonce":2542681064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6808","nonce":2542681064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6805","nonce":2542681064}]},"public_addr":"172.21.15.73:6801/2542681064","cluster_addr":"172.21.15.73:6803/2542681064","heartbeat_back_addr":"172.21.15.73:6808/2542681064","heartbeat_front_addr":"172.21.15.73:6805/2542681064","state":["exists","up"]},{"osd":3,"uuid":"bd3d2148-6e38-41b4-a453-fe6e3b8b4ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6822","nonce":1604042718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6826","nonce":1604042718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6831","nonce":1604042718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6829","nonce":1604042718}]},"public_addr":"172.21.15.73:6822/1604042718","cluster_addr":"172.21.15.73:6826/1604042718","heartbeat_back_addr":"172.21.15.73:6831/1604042718","heartbeat_front_addr":"172.21.15.73:6829/1604042718","state":["exists","up"]},{"osd":4,"uuid":"e8668dc8-a23f-46d2-8311-cb882da08e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6817","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6819","nonce":3591055561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6824","nonce":3591055561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6831","nonce":3591055561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6828","nonce":3591055561}]},"public_addr":"172.21.15.120:6819/3591055561","cluster_addr":"172.21.15.120:6824/3591055561","heartbeat_back_addr":"172.21.15.120:6831/3591055561","heartbeat_front_addr":"172.21.15.120:6828/3591055561","state":["exists","up"]},{"osd":5,"uuid":"cbac0bdb-96bd-4852-bcce-e564495c3fcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6809","nonce":819450433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6811","nonce":819450433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6815","nonce":819450433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6813","nonce":819450433}]},"public_addr":"172.21.15.120:6809/819450433","cluster_addr":"172.21.15.120:6811/819450433","heartbeat_back_addr":"172.21.15.120:6815/819450433","heartbeat_front_addr":"172.21.15.120:6813/819450433","state":["exists","up"]},{"osd":6,"uuid":"1884b2a8-569f-4272-96a9-f25a09364dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6801","nonce":3946210727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6803","nonce":3946210727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6807","nonce":3946210727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6805","nonce":3946210727}]},"public_addr":"172.21.15.120:6801/3946210727","cluster_addr":"172.21.15.120:6803/3946210727","heartbeat_back_addr":"172.21.15.120:6807/3946210727","heartbeat_front_addr":"172.21.15.120:6805/3946210727","state":["exists","up"]},{"osd":7,"uuid":"c0c3bf6d-a4c5-4f1e-aa2d-330951830666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6818","nonce":2915574674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6821","nonce":2915574674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6827","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6829","nonce":2915574674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6823","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6825","nonce":2915574674}]},"public_addr":"172.21.15.120:6818/2915574674","cluster_addr":"172.21.15.120:6821/2915574674","heartbeat_back_addr":"172.21.15.120:6829/2915574674","heartbeat_front_addr":"172.21.15.120:6825/2915574674","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.218216+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.208710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.112705+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.182789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.164324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.174212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.225396+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.261559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:51:17.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.874+0000 7f878d247640 1 -- 172.21.15.73:0/61955713 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f877406c8f0 msgr2=0x7f877406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:17.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.874+0000 7f878d247640 1 --2- 172.21.15.73:0/61955713 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f877406c8f0 0x7f877406edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8778000f60 tx=0x7f8778009040 comp rx=0 tx=0).stop 2024-04-26T20:51:17.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.874+0000 7f878d247640 1 -- 172.21.15.73:0/61955713 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8788162810 msgr2=0x7f878815f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:17.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.874+0000 7f878d247640 1 --2- 172.21.15.73:0/61955713 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8788162810 0x7f878815f860 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8780009ea0 tx=0x7f8780009f90 comp rx=0 tx=0).stop 2024-04-26T20:51:17.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.874+0000 7f878d247640 1 -- 172.21.15.73:0/61955713 shutdown_connections 2024-04-26T20:51:17.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.874+0000 7f878d247640 1 --2- 172.21.15.73:0/61955713 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8788162810 0x7f878815f860 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:17.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.874+0000 7f878d247640 1 --2- 172.21.15.73:0/61955713 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f877406c8f0 0x7f877406edb0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:17.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.874+0000 7f878d247640 1 --2- 172.21.15.73:0/61955713 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f87880cd930 0x7f87880c7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:17.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.874+0000 7f878d247640 1 --2- 172.21.15.73:0/61955713 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f87880b2340 0x7f8788150f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:17.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.874+0000 7f878d247640 1 -- 172.21.15.73:0/61955713 >> 172.21.15.73:0/61955713 conn(0x7f87880d2da0 msgr2=0x7f87880b3c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:17.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.875+0000 7f878d247640 1 -- 172.21.15.73:0/61955713 shutdown_connections 2024-04-26T20:51:17.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.875+0000 7f878d247640 1 -- 172.21.15.73:0/61955713 wait complete. 2024-04-26T20:51:17.887 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:50:19.993731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:50:29.964420+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:50:30.920082+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T20:50:34.650207+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T20:51:17.887 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:51:17.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.986+0000 7f0457a52640 1 Processor -- start 2024-04-26T20:51:17.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.986+0000 7f0457a52640 1 -- start start 2024-04-26T20:51:17.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.987+0000 7f0457a52640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f045015d250 0x7f04500d0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:17.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.987+0000 7f0457a52640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f04500d39f0 0x7f04500d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:17.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.987+0000 7f0457a52640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04500701e0 con 0x7f04500d1550 2024-04-26T20:51:17.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.987+0000 7f0457a52640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f04500d19c0 con 0x7f045015d250 2024-04-26T20:51:17.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.987+0000 7f0457a52640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f045015d630 con 0x7f04500d39f0 2024-04-26T20:51:17.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.987+0000 7f04557ee640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f045015d250 0x7f04500d0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:17.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.987+0000 7f04557ee640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f045015d250 0x7f04500d0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52004/0 (socket says 172.21.15.73:52004) 2024-04-26T20:51:17.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.987+0000 7f04557ee640 1 -- 172.21.15.73:0/3803932645 learned_addr learned my addr 172.21.15.73:0/3803932645 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:17.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.987+0000 7f0454fed640 1 --2- 172.21.15.73:0/3803932645 >> v2:172.21.15.120:3300/0 conn(0x7f04500d39f0 0x7f04500d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:17.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.987+0000 7f04557ee640 1 -- 172.21.15.73:0/3803932645 >> v2:172.21.15.120:3300/0 conn(0x7f04500d39f0 msgr2=0x7f04500d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:17.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.987+0000 7f04557ee640 1 --2- 172.21.15.73:0/3803932645 >> v2:172.21.15.120:3300/0 conn(0x7f04500d39f0 0x7f04500d1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:17.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.987+0000 7f04557ee640 1 -- 172.21.15.73:0/3803932645 >> v1:172.21.15.120:6789/0 conn(0x7f04500d1550 legacy=0x7f04500cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:17.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.987+0000 7f04557ee640 1 -- 172.21.15.73:0/3803932645 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04500ce370 con 0x7f045015d250 2024-04-26T20:51:17.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.988+0000 7f04557ee640 1 --2- 172.21.15.73:0/3803932645 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f045015d250 0x7f04500d0950 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0450074050 tx=0x7f044c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f865ce2a65f128b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:17.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.988+0000 7f0447fff640 1 -- 172.21.15.73:0/3803932645 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044c037070 con 0x7f045015d250 2024-04-26T20:51:17.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.988+0000 7f0447fff640 1 -- 172.21.15.73:0/3803932645 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f044c032440 con 0x7f045015d250 2024-04-26T20:51:17.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.988+0000 7f0447fff640 1 -- 172.21.15.73:0/3803932645 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044c032810 con 0x7f045015d250 2024-04-26T20:51:17.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.989+0000 7f0457a52640 1 -- 172.21.15.73:0/3803932645 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f045015d250 msgr2=0x7f04500d0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:17.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.989+0000 7f0457a52640 1 --2- 172.21.15.73:0/3803932645 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f045015d250 0x7f04500d0950 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0450074050 tx=0x7f044c02a1a0 comp rx=0 tx=0).stop 2024-04-26T20:51:17.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.989+0000 7f0457a52640 1 -- 172.21.15.73:0/3803932645 shutdown_connections 2024-04-26T20:51:17.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.989+0000 7f0457a52640 1 --2- 172.21.15.73:0/3803932645 >> v2:172.21.15.120:3300/0 conn(0x7f04500d39f0 0x7f04500d1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:17.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.989+0000 7f0457a52640 1 --2- 172.21.15.73:0/3803932645 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f045015d250 0x7f04500d0950 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:17.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.989+0000 7f0457a52640 1 -- 172.21.15.73:0/3803932645 >> 172.21.15.73:0/3803932645 conn(0x7f04500d2d80 msgr2=0x7f04500d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:17.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.989+0000 7f0457a52640 1 -- 172.21.15.73:0/3803932645 shutdown_connections 2024-04-26T20:51:17.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.989+0000 7f0457a52640 1 -- 172.21.15.73:0/3803932645 wait complete. 2024-04-26T20:51:17.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.990+0000 7f0457a52640 1 Processor -- start 2024-04-26T20:51:17.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.990+0000 7f0457a52640 1 -- start start 2024-04-26T20:51:17.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.991+0000 7f0457a52640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04500d1550 0x7f045014e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:17.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.991+0000 7f0457a52640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f04500d39f0 0x7f04500c7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:17.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.991+0000 7f0457a52640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f045015d250 0x7f045015f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:17.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.991+0000 7f0457a52640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f04500ce8d0 con 0x7f04500d1550 2024-04-26T20:51:17.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.991+0000 7f0457a52640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f04500704e0 con 0x7f045015d250 2024-04-26T20:51:17.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.991+0000 7f0457a52640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f04500cebb0 con 0x7f04500d39f0 2024-04-26T20:51:17.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.991+0000 7f04557ee640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04500d1550 0x7f045014e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:17.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.991+0000 7f04557ee640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04500d1550 0x7f045014e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52010/0 (socket says 172.21.15.73:52010) 2024-04-26T20:51:17.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.991+0000 7f04557ee640 1 -- 172.21.15.73:0/1260803249 learned_addr learned my addr 172.21.15.73:0/1260803249 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:17.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.991+0000 7f0455fef640 1 --2- 172.21.15.73:0/1260803249 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f045015d250 0x7f045015f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:17.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.991+0000 7f0454fed640 1 --2- 172.21.15.73:0/1260803249 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f04500d39f0 0x7f04500c7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:17.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.991+0000 7f04557ee640 1 -- 172.21.15.73:0/1260803249 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f04500d39f0 msgr2=0x7f04500c7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:17.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.991+0000 7f04557ee640 1 --2- 172.21.15.73:0/1260803249 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f04500d39f0 0x7f04500c7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:17.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.991+0000 7f04557ee640 1 -- 172.21.15.73:0/1260803249 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f045015d250 msgr2=0x7f045015f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:17.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.991+0000 7f04557ee640 1 --2- 172.21.15.73:0/1260803249 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f045015d250 0x7f045015f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:17.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.991+0000 7f04557ee640 1 -- 172.21.15.73:0/1260803249 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04500c9120 con 0x7f04500d1550 2024-04-26T20:51:17.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.991+0000 7f0455fef640 1 --2- 172.21.15.73:0/1260803249 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f045015d250 0x7f045015f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:17.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.991+0000 7f0454fed640 1 --2- 172.21.15.73:0/1260803249 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f04500d39f0 0x7f04500c7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:17.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.991+0000 7f04557ee640 1 --2- 172.21.15.73:0/1260803249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04500d1550 0x7f045014e240 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f044c000c00 tx=0x7f044c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:17.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.991+0000 7f0445ffb640 1 -- 172.21.15.73:0/1260803249 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044c041020 con 0x7f04500d1550 2024-04-26T20:51:17.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.991+0000 7f0445ffb640 1 -- 172.21.15.73:0/1260803249 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f044c0306c0 con 0x7f04500d1550 2024-04-26T20:51:17.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.991+0000 7f0445ffb640 1 -- 172.21.15.73:0/1260803249 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044c037040 con 0x7f04500d1550 2024-04-26T20:51:17.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.991+0000 7f0457a52640 1 -- 172.21.15.73:0/1260803249 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04500c6950 con 0x7f04500d1550 2024-04-26T20:51:17.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.992+0000 7f0457a52640 1 -- 172.21.15.73:0/1260803249 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04500c6f10 con 0x7f04500d1550 2024-04-26T20:51:17.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.993+0000 7f0457a52640 1 -- 172.21.15.73:0/1260803249 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04500694f0 con 0x7f04500d1550 2024-04-26T20:51:17.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.993+0000 7f0445ffb640 1 -- 172.21.15.73:0/1260803249 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f044c044020 con 0x7f04500d1550 2024-04-26T20:51:17.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.993+0000 7f0445ffb640 1 --2- 172.21.15.73:0/1260803249 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f043406c8f0 0x7f043406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:17.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.994+0000 7f0445ffb640 1 -- 172.21.15.73:0/1260803249 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f044c0ac780 con 0x7f04500d1550 2024-04-26T20:51:17.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.994+0000 7f0454fed640 1 --2- 172.21.15.73:0/1260803249 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f043406c8f0 0x7f043406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:17.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.994+0000 7f0454fed640 1 --2- 172.21.15.73:0/1260803249 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f043406c8f0 0x7f043406edb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0440006fd0 tx=0x7f0440008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:17.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:17.998+0000 7f0445ffb640 1 -- 172.21.15.73:0/1260803249 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f044c04b030 con 0x7f04500d1550 2024-04-26T20:51:18.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.173+0000 7f0457a52640 1 -- 172.21.15.73:0/1260803249 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f04500b36d0 con 0x7f04500d1550 2024-04-26T20:51:18.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.173+0000 7f0445ffb640 1 -- 172.21.15.73:0/1260803249 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f04500ce370 con 0x7f04500d1550 2024-04-26T20:51:18.174 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-26T20:51:18.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.175+0000 7f0457a52640 1 -- 172.21.15.73:0/1260803249 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f043406c8f0 msgr2=0x7f043406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:18.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.175+0000 7f0457a52640 1 --2- 172.21.15.73:0/1260803249 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f043406c8f0 0x7f043406edb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0440006fd0 tx=0x7f0440008040 comp rx=0 tx=0).stop 2024-04-26T20:51:18.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.175+0000 7f0457a52640 1 -- 172.21.15.73:0/1260803249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04500d1550 msgr2=0x7f045014e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:18.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.175+0000 7f0457a52640 1 --2- 172.21.15.73:0/1260803249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04500d1550 0x7f045014e240 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f044c000c00 tx=0x7f044c0046c0 comp rx=0 tx=0).stop 2024-04-26T20:51:18.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.175+0000 7f0457a52640 1 -- 172.21.15.73:0/1260803249 shutdown_connections 2024-04-26T20:51:18.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.175+0000 7f0457a52640 1 --2- 172.21.15.73:0/1260803249 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f043406c8f0 0x7f043406edb0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:18.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.175+0000 7f0457a52640 1 --2- 172.21.15.73:0/1260803249 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f045015d250 0x7f045015f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:18.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.175+0000 7f0457a52640 1 --2- 172.21.15.73:0/1260803249 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f04500d39f0 0x7f04500c7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:18.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.175+0000 7f0457a52640 1 --2- 172.21.15.73:0/1260803249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04500d1550 0x7f045014e240 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:18.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.175+0000 7f0457a52640 1 -- 172.21.15.73:0/1260803249 >> 172.21.15.73:0/1260803249 conn(0x7f04500d2d80 msgr2=0x7f04500afab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:18.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.175+0000 7f0457a52640 1 -- 172.21.15.73:0/1260803249 shutdown_connections 2024-04-26T20:51:18.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.175+0000 7f0457a52640 1 -- 172.21.15.73:0/1260803249 wait complete. 2024-04-26T20:51:18.187 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:51:18.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.287+0000 7f7eb8f46640 1 Processor -- start 2024-04-26T20:51:18.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.287+0000 7f7eb8f46640 1 -- start start 2024-04-26T20:51:18.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.288+0000 7f7eb8f46640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7eb415b600 0x7f7eb4071f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:18.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.288+0000 7f7eb8f46640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f7eb40cf0e0 0x7f7eb406f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:18.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.288+0000 7f7eb8f46640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7eb4072450 con 0x7f7eb40cf5e0 2024-04-26T20:51:18.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.288+0000 7f7eb8f46640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f7eb40cb9b0 con 0x7f7eb415b600 2024-04-26T20:51:18.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.288+0000 7f7eb8f46640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f7eb415b9e0 con 0x7f7eb40cf0e0 2024-04-26T20:51:18.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.288+0000 7f7eb259c640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7eb415b600 0x7f7eb4071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:18.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.288+0000 7f7eabfff640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f7eb40cf5e0 0x7f7eb40ce9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40240/0 (socket says 172.21.15.73:40240) 2024-04-26T20:51:18.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.288+0000 7f7eabfff640 1 -- 172.21.15.73:0/3659324572 learned_addr learned my addr 172.21.15.73:0/3659324572 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:18.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.288+0000 7f7eb259c640 1 -- 172.21.15.73:0/3659324572 >> v2:172.21.15.120:3300/0 conn(0x7f7eb40cf0e0 msgr2=0x7f7eb406f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:18.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.288+0000 7f7eb2d9d640 1 --2- 172.21.15.73:0/3659324572 >> v2:172.21.15.120:3300/0 conn(0x7f7eb40cf0e0 0x7f7eb406f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:18.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.288+0000 7f7eb1d9b640 1 -- 172.21.15.73:0/3659324572 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 205603939 0 0) 0x7f7e9c001110 con 0x7f7eb40cf5e0 2024-04-26T20:51:18.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.288+0000 7f7eb259c640 1 --2- 172.21.15.73:0/3659324572 >> v2:172.21.15.120:3300/0 conn(0x7f7eb40cf0e0 0x7f7eb406f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:18.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.288+0000 7f7eb259c640 1 -- 172.21.15.73:0/3659324572 >> v1:172.21.15.73:6789/0 conn(0x7f7eb40cf5e0 legacy=0x7f7eb40ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:18.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.288+0000 7f7eb259c640 1 -- 172.21.15.73:0/3659324572 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eb40d0740 con 0x7f7eb415b600 2024-04-26T20:51:18.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.288+0000 7f7eb2d9d640 1 --2- 172.21.15.73:0/3659324572 >> v2:172.21.15.120:3300/0 conn(0x7f7eb40cf0e0 0x7f7eb406f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:18.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.289+0000 7f7eb259c640 1 --2- 172.21.15.73:0/3659324572 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7eb415b600 0x7f7eb4071f10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4071a80 tx=0x7f7ea402b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71d97d45b3fa839d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:18.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.289+0000 7f7eb1d9b640 1 -- 172.21.15.73:0/3659324572 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea400b040 con 0x7f7eb415b600 2024-04-26T20:51:18.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.289+0000 7f7eb1d9b640 1 -- 172.21.15.73:0/3659324572 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ea4030710 con 0x7f7eb415b600 2024-04-26T20:51:18.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.289+0000 7f7eb1d9b640 1 -- 172.21.15.73:0/3659324572 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea4030ae0 con 0x7f7eb415b600 2024-04-26T20:51:18.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.289+0000 7f7eb8f46640 1 -- 172.21.15.73:0/3659324572 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7eb415b600 msgr2=0x7f7eb4071f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:18.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.289+0000 7f7eb8f46640 1 --2- 172.21.15.73:0/3659324572 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7eb415b600 0x7f7eb4071f10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4071a80 tx=0x7f7ea402b560 comp rx=0 tx=0).stop 2024-04-26T20:51:18.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.289+0000 7f7eb8f46640 1 -- 172.21.15.73:0/3659324572 shutdown_connections 2024-04-26T20:51:18.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.289+0000 7f7eb8f46640 1 --2- 172.21.15.73:0/3659324572 >> v2:172.21.15.120:3300/0 conn(0x7f7eb40cf0e0 0x7f7eb406f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:18.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.289+0000 7f7eb8f46640 1 --2- 172.21.15.73:0/3659324572 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7eb415b600 0x7f7eb4071f10 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:18.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.289+0000 7f7eb8f46640 1 -- 172.21.15.73:0/3659324572 >> 172.21.15.73:0/3659324572 conn(0x7f7eb40d0be0 msgr2=0x7f7eb40d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:18.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.290+0000 7f7eb8f46640 1 -- 172.21.15.73:0/3659324572 shutdown_connections 2024-04-26T20:51:18.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.290+0000 7f7eb8f46640 1 -- 172.21.15.73:0/3659324572 wait complete. 2024-04-26T20:51:18.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.291+0000 7f7eb8f46640 1 Processor -- start 2024-04-26T20:51:18.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.291+0000 7f7eb8f46640 1 -- start start 2024-04-26T20:51:18.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.291+0000 7f7eb8f46640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7eb40cf0e0 0x7f7eb41f0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:18.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.291+0000 7f7eb8f46640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7eb40cf5e0 0x7f7eb40b5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:18.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.291+0000 7f7eb8f46640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7eb415b600 0x7f7eb40b6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:18.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.291+0000 7f7eb8f46640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb40d02f0 con 0x7f7eb40cf5e0 2024-04-26T20:51:18.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.291+0000 7f7eb8f46640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb40cb640 con 0x7f7eb415b600 2024-04-26T20:51:18.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.291+0000 7f7eb8f46640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f7eb40d0470 con 0x7f7eb40cf0e0 2024-04-26T20:51:18.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.292+0000 7f7eabfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7eb40cf5e0 0x7f7eb40b5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:18.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.292+0000 7f7eabfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7eb40cf5e0 0x7f7eb40b5fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52030/0 (socket says 172.21.15.73:52030) 2024-04-26T20:51:18.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.292+0000 7f7eabfff640 1 -- 172.21.15.73:0/2094754551 learned_addr learned my addr 172.21.15.73:0/2094754551 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:18.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.292+0000 7f7eb2d9d640 1 --2- 172.21.15.73:0/2094754551 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7eb415b600 0x7f7eb40b6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:18.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.292+0000 7f7eb259c640 1 --2- 172.21.15.73:0/2094754551 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7eb40cf0e0 0x7f7eb41f0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:18.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.292+0000 7f7eabfff640 1 -- 172.21.15.73:0/2094754551 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7eb40cf0e0 msgr2=0x7f7eb41f0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:18.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.292+0000 7f7eabfff640 1 --2- 172.21.15.73:0/2094754551 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7eb40cf0e0 0x7f7eb41f0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:18.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.292+0000 7f7eabfff640 1 -- 172.21.15.73:0/2094754551 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7eb415b600 msgr2=0x7f7eb40b6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:18.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.292+0000 7f7eabfff640 1 --2- 172.21.15.73:0/2094754551 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7eb415b600 0x7f7eb40b6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:18.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.292+0000 7f7eabfff640 1 -- 172.21.15.73:0/2094754551 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eb41f0ab0 con 0x7f7eb40cf5e0 2024-04-26T20:51:18.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.292+0000 7f7eabfff640 1 --2- 172.21.15.73:0/2094754551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7eb40cf5e0 0x7f7eb40b5fe0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c00c780 tx=0x7f7e9c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:18.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.292+0000 7f7eaaffd640 1 -- 172.21.15.73:0/2094754551 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c018070 con 0x7f7eb40cf5e0 2024-04-26T20:51:18.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.292+0000 7f7eaaffd640 1 -- 172.21.15.73:0/2094754551 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e9c0072d0 con 0x7f7eb40cf5e0 2024-04-26T20:51:18.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.292+0000 7f7eb8f46640 1 -- 172.21.15.73:0/2094754551 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eb40c8670 con 0x7f7eb40cf5e0 2024-04-26T20:51:18.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.292+0000 7f7eaaffd640 1 -- 172.21.15.73:0/2094754551 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c0047e0 con 0x7f7eb40cf5e0 2024-04-26T20:51:18.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.292+0000 7f7eb8f46640 1 -- 172.21.15.73:0/2094754551 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eb40c8c30 con 0x7f7eb40cf5e0 2024-04-26T20:51:18.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.293+0000 7f7eb8f46640 1 -- 172.21.15.73:0/2094754551 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7eb40694f0 con 0x7f7eb40cf5e0 2024-04-26T20:51:18.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.294+0000 7f7eaaffd640 1 -- 172.21.15.73:0/2094754551 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7e9c01b070 con 0x7f7eb40cf5e0 2024-04-26T20:51:18.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.294+0000 7f7eaaffd640 1 --2- 172.21.15.73:0/2094754551 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f7e9406c8f0 0x7f7e9406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:18.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.294+0000 7f7eaaffd640 1 -- 172.21.15.73:0/2094754551 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f7e9c08b280 con 0x7f7eb40cf5e0 2024-04-26T20:51:18.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.294+0000 7f7eb259c640 1 --2- 172.21.15.73:0/2094754551 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f7e9406c8f0 0x7f7e9406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:18.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.295+0000 7f7eb259c640 1 --2- 172.21.15.73:0/2094754551 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f7e9406c8f0 0x7f7e9406edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4035000 tx=0x7f7ea400c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:18.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.298+0000 7f7eaaffd640 1 -- 172.21.15.73:0/2094754551 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7eb4072450 con 0x7f7eb40cf5e0 2024-04-26T20:51:18.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.472+0000 7f7eb8f46640 1 -- 172.21.15.73:0/2094754551 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7eb40ce7b0 con 0x7f7eb40cf5e0 2024-04-26T20:51:18.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.472+0000 7f7eaaffd640 1 -- 172.21.15.73:0/2094754551 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f7e9c0592b0 con 0x7f7eb40cf5e0 2024-04-26T20:51:18.473 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-26T20:51:18.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.474+0000 7f7eb8f46640 1 -- 172.21.15.73:0/2094754551 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f7e9406c8f0 msgr2=0x7f7e9406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:18.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.474+0000 7f7eb8f46640 1 --2- 172.21.15.73:0/2094754551 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f7e9406c8f0 0x7f7e9406edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4035000 tx=0x7f7ea400c040 comp rx=0 tx=0).stop 2024-04-26T20:51:18.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.474+0000 7f7eb8f46640 1 -- 172.21.15.73:0/2094754551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7eb40cf5e0 msgr2=0x7f7eb40b5fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:18.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.474+0000 7f7eb8f46640 1 --2- 172.21.15.73:0/2094754551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7eb40cf5e0 0x7f7eb40b5fe0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c00c780 tx=0x7f7e9c00cc40 comp rx=0 tx=0).stop 2024-04-26T20:51:18.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.474+0000 7f7eb8f46640 1 -- 172.21.15.73:0/2094754551 shutdown_connections 2024-04-26T20:51:18.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.474+0000 7f7eb8f46640 1 --2- 172.21.15.73:0/2094754551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7eb40cf5e0 0x7f7eb40b5fe0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:18.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.474+0000 7f7eb8f46640 1 --2- 172.21.15.73:0/2094754551 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f7e9406c8f0 0x7f7e9406edb0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:18.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.474+0000 7f7eb8f46640 1 --2- 172.21.15.73:0/2094754551 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7eb415b600 0x7f7eb40b6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:18.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.474+0000 7f7eb8f46640 1 --2- 172.21.15.73:0/2094754551 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7eb40cf0e0 0x7f7eb41f0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:18.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.474+0000 7f7eb8f46640 1 -- 172.21.15.73:0/2094754551 >> 172.21.15.73:0/2094754551 conn(0x7f7eb40d0be0 msgr2=0x7f7eb40b1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:18.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.474+0000 7f7eb8f46640 1 -- 172.21.15.73:0/2094754551 shutdown_connections 2024-04-26T20:51:18.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.474+0000 7f7eb8f46640 1 -- 172.21.15.73:0/2094754551 wait complete. 2024-04-26T20:51:18.486 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:51:18.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.584+0000 7efd60852640 1 Processor -- start 2024-04-26T20:51:18.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.584+0000 7efd60852640 1 -- start start 2024-04-26T20:51:18.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.585+0000 7efd60852640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efd5c0b10b0 0x7efd5c0ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:18.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.585+0000 7efd60852640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7efd5c06c950 0x7efd5c15e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:18.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.585+0000 7efd60852640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd5c0cfd80 con 0x7efd5c0b0380 2024-04-26T20:51:18.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.585+0000 7efd60852640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7efd5c0af8e0 con 0x7efd5c06c950 2024-04-26T20:51:18.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.585+0000 7efd60852640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7efd5c0d0020 con 0x7efd5c0b10b0 2024-04-26T20:51:18.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.586+0000 7efd5a59c640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7efd5c0b0380 0x7efd5c06c240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40256/0 (socket says 172.21.15.73:40256) 2024-04-26T20:51:18.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.586+0000 7efd5a59c640 1 -- 172.21.15.73:0/943230677 learned_addr learned my addr 172.21.15.73:0/943230677 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:18.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.586+0000 7efd5ad9d640 1 --2- 172.21.15.73:0/943230677 >> v2:172.21.15.120:3300/0 conn(0x7efd5c06c950 0x7efd5c15e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:18.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.586+0000 7efd59d9b640 1 --2- 172.21.15.73:0/943230677 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efd5c0b10b0 0x7efd5c0ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:18.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.586+0000 7efd5959a640 1 -- 172.21.15.73:0/943230677 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3673675625 0 0) 0x7efd5c0cfd80 con 0x7efd5c0b0380 2024-04-26T20:51:18.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.586+0000 7efd5959a640 1 -- 172.21.15.73:0/943230677 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd44003520 con 0x7efd5c0b0380 2024-04-26T20:51:18.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.586+0000 7efd59d9b640 1 -- 172.21.15.73:0/943230677 >> v2:172.21.15.120:3300/0 conn(0x7efd5c06c950 msgr2=0x7efd5c15e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:18.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.586+0000 7efd59d9b640 1 --2- 172.21.15.73:0/943230677 >> v2:172.21.15.120:3300/0 conn(0x7efd5c06c950 0x7efd5c15e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:18.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.586+0000 7efd59d9b640 1 -- 172.21.15.73:0/943230677 >> v1:172.21.15.73:6789/0 conn(0x7efd5c0b0380 legacy=0x7efd5c06c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:18.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.586+0000 7efd59d9b640 1 -- 172.21.15.73:0/943230677 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd5c0d0300 con 0x7efd5c0b10b0 2024-04-26T20:51:18.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.587+0000 7efd59d9b640 1 --2- 172.21.15.73:0/943230677 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efd5c0b10b0 0x7efd5c0ccd50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7efd5c0afa60 tx=0x7efd5002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=114982039cc7b2e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:18.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.587+0000 7efd5959a640 1 -- 172.21.15.73:0/943230677 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd50037070 con 0x7efd5c0b10b0 2024-04-26T20:51:18.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.587+0000 7efd5959a640 1 -- 172.21.15.73:0/943230677 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd5002ae50 con 0x7efd5c0b10b0 2024-04-26T20:51:18.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.587+0000 7efd5959a640 1 -- 172.21.15.73:0/943230677 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd50032640 con 0x7efd5c0b10b0 2024-04-26T20:51:18.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.587+0000 7efd60852640 1 -- 172.21.15.73:0/943230677 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efd5c0b10b0 msgr2=0x7efd5c0ccd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:18.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.587+0000 7efd60852640 1 --2- 172.21.15.73:0/943230677 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efd5c0b10b0 0x7efd5c0ccd50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7efd5c0afa60 tx=0x7efd5002a100 comp rx=0 tx=0).stop 2024-04-26T20:51:18.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.588+0000 7efd60852640 1 -- 172.21.15.73:0/943230677 shutdown_connections 2024-04-26T20:51:18.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.588+0000 7efd60852640 1 --2- 172.21.15.73:0/943230677 >> v2:172.21.15.120:3300/0 conn(0x7efd5c06c950 0x7efd5c15e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:18.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.588+0000 7efd60852640 1 --2- 172.21.15.73:0/943230677 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efd5c0b10b0 0x7efd5c0ccd50 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:18.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.588+0000 7efd60852640 1 -- 172.21.15.73:0/943230677 >> 172.21.15.73:0/943230677 conn(0x7efd5c0d1cf0 msgr2=0x7efd5c0d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:18.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.588+0000 7efd60852640 1 -- 172.21.15.73:0/943230677 shutdown_connections 2024-04-26T20:51:18.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.588+0000 7efd60852640 1 -- 172.21.15.73:0/943230677 wait complete. 2024-04-26T20:51:18.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.589+0000 7efd60852640 1 Processor -- start 2024-04-26T20:51:18.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.589+0000 7efd60852640 1 -- start start 2024-04-26T20:51:18.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.589+0000 7efd60852640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efd5c06c950 0x7efd5c1f7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:18.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.590+0000 7efd60852640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd5c0b0380 0x7efd5c162420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:18.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.590+0000 7efd60852640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd5c0b10b0 0x7efd5c0c9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:18.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.590+0000 7efd60852640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efd5c0b0760 con 0x7efd5c0b0380 2024-04-26T20:51:18.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.590+0000 7efd60852640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efd5c0d1980 con 0x7efd5c0b10b0 2024-04-26T20:51:18.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.590+0000 7efd60852640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7efd5c0d15c0 con 0x7efd5c06c950 2024-04-26T20:51:18.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.590+0000 7efd59d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd5c0b0380 0x7efd5c162420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:18.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.590+0000 7efd59d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd5c0b0380 0x7efd5c162420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52046/0 (socket says 172.21.15.73:52046) 2024-04-26T20:51:18.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.590+0000 7efd59d9b640 1 -- 172.21.15.73:0/2191184525 learned_addr learned my addr 172.21.15.73:0/2191184525 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:18.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.590+0000 7efd5a59c640 1 --2- 172.21.15.73:0/2191184525 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efd5c06c950 0x7efd5c1f7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:18.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.590+0000 7efd5ad9d640 1 --2- 172.21.15.73:0/2191184525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd5c0b10b0 0x7efd5c0c9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:18.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.590+0000 7efd59d9b640 1 -- 172.21.15.73:0/2191184525 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efd5c06c950 msgr2=0x7efd5c1f7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:18.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.590+0000 7efd59d9b640 1 --2- 172.21.15.73:0/2191184525 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efd5c06c950 0x7efd5c1f7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:18.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.590+0000 7efd59d9b640 1 -- 172.21.15.73:0/2191184525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd5c0b10b0 msgr2=0x7efd5c0c9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:18.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.590+0000 7efd59d9b640 1 --2- 172.21.15.73:0/2191184525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd5c0b10b0 0x7efd5c0c9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:18.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.590+0000 7efd59d9b640 1 -- 172.21.15.73:0/2191184525 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd5c1f83d0 con 0x7efd5c0b0380 2024-04-26T20:51:18.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.590+0000 7efd59d9b640 1 --2- 172.21.15.73:0/2191184525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd5c0b0380 0x7efd5c162420 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7efd50000c00 tx=0x7efd500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:18.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.590+0000 7efd4affd640 1 -- 172.21.15.73:0/2191184525 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd50041030 con 0x7efd5c0b0380 2024-04-26T20:51:18.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.590+0000 7efd4affd640 1 -- 172.21.15.73:0/2191184525 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd5002ac50 con 0x7efd5c0b0380 2024-04-26T20:51:18.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.590+0000 7efd60852640 1 -- 172.21.15.73:0/2191184525 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd5c1f86b0 con 0x7efd5c0b0380 2024-04-26T20:51:18.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.590+0000 7efd4affd640 1 -- 172.21.15.73:0/2191184525 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd50037040 con 0x7efd5c0b0380 2024-04-26T20:51:18.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.590+0000 7efd60852640 1 -- 172.21.15.73:0/2191184525 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd5c1f8c70 con 0x7efd5c0b0380 2024-04-26T20:51:18.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.591+0000 7efd60852640 1 -- 172.21.15.73:0/2191184525 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd5c0694f0 con 0x7efd5c0b0380 2024-04-26T20:51:18.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.592+0000 7efd4affd640 1 -- 172.21.15.73:0/2191184525 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efd50044070 con 0x7efd5c0b0380 2024-04-26T20:51:18.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.592+0000 7efd4affd640 1 --2- 172.21.15.73:0/2191184525 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7efd3c06c8f0 0x7efd3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:18.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.592+0000 7efd4affd640 1 -- 172.21.15.73:0/2191184525 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7efd500ac440 con 0x7efd5c0b0380 2024-04-26T20:51:18.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.592+0000 7efd5a59c640 1 --2- 172.21.15.73:0/2191184525 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7efd3c06c8f0 0x7efd3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:18.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.593+0000 7efd5a59c640 1 --2- 172.21.15.73:0/2191184525 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7efd3c06c8f0 0x7efd3c06edb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7efd54005880 tx=0x7efd54009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:18.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.596+0000 7efd4affd640 1 -- 172.21.15.73:0/2191184525 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efd50079f40 con 0x7efd5c0b0380 2024-04-26T20:51:18.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.771+0000 7efd60852640 1 -- 172.21.15.73:0/2191184525 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7efd5c0c8ff0 con 0x7efd5c0b0380 2024-04-26T20:51:18.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.771+0000 7efd4affd640 1 -- 172.21.15.73:0/2191184525 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7efd5002ae20 con 0x7efd5c0b0380 2024-04-26T20:51:18.772 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-26T20:51:18.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.773+0000 7efd60852640 1 -- 172.21.15.73:0/2191184525 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7efd3c06c8f0 msgr2=0x7efd3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:18.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.773+0000 7efd60852640 1 --2- 172.21.15.73:0/2191184525 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7efd3c06c8f0 0x7efd3c06edb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7efd54005880 tx=0x7efd54009040 comp rx=0 tx=0).stop 2024-04-26T20:51:18.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.773+0000 7efd60852640 1 -- 172.21.15.73:0/2191184525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd5c0b0380 msgr2=0x7efd5c162420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:18.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.773+0000 7efd60852640 1 --2- 172.21.15.73:0/2191184525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd5c0b0380 0x7efd5c162420 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7efd50000c00 tx=0x7efd500046c0 comp rx=0 tx=0).stop 2024-04-26T20:51:18.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.773+0000 7efd60852640 1 -- 172.21.15.73:0/2191184525 shutdown_connections 2024-04-26T20:51:18.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.773+0000 7efd60852640 1 --2- 172.21.15.73:0/2191184525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd5c0b0380 0x7efd5c162420 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:18.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.773+0000 7efd60852640 1 --2- 172.21.15.73:0/2191184525 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7efd3c06c8f0 0x7efd3c06edb0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:18.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.773+0000 7efd60852640 1 --2- 172.21.15.73:0/2191184525 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd5c0b10b0 0x7efd5c0c9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:18.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.773+0000 7efd60852640 1 --2- 172.21.15.73:0/2191184525 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efd5c06c950 0x7efd5c1f7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:18.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.773+0000 7efd60852640 1 -- 172.21.15.73:0/2191184525 >> 172.21.15.73:0/2191184525 conn(0x7efd5c0d1cf0 msgr2=0x7efd5c072e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:18.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.773+0000 7efd60852640 1 -- 172.21.15.73:0/2191184525 shutdown_connections 2024-04-26T20:51:18.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.773+0000 7efd60852640 1 -- 172.21.15.73:0/2191184525 wait complete. 2024-04-26T20:51:18.785 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T20:51:18.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.884+0000 7f7ae6283640 1 Processor -- start 2024-04-26T20:51:18.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.884+0000 7f7ae6283640 1 -- start start 2024-04-26T20:51:18.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.885+0000 7f7ae6283640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f7ae00d1a00 0x7f7ae00d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:18.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.885+0000 7f7ae6283640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7ae00d2320 0x7f7ae01625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:18.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.885+0000 7f7ae6283640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7ae00b40d0 0x7f7ae0162b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:18.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.885+0000 7f7ae6283640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f7ae0163070 con 0x7f7ae00d2320 2024-04-26T20:51:18.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.885+0000 7f7ae6283640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f7ae00d0a10 con 0x7f7ae00d1a00 2024-04-26T20:51:18.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.885+0000 7f7ae6283640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f7ae00d0cf0 con 0x7f7ae00b40d0 2024-04-26T20:51:18.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.885+0000 7f7adf7fe640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7ae00d2320 0x7f7ae01625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:18.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.885+0000 7f7adf7fe640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7ae00d2320 0x7f7ae01625f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52048/0 (socket says 172.21.15.73:52048) 2024-04-26T20:51:18.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.885+0000 7f7adf7fe640 1 -- 172.21.15.73:0/4140234008 learned_addr learned my addr 172.21.15.73:0/4140234008 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:18.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.885+0000 7f7adffff640 1 --2- 172.21.15.73:0/4140234008 >> v2:172.21.15.120:3300/0 conn(0x7f7ae00d1a00 0x7f7ae00d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:18.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.885+0000 7f7ae4820640 1 --2- 172.21.15.73:0/4140234008 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7ae00b40d0 0x7f7ae0162b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:18.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.885+0000 7f7adf7fe640 1 -- 172.21.15.73:0/4140234008 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7ae00b40d0 msgr2=0x7f7ae0162b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:18.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.886+0000 7f7adf7fe640 1 --2- 172.21.15.73:0/4140234008 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7ae00b40d0 0x7f7ae0162b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:18.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.886+0000 7f7adf7fe640 1 -- 172.21.15.73:0/4140234008 >> v2:172.21.15.120:3300/0 conn(0x7f7ae00d1a00 msgr2=0x7f7ae00d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:18.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.886+0000 7f7adf7fe640 1 --2- 172.21.15.73:0/4140234008 >> v2:172.21.15.120:3300/0 conn(0x7f7ae00d1a00 0x7f7ae00d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:18.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.886+0000 7f7adf7fe640 1 -- 172.21.15.73:0/4140234008 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ae00d0fd0 con 0x7f7ae00d2320 2024-04-26T20:51:18.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.886+0000 7f7ae4820640 1 --2- 172.21.15.73:0/4140234008 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7ae00b40d0 0x7f7ae0162b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:18.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.886+0000 7f7adf7fe640 1 --2- 172.21.15.73:0/4140234008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ae00d2320 0x7f7ae01625f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7ae006c650 tx=0x7f7ad402a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb34555c64d2d60c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:18.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.886+0000 7f7adeffd640 1 -- 172.21.15.73:0/4140234008 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad4037070 con 0x7f7ae00d2320 2024-04-26T20:51:18.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.886+0000 7f7adeffd640 1 -- 172.21.15.73:0/4140234008 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ad402ae00 con 0x7f7ae00d2320 2024-04-26T20:51:18.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.886+0000 7f7adeffd640 1 -- 172.21.15.73:0/4140234008 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad4032660 con 0x7f7ae00d2320 2024-04-26T20:51:18.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.887+0000 7f7ae6283640 1 -- 172.21.15.73:0/4140234008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ae00d2320 msgr2=0x7f7ae01625f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:18.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.887+0000 7f7ae6283640 1 --2- 172.21.15.73:0/4140234008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ae00d2320 0x7f7ae01625f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7ae006c650 tx=0x7f7ad402a0b0 comp rx=0 tx=0).stop 2024-04-26T20:51:18.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.887+0000 7f7ae6283640 1 -- 172.21.15.73:0/4140234008 shutdown_connections 2024-04-26T20:51:18.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.887+0000 7f7ae6283640 1 --2- 172.21.15.73:0/4140234008 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7ae00b40d0 0x7f7ae0162b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:18.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.887+0000 7f7ae6283640 1 --2- 172.21.15.73:0/4140234008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ae00d2320 0x7f7ae01625f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:18.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.887+0000 7f7ae6283640 1 --2- 172.21.15.73:0/4140234008 >> v2:172.21.15.120:3300/0 conn(0x7f7ae00d1a00 0x7f7ae00d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:18.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.887+0000 7f7ae6283640 1 -- 172.21.15.73:0/4140234008 >> 172.21.15.73:0/4140234008 conn(0x7f7ae00d4fd0 msgr2=0x7f7ae00d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:18.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.887+0000 7f7ae6283640 1 -- 172.21.15.73:0/4140234008 shutdown_connections 2024-04-26T20:51:18.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.887+0000 7f7ae6283640 1 -- 172.21.15.73:0/4140234008 wait complete. 2024-04-26T20:51:18.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.888+0000 7f7ae6283640 1 Processor -- start 2024-04-26T20:51:18.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.888+0000 7f7ae6283640 1 -- start start 2024-04-26T20:51:18.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.889+0000 7f7ae6283640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ae00b40d0 0x7f7ae00c9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:18.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.889+0000 7f7ae6283640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7ae00d1a00 0x7f7ae00b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:18.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.889+0000 7f7ae6283640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ae00d2320 0x7f7ae00b2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:18.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.889+0000 7f7ae6283640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae00d3050 con 0x7f7ae00b40d0 2024-04-26T20:51:18.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.889+0000 7f7ae6283640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae00d2ed0 con 0x7f7ae00d2320 2024-04-26T20:51:18.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.889+0000 7f7ae6283640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f7ae00d31d0 con 0x7f7ae00d1a00 2024-04-26T20:51:18.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.889+0000 7f7adffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ae00b40d0 0x7f7ae00c9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:18.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.889+0000 7f7adffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ae00b40d0 0x7f7ae00c9370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52062/0 (socket says 172.21.15.73:52062) 2024-04-26T20:51:18.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.889+0000 7f7adffff640 1 -- 172.21.15.73:0/2815045724 learned_addr learned my addr 172.21.15.73:0/2815045724 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:18.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.889+0000 7f7adf7fe640 1 --2- 172.21.15.73:0/2815045724 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7ae00d1a00 0x7f7ae00b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:18.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.889+0000 7f7ae4820640 1 --2- 172.21.15.73:0/2815045724 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ae00d2320 0x7f7ae00b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:18.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.889+0000 7f7adffff640 1 -- 172.21.15.73:0/2815045724 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7ae00d1a00 msgr2=0x7f7ae00b8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:18.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.889+0000 7f7adffff640 1 --2- 172.21.15.73:0/2815045724 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7ae00d1a00 0x7f7ae00b8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:18.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.889+0000 7f7adffff640 1 -- 172.21.15.73:0/2815045724 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ae00d2320 msgr2=0x7f7ae00b2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:18.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.889+0000 7f7adffff640 1 --2- 172.21.15.73:0/2815045724 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ae00d2320 0x7f7ae00b2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:18.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.889+0000 7f7adffff640 1 -- 172.21.15.73:0/2815045724 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ae00c98b0 con 0x7f7ae00b40d0 2024-04-26T20:51:18.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.889+0000 7f7ae4820640 1 --2- 172.21.15.73:0/2815045724 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ae00d2320 0x7f7ae00b2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:18.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.889+0000 7f7adf7fe640 1 --2- 172.21.15.73:0/2815045724 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7ae00d1a00 0x7f7ae00b8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:18.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.890+0000 7f7adffff640 1 --2- 172.21.15.73:0/2815045724 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ae00b40d0 0x7f7ae00c9370 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7ad000d800 tx=0x7f7ad000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:18.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.890+0000 7f7adcff9640 1 -- 172.21.15.73:0/2815045724 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad000b6c0 con 0x7f7ae00b40d0 2024-04-26T20:51:18.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.890+0000 7f7adcff9640 1 -- 172.21.15.73:0/2815045724 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ad0019070 con 0x7f7ae00b40d0 2024-04-26T20:51:18.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.890+0000 7f7ae6283640 1 -- 172.21.15.73:0/2815045724 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ae00b95b0 con 0x7f7ae00b40d0 2024-04-26T20:51:18.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.890+0000 7f7adcff9640 1 -- 172.21.15.73:0/2815045724 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad000ba80 con 0x7f7ae00b40d0 2024-04-26T20:51:18.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.890+0000 7f7ae6283640 1 -- 172.21.15.73:0/2815045724 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ae00b9b70 con 0x7f7ae00b40d0 2024-04-26T20:51:18.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.891+0000 7f7ae6283640 1 -- 172.21.15.73:0/2815045724 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ae00b9db0 con 0x7f7ae00b40d0 2024-04-26T20:51:18.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.891+0000 7f7adcff9640 1 -- 172.21.15.73:0/2815045724 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7ad000bc20 con 0x7f7ae00b40d0 2024-04-26T20:51:18.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.891+0000 7f7adcff9640 1 --2- 172.21.15.73:0/2815045724 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f7acc06c8f0 0x7f7acc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:18.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.892+0000 7f7adcff9640 1 -- 172.21.15.73:0/2815045724 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f7ad008c420 con 0x7f7ae00b40d0 2024-04-26T20:51:18.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.892+0000 7f7adf7fe640 1 --2- 172.21.15.73:0/2815045724 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f7acc06c8f0 0x7f7acc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:18.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.892+0000 7f7adf7fe640 1 --2- 172.21.15.73:0/2815045724 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f7acc06c8f0 0x7f7acc06edb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4034000 tx=0x7f7ad402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:18.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:18.896+0000 7f7adcff9640 1 -- 172.21.15.73:0/2815045724 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7ad005a450 con 0x7f7ae00b40d0 2024-04-26T20:51:19.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.073+0000 7f7ae6283640 1 -- 172.21.15.73:0/2815045724 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f7ae00694f0 con 0x7f7ae00b40d0 2024-04-26T20:51:19.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.073+0000 7f7adcff9640 1 -- 172.21.15.73:0/2815045724 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f7ad0059d50 con 0x7f7ae00b40d0 2024-04-26T20:51:19.074 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-26T20:51:19.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.075+0000 7f7ae6283640 1 -- 172.21.15.73:0/2815045724 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f7acc06c8f0 msgr2=0x7f7acc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:19.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.075+0000 7f7ae6283640 1 --2- 172.21.15.73:0/2815045724 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f7acc06c8f0 0x7f7acc06edb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4034000 tx=0x7f7ad402e000 comp rx=0 tx=0).stop 2024-04-26T20:51:19.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.075+0000 7f7ae6283640 1 -- 172.21.15.73:0/2815045724 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ae00b40d0 msgr2=0x7f7ae00c9370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:19.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.075+0000 7f7ae6283640 1 --2- 172.21.15.73:0/2815045724 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ae00b40d0 0x7f7ae00c9370 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7ad000d800 tx=0x7f7ad000dcc0 comp rx=0 tx=0).stop 2024-04-26T20:51:19.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.075+0000 7f7ae6283640 1 -- 172.21.15.73:0/2815045724 shutdown_connections 2024-04-26T20:51:19.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.075+0000 7f7ae6283640 1 --2- 172.21.15.73:0/2815045724 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f7acc06c8f0 0x7f7acc06edb0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:19.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.075+0000 7f7ae6283640 1 --2- 172.21.15.73:0/2815045724 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7ae00d2320 0x7f7ae00b2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:19.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.075+0000 7f7ae6283640 1 --2- 172.21.15.73:0/2815045724 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7ae00d1a00 0x7f7ae00b8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:19.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.075+0000 7f7ae6283640 1 --2- 172.21.15.73:0/2815045724 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ae00b40d0 0x7f7ae00c9370 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:19.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.075+0000 7f7ae6283640 1 -- 172.21.15.73:0/2815045724 >> 172.21.15.73:0/2815045724 conn(0x7f7ae00d4fd0 msgr2=0x7f7ae00cfe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:19.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.075+0000 7f7ae6283640 1 -- 172.21.15.73:0/2815045724 shutdown_connections 2024-04-26T20:51:19.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.075+0000 7f7ae6283640 1 -- 172.21.15.73:0/2815045724 wait complete. 2024-04-26T20:51:19.112 INFO:tasks.cephfs_test_runner:Starting test: test_backtrace_repair (tasks.cephfs.test_auto_repair.TestMDSAutoRepair) MDS should verify/fix backtrace on fetch dirfrag 2024-04-26T20:51:19.112 INFO:tasks.cephfs_test_runner:test_backtrace_repair (tasks.cephfs.test_auto_repair.TestMDSAutoRepair) 2024-04-26T20:51:19.113 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:51:19.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.207+0000 7f70e8c64640 1 Processor -- start 2024-04-26T20:51:19.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.207+0000 7f70e8c64640 1 -- start start 2024-04-26T20:51:19.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.208+0000 7f70e8c64640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f70e4162810 0x7f70e4070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:19.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.208+0000 7f70e8c64640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f70e40d1b30 0x7f70e40d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:19.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.208+0000 7f70e8c64640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70e40d10d0 con 0x7f70e40b2340 2024-04-26T20:51:19.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.208+0000 7f70e8c64640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f70e40d0750 con 0x7f70e4162810 2024-04-26T20:51:19.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.208+0000 7f70e8c64640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f70e40d16a0 con 0x7f70e40d1b30 2024-04-26T20:51:19.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.208+0000 7f70e1d9b640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f70e40b2340 0x7f70e40cd3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40260/0 (socket says 172.21.15.73:40260) 2024-04-26T20:51:19.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.208+0000 7f70e1d9b640 1 -- 172.21.15.73:0/2794619136 learned_addr learned my addr 172.21.15.73:0/2794619136 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:19.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.208+0000 7f70e2d9d640 1 --2- 172.21.15.73:0/2794619136 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f70e40d1b30 0x7f70e40d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:19.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.208+0000 7f70e259c640 1 --2- 172.21.15.73:0/2794619136 >> v2:172.21.15.120:3300/0 conn(0x7f70e4162810 0x7f70e4070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:19.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.209+0000 7f70e159a640 1 -- 172.21.15.73:0/2794619136 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2080461503 0 0) 0x7f70e40d10d0 con 0x7f70e40b2340 2024-04-26T20:51:19.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.209+0000 7f70e159a640 1 -- 172.21.15.73:0/2794619136 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f70cc003520 con 0x7f70e40b2340 2024-04-26T20:51:19.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.209+0000 7f70e159a640 1 -- 172.21.15.73:0/2794619136 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1808075512 0 0) 0x7f70cc003520 con 0x7f70e40b2340 2024-04-26T20:51:19.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.209+0000 7f70e159a640 1 -- 172.21.15.73:0/2794619136 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f70e40d1b30 msgr2=0x7f70e40d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:19.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.209+0000 7f70e159a640 1 --2- 172.21.15.73:0/2794619136 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f70e40d1b30 0x7f70e40d1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:19.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.209+0000 7f70e159a640 1 -- 172.21.15.73:0/2794619136 >> v2:172.21.15.120:3300/0 conn(0x7f70e4162810 msgr2=0x7f70e4070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:19.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.209+0000 7f70e159a640 1 --2- 172.21.15.73:0/2794619136 >> v2:172.21.15.120:3300/0 conn(0x7f70e4162810 0x7f70e4070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:19.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.209+0000 7f70e159a640 1 -- 172.21.15.73:0/2794619136 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70e40d1370 con 0x7f70e40b2340 2024-04-26T20:51:19.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.209+0000 7f70e159a640 1 -- 172.21.15.73:0/2794619136 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7f70d8001670 con 0x7f70e40b2340 2024-04-26T20:51:19.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.209+0000 7f70e159a640 1 -- 172.21.15.73:0/2794619136 >> v1:172.21.15.73:6789/0 conn(0x7f70e40b2340 legacy=0x7f70e40cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:19.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.209+0000 7f70e159a640 1 --2- 172.21.15.73:0/2794619136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70cc0039b0 0x7f70cc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:19.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.209+0000 7f70e159a640 1 --2- 172.21.15.73:0/2794619136 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f70cc004800 0x7f70cc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:19.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.209+0000 7f70e159a640 1 --2- 172.21.15.73:0/2794619136 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f70e40d1b30 0x7f70cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:19.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.209+0000 7f70e159a640 1 -- 172.21.15.73:0/2794619136 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f70e40d0750 con 0x7f70cc0039b0 2024-04-26T20:51:19.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.209+0000 7f70e159a640 1 -- 172.21.15.73:0/2794619136 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f70e40d16a0 con 0x7f70cc004800 2024-04-26T20:51:19.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.209+0000 7f70e159a640 1 -- 172.21.15.73:0/2794619136 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f70cc00b9c0 con 0x7f70e40d1b30 2024-04-26T20:51:19.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.209+0000 7f70e259c640 1 --2- 172.21.15.73:0/2794619136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70cc0039b0 0x7f70cc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:19.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.209+0000 7f70e1d9b640 1 --2- 172.21.15.73:0/2794619136 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f70cc004800 0x7f70cc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:19.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.209+0000 7f70e2d9d640 1 --2- 172.21.15.73:0/2794619136 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f70e40d1b30 0x7f70cc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:19.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.209+0000 7f70e259c640 1 -- 172.21.15.73:0/2794619136 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f70e40d1b30 msgr2=0x7f70cc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:19.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.209+0000 7f70e259c640 1 --2- 172.21.15.73:0/2794619136 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f70e40d1b30 0x7f70cc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:19.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.209+0000 7f70e259c640 1 -- 172.21.15.73:0/2794619136 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f70cc004800 msgr2=0x7f70cc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:19.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.209+0000 7f70e259c640 1 --2- 172.21.15.73:0/2794619136 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f70cc004800 0x7f70cc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:19.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.209+0000 7f70e259c640 1 -- 172.21.15.73:0/2794619136 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70cc003520 con 0x7f70cc0039b0 2024-04-26T20:51:19.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.210+0000 7f70e259c640 1 --2- 172.21.15.73:0/2794619136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70cc0039b0 0x7f70cc003dd0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f70d4009fc0 tx=0x7f70cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40d5dff52f6e64ec server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:19.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.210+0000 7f70e159a640 1 -- 172.21.15.73:0/2794619136 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d4009c10 con 0x7f70cc0039b0 2024-04-26T20:51:19.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.210+0000 7f70e159a640 1 -- 172.21.15.73:0/2794619136 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70d4036070 con 0x7f70cc0039b0 2024-04-26T20:51:19.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.210+0000 7f70e159a640 1 -- 172.21.15.73:0/2794619136 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d402d070 con 0x7f70cc0039b0 2024-04-26T20:51:19.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.211+0000 7f70e8c64640 1 -- 172.21.15.73:0/2794619136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70cc0039b0 msgr2=0x7f70cc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:19.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.211+0000 7f70e8c64640 1 --2- 172.21.15.73:0/2794619136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70cc0039b0 0x7f70cc003dd0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f70d4009fc0 tx=0x7f70cc005830 comp rx=0 tx=0).stop 2024-04-26T20:51:19.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.211+0000 7f70e2d9d640 1 -- 172.21.15.73:0/2794619136 reap_dead start 2024-04-26T20:51:19.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.211+0000 7f70e8c64640 1 -- 172.21.15.73:0/2794619136 shutdown_connections 2024-04-26T20:51:19.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.211+0000 7f70e8c64640 1 -- 172.21.15.73:0/2794619136 >> 172.21.15.73:0/2794619136 conn(0x7f70e40d2da0 msgr2=0x7f70e40d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:19.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.211+0000 7f70e8c64640 1 -- 172.21.15.73:0/2794619136 shutdown_connections 2024-04-26T20:51:19.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.211+0000 7f70e8c64640 1 -- 172.21.15.73:0/2794619136 wait complete. 2024-04-26T20:51:19.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.212+0000 7f70e8c64640 1 Processor -- start 2024-04-26T20:51:19.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.212+0000 7f70e8c64640 1 -- start start 2024-04-26T20:51:19.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.213+0000 7f70e8c64640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f70cc0039b0 0x7f70e414cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:19.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.213+0000 7f70e8c64640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70e40ca880 0x7f70e40c82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:19.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.213+0000 7f70e8c64640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f70e415e5c0 0x7f70e415ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:19.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.213+0000 7f70e8c64640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f70e4070890 con 0x7f70e40ca880 2024-04-26T20:51:19.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.213+0000 7f70e8c64640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f70e40d0470 con 0x7f70e415e5c0 2024-04-26T20:51:19.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.213+0000 7f70e8c64640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f70e40cdb00 con 0x7f70cc0039b0 2024-04-26T20:51:19.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.213+0000 7f70e1d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70e40ca880 0x7f70e40c82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:19.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.213+0000 7f70e1d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70e40ca880 0x7f70e40c82e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52076/0 (socket says 172.21.15.73:52076) 2024-04-26T20:51:19.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.213+0000 7f70e1d9b640 1 -- 172.21.15.73:0/2916340430 learned_addr learned my addr 172.21.15.73:0/2916340430 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:19.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.213+0000 7f70e259c640 1 --2- 172.21.15.73:0/2916340430 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f70cc0039b0 0x7f70e414cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:19.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.213+0000 7f70e2d9d640 1 --2- 172.21.15.73:0/2916340430 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f70e415e5c0 0x7f70e415ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:19.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.213+0000 7f70e1d9b640 1 -- 172.21.15.73:0/2916340430 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f70cc0039b0 msgr2=0x7f70e414cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:19.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.213+0000 7f70e1d9b640 1 --2- 172.21.15.73:0/2916340430 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f70cc0039b0 0x7f70e414cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:19.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.213+0000 7f70e1d9b640 1 -- 172.21.15.73:0/2916340430 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f70e415e5c0 msgr2=0x7f70e415ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:19.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.213+0000 7f70e1d9b640 1 --2- 172.21.15.73:0/2916340430 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f70e415e5c0 0x7f70e415ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:19.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.213+0000 7f70e1d9b640 1 -- 172.21.15.73:0/2916340430 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70e40c8b60 con 0x7f70e40ca880 2024-04-26T20:51:19.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.213+0000 7f70e1d9b640 1 --2- 172.21.15.73:0/2916340430 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70e40ca880 0x7f70e40c82e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f70d8007310 tx=0x7f70d8007710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:19.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.213+0000 7f70d2ffd640 1 -- 172.21.15.73:0/2916340430 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d800f9c0 con 0x7f70e40ca880 2024-04-26T20:51:19.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.213+0000 7f70d2ffd640 1 -- 172.21.15.73:0/2916340430 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70d8018070 con 0x7f70e40ca880 2024-04-26T20:51:19.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.213+0000 7f70e8c64640 1 -- 172.21.15.73:0/2916340430 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70e40c8e40 con 0x7f70e40ca880 2024-04-26T20:51:19.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.213+0000 7f70d2ffd640 1 -- 172.21.15.73:0/2916340430 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d800fde0 con 0x7f70e40ca880 2024-04-26T20:51:19.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.214+0000 7f70e8c64640 1 -- 172.21.15.73:0/2916340430 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70e40c8820 con 0x7f70e40ca880 2024-04-26T20:51:19.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.215+0000 7f70e8c64640 1 -- 172.21.15.73:0/2916340430 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70e40694f0 con 0x7f70e40ca880 2024-04-26T20:51:19.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.215+0000 7f70d2ffd640 1 -- 172.21.15.73:0/2916340430 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f70d801d460 con 0x7f70e40ca880 2024-04-26T20:51:19.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.215+0000 7f70d2ffd640 1 --2- 172.21.15.73:0/2916340430 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f70c406c8f0 0x7f70c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:19.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.215+0000 7f70d2ffd640 1 -- 172.21.15.73:0/2916340430 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f70d808beb0 con 0x7f70e40ca880 2024-04-26T20:51:19.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.215+0000 7f70e259c640 1 --2- 172.21.15.73:0/2916340430 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f70c406c8f0 0x7f70c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:19.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.216+0000 7f70e259c640 1 --2- 172.21.15.73:0/2916340430 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f70c406c8f0 0x7f70c406edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f70d4035000 tx=0x7f70d402f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:19.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.220+0000 7f70d2ffd640 1 -- 172.21.15.73:0/2916340430 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f70d8059f70 con 0x7f70e40ca880 2024-04-26T20:51:19.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.395+0000 7f70e8c64640 1 -- 172.21.15.73:0/2916340430 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f70e40c9b10 con 0x7f70e40ca880 2024-04-26T20:51:19.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.396+0000 7f70d2ffd640 1 -- 172.21.15.73:0/2916340430 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18902 (secure 0 0 0) 0x7f70d8059870 con 0x7f70e40ca880 2024-04-26T20:51:19.398 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:51:19.398 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":27,"fsid":"fbd3f548-b547-4713-856b-09f98f5d1764","created":"2024-04-26T20:50:07.876786+0000","modified":"2024-04-26T20:51:13.783133+0000","last_up_change":"2024-04-26T20:50:19.082741+0000","last_in_change":"2024-04-26T20:50:10.018400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:50:19.993731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:50:29.964420+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:50:30.920082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:50:34.650207+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"72d6f4f3-214a-4392-a02f-c33cbcab3c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6809","nonce":3461705124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6811","nonce":3461705124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6815","nonce":3461705124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6813","nonce":3461705124}]},"public_addr":"172.21.15.73:6809/3461705124","cluster_addr":"172.21.15.73:6811/3461705124","heartbeat_back_addr":"172.21.15.73:6815/3461705124","heartbeat_front_addr":"172.21.15.73:6813/3461705124","state":["exists","up"]},{"osd":1,"uuid":"786bf8ca-6ddc-4b3c-be6c-9152398bd5fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6817","nonce":2827619403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6819","nonce":2827619403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6825","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6827","nonce":2827619403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6821","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6823","nonce":2827619403}]},"public_addr":"172.21.15.73:6817/2827619403","cluster_addr":"172.21.15.73:6819/2827619403","heartbeat_back_addr":"172.21.15.73:6827/2827619403","heartbeat_front_addr":"172.21.15.73:6823/2827619403","state":["exists","up"]},{"osd":2,"uuid":"90963b1a-5437-4ef8-ad8b-6b74c534eeb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6801","nonce":2542681064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6803","nonce":2542681064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6808","nonce":2542681064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6805","nonce":2542681064}]},"public_addr":"172.21.15.73:6801/2542681064","cluster_addr":"172.21.15.73:6803/2542681064","heartbeat_back_addr":"172.21.15.73:6808/2542681064","heartbeat_front_addr":"172.21.15.73:6805/2542681064","state":["exists","up"]},{"osd":3,"uuid":"bd3d2148-6e38-41b4-a453-fe6e3b8b4ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6822","nonce":1604042718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6826","nonce":1604042718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6831","nonce":1604042718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6829","nonce":1604042718}]},"public_addr":"172.21.15.73:6822/1604042718","cluster_addr":"172.21.15.73:6826/1604042718","heartbeat_back_addr":"172.21.15.73:6831/1604042718","heartbeat_front_addr":"172.21.15.73:6829/1604042718","state":["exists","up"]},{"osd":4,"uuid":"e8668dc8-a23f-46d2-8311-cb882da08e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6817","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6819","nonce":3591055561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6824","nonce":3591055561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6831","nonce":3591055561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6828","nonce":3591055561}]},"public_addr":"172.21.15.120:6819/3591055561","cluster_addr":"172.21.15.120:6824/3591055561","heartbeat_back_addr":"172.21.15.120:6831/3591055561","heartbeat_front_addr":"172.21.15.120:6828/3591055561","state":["exists","up"]},{"osd":5,"uuid":"cbac0bdb-96bd-4852-bcce-e564495c3fcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6809","nonce":819450433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6811","nonce":819450433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6815","nonce":819450433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6813","nonce":819450433}]},"public_addr":"172.21.15.120:6809/819450433","cluster_addr":"172.21.15.120:6811/819450433","heartbeat_back_addr":"172.21.15.120:6815/819450433","heartbeat_front_addr":"172.21.15.120:6813/819450433","state":["exists","up"]},{"osd":6,"uuid":"1884b2a8-569f-4272-96a9-f25a09364dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6801","nonce":3946210727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6803","nonce":3946210727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6807","nonce":3946210727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6805","nonce":3946210727}]},"public_addr":"172.21.15.120:6801/3946210727","cluster_addr":"172.21.15.120:6803/3946210727","heartbeat_back_addr":"172.21.15.120:6807/3946210727","heartbeat_front_addr":"172.21.15.120:6805/3946210727","state":["exists","up"]},{"osd":7,"uuid":"c0c3bf6d-a4c5-4f1e-aa2d-330951830666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6818","nonce":2915574674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6821","nonce":2915574674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6827","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6829","nonce":2915574674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6823","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6825","nonce":2915574674}]},"public_addr":"172.21.15.120:6818/2915574674","cluster_addr":"172.21.15.120:6821/2915574674","heartbeat_back_addr":"172.21.15.120:6829/2915574674","heartbeat_front_addr":"172.21.15.120:6825/2915574674","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.218216+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.208710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.112705+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.182789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.164324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.174212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.225396+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.261559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:51:19.399 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.398+0000 7f70e8c64640 1 -- 172.21.15.73:0/2916340430 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f70c406c8f0 msgr2=0x7f70c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:19.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.398+0000 7f70e8c64640 1 --2- 172.21.15.73:0/2916340430 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f70c406c8f0 0x7f70c406edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f70d4035000 tx=0x7f70d402f000 comp rx=0 tx=0).stop 2024-04-26T20:51:19.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.398+0000 7f70e8c64640 1 -- 172.21.15.73:0/2916340430 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70e40ca880 msgr2=0x7f70e40c82e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:19.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.398+0000 7f70e8c64640 1 --2- 172.21.15.73:0/2916340430 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70e40ca880 0x7f70e40c82e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f70d8007310 tx=0x7f70d8007710 comp rx=0 tx=0).stop 2024-04-26T20:51:19.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.398+0000 7f70e8c64640 1 -- 172.21.15.73:0/2916340430 shutdown_connections 2024-04-26T20:51:19.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.398+0000 7f70e8c64640 1 --2- 172.21.15.73:0/2916340430 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70e40ca880 0x7f70e40c82e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:19.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.398+0000 7f70e8c64640 1 --2- 172.21.15.73:0/2916340430 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f70c406c8f0 0x7f70c406edb0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:19.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.398+0000 7f70e8c64640 1 --2- 172.21.15.73:0/2916340430 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f70e415e5c0 0x7f70e415ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:19.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.398+0000 7f70e8c64640 1 --2- 172.21.15.73:0/2916340430 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f70cc0039b0 0x7f70e414cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:19.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.398+0000 7f70e8c64640 1 -- 172.21.15.73:0/2916340430 >> 172.21.15.73:0/2916340430 conn(0x7f70e40d2da0 msgr2=0x7f70e4070b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:19.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.398+0000 7f70e8c64640 1 -- 172.21.15.73:0/2916340430 shutdown_connections 2024-04-26T20:51:19.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.398+0000 7f70e8c64640 1 -- 172.21.15.73:0/2916340430 wait complete. 2024-04-26T20:51:19.411 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:50:19.993731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:50:29.964420+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:50:30.920082+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T20:50:34.650207+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T20:51:19.411 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:51:19.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.510+0000 7fc4563e9640 1 Processor -- start 2024-04-26T20:51:19.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.510+0000 7fc4563e9640 1 -- start start 2024-04-26T20:51:19.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.511+0000 7fc4563e9640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fc450070350 0x7fc450070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:19.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.511+0000 7fc4563e9640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc450071090 0x7fc45015e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:19.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.511+0000 7fc4563e9640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4500d2390 con 0x7fc4500cfde0 2024-04-26T20:51:19.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.511+0000 7fc4563e9640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7fc4500afab0 con 0x7fc450071090 2024-04-26T20:51:19.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.511+0000 7fc4563e9640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fc45006dd20 con 0x7fc450070350 2024-04-26T20:51:19.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.511+0000 7fc454986640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc450071090 0x7fc45015e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:19.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.511+0000 7fc454986640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc450071090 0x7fc45015e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59752/0 (socket says 172.21.15.73:59752) 2024-04-26T20:51:19.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.511+0000 7fc454986640 1 -- 172.21.15.73:0/1405262429 learned_addr learned my addr 172.21.15.73:0/1405262429 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:19.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.511+0000 7fc454986640 1 -- 172.21.15.73:0/1405262429 >> v2:172.21.15.120:3300/0 conn(0x7fc450070350 msgr2=0x7fc450070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:19.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.511+0000 7fc454986640 1 --2- 172.21.15.73:0/1405262429 >> v2:172.21.15.120:3300/0 conn(0x7fc450070350 0x7fc450070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:19.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.511+0000 7fc454986640 1 -- 172.21.15.73:0/1405262429 >> v1:172.21.15.120:6789/0 conn(0x7fc4500cfde0 legacy=0x7fc4500d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:19.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.511+0000 7fc454986640 1 -- 172.21.15.73:0/1405262429 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc45006d3f0 con 0x7fc450071090 2024-04-26T20:51:19.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.512+0000 7fc454986640 1 --2- 172.21.15.73:0/1405262429 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc450071090 0x7fc45015e9f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc45006cb80 tx=0x7fc44802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f0b6ba1516122ae server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:19.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.512+0000 7fc44effd640 1 -- 172.21.15.73:0/1405262429 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc448036070 con 0x7fc450071090 2024-04-26T20:51:19.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.512+0000 7fc44effd640 1 -- 172.21.15.73:0/1405262429 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc448031600 con 0x7fc450071090 2024-04-26T20:51:19.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.512+0000 7fc44effd640 1 -- 172.21.15.73:0/1405262429 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4480319d0 con 0x7fc450071090 2024-04-26T20:51:19.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.513+0000 7fc4563e9640 1 -- 172.21.15.73:0/1405262429 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc450071090 msgr2=0x7fc45015e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:19.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.513+0000 7fc4563e9640 1 --2- 172.21.15.73:0/1405262429 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc450071090 0x7fc45015e9f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc45006cb80 tx=0x7fc44802a660 comp rx=0 tx=0).stop 2024-04-26T20:51:19.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.513+0000 7fc4563e9640 1 -- 172.21.15.73:0/1405262429 shutdown_connections 2024-04-26T20:51:19.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.513+0000 7fc4563e9640 1 --2- 172.21.15.73:0/1405262429 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc450071090 0x7fc45015e9f0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:19.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.513+0000 7fc4563e9640 1 --2- 172.21.15.73:0/1405262429 >> v2:172.21.15.120:3300/0 conn(0x7fc450070350 0x7fc450070b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:19.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.513+0000 7fc4563e9640 1 -- 172.21.15.73:0/1405262429 >> 172.21.15.73:0/1405262429 conn(0x7fc4500d2d80 msgr2=0x7fc4500d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:19.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.513+0000 7fc4563e9640 1 -- 172.21.15.73:0/1405262429 shutdown_connections 2024-04-26T20:51:19.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.513+0000 7fc4563e9640 1 -- 172.21.15.73:0/1405262429 wait complete. 2024-04-26T20:51:19.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.514+0000 7fc4563e9640 1 Processor -- start 2024-04-26T20:51:19.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.514+0000 7fc4563e9640 1 -- start start 2024-04-26T20:51:19.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.515+0000 7fc4563e9640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc450070350 0x7fc45014c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:19.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.515+0000 7fc4563e9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc450071090 0x7fc4500c71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:19.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.515+0000 7fc4563e9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4500cfde0 0x7fc4501619a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:19.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.515+0000 7fc4563e9640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc45006d9b0 con 0x7fc4500cfde0 2024-04-26T20:51:19.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.515+0000 7fc4563e9640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc4500714c0 con 0x7fc450071090 2024-04-26T20:51:19.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.515+0000 7fc4563e9640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fc45015ef30 con 0x7fc450070350 2024-04-26T20:51:19.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.515+0000 7fc454986640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4500cfde0 0x7fc4501619a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:19.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.515+0000 7fc454986640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4500cfde0 0x7fc4501619a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59762/0 (socket says 172.21.15.73:59762) 2024-04-26T20:51:19.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.515+0000 7fc454986640 1 -- 172.21.15.73:0/3305284174 learned_addr learned my addr 172.21.15.73:0/3305284174 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:19.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.515+0000 7fc44ffff640 1 --2- 172.21.15.73:0/3305284174 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc450070350 0x7fc45014c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:19.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.515+0000 7fc44f7fe640 1 --2- 172.21.15.73:0/3305284174 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc450071090 0x7fc4500c71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:19.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.515+0000 7fc454986640 1 -- 172.21.15.73:0/3305284174 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc450070350 msgr2=0x7fc45014c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:19.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.515+0000 7fc454986640 1 --2- 172.21.15.73:0/3305284174 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc450070350 0x7fc45014c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:19.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.515+0000 7fc454986640 1 -- 172.21.15.73:0/3305284174 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc450071090 msgr2=0x7fc4500c71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:19.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.515+0000 7fc454986640 1 --2- 172.21.15.73:0/3305284174 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc450071090 0x7fc4500c71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:19.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.515+0000 7fc454986640 1 -- 172.21.15.73:0/3305284174 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc450160be0 con 0x7fc4500cfde0 2024-04-26T20:51:19.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.515+0000 7fc44ffff640 1 --2- 172.21.15.73:0/3305284174 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc450070350 0x7fc45014c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:19.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.515+0000 7fc44f7fe640 1 --2- 172.21.15.73:0/3305284174 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc450071090 0x7fc4500c71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:19.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.515+0000 7fc454986640 1 --2- 172.21.15.73:0/3305284174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4500cfde0 0x7fc4501619a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc448034040 tx=0x7fc448004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:19.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.516+0000 7fc44cff9640 1 -- 172.21.15.73:0/3305284174 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc448041020 con 0x7fc4500cfde0 2024-04-26T20:51:19.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.516+0000 7fc44cff9640 1 -- 172.21.15.73:0/3305284174 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4480077a0 con 0x7fc4500cfde0 2024-04-26T20:51:19.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.516+0000 7fc4563e9640 1 -- 172.21.15.73:0/3305284174 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc450160ec0 con 0x7fc4500cfde0 2024-04-26T20:51:19.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.516+0000 7fc44cff9640 1 -- 172.21.15.73:0/3305284174 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc448036040 con 0x7fc4500cfde0 2024-04-26T20:51:19.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.516+0000 7fc4563e9640 1 -- 172.21.15.73:0/3305284174 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc450161480 con 0x7fc4500cfde0 2024-04-26T20:51:19.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.517+0000 7fc4563e9640 1 -- 172.21.15.73:0/3305284174 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4500694f0 con 0x7fc4500cfde0 2024-04-26T20:51:19.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.517+0000 7fc44cff9640 1 -- 172.21.15.73:0/3305284174 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc448044020 con 0x7fc4500cfde0 2024-04-26T20:51:19.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.518+0000 7fc44cff9640 1 --2- 172.21.15.73:0/3305284174 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc43c06c8f0 0x7fc43c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:19.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.518+0000 7fc44ffff640 1 --2- 172.21.15.73:0/3305284174 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc43c06c8f0 0x7fc43c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:19.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.518+0000 7fc44cff9640 1 -- 172.21.15.73:0/3305284174 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fc4480acee0 con 0x7fc4500cfde0 2024-04-26T20:51:19.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.518+0000 7fc44ffff640 1 --2- 172.21.15.73:0/3305284174 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc43c06c8f0 0x7fc43c06edb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc44000a810 tx=0x7fc440008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:19.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.522+0000 7fc44cff9640 1 -- 172.21.15.73:0/3305284174 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc44804b030 con 0x7fc4500cfde0 2024-04-26T20:51:19.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.698+0000 7fc4563e9640 1 -- 172.21.15.73:0/3305284174 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc450161650 con 0x7fc4500cfde0 2024-04-26T20:51:19.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.699+0000 7fc44cff9640 1 -- 172.21.15.73:0/3305284174 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7fc44807afd0 con 0x7fc4500cfde0 2024-04-26T20:51:19.700 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-26T20:51:19.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.701+0000 7fc4563e9640 1 -- 172.21.15.73:0/3305284174 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc43c06c8f0 msgr2=0x7fc43c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:19.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.701+0000 7fc4563e9640 1 --2- 172.21.15.73:0/3305284174 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc43c06c8f0 0x7fc43c06edb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc44000a810 tx=0x7fc440008040 comp rx=0 tx=0).stop 2024-04-26T20:51:19.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.701+0000 7fc4563e9640 1 -- 172.21.15.73:0/3305284174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4500cfde0 msgr2=0x7fc4501619a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:19.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.701+0000 7fc4563e9640 1 --2- 172.21.15.73:0/3305284174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4500cfde0 0x7fc4501619a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc448034040 tx=0x7fc448004a40 comp rx=0 tx=0).stop 2024-04-26T20:51:19.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.701+0000 7fc4563e9640 1 -- 172.21.15.73:0/3305284174 shutdown_connections 2024-04-26T20:51:19.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.701+0000 7fc4563e9640 1 --2- 172.21.15.73:0/3305284174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4500cfde0 0x7fc4501619a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:19.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.701+0000 7fc4563e9640 1 --2- 172.21.15.73:0/3305284174 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc43c06c8f0 0x7fc43c06edb0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:19.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.701+0000 7fc4563e9640 1 --2- 172.21.15.73:0/3305284174 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc450071090 0x7fc4500c71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:19.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.701+0000 7fc4563e9640 1 --2- 172.21.15.73:0/3305284174 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc450070350 0x7fc45014c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:19.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.701+0000 7fc4563e9640 1 -- 172.21.15.73:0/3305284174 >> 172.21.15.73:0/3305284174 conn(0x7fc4500d2d80 msgr2=0x7fc4500b1dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:19.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.701+0000 7fc4563e9640 1 -- 172.21.15.73:0/3305284174 shutdown_connections 2024-04-26T20:51:19.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.701+0000 7fc4563e9640 1 -- 172.21.15.73:0/3305284174 wait complete. 2024-04-26T20:51:19.713 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:51:19.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.812+0000 7efe7ddbd640 1 Processor -- start 2024-04-26T20:51:19.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.812+0000 7efe7ddbd640 1 -- start start 2024-04-26T20:51:19.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.813+0000 7efe7ddbd640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efe7815d100 0x7efe78160ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:19.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.813+0000 7efe7ddbd640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7efe78070930 0x7efe780aed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:19.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.813+0000 7efe7ddbd640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe780b0990 con 0x7efe7806c240 2024-04-26T20:51:19.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.813+0000 7efe7ddbd640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7efe780af2d0 con 0x7efe78070930 2024-04-26T20:51:19.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.813+0000 7efe7ddbd640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7efe780b0c70 con 0x7efe7815d100 2024-04-26T20:51:19.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.813+0000 7efe76ffd640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7efe78070930 0x7efe780aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:19.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.813+0000 7efe76ffd640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7efe78070930 0x7efe780aed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59776/0 (socket says 172.21.15.73:59776) 2024-04-26T20:51:19.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.813+0000 7efe76ffd640 1 -- 172.21.15.73:0/582464845 learned_addr learned my addr 172.21.15.73:0/582464845 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:19.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.813+0000 7efe777fe640 1 --2- 172.21.15.73:0/582464845 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efe7815d100 0x7efe78160ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:19.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.813+0000 7efe76ffd640 1 -- 172.21.15.73:0/582464845 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efe7815d100 msgr2=0x7efe78160ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:19.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.813+0000 7efe76ffd640 1 --2- 172.21.15.73:0/582464845 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efe7815d100 0x7efe78160ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:19.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.813+0000 7efe76ffd640 1 -- 172.21.15.73:0/582464845 >> v1:172.21.15.120:6789/0 conn(0x7efe7806c240 legacy=0x7efe78073160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:19.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.813+0000 7efe76ffd640 1 -- 172.21.15.73:0/582464845 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe780b0f50 con 0x7efe78070930 2024-04-26T20:51:19.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.813+0000 7efe777fe640 1 --2- 172.21.15.73:0/582464845 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efe7815d100 0x7efe78160ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:19.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.814+0000 7efe76ffd640 1 --2- 172.21.15.73:0/582464845 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe78070930 0x7efe780aed90 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7efe6c009870 tx=0x7efe6c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11e33df074624d05 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:19.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.814+0000 7efe767fc640 1 -- 172.21.15.73:0/582464845 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c037070 con 0x7efe78070930 2024-04-26T20:51:19.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.814+0000 7efe767fc640 1 -- 172.21.15.73:0/582464845 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe6c032440 con 0x7efe78070930 2024-04-26T20:51:19.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.814+0000 7efe767fc640 1 -- 172.21.15.73:0/582464845 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c032810 con 0x7efe78070930 2024-04-26T20:51:19.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.815+0000 7efe7ddbd640 1 -- 172.21.15.73:0/582464845 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe78070930 msgr2=0x7efe780aed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:19.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.815+0000 7efe7ddbd640 1 --2- 172.21.15.73:0/582464845 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe78070930 0x7efe780aed90 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7efe6c009870 tx=0x7efe6c02a270 comp rx=0 tx=0).stop 2024-04-26T20:51:19.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.815+0000 7efe7ddbd640 1 -- 172.21.15.73:0/582464845 shutdown_connections 2024-04-26T20:51:19.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.815+0000 7efe7ddbd640 1 --2- 172.21.15.73:0/582464845 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe78070930 0x7efe780aed90 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:19.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.815+0000 7efe7ddbd640 1 --2- 172.21.15.73:0/582464845 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efe7815d100 0x7efe78160ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:19.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.815+0000 7efe7ddbd640 1 -- 172.21.15.73:0/582464845 >> 172.21.15.73:0/582464845 conn(0x7efe780b39b0 msgr2=0x7efe780b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:19.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.815+0000 7efe7ddbd640 1 -- 172.21.15.73:0/582464845 shutdown_connections 2024-04-26T20:51:19.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.815+0000 7efe7ddbd640 1 -- 172.21.15.73:0/582464845 wait complete. 2024-04-26T20:51:19.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.816+0000 7efe7ddbd640 1 Processor -- start 2024-04-26T20:51:19.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.816+0000 7efe7ddbd640 1 -- start start 2024-04-26T20:51:19.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.817+0000 7efe7ddbd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe7806c240 0x7efe78141480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:19.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.817+0000 7efe7ddbd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe78070930 0x7efe78153d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:19.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.817+0000 7efe7ddbd640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efe7815d100 0x7efe7813abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:19.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.817+0000 7efe7ddbd640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efe7806c6d0 con 0x7efe78070930 2024-04-26T20:51:19.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.817+0000 7efe7ddbd640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efe780b1510 con 0x7efe7806c240 2024-04-26T20:51:19.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.817+0000 7efe7ddbd640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7efe7806c850 con 0x7efe7815d100 2024-04-26T20:51:19.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.817+0000 7efe76ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe78070930 0x7efe78153d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:19.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.817+0000 7efe76ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe78070930 0x7efe78153d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59778/0 (socket says 172.21.15.73:59778) 2024-04-26T20:51:19.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.817+0000 7efe76ffd640 1 -- 172.21.15.73:0/108739777 learned_addr learned my addr 172.21.15.73:0/108739777 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:19.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.817+0000 7efe777fe640 1 --2- 172.21.15.73:0/108739777 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe7806c240 0x7efe78141480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:19.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.817+0000 7efe77fff640 1 --2- 172.21.15.73:0/108739777 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efe7815d100 0x7efe7813abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:19.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.817+0000 7efe76ffd640 1 -- 172.21.15.73:0/108739777 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efe7815d100 msgr2=0x7efe7813abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:19.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.817+0000 7efe76ffd640 1 --2- 172.21.15.73:0/108739777 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efe7815d100 0x7efe7813abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:19.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.817+0000 7efe76ffd640 1 -- 172.21.15.73:0/108739777 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe7806c240 msgr2=0x7efe78141480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:19.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.817+0000 7efe76ffd640 1 --2- 172.21.15.73:0/108739777 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe7806c240 0x7efe78141480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:19.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.817+0000 7efe76ffd640 1 -- 172.21.15.73:0/108739777 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe7813b100 con 0x7efe78070930 2024-04-26T20:51:19.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.817+0000 7efe777fe640 1 --2- 172.21.15.73:0/108739777 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe7806c240 0x7efe78141480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:19.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.817+0000 7efe76ffd640 1 --2- 172.21.15.73:0/108739777 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe78070930 0x7efe78153d00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7efe6c000c00 tx=0x7efe6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:19.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.817+0000 7efe5ffff640 1 -- 172.21.15.73:0/108739777 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c041020 con 0x7efe78070930 2024-04-26T20:51:19.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.817+0000 7efe5ffff640 1 -- 172.21.15.73:0/108739777 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe6c0306c0 con 0x7efe78070930 2024-04-26T20:51:19.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.817+0000 7efe7ddbd640 1 -- 172.21.15.73:0/108739777 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe78157590 con 0x7efe78070930 2024-04-26T20:51:19.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.817+0000 7efe5ffff640 1 -- 172.21.15.73:0/108739777 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c037040 con 0x7efe78070930 2024-04-26T20:51:19.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.817+0000 7efe7ddbd640 1 -- 172.21.15.73:0/108739777 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe78153500 con 0x7efe78070930 2024-04-26T20:51:19.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.818+0000 7efe7ddbd640 1 -- 172.21.15.73:0/108739777 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe780694f0 con 0x7efe78070930 2024-04-26T20:51:19.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.819+0000 7efe5ffff640 1 -- 172.21.15.73:0/108739777 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efe6c044020 con 0x7efe78070930 2024-04-26T20:51:19.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.819+0000 7efe5ffff640 1 --2- 172.21.15.73:0/108739777 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7efe6406c8f0 0x7efe6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:19.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.819+0000 7efe777fe640 1 --2- 172.21.15.73:0/108739777 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7efe6406c8f0 0x7efe6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:19.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.819+0000 7efe5ffff640 1 -- 172.21.15.73:0/108739777 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7efe6c0ac820 con 0x7efe78070930 2024-04-26T20:51:19.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.820+0000 7efe777fe640 1 --2- 172.21.15.73:0/108739777 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7efe6406c8f0 0x7efe6406edb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7efe68006fd0 tx=0x7efe68008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:19.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:19.823+0000 7efe5ffff640 1 -- 172.21.15.73:0/108739777 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efe6c04b030 con 0x7efe78070930 2024-04-26T20:51:20.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.000+0000 7efe7ddbd640 1 -- 172.21.15.73:0/108739777 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7efe7815c710 con 0x7efe78070930 2024-04-26T20:51:20.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.000+0000 7efe5ffff640 1 -- 172.21.15.73:0/108739777 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7efe7813b100 con 0x7efe78070930 2024-04-26T20:51:20.001 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-26T20:51:20.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.002+0000 7efe7ddbd640 1 -- 172.21.15.73:0/108739777 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7efe6406c8f0 msgr2=0x7efe6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:20.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.002+0000 7efe7ddbd640 1 --2- 172.21.15.73:0/108739777 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7efe6406c8f0 0x7efe6406edb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7efe68006fd0 tx=0x7efe68008040 comp rx=0 tx=0).stop 2024-04-26T20:51:20.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.002+0000 7efe7ddbd640 1 -- 172.21.15.73:0/108739777 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe78070930 msgr2=0x7efe78153d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:20.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.002+0000 7efe7ddbd640 1 --2- 172.21.15.73:0/108739777 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe78070930 0x7efe78153d00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7efe6c000c00 tx=0x7efe6c0046c0 comp rx=0 tx=0).stop 2024-04-26T20:51:20.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.002+0000 7efe7ddbd640 1 -- 172.21.15.73:0/108739777 shutdown_connections 2024-04-26T20:51:20.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.002+0000 7efe7ddbd640 1 --2- 172.21.15.73:0/108739777 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe78070930 0x7efe78153d00 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:20.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.002+0000 7efe7ddbd640 1 --2- 172.21.15.73:0/108739777 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7efe6406c8f0 0x7efe6406edb0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:20.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.002+0000 7efe7ddbd640 1 --2- 172.21.15.73:0/108739777 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efe7815d100 0x7efe7813abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:20.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.002+0000 7efe7ddbd640 1 --2- 172.21.15.73:0/108739777 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe7806c240 0x7efe78141480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:20.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.002+0000 7efe7ddbd640 1 -- 172.21.15.73:0/108739777 >> 172.21.15.73:0/108739777 conn(0x7efe780b39b0 msgr2=0x7efe78160da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:20.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.002+0000 7efe7ddbd640 1 -- 172.21.15.73:0/108739777 shutdown_connections 2024-04-26T20:51:20.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.002+0000 7efe7ddbd640 1 -- 172.21.15.73:0/108739777 wait complete. 2024-04-26T20:51:20.014 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:51:20.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.113+0000 7f17ec996640 1 Processor -- start 2024-04-26T20:51:20.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.113+0000 7f17ec996640 1 -- start start 2024-04-26T20:51:20.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.114+0000 7f17ec996640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f17e8071e60 0x7f17e806f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:20.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.114+0000 7f17ec996640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f17e806fe20 0x7f17e8072240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:20.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.114+0000 7f17ec996640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17e815d7c0 con 0x7f17e8072780 2024-04-26T20:51:20.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.114+0000 7f17ec996640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f17e815daa0 con 0x7f17e806fe20 2024-04-26T20:51:20.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.114+0000 7f17ec996640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f17e815dd80 con 0x7f17e8071e60 2024-04-26T20:51:20.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.114+0000 7f17e6d9d640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f17e8072780 0x7f17e80cb470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:34504/0 (socket says 172.21.15.73:34504) 2024-04-26T20:51:20.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.114+0000 7f17e6d9d640 1 -- 172.21.15.73:0/1674467488 learned_addr learned my addr 172.21.15.73:0/1674467488 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:20.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.115+0000 7f17e559a640 1 -- 172.21.15.73:0/1674467488 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3719305451 0 0) 0x7f17e815d7c0 con 0x7f17e8072780 2024-04-26T20:51:20.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.115+0000 7f17e5d9b640 1 --2- 172.21.15.73:0/1674467488 >> v2:172.21.15.120:3300/0 conn(0x7f17e806fe20 0x7f17e8072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:20.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.115+0000 7f17e659c640 1 --2- 172.21.15.73:0/1674467488 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f17e8071e60 0x7f17e806f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:20.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.115+0000 7f17e559a640 1 -- 172.21.15.73:0/1674467488 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f17d0003520 con 0x7f17e8072780 2024-04-26T20:51:20.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.115+0000 7f17e559a640 1 -- 172.21.15.73:0/1674467488 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3185156648 0 0) 0x7f17d0003520 con 0x7f17e8072780 2024-04-26T20:51:20.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.115+0000 7f17e559a640 1 -- 172.21.15.73:0/1674467488 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f17e8071e60 msgr2=0x7f17e806f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:20.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.115+0000 7f17e559a640 1 --2- 172.21.15.73:0/1674467488 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f17e8071e60 0x7f17e806f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:20.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.115+0000 7f17e559a640 1 -- 172.21.15.73:0/1674467488 >> v2:172.21.15.120:3300/0 conn(0x7f17e806fe20 msgr2=0x7f17e8072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:20.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.115+0000 7f17e559a640 1 --2- 172.21.15.73:0/1674467488 >> v2:172.21.15.120:3300/0 conn(0x7f17e806fe20 0x7f17e8072240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:20.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.115+0000 7f17e559a640 1 -- 172.21.15.73:0/1674467488 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17e815e060 con 0x7f17e8072780 2024-04-26T20:51:20.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.115+0000 7f17e559a640 1 -- 172.21.15.73:0/1674467488 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7f17e0001670 con 0x7f17e8072780 2024-04-26T20:51:20.119 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.115+0000 7f17e5d9b640 1 --2- 172.21.15.73:0/1674467488 >> v2:172.21.15.120:3300/0 conn(0x7f17e806fe20 0x7f17e8072240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:20.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.115+0000 7f17e559a640 1 -- 172.21.15.73:0/1674467488 >> v1:172.21.15.73:6789/0 conn(0x7f17e8072780 legacy=0x7f17e80cb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:20.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.115+0000 7f17e559a640 1 --2- 172.21.15.73:0/1674467488 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17d00039b0 0x7f17d0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:20.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.115+0000 7f17e559a640 1 --2- 172.21.15.73:0/1674467488 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17d0004800 0x7f17d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:20.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.115+0000 7f17e559a640 1 --2- 172.21.15.73:0/1674467488 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f17e8071e60 0x7f17d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:20.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.115+0000 7f17e559a640 1 -- 172.21.15.73:0/1674467488 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f17e815daa0 con 0x7f17d00039b0 2024-04-26T20:51:20.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.115+0000 7f17e559a640 1 -- 172.21.15.73:0/1674467488 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f17e815dd80 con 0x7f17d0004800 2024-04-26T20:51:20.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.115+0000 7f17e559a640 1 -- 172.21.15.73:0/1674467488 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f17d000ba70 con 0x7f17e8071e60 2024-04-26T20:51:20.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.115+0000 7f17e659c640 1 --2- 172.21.15.73:0/1674467488 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17d00039b0 0x7f17d0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:20.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.115+0000 7f17e5d9b640 1 --2- 172.21.15.73:0/1674467488 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17d0004800 0x7f17d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:20.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.115+0000 7f17e6d9d640 1 --2- 172.21.15.73:0/1674467488 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f17e8071e60 0x7f17d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:20.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.115+0000 7f17e659c640 1 -- 172.21.15.73:0/1674467488 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f17e8071e60 msgr2=0x7f17d00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:20.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.115+0000 7f17e659c640 1 --2- 172.21.15.73:0/1674467488 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f17e8071e60 0x7f17d00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:20.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.115+0000 7f17e659c640 1 -- 172.21.15.73:0/1674467488 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17d0004800 msgr2=0x7f17d0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:20.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.115+0000 7f17e659c640 1 --2- 172.21.15.73:0/1674467488 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17d0004800 0x7f17d0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:20.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.115+0000 7f17e659c640 1 -- 172.21.15.73:0/1674467488 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17d0003520 con 0x7f17d00039b0 2024-04-26T20:51:20.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.115+0000 7f17e6d9d640 1 --2- 172.21.15.73:0/1674467488 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f17e8071e60 0x7f17d00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:20.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.116+0000 7f17e659c640 1 --2- 172.21.15.73:0/1674467488 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17d00039b0 0x7f17d0003dd0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f17d8007330 tx=0x7f17d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22e43fece2b26f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:20.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.116+0000 7f17e559a640 1 -- 172.21.15.73:0/1674467488 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d8033070 con 0x7f17d00039b0 2024-04-26T20:51:20.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.116+0000 7f17e559a640 1 -- 172.21.15.73:0/1674467488 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17d802f630 con 0x7f17d00039b0 2024-04-26T20:51:20.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.116+0000 7f17e559a640 1 -- 172.21.15.73:0/1674467488 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d8009be0 con 0x7f17d00039b0 2024-04-26T20:51:20.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.117+0000 7f17ec996640 1 -- 172.21.15.73:0/1674467488 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17d00039b0 msgr2=0x7f17d0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:20.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.117+0000 7f17ec996640 1 --2- 172.21.15.73:0/1674467488 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17d00039b0 0x7f17d0003dd0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f17d8007330 tx=0x7f17d0005830 comp rx=0 tx=0).stop 2024-04-26T20:51:20.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.117+0000 7f17e6d9d640 1 -- 172.21.15.73:0/1674467488 reap_dead start 2024-04-26T20:51:20.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.117+0000 7f17ec996640 1 -- 172.21.15.73:0/1674467488 shutdown_connections 2024-04-26T20:51:20.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.117+0000 7f17ec996640 1 -- 172.21.15.73:0/1674467488 >> 172.21.15.73:0/1674467488 conn(0x7f17e80d0be0 msgr2=0x7f17e80d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:20.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.117+0000 7f17ec996640 1 -- 172.21.15.73:0/1674467488 shutdown_connections 2024-04-26T20:51:20.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.117+0000 7f17ec996640 1 -- 172.21.15.73:0/1674467488 wait complete. 2024-04-26T20:51:20.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.118+0000 7f17ec996640 1 Processor -- start 2024-04-26T20:51:20.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.118+0000 7f17ec996640 1 -- start start 2024-04-26T20:51:20.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.119+0000 7f17ec996640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f17d00039b0 0x7f17e815a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:20.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.119+0000 7f17ec996640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17e80c9f80 0x7f17e815c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:20.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.119+0000 7f17ec996640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17e81f5610 0x7f17e81f5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:20.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.119+0000 7f17ec996640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f17e815d3f0 con 0x7f17e81f5610 2024-04-26T20:51:20.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.119+0000 7f17ec996640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f17e8072c10 con 0x7f17e80c9f80 2024-04-26T20:51:20.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.119+0000 7f17ec996640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f17e815d570 con 0x7f17d00039b0 2024-04-26T20:51:20.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.119+0000 7f17e6d9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17e81f5610 0x7f17e81f5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:20.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.119+0000 7f17e6d9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17e81f5610 0x7f17e81f5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59804/0 (socket says 172.21.15.73:59804) 2024-04-26T20:51:20.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.119+0000 7f17e6d9d640 1 -- 172.21.15.73:0/995765396 learned_addr learned my addr 172.21.15.73:0/995765396 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:20.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.119+0000 7f17e659c640 1 --2- 172.21.15.73:0/995765396 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f17d00039b0 0x7f17e815a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:20.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.119+0000 7f17e5d9b640 1 --2- 172.21.15.73:0/995765396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17e80c9f80 0x7f17e815c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:20.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.119+0000 7f17e6d9d640 1 -- 172.21.15.73:0/995765396 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f17d00039b0 msgr2=0x7f17e815a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:20.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.119+0000 7f17e6d9d640 1 --2- 172.21.15.73:0/995765396 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f17d00039b0 0x7f17e815a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:20.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.119+0000 7f17e6d9d640 1 -- 172.21.15.73:0/995765396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17e80c9f80 msgr2=0x7f17e815c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:20.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.119+0000 7f17e6d9d640 1 --2- 172.21.15.73:0/995765396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17e80c9f80 0x7f17e815c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:20.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.119+0000 7f17e6d9d640 1 -- 172.21.15.73:0/995765396 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17e815aa90 con 0x7f17e81f5610 2024-04-26T20:51:20.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.119+0000 7f17e5d9b640 1 --2- 172.21.15.73:0/995765396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17e80c9f80 0x7f17e815c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:20.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.119+0000 7f17e6d9d640 1 --2- 172.21.15.73:0/995765396 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17e81f5610 0x7f17e81f5a70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f17e8072b60 tx=0x7f17e000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:20.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.119+0000 7f17d6ffd640 1 -- 172.21.15.73:0/995765396 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e000c5b0 con 0x7f17e81f5610 2024-04-26T20:51:20.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.120+0000 7f17d6ffd640 1 -- 172.21.15.73:0/995765396 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17e0018070 con 0x7f17e81f5610 2024-04-26T20:51:20.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.120+0000 7f17ec996640 1 -- 172.21.15.73:0/995765396 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17e815ad70 con 0x7f17e81f5610 2024-04-26T20:51:20.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.120+0000 7f17d6ffd640 1 -- 172.21.15.73:0/995765396 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e000c9b0 con 0x7f17e81f5610 2024-04-26T20:51:20.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.120+0000 7f17ec996640 1 -- 172.21.15.73:0/995765396 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17e815b330 con 0x7f17e81f5610 2024-04-26T20:51:20.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.120+0000 7f17ec996640 1 -- 172.21.15.73:0/995765396 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17e80694f0 con 0x7f17e81f5610 2024-04-26T20:51:20.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.121+0000 7f17d6ffd640 1 -- 172.21.15.73:0/995765396 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f17e000cb50 con 0x7f17e81f5610 2024-04-26T20:51:20.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.121+0000 7f17d6ffd640 1 --2- 172.21.15.73:0/995765396 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f17c806c8f0 0x7f17c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:20.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.121+0000 7f17d6ffd640 1 -- 172.21.15.73:0/995765396 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f17e008ba40 con 0x7f17e81f5610 2024-04-26T20:51:20.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.121+0000 7f17e659c640 1 --2- 172.21.15.73:0/995765396 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f17c806c8f0 0x7f17c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:20.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.122+0000 7f17e659c640 1 --2- 172.21.15.73:0/995765396 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f17c806c8f0 0x7f17c806edb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f17d8035000 tx=0x7f17d800b030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:20.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.126+0000 7f17d6ffd640 1 -- 172.21.15.73:0/995765396 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f17e0059b00 con 0x7f17e81f5610 2024-04-26T20:51:20.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.302+0000 7f17ec996640 1 -- 172.21.15.73:0/995765396 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f17e80cc420 con 0x7f17e81f5610 2024-04-26T20:51:20.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.302+0000 7f17d6ffd640 1 -- 172.21.15.73:0/995765396 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f17e008be20 con 0x7f17e81f5610 2024-04-26T20:51:20.303 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-26T20:51:20.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.304+0000 7f17ec996640 1 -- 172.21.15.73:0/995765396 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f17c806c8f0 msgr2=0x7f17c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:20.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.304+0000 7f17ec996640 1 --2- 172.21.15.73:0/995765396 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f17c806c8f0 0x7f17c806edb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f17d8035000 tx=0x7f17d800b030 comp rx=0 tx=0).stop 2024-04-26T20:51:20.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.304+0000 7f17ec996640 1 -- 172.21.15.73:0/995765396 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17e81f5610 msgr2=0x7f17e81f5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:20.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.304+0000 7f17ec996640 1 --2- 172.21.15.73:0/995765396 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17e81f5610 0x7f17e81f5a70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f17e8072b60 tx=0x7f17e000eb90 comp rx=0 tx=0).stop 2024-04-26T20:51:20.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.304+0000 7f17ec996640 1 -- 172.21.15.73:0/995765396 shutdown_connections 2024-04-26T20:51:20.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.304+0000 7f17ec996640 1 --2- 172.21.15.73:0/995765396 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17e81f5610 0x7f17e81f5a70 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:20.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.304+0000 7f17ec996640 1 --2- 172.21.15.73:0/995765396 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f17c806c8f0 0x7f17c806edb0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:20.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.304+0000 7f17ec996640 1 --2- 172.21.15.73:0/995765396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f17e80c9f80 0x7f17e815c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:20.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.304+0000 7f17ec996640 1 --2- 172.21.15.73:0/995765396 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f17d00039b0 0x7f17e815a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:20.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.304+0000 7f17ec996640 1 -- 172.21.15.73:0/995765396 >> 172.21.15.73:0/995765396 conn(0x7f17e80d0be0 msgr2=0x7f17e80b06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:20.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.304+0000 7f17ec996640 1 -- 172.21.15.73:0/995765396 shutdown_connections 2024-04-26T20:51:20.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.304+0000 7f17ec996640 1 -- 172.21.15.73:0/995765396 wait complete. 2024-04-26T20:51:20.316 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T20:51:20.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.415+0000 7f5fefa93640 1 Processor -- start 2024-04-26T20:51:20.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.415+0000 7f5fefa93640 1 -- start start 2024-04-26T20:51:20.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.416+0000 7f5fefa93640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f5fe80b0380 0x7f5fe80ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:20.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.416+0000 7f5fefa93640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5fe806c950 0x7f5fe815e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:20.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.416+0000 7f5fefa93640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fe80cfd80 con 0x7f5fe80b10b0 2024-04-26T20:51:20.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.416+0000 7f5fefa93640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f5fe80af8e0 con 0x7f5fe806c950 2024-04-26T20:51:20.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.416+0000 7f5fefa93640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f5fe80d0020 con 0x7f5fe80b0380 2024-04-26T20:51:20.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.416+0000 7f5fee030640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5fe806c950 0x7f5fe815e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:20.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.416+0000 7f5fee030640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5fe806c950 0x7f5fe815e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59812/0 (socket says 172.21.15.73:59812) 2024-04-26T20:51:20.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.416+0000 7f5fee030640 1 -- 172.21.15.73:0/3059443418 learned_addr learned my addr 172.21.15.73:0/3059443418 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:20.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.416+0000 7f5fed82f640 1 --2- 172.21.15.73:0/3059443418 >> v2:172.21.15.120:3300/0 conn(0x7f5fe80b0380 0x7f5fe80ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:20.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.416+0000 7f5fee030640 1 -- 172.21.15.73:0/3059443418 >> v2:172.21.15.120:3300/0 conn(0x7f5fe80b0380 msgr2=0x7f5fe80ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:20.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.416+0000 7f5fee030640 1 --2- 172.21.15.73:0/3059443418 >> v2:172.21.15.120:3300/0 conn(0x7f5fe80b0380 0x7f5fe80ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:20.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.416+0000 7f5fee030640 1 -- 172.21.15.73:0/3059443418 >> v1:172.21.15.120:6789/0 conn(0x7f5fe80b10b0 legacy=0x7f5fe806c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:20.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.416+0000 7f5fee030640 1 -- 172.21.15.73:0/3059443418 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fe80d0300 con 0x7f5fe806c950 2024-04-26T20:51:20.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.416+0000 7f5fed82f640 1 --2- 172.21.15.73:0/3059443418 >> v2:172.21.15.120:3300/0 conn(0x7f5fe80b0380 0x7f5fe80ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:20.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.417+0000 7f5fee030640 1 --2- 172.21.15.73:0/3059443418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5fe806c950 0x7f5fe815e970 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8072f00 tx=0x7f5fe002a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c620ee8a9f992f41 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:20.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.417+0000 7f5fec82d640 1 -- 172.21.15.73:0/3059443418 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe0037070 con 0x7f5fe806c950 2024-04-26T20:51:20.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.417+0000 7f5fec82d640 1 -- 172.21.15.73:0/3059443418 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fe0032440 con 0x7f5fe806c950 2024-04-26T20:51:20.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.417+0000 7f5fec82d640 1 -- 172.21.15.73:0/3059443418 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe0032810 con 0x7f5fe806c950 2024-04-26T20:51:20.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.418+0000 7f5fefa93640 1 -- 172.21.15.73:0/3059443418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5fe806c950 msgr2=0x7f5fe815e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:20.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.418+0000 7f5fefa93640 1 --2- 172.21.15.73:0/3059443418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5fe806c950 0x7f5fe815e970 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8072f00 tx=0x7f5fe002a160 comp rx=0 tx=0).stop 2024-04-26T20:51:20.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.418+0000 7f5fefa93640 1 -- 172.21.15.73:0/3059443418 shutdown_connections 2024-04-26T20:51:20.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.418+0000 7f5fefa93640 1 --2- 172.21.15.73:0/3059443418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5fe806c950 0x7f5fe815e970 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:20.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.418+0000 7f5fefa93640 1 --2- 172.21.15.73:0/3059443418 >> v2:172.21.15.120:3300/0 conn(0x7f5fe80b0380 0x7f5fe80ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:20.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.418+0000 7f5fefa93640 1 -- 172.21.15.73:0/3059443418 >> 172.21.15.73:0/3059443418 conn(0x7f5fe80d1cf0 msgr2=0x7f5fe80d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:20.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.418+0000 7f5fefa93640 1 -- 172.21.15.73:0/3059443418 shutdown_connections 2024-04-26T20:51:20.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.418+0000 7f5fefa93640 1 -- 172.21.15.73:0/3059443418 wait complete. 2024-04-26T20:51:20.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.419+0000 7f5fefa93640 1 Processor -- start 2024-04-26T20:51:20.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.419+0000 7f5fefa93640 1 -- start start 2024-04-26T20:51:20.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.420+0000 7f5fefa93640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5fe806c950 0x7f5fe80c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:20.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.420+0000 7f5fefa93640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5fe80b0380 0x7f5fe80b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:20.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.420+0000 7f5fefa93640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5fe80b10b0 0x7f5fe81550e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:20.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.420+0000 7f5fefa93640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe80b0760 con 0x7f5fe80b0380 2024-04-26T20:51:20.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.420+0000 7f5fefa93640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe80d1980 con 0x7f5fe806c950 2024-04-26T20:51:20.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.420+0000 7f5fefa93640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f5fe80d15c0 con 0x7f5fe80b10b0 2024-04-26T20:51:20.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.420+0000 7f5fed02e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5fe80b0380 0x7f5fe80b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:20.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.420+0000 7f5fed02e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5fe80b0380 0x7f5fe80b6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59826/0 (socket says 172.21.15.73:59826) 2024-04-26T20:51:20.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.420+0000 7f5fed02e640 1 -- 172.21.15.73:0/609252227 learned_addr learned my addr 172.21.15.73:0/609252227 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:20.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.420+0000 7f5fed82f640 1 --2- 172.21.15.73:0/609252227 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5fe806c950 0x7f5fe80c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:20.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.420+0000 7f5fee030640 1 --2- 172.21.15.73:0/609252227 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5fe80b10b0 0x7f5fe81550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:20.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.420+0000 7f5fed02e640 1 -- 172.21.15.73:0/609252227 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5fe80b10b0 msgr2=0x7f5fe81550e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:20.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.420+0000 7f5fed02e640 1 --2- 172.21.15.73:0/609252227 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5fe80b10b0 0x7f5fe81550e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:20.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.420+0000 7f5fed02e640 1 -- 172.21.15.73:0/609252227 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5fe806c950 msgr2=0x7f5fe80c83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:20.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.420+0000 7f5fed02e640 1 --2- 172.21.15.73:0/609252227 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5fe806c950 0x7f5fe80c83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:20.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.420+0000 7f5fed02e640 1 -- 172.21.15.73:0/609252227 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fe81f86a0 con 0x7f5fe80b0380 2024-04-26T20:51:20.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.420+0000 7f5fed02e640 1 --2- 172.21.15.73:0/609252227 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5fe80b0380 0x7f5fe80b6b70 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5fd800b420 tx=0x7f5fd800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:20.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.420+0000 7f5fde7fc640 1 -- 172.21.15.73:0/609252227 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd8011830 con 0x7f5fe80b0380 2024-04-26T20:51:20.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.420+0000 7f5fde7fc640 1 -- 172.21.15.73:0/609252227 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fd8016070 con 0x7f5fe80b0380 2024-04-26T20:51:20.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.420+0000 7f5fefa93640 1 -- 172.21.15.73:0/609252227 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fe81f8980 con 0x7f5fe80b0380 2024-04-26T20:51:20.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.420+0000 7f5fde7fc640 1 -- 172.21.15.73:0/609252227 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd8011c10 con 0x7f5fe80b0380 2024-04-26T20:51:20.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.420+0000 7f5fefa93640 1 -- 172.21.15.73:0/609252227 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fe81f8f40 con 0x7f5fe80b0380 2024-04-26T20:51:20.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.421+0000 7f5fefa93640 1 -- 172.21.15.73:0/609252227 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fe80cc510 con 0x7f5fe80b0380 2024-04-26T20:51:20.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.422+0000 7f5fde7fc640 1 -- 172.21.15.73:0/609252227 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5fd801b460 con 0x7f5fe80b0380 2024-04-26T20:51:20.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.422+0000 7f5fde7fc640 1 --2- 172.21.15.73:0/609252227 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f5fcc06c8f0 0x7f5fcc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:20.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.422+0000 7f5fed82f640 1 --2- 172.21.15.73:0/609252227 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f5fcc06c8f0 0x7f5fcc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:20.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.422+0000 7f5fde7fc640 1 -- 172.21.15.73:0/609252227 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f5fd808ba90 con 0x7f5fe80b0380 2024-04-26T20:51:20.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.423+0000 7f5fed82f640 1 --2- 172.21.15.73:0/609252227 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f5fcc06c8f0 0x7f5fcc06edb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5fe40075b0 tx=0x7f5fe4007540 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:20.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.427+0000 7f5fde7fc640 1 -- 172.21.15.73:0/609252227 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5fd8020020 con 0x7f5fe80b0380 2024-04-26T20:51:20.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.600+0000 7f5fefa93640 1 -- 172.21.15.73:0/609252227 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f5fe80694f0 con 0x7f5fe80b0380 2024-04-26T20:51:20.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.600+0000 7f5fde7fc640 1 -- 172.21.15.73:0/609252227 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f5fd8059a20 con 0x7f5fe80b0380 2024-04-26T20:51:20.601 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-26T20:51:20.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.602+0000 7f5fefa93640 1 -- 172.21.15.73:0/609252227 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f5fcc06c8f0 msgr2=0x7f5fcc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:20.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.602+0000 7f5fefa93640 1 --2- 172.21.15.73:0/609252227 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f5fcc06c8f0 0x7f5fcc06edb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5fe40075b0 tx=0x7f5fe4007540 comp rx=0 tx=0).stop 2024-04-26T20:51:20.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.602+0000 7f5fefa93640 1 -- 172.21.15.73:0/609252227 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5fe80b0380 msgr2=0x7f5fe80b6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:20.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.602+0000 7f5fefa93640 1 --2- 172.21.15.73:0/609252227 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5fe80b0380 0x7f5fe80b6b70 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5fd800b420 tx=0x7f5fd800b8e0 comp rx=0 tx=0).stop 2024-04-26T20:51:20.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.602+0000 7f5fefa93640 1 -- 172.21.15.73:0/609252227 shutdown_connections 2024-04-26T20:51:20.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.602+0000 7f5fefa93640 1 --2- 172.21.15.73:0/609252227 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5fe80b0380 0x7f5fe80b6b70 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:20.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.602+0000 7f5fefa93640 1 --2- 172.21.15.73:0/609252227 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f5fcc06c8f0 0x7f5fcc06edb0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:20.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.602+0000 7f5fefa93640 1 --2- 172.21.15.73:0/609252227 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5fe80b10b0 0x7f5fe81550e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:20.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.602+0000 7f5fefa93640 1 --2- 172.21.15.73:0/609252227 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5fe806c950 0x7f5fe80c83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:20.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.602+0000 7f5fefa93640 1 -- 172.21.15.73:0/609252227 >> 172.21.15.73:0/609252227 conn(0x7f5fe80d1cf0 msgr2=0x7f5fe8070fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:20.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.602+0000 7f5fefa93640 1 -- 172.21.15.73:0/609252227 shutdown_connections 2024-04-26T20:51:20.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.602+0000 7f5fefa93640 1 -- 172.21.15.73:0/609252227 wait complete. 2024-04-26T20:51:20.614 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_auto_repair.TestMDSAutoRepair.test_backtrace_repair' 2024-04-26T20:51:20.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.712+0000 7f9997baf640 1 Processor -- start 2024-04-26T20:51:20.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.712+0000 7f9997baf640 1 -- start start 2024-04-26T20:51:20.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.713+0000 7f9997baf640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f999015f540 0x7f99900d3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:20.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.713+0000 7f9997baf640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f99900b3ff0 0x7f99900d39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:20.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.713+0000 7f9997baf640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99900d2d70 con 0x7f99900d1770 2024-04-26T20:51:20.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.713+0000 7f9997baf640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f999015f920 con 0x7f999015f540 2024-04-26T20:51:20.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.713+0000 7f9997baf640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f99900d0f30 con 0x7f99900b3ff0 2024-04-26T20:51:20.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.713+0000 7f999594b640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f999015f540 0x7f99900d3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:20.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.713+0000 7f999594b640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f999015f540 0x7f99900d3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59834/0 (socket says 172.21.15.73:59834) 2024-04-26T20:51:20.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.713+0000 7f999614c640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f99900d1770 0x7f99900d1ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:34520/0 (socket says 172.21.15.73:34520) 2024-04-26T20:51:20.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.713+0000 7f999594b640 1 -- 172.21.15.73:0/4088299311 learned_addr learned my addr 172.21.15.73:0/4088299311 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:20.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.714+0000 7f999514a640 1 --2- 172.21.15.73:0/4088299311 >> v2:172.21.15.120:3300/0 conn(0x7f99900b3ff0 0x7f99900d39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:20.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.714+0000 7f999594b640 1 -- 172.21.15.73:0/4088299311 >> v2:172.21.15.120:3300/0 conn(0x7f99900b3ff0 msgr2=0x7f99900d39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:20.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.714+0000 7f999594b640 1 --2- 172.21.15.73:0/4088299311 >> v2:172.21.15.120:3300/0 conn(0x7f99900b3ff0 0x7f99900d39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:20.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.714+0000 7f999594b640 1 -- 172.21.15.73:0/4088299311 >> v1:172.21.15.73:6789/0 conn(0x7f99900d1770 legacy=0x7f99900d1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:20.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.714+0000 7f999594b640 1 -- 172.21.15.73:0/4088299311 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99900d3050 con 0x7f999015f540 2024-04-26T20:51:20.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.714+0000 7f999514a640 1 --2- 172.21.15.73:0/4088299311 >> v2:172.21.15.120:3300/0 conn(0x7f99900b3ff0 0x7f99900d39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:20.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.714+0000 7f999594b640 1 --2- 172.21.15.73:0/4088299311 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f999015f540 0x7f99900d3470 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f99900749e0 tx=0x7f998802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=971680f1b07f827a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:20.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.714+0000 7f9994949640 1 -- 172.21.15.73:0/4088299311 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9988036070 con 0x7f999015f540 2024-04-26T20:51:20.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.715+0000 7f9994949640 1 -- 172.21.15.73:0/4088299311 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9988031600 con 0x7f999015f540 2024-04-26T20:51:20.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.715+0000 7f9994949640 1 -- 172.21.15.73:0/4088299311 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99880319d0 con 0x7f999015f540 2024-04-26T20:51:20.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.715+0000 7f9997baf640 1 -- 172.21.15.73:0/4088299311 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f999015f540 msgr2=0x7f99900d3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:20.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.715+0000 7f9997baf640 1 --2- 172.21.15.73:0/4088299311 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f999015f540 0x7f99900d3470 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f99900749e0 tx=0x7f998802a730 comp rx=0 tx=0).stop 2024-04-26T20:51:20.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.715+0000 7f9997baf640 1 -- 172.21.15.73:0/4088299311 shutdown_connections 2024-04-26T20:51:20.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.715+0000 7f9997baf640 1 --2- 172.21.15.73:0/4088299311 >> v2:172.21.15.120:3300/0 conn(0x7f99900b3ff0 0x7f99900d39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:20.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.715+0000 7f9997baf640 1 --2- 172.21.15.73:0/4088299311 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f999015f540 0x7f99900d3470 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:20.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.715+0000 7f9997baf640 1 -- 172.21.15.73:0/4088299311 >> 172.21.15.73:0/4088299311 conn(0x7f99900d4f90 msgr2=0x7f99900d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:20.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.715+0000 7f9997baf640 1 -- 172.21.15.73:0/4088299311 shutdown_connections 2024-04-26T20:51:20.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.715+0000 7f9997baf640 1 -- 172.21.15.73:0/4088299311 wait complete. 2024-04-26T20:51:20.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.716+0000 7f9997baf640 1 Processor -- start 2024-04-26T20:51:20.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.716+0000 7f9997baf640 1 -- start start 2024-04-26T20:51:20.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.717+0000 7f9997baf640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99900b3ff0 0x7f99901f4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:20.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.717+0000 7f9997baf640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99900d1770 0x7f99900a12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:20.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.717+0000 7f9997baf640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f999015f540 0x7f99900a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:20.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.717+0000 7f9997baf640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9990068e50 con 0x7f99900d1770 2024-04-26T20:51:20.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.717+0000 7f9997baf640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f99900d0bc0 con 0x7f99900b3ff0 2024-04-26T20:51:20.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.717+0000 7f9997baf640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f99900b5ab0 con 0x7f999015f540 2024-04-26T20:51:20.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.717+0000 7f999514a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99900d1770 0x7f99900a12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:20.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.717+0000 7f999514a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99900d1770 0x7f99900a12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59842/0 (socket says 172.21.15.73:59842) 2024-04-26T20:51:20.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.717+0000 7f999514a640 1 -- 172.21.15.73:0/3830697477 learned_addr learned my addr 172.21.15.73:0/3830697477 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:20.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.717+0000 7f999594b640 1 --2- 172.21.15.73:0/3830697477 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99900b3ff0 0x7f99901f4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:20.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.717+0000 7f999614c640 1 --2- 172.21.15.73:0/3830697477 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f999015f540 0x7f99900a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:20.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.717+0000 7f999514a640 1 -- 172.21.15.73:0/3830697477 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f999015f540 msgr2=0x7f99900a17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:20.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.717+0000 7f999514a640 1 --2- 172.21.15.73:0/3830697477 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f999015f540 0x7f99900a17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:20.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.717+0000 7f999514a640 1 -- 172.21.15.73:0/3830697477 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99900b3ff0 msgr2=0x7f99901f4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:20.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.717+0000 7f999514a640 1 --2- 172.21.15.73:0/3830697477 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99900b3ff0 0x7f99901f4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:20.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.717+0000 7f999514a640 1 -- 172.21.15.73:0/3830697477 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99900ccdd0 con 0x7f99900d1770 2024-04-26T20:51:20.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.717+0000 7f999514a640 1 --2- 172.21.15.73:0/3830697477 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99900d1770 0x7f99900a12b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f998000c7f0 tx=0x7f998000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:20.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.718+0000 7f99867fc640 1 -- 172.21.15.73:0/3830697477 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9980015070 con 0x7f99900d1770 2024-04-26T20:51:20.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.718+0000 7f99867fc640 1 -- 172.21.15.73:0/3830697477 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9980007a30 con 0x7f99900d1770 2024-04-26T20:51:20.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.718+0000 7f9997baf640 1 -- 172.21.15.73:0/3830697477 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99900cd0b0 con 0x7f99900d1770 2024-04-26T20:51:20.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.718+0000 7f99867fc640 1 -- 172.21.15.73:0/3830697477 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9980007e00 con 0x7f99900d1770 2024-04-26T20:51:20.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.718+0000 7f9997baf640 1 -- 172.21.15.73:0/3830697477 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99900cd670 con 0x7f99900d1770 2024-04-26T20:51:20.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.719+0000 7f9997baf640 1 -- 172.21.15.73:0/3830697477 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99900694f0 con 0x7f99900d1770 2024-04-26T20:51:20.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.719+0000 7f99867fc640 1 -- 172.21.15.73:0/3830697477 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9980018070 con 0x7f99900d1770 2024-04-26T20:51:20.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.720+0000 7f99867fc640 1 --2- 172.21.15.73:0/3830697477 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f997406c920 0x7f997406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:20.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.720+0000 7f999594b640 1 --2- 172.21.15.73:0/3830697477 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f997406c920 0x7f997406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:20.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.720+0000 7f99867fc640 1 -- 172.21.15.73:0/3830697477 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f998008bef0 con 0x7f99900d1770 2024-04-26T20:51:20.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.720+0000 7f999594b640 1 --2- 172.21.15.73:0/3830697477 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f997406c920 0x7f997406ede0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9988033000 tx=0x7f9988009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:20.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.724+0000 7f99867fc640 1 -- 172.21.15.73:0/3830697477 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9980059f20 con 0x7f99900d1770 2024-04-26T20:51:20.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:20.974+0000 7f9997baf640 1 -- 172.21.15.73:0/3830697477 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_auto_repair.TestMDSAutoRepair.test_backtrace_repair"]} v 0) -- 0x7f99900d4b20 con 0x7f99900d1770 2024-04-26T20:51:21.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.817+0000 7f99867fc640 1 -- 172.21.15.73:0/3830697477 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_auto_repair.TestMDSAutoRepair.test_backtrace_repair"]}]=0 v71) ==== 151+0+0 (secure 0 0 0) 0x7f9980059820 con 0x7f99900d1770 2024-04-26T20:51:21.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.819+0000 7f9997baf640 1 -- 172.21.15.73:0/3830697477 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f997406c920 msgr2=0x7f997406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:21.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.819+0000 7f9997baf640 1 --2- 172.21.15.73:0/3830697477 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f997406c920 0x7f997406ede0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9988033000 tx=0x7f9988009000 comp rx=0 tx=0).stop 2024-04-26T20:51:21.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.819+0000 7f9997baf640 1 -- 172.21.15.73:0/3830697477 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99900d1770 msgr2=0x7f99900a12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:21.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.819+0000 7f9997baf640 1 --2- 172.21.15.73:0/3830697477 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99900d1770 0x7f99900a12b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f998000c7f0 tx=0x7f998000cbf0 comp rx=0 tx=0).stop 2024-04-26T20:51:21.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.820+0000 7f9997baf640 1 -- 172.21.15.73:0/3830697477 shutdown_connections 2024-04-26T20:51:21.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.820+0000 7f9997baf640 1 --2- 172.21.15.73:0/3830697477 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99900d1770 0x7f99900a12b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:21.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.820+0000 7f9997baf640 1 --2- 172.21.15.73:0/3830697477 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f997406c920 0x7f997406ede0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:21.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.820+0000 7f9997baf640 1 --2- 172.21.15.73:0/3830697477 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f999015f540 0x7f99900a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:21.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.820+0000 7f9997baf640 1 --2- 172.21.15.73:0/3830697477 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99900b3ff0 0x7f99901f4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:21.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.820+0000 7f9997baf640 1 -- 172.21.15.73:0/3830697477 >> 172.21.15.73:0/3830697477 conn(0x7f99900d4f90 msgr2=0x7f99900d5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:21.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.820+0000 7f9997baf640 1 -- 172.21.15.73:0/3830697477 shutdown_connections 2024-04-26T20:51:21.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.820+0000 7f9997baf640 1 -- 172.21.15.73:0/3830697477 wait complete. 2024-04-26T20:51:21.834 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T20:51:21.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.938+0000 7fb1da4a9640 1 Processor -- start 2024-04-26T20:51:21.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.938+0000 7fb1da4a9640 1 -- start start 2024-04-26T20:51:21.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.939+0000 7fb1da4a9640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fb1d4070350 0x7fb1d4070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:21.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.939+0000 7fb1da4a9640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1d40d2390 con 0x7fb1d4071090 2024-04-26T20:51:21.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.939+0000 7fb1da4a9640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1d406d3f0 con 0x7fb1d40cfde0 2024-04-26T20:51:21.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.939+0000 7fb1da4a9640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fb1d40afab0 con 0x7fb1d4070350 2024-04-26T20:51:21.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.939+0000 7fb1d8a46640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fb1d4071090 0x7fb1d415e9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:34528/0 (socket says 172.21.15.73:34528) 2024-04-26T20:51:21.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.939+0000 7fb1d8a46640 1 -- 172.21.15.73:0/1766378199 learned_addr learned my addr 172.21.15.73:0/1766378199 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:21.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.939+0000 7fb1d2ffd640 1 -- 172.21.15.73:0/1766378199 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3322908895 0 0) 0x7fb1d40d2390 con 0x7fb1d4071090 2024-04-26T20:51:21.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.939+0000 7fb1d2ffd640 1 -- 172.21.15.73:0/1766378199 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb1bc003520 con 0x7fb1d4071090 2024-04-26T20:51:21.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.939+0000 7fb1d37fe640 1 --2- 172.21.15.73:0/1766378199 >> v2:172.21.15.120:3300/0 conn(0x7fb1d4070350 0x7fb1d4070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:21.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.940+0000 7fb1d2ffd640 1 -- 172.21.15.73:0/1766378199 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 886589629 0 0) 0x7fb1bc003520 con 0x7fb1d4071090 2024-04-26T20:51:21.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.940+0000 7fb1d2ffd640 1 -- 172.21.15.73:0/1766378199 >> v2:172.21.15.120:3300/0 conn(0x7fb1d4070350 msgr2=0x7fb1d4070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:21.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.940+0000 7fb1d2ffd640 1 --2- 172.21.15.73:0/1766378199 >> v2:172.21.15.120:3300/0 conn(0x7fb1d4070350 0x7fb1d4070b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:21.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.940+0000 7fb1d2ffd640 1 -- 172.21.15.73:0/1766378199 >> v1:172.21.15.120:6789/0 conn(0x7fb1d40cfde0 legacy=0x7fb1d40d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:21.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.940+0000 7fb1d2ffd640 1 -- 172.21.15.73:0/1766378199 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1d406d6d0 con 0x7fb1d4071090 2024-04-26T20:51:21.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.940+0000 7fb1d2ffd640 1 -- 172.21.15.73:0/1766378199 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7fb1cc0019f0 con 0x7fb1d4071090 2024-04-26T20:51:21.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.940+0000 7fb1d2ffd640 1 -- 172.21.15.73:0/1766378199 >> v1:172.21.15.73:6789/0 conn(0x7fb1d4071090 legacy=0x7fb1d415e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:21.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.940+0000 7fb1d2ffd640 1 --2- 172.21.15.73:0/1766378199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1bc0039b0 0x7fb1bc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:21.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.940+0000 7fb1d2ffd640 1 --2- 172.21.15.73:0/1766378199 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb1bc004800 0x7fb1bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:21.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.940+0000 7fb1d2ffd640 1 --2- 172.21.15.73:0/1766378199 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb1bc0051c0 0x7fb1bc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:21.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.940+0000 7fb1d2ffd640 1 -- 172.21.15.73:0/1766378199 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb1bc00c130 con 0x7fb1bc0039b0 2024-04-26T20:51:21.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.940+0000 7fb1d2ffd640 1 -- 172.21.15.73:0/1766378199 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb1bc00c2e0 con 0x7fb1bc0051c0 2024-04-26T20:51:21.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.940+0000 7fb1d2ffd640 1 -- 172.21.15.73:0/1766378199 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fb1bc00c4e0 con 0x7fb1bc004800 2024-04-26T20:51:21.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.940+0000 7fb1d3fff640 1 --2- 172.21.15.73:0/1766378199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1bc0039b0 0x7fb1bc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:21.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.940+0000 7fb1d8a46640 1 --2- 172.21.15.73:0/1766378199 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb1bc0051c0 0x7fb1bc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:21.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.940+0000 7fb1d37fe640 1 --2- 172.21.15.73:0/1766378199 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb1bc004800 0x7fb1bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:21.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.940+0000 7fb1d3fff640 1 -- 172.21.15.73:0/1766378199 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb1bc004800 msgr2=0x7fb1bc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:21.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.940+0000 7fb1d3fff640 1 --2- 172.21.15.73:0/1766378199 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb1bc004800 0x7fb1bc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:21.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.940+0000 7fb1d3fff640 1 -- 172.21.15.73:0/1766378199 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb1bc0051c0 msgr2=0x7fb1bc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:21.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.940+0000 7fb1d3fff640 1 --2- 172.21.15.73:0/1766378199 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb1bc0051c0 0x7fb1bc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:21.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.940+0000 7fb1d8a46640 1 -- 172.21.15.73:0/1766378199 reap_dead start 2024-04-26T20:51:21.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.940+0000 7fb1d3fff640 1 -- 172.21.15.73:0/1766378199 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1bc003520 con 0x7fb1bc0039b0 2024-04-26T20:51:21.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.941+0000 7fb1d3fff640 1 --2- 172.21.15.73:0/1766378199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1bc0039b0 0x7fb1bc003dd0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb1c4001750 tx=0x7fb1bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c2657f062162d9b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:21.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.941+0000 7fb1d2ffd640 1 -- 172.21.15.73:0/1766378199 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c4001370 con 0x7fb1bc0039b0 2024-04-26T20:51:21.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.941+0000 7fb1d2ffd640 1 -- 172.21.15.73:0/1766378199 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1c40052d0 con 0x7fb1bc0039b0 2024-04-26T20:51:21.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.941+0000 7fb1d2ffd640 1 -- 172.21.15.73:0/1766378199 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c402abc0 con 0x7fb1bc0039b0 2024-04-26T20:51:21.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.941+0000 7fb1da4a9640 1 -- 172.21.15.73:0/1766378199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1bc0039b0 msgr2=0x7fb1bc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:21.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.941+0000 7fb1da4a9640 1 --2- 172.21.15.73:0/1766378199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1bc0039b0 0x7fb1bc003dd0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb1c4001750 tx=0x7fb1bc005830 comp rx=0 tx=0).stop 2024-04-26T20:51:21.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.941+0000 7fb1da4a9640 1 -- 172.21.15.73:0/1766378199 shutdown_connections 2024-04-26T20:51:21.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.941+0000 7fb1da4a9640 1 --2- 172.21.15.73:0/1766378199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1bc0039b0 0x7fb1bc003dd0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:21.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.941+0000 7fb1da4a9640 1 -- 172.21.15.73:0/1766378199 >> 172.21.15.73:0/1766378199 conn(0x7fb1d40d2d80 msgr2=0x7fb1d40d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:21.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.941+0000 7fb1da4a9640 1 -- 172.21.15.73:0/1766378199 shutdown_connections 2024-04-26T20:51:21.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.942+0000 7fb1da4a9640 1 -- 172.21.15.73:0/1766378199 wait complete. 2024-04-26T20:51:21.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.943+0000 7fb1da4a9640 1 Processor -- start 2024-04-26T20:51:21.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.943+0000 7fb1da4a9640 1 -- start start 2024-04-26T20:51:21.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.944+0000 7fb1da4a9640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb1bc0039b0 0x7fb1d41f4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:21.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.944+0000 7fb1da4a9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb1d4152ff0 0x7fb1d415dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:21.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.944+0000 7fb1da4a9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1d4153470 0x7fb1d415e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:21.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.944+0000 7fb1da4a9640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d40714c0 con 0x7fb1d4153470 2024-04-26T20:51:21.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.944+0000 7fb1da4a9640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d406dd20 con 0x7fb1d4152ff0 2024-04-26T20:51:21.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.944+0000 7fb1da4a9640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fb1d4070b50 con 0x7fb1bc0039b0 2024-04-26T20:51:21.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.944+0000 7fb1d8a46640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1d4153470 0x7fb1d415e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:21.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.944+0000 7fb1d8a46640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1d4153470 0x7fb1d415e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59854/0 (socket says 172.21.15.73:59854) 2024-04-26T20:51:21.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.944+0000 7fb1d8a46640 1 -- 172.21.15.73:0/957505582 learned_addr learned my addr 172.21.15.73:0/957505582 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:21.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.944+0000 7fb1d3fff640 1 --2- 172.21.15.73:0/957505582 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb1bc0039b0 0x7fb1d41f4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:21.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.944+0000 7fb1d37fe640 1 --2- 172.21.15.73:0/957505582 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb1d4152ff0 0x7fb1d415dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:21.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.944+0000 7fb1d8a46640 1 -- 172.21.15.73:0/957505582 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb1bc0039b0 msgr2=0x7fb1d41f4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:21.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.944+0000 7fb1d8a46640 1 --2- 172.21.15.73:0/957505582 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb1bc0039b0 0x7fb1d41f4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:21.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.944+0000 7fb1d8a46640 1 -- 172.21.15.73:0/957505582 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb1d4152ff0 msgr2=0x7fb1d415dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:21.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.944+0000 7fb1d8a46640 1 --2- 172.21.15.73:0/957505582 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb1d4152ff0 0x7fb1d415dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:21.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.944+0000 7fb1d8a46640 1 -- 172.21.15.73:0/957505582 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1d409cbc0 con 0x7fb1d4153470 2024-04-26T20:51:21.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.944+0000 7fb1d3fff640 1 --2- 172.21.15.73:0/957505582 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb1bc0039b0 0x7fb1d41f4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:21.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.944+0000 7fb1d8a46640 1 --2- 172.21.15.73:0/957505582 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1d4153470 0x7fb1d415e520 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb1d406dc90 tx=0x7fb1cc007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:21.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.944+0000 7fb1d0ff9640 1 -- 172.21.15.73:0/957505582 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc00abc0 con 0x7fb1d4153470 2024-04-26T20:51:21.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.944+0000 7fb1d0ff9640 1 -- 172.21.15.73:0/957505582 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1cc019070 con 0x7fb1d4153470 2024-04-26T20:51:21.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.944+0000 7fb1da4a9640 1 -- 172.21.15.73:0/957505582 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1d409cea0 con 0x7fb1d4153470 2024-04-26T20:51:21.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.944+0000 7fb1d0ff9640 1 -- 172.21.15.73:0/957505582 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc0133f0 con 0x7fb1d4153470 2024-04-26T20:51:21.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.944+0000 7fb1da4a9640 1 -- 172.21.15.73:0/957505582 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1d409d740 con 0x7fb1d4153470 2024-04-26T20:51:21.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.945+0000 7fb1da4a9640 1 -- 172.21.15.73:0/957505582 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1d4069a80 con 0x7fb1d4153470 2024-04-26T20:51:21.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.946+0000 7fb1d0ff9640 1 -- 172.21.15.73:0/957505582 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb1cc013590 con 0x7fb1d4153470 2024-04-26T20:51:21.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.946+0000 7fb1d0ff9640 1 --2- 172.21.15.73:0/957505582 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb1c006c920 0x7fb1c006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:21.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.946+0000 7fb1d0ff9640 1 -- 172.21.15.73:0/957505582 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fb1cc08c3d0 con 0x7fb1d4153470 2024-04-26T20:51:21.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.946+0000 7fb1d3fff640 1 --2- 172.21.15.73:0/957505582 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb1c006c920 0x7fb1c006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:21.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.946+0000 7fb1d3fff640 1 --2- 172.21.15.73:0/957505582 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb1c006c920 0x7fb1c006ede0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb1c4032000 tx=0x7fb1c402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:21.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:21.950+0000 7fb1d0ff9640 1 -- 172.21.15.73:0/957505582 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb1d4069a80 con 0x7fb1d4153470 2024-04-26T20:51:22.090 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:51:22.093 DEBUG:teuthology.orchestra.run.smithi120:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:51:22.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.127+0000 7fb1da4a9640 1 -- 172.21.15.73:0/957505582 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fb1d4162990 con 0x7fb1d4153470 2024-04-26T20:51:22.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.133+0000 7fb1d0ff9640 1 -- 172.21.15.73:0/957505582 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7fb1cc05a490 con 0x7fb1d4153470 2024-04-26T20:51:22.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.135+0000 7fb1da4a9640 1 -- 172.21.15.73:0/957505582 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb1c006c920 msgr2=0x7fb1c006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:22.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.135+0000 7fb1da4a9640 1 --2- 172.21.15.73:0/957505582 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb1c006c920 0x7fb1c006ede0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb1c4032000 tx=0x7fb1c402e000 comp rx=0 tx=0).stop 2024-04-26T20:51:22.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.135+0000 7fb1da4a9640 1 -- 172.21.15.73:0/957505582 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1d4153470 msgr2=0x7fb1d415e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:22.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.135+0000 7fb1da4a9640 1 --2- 172.21.15.73:0/957505582 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1d4153470 0x7fb1d415e520 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb1d406dc90 tx=0x7fb1cc007a90 comp rx=0 tx=0).stop 2024-04-26T20:51:22.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.135+0000 7fb1da4a9640 1 -- 172.21.15.73:0/957505582 shutdown_connections 2024-04-26T20:51:22.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.135+0000 7fb1da4a9640 1 --2- 172.21.15.73:0/957505582 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1d4153470 0x7fb1d415e520 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:22.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.135+0000 7fb1da4a9640 1 --2- 172.21.15.73:0/957505582 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb1c006c920 0x7fb1c006ede0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:22.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.135+0000 7fb1da4a9640 1 --2- 172.21.15.73:0/957505582 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb1d4152ff0 0x7fb1d415dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:22.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.135+0000 7fb1da4a9640 1 --2- 172.21.15.73:0/957505582 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb1bc0039b0 0x7fb1d41f4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:22.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.135+0000 7fb1da4a9640 1 -- 172.21.15.73:0/957505582 >> 172.21.15.73:0/957505582 conn(0x7fb1d40d2d80 msgr2=0x7fb1d40b0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:22.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.135+0000 7fb1da4a9640 1 -- 172.21.15.73:0/957505582 shutdown_connections 2024-04-26T20:51:22.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.135+0000 7fb1da4a9640 1 -- 172.21.15.73:0/957505582 wait complete. 2024-04-26T20:51:22.148 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T20:51:22.148 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T20:51:22.226 DEBUG:teuthology.orchestra.run.smithi120:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T20:51:22.266 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T20:51:22.267 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-26T20:51:22.267 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:51:22.282 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-26T20:51:22.282 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:51:22.338 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-26T20:51:22.338 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:51:22.393 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:51:22.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.532+0000 7f3b46346640 1 Processor -- start 2024-04-26T20:51:22.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.532+0000 7f3b46346640 1 -- start start 2024-04-26T20:51:22.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.533+0000 7f3b46346640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f3b4006fc40 0x7f3b40073360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:22.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.533+0000 7f3b46346640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b40156840 con 0x7f3b40156460 2024-04-26T20:51:22.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.533+0000 7f3b46346640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b400cbf20 con 0x7f3b4006f790 2024-04-26T20:51:22.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.533+0000 7f3b46346640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f3b400cc200 con 0x7f3b4006fc40 2024-04-26T20:51:22.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.533+0000 7f3b3ffff640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f3b40156460 0x7f3b400cb4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:34530/0 (socket says 172.21.15.73:34530) 2024-04-26T20:51:22.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.533+0000 7f3b3ffff640 1 -- 172.21.15.73:0/664239767 learned_addr learned my addr 172.21.15.73:0/664239767 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:22.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.533+0000 7f3b3effd640 1 -- 172.21.15.73:0/664239767 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1762768074 0 0) 0x7f3b40156840 con 0x7f3b40156460 2024-04-26T20:51:22.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.533+0000 7f3b3effd640 1 -- 172.21.15.73:0/664239767 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b28003520 con 0x7f3b40156460 2024-04-26T20:51:22.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.533+0000 7f3b448e3640 1 --2- 172.21.15.73:0/664239767 >> v2:172.21.15.120:3300/0 conn(0x7f3b4006fc40 0x7f3b40073360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:22.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.533+0000 7f3b3effd640 1 -- 172.21.15.73:0/664239767 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3985312165 0 0) 0x7f3b28003520 con 0x7f3b40156460 2024-04-26T20:51:22.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.533+0000 7f3b3effd640 1 -- 172.21.15.73:0/664239767 >> v2:172.21.15.120:3300/0 conn(0x7f3b4006fc40 msgr2=0x7f3b40073360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:22.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.533+0000 7f3b3effd640 1 --2- 172.21.15.73:0/664239767 >> v2:172.21.15.120:3300/0 conn(0x7f3b4006fc40 0x7f3b40073360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:22.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.533+0000 7f3b3effd640 1 -- 172.21.15.73:0/664239767 >> v1:172.21.15.120:6789/0 conn(0x7f3b4006f790 legacy=0x7f3b40072c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:22.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.533+0000 7f3b3effd640 1 -- 172.21.15.73:0/664239767 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b400cc4e0 con 0x7f3b40156460 2024-04-26T20:51:22.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.533+0000 7f3b3effd640 1 -- 172.21.15.73:0/664239767 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7f3b34001670 con 0x7f3b40156460 2024-04-26T20:51:22.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.534+0000 7f3b3effd640 1 -- 172.21.15.73:0/664239767 >> v1:172.21.15.73:6789/0 conn(0x7f3b40156460 legacy=0x7f3b400cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:22.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.534+0000 7f3b3effd640 1 --2- 172.21.15.73:0/664239767 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b280039b0 0x7f3b28003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:22.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.534+0000 7f3b3effd640 1 --2- 172.21.15.73:0/664239767 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3b28004800 0x7f3b28004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:22.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.534+0000 7f3b3effd640 1 --2- 172.21.15.73:0/664239767 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b280051c0 0x7f3b2800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:22.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.534+0000 7f3b3effd640 1 -- 172.21.15.73:0/664239767 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3b400cc200 con 0x7f3b280051c0 2024-04-26T20:51:22.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.534+0000 7f3b3effd640 1 -- 172.21.15.73:0/664239767 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2800c060 con 0x7f3b280039b0 2024-04-26T20:51:22.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.534+0000 7f3b3effd640 1 -- 172.21.15.73:0/664239767 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f3b2800c260 con 0x7f3b28004800 2024-04-26T20:51:22.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.534+0000 7f3b448e3640 1 --2- 172.21.15.73:0/664239767 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b280051c0 0x7f3b2800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:22.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.534+0000 7f3b3ffff640 1 --2- 172.21.15.73:0/664239767 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b280039b0 0x7f3b28003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:22.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.534+0000 7f3b3f7fe640 1 --2- 172.21.15.73:0/664239767 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3b28004800 0x7f3b28004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:22.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.534+0000 7f3b448e3640 1 -- 172.21.15.73:0/664239767 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3b28004800 msgr2=0x7f3b28004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:22.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.534+0000 7f3b448e3640 1 --2- 172.21.15.73:0/664239767 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3b28004800 0x7f3b28004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:22.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.534+0000 7f3b448e3640 1 -- 172.21.15.73:0/664239767 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b280039b0 msgr2=0x7f3b28003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:22.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.534+0000 7f3b448e3640 1 --2- 172.21.15.73:0/664239767 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b280039b0 0x7f3b28003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:22.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.534+0000 7f3b448e3640 1 -- 172.21.15.73:0/664239767 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b28003520 con 0x7f3b280051c0 2024-04-26T20:51:22.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.534+0000 7f3b3f7fe640 1 --2- 172.21.15.73:0/664239767 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3b28004800 0x7f3b28004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:22.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.535+0000 7f3b448e3640 1 -- 172.21.15.73:0/664239767 reap_dead start 2024-04-26T20:51:22.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.535+0000 7f3b448e3640 1 --2- 172.21.15.73:0/664239767 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b280051c0 0x7f3b2800b9c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3b38009fc0 tx=0x7f3b28005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcb4047eea59bc96 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:22.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.535+0000 7f3b3effd640 1 -- 172.21.15.73:0/664239767 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b38035020 con 0x7f3b280051c0 2024-04-26T20:51:22.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.535+0000 7f3b3effd640 1 -- 172.21.15.73:0/664239767 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b38003bd0 con 0x7f3b280051c0 2024-04-26T20:51:22.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.535+0000 7f3b3effd640 1 -- 172.21.15.73:0/664239767 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b38009c10 con 0x7f3b280051c0 2024-04-26T20:51:22.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.536+0000 7f3b46346640 1 -- 172.21.15.73:0/664239767 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b280051c0 msgr2=0x7f3b2800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:22.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.536+0000 7f3b46346640 1 --2- 172.21.15.73:0/664239767 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b280051c0 0x7f3b2800b9c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3b38009fc0 tx=0x7f3b28005830 comp rx=0 tx=0).stop 2024-04-26T20:51:22.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.536+0000 7f3b46346640 1 -- 172.21.15.73:0/664239767 shutdown_connections 2024-04-26T20:51:22.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.536+0000 7f3b46346640 1 --2- 172.21.15.73:0/664239767 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b280051c0 0x7f3b2800b9c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:22.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.536+0000 7f3b46346640 1 -- 172.21.15.73:0/664239767 >> 172.21.15.73:0/664239767 conn(0x7f3b400d0be0 msgr2=0x7f3b400d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:22.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.536+0000 7f3b46346640 1 -- 172.21.15.73:0/664239767 shutdown_connections 2024-04-26T20:51:22.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.536+0000 7f3b46346640 1 -- 172.21.15.73:0/664239767 wait complete. 2024-04-26T20:51:22.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.537+0000 7f3b46346640 1 Processor -- start 2024-04-26T20:51:22.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.537+0000 7f3b46346640 1 -- start start 2024-04-26T20:51:22.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.538+0000 7f3b46346640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b280051c0 0x7f3b401f3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:22.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.538+0000 7f3b46346640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b40151a10 0x7f3b401f63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:22.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.538+0000 7f3b46346640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3b400c60e0 0x7f3b401f5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:22.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.538+0000 7f3b46346640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3b40070050 con 0x7f3b40151a10 2024-04-26T20:51:22.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.538+0000 7f3b46346640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3b40072700 con 0x7f3b280051c0 2024-04-26T20:51:22.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.538+0000 7f3b46346640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f3b4006e260 con 0x7f3b400c60e0 2024-04-26T20:51:22.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.538+0000 7f3b3f7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b40151a10 0x7f3b401f63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:22.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.538+0000 7f3b3f7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b40151a10 0x7f3b401f63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59868/0 (socket says 172.21.15.73:59868) 2024-04-26T20:51:22.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.538+0000 7f3b3f7fe640 1 -- 172.21.15.73:0/3538147149 learned_addr learned my addr 172.21.15.73:0/3538147149 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:22.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.538+0000 7f3b3ffff640 1 --2- 172.21.15.73:0/3538147149 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b280051c0 0x7f3b401f3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:22.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.538+0000 7f3b448e3640 1 --2- 172.21.15.73:0/3538147149 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3b400c60e0 0x7f3b401f5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:22.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.538+0000 7f3b3f7fe640 1 -- 172.21.15.73:0/3538147149 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3b400c60e0 msgr2=0x7f3b401f5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:22.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.538+0000 7f3b3f7fe640 1 --2- 172.21.15.73:0/3538147149 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3b400c60e0 0x7f3b401f5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:22.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.538+0000 7f3b3f7fe640 1 -- 172.21.15.73:0/3538147149 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b280051c0 msgr2=0x7f3b401f3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:22.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.538+0000 7f3b3f7fe640 1 --2- 172.21.15.73:0/3538147149 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b280051c0 0x7f3b401f3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:22.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.538+0000 7f3b3f7fe640 1 -- 172.21.15.73:0/3538147149 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b400c6a60 con 0x7f3b40151a10 2024-04-26T20:51:22.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.538+0000 7f3b448e3640 1 --2- 172.21.15.73:0/3538147149 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3b400c60e0 0x7f3b401f5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:22.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.538+0000 7f3b3ffff640 1 --2- 172.21.15.73:0/3538147149 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b280051c0 0x7f3b401f3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:22.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.538+0000 7f3b3f7fe640 1 --2- 172.21.15.73:0/3538147149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b40151a10 0x7f3b401f63b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3b3000e440 tx=0x7f3b3000e900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:22.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.538+0000 7f3b3cff9640 1 -- 172.21.15.73:0/3538147149 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b300052e0 con 0x7f3b40151a10 2024-04-26T20:51:22.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.538+0000 7f3b3cff9640 1 -- 172.21.15.73:0/3538147149 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b30018070 con 0x7f3b40151a10 2024-04-26T20:51:22.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.538+0000 7f3b3cff9640 1 -- 172.21.15.73:0/3538147149 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b30003830 con 0x7f3b40151a10 2024-04-26T20:51:22.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.539+0000 7f3b46346640 1 -- 172.21.15.73:0/3538147149 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b401536e0 con 0x7f3b40151a10 2024-04-26T20:51:22.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.539+0000 7f3b46346640 1 -- 172.21.15.73:0/3538147149 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b40153ca0 con 0x7f3b40151a10 2024-04-26T20:51:22.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.540+0000 7f3b46346640 1 -- 172.21.15.73:0/3538147149 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b400694f0 con 0x7f3b40151a10 2024-04-26T20:51:22.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.540+0000 7f3b3cff9640 1 -- 172.21.15.73:0/3538147149 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3b300039d0 con 0x7f3b40151a10 2024-04-26T20:51:22.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.540+0000 7f3b3cff9640 1 --2- 172.21.15.73:0/3538147149 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f3b2c06c8f0 0x7f3b2c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:22.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.540+0000 7f3b3cff9640 1 -- 172.21.15.73:0/3538147149 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f3b3008bb30 con 0x7f3b40151a10 2024-04-26T20:51:22.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.540+0000 7f3b3ffff640 1 --2- 172.21.15.73:0/3538147149 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f3b2c06c8f0 0x7f3b2c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:22.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.541+0000 7f3b3ffff640 1 --2- 172.21.15.73:0/3538147149 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f3b2c06c8f0 0x7f3b2c06edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3b400b6a60 tx=0x7f3b34009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:22.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.545+0000 7f3b3cff9640 1 -- 172.21.15.73:0/3538147149 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3b30059b60 con 0x7f3b40151a10 2024-04-26T20:51:22.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.728+0000 7f3b46346640 1 -- 172.21.15.73:0/3538147149 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3b4006de80 con 0x7f3b40151a10 2024-04-26T20:51:22.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.729+0000 7f3b3cff9640 1 -- 172.21.15.73:0/3538147149 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18902 (secure 0 0 0) 0x7f3b30059460 con 0x7f3b40151a10 2024-04-26T20:51:22.730 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:51:22.730 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":27,"fsid":"fbd3f548-b547-4713-856b-09f98f5d1764","created":"2024-04-26T20:50:07.876786+0000","modified":"2024-04-26T20:51:13.783133+0000","last_up_change":"2024-04-26T20:50:19.082741+0000","last_in_change":"2024-04-26T20:50:10.018400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:50:19.993731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:50:29.964420+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:50:30.920082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:50:34.650207+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"72d6f4f3-214a-4392-a02f-c33cbcab3c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6809","nonce":3461705124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6811","nonce":3461705124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6815","nonce":3461705124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6813","nonce":3461705124}]},"public_addr":"172.21.15.73:6809/3461705124","cluster_addr":"172.21.15.73:6811/3461705124","heartbeat_back_addr":"172.21.15.73:6815/3461705124","heartbeat_front_addr":"172.21.15.73:6813/3461705124","state":["exists","up"]},{"osd":1,"uuid":"786bf8ca-6ddc-4b3c-be6c-9152398bd5fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6817","nonce":2827619403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6819","nonce":2827619403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6825","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6827","nonce":2827619403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6821","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6823","nonce":2827619403}]},"public_addr":"172.21.15.73:6817/2827619403","cluster_addr":"172.21.15.73:6819/2827619403","heartbeat_back_addr":"172.21.15.73:6827/2827619403","heartbeat_front_addr":"172.21.15.73:6823/2827619403","state":["exists","up"]},{"osd":2,"uuid":"90963b1a-5437-4ef8-ad8b-6b74c534eeb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6801","nonce":2542681064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6803","nonce":2542681064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6808","nonce":2542681064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6805","nonce":2542681064}]},"public_addr":"172.21.15.73:6801/2542681064","cluster_addr":"172.21.15.73:6803/2542681064","heartbeat_back_addr":"172.21.15.73:6808/2542681064","heartbeat_front_addr":"172.21.15.73:6805/2542681064","state":["exists","up"]},{"osd":3,"uuid":"bd3d2148-6e38-41b4-a453-fe6e3b8b4ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6822","nonce":1604042718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6826","nonce":1604042718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6831","nonce":1604042718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6829","nonce":1604042718}]},"public_addr":"172.21.15.73:6822/1604042718","cluster_addr":"172.21.15.73:6826/1604042718","heartbeat_back_addr":"172.21.15.73:6831/1604042718","heartbeat_front_addr":"172.21.15.73:6829/1604042718","state":["exists","up"]},{"osd":4,"uuid":"e8668dc8-a23f-46d2-8311-cb882da08e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6817","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6819","nonce":3591055561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6824","nonce":3591055561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6831","nonce":3591055561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6828","nonce":3591055561}]},"public_addr":"172.21.15.120:6819/3591055561","cluster_addr":"172.21.15.120:6824/3591055561","heartbeat_back_addr":"172.21.15.120:6831/3591055561","heartbeat_front_addr":"172.21.15.120:6828/3591055561","state":["exists","up"]},{"osd":5,"uuid":"cbac0bdb-96bd-4852-bcce-e564495c3fcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6809","nonce":819450433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6811","nonce":819450433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6815","nonce":819450433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6813","nonce":819450433}]},"public_addr":"172.21.15.120:6809/819450433","cluster_addr":"172.21.15.120:6811/819450433","heartbeat_back_addr":"172.21.15.120:6815/819450433","heartbeat_front_addr":"172.21.15.120:6813/819450433","state":["exists","up"]},{"osd":6,"uuid":"1884b2a8-569f-4272-96a9-f25a09364dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6801","nonce":3946210727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6803","nonce":3946210727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6807","nonce":3946210727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6805","nonce":3946210727}]},"public_addr":"172.21.15.120:6801/3946210727","cluster_addr":"172.21.15.120:6803/3946210727","heartbeat_back_addr":"172.21.15.120:6807/3946210727","heartbeat_front_addr":"172.21.15.120:6805/3946210727","state":["exists","up"]},{"osd":7,"uuid":"c0c3bf6d-a4c5-4f1e-aa2d-330951830666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6818","nonce":2915574674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6821","nonce":2915574674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6827","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6829","nonce":2915574674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6823","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6825","nonce":2915574674}]},"public_addr":"172.21.15.120:6818/2915574674","cluster_addr":"172.21.15.120:6821/2915574674","heartbeat_back_addr":"172.21.15.120:6829/2915574674","heartbeat_front_addr":"172.21.15.120:6825/2915574674","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.218216+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.208710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.112705+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.182789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.164324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.174212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.225396+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.261559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:51:22.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.731+0000 7f3b46346640 1 -- 172.21.15.73:0/3538147149 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f3b2c06c8f0 msgr2=0x7f3b2c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:22.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.731+0000 7f3b46346640 1 --2- 172.21.15.73:0/3538147149 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f3b2c06c8f0 0x7f3b2c06edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3b400b6a60 tx=0x7f3b34009040 comp rx=0 tx=0).stop 2024-04-26T20:51:22.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.731+0000 7f3b46346640 1 -- 172.21.15.73:0/3538147149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b40151a10 msgr2=0x7f3b401f63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:22.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.731+0000 7f3b46346640 1 --2- 172.21.15.73:0/3538147149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b40151a10 0x7f3b401f63b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3b3000e440 tx=0x7f3b3000e900 comp rx=0 tx=0).stop 2024-04-26T20:51:22.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.731+0000 7f3b46346640 1 -- 172.21.15.73:0/3538147149 shutdown_connections 2024-04-26T20:51:22.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.731+0000 7f3b46346640 1 --2- 172.21.15.73:0/3538147149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b40151a10 0x7f3b401f63b0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:22.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.731+0000 7f3b46346640 1 --2- 172.21.15.73:0/3538147149 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f3b2c06c8f0 0x7f3b2c06edb0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:22.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.731+0000 7f3b46346640 1 --2- 172.21.15.73:0/3538147149 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3b400c60e0 0x7f3b401f5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:22.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.731+0000 7f3b46346640 1 --2- 172.21.15.73:0/3538147149 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b280051c0 0x7f3b401f3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:22.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.731+0000 7f3b46346640 1 -- 172.21.15.73:0/3538147149 >> 172.21.15.73:0/3538147149 conn(0x7f3b400d0be0 msgr2=0x7f3b4015e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:22.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.731+0000 7f3b46346640 1 -- 172.21.15.73:0/3538147149 shutdown_connections 2024-04-26T20:51:22.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.731+0000 7f3b46346640 1 -- 172.21.15.73:0/3538147149 wait complete. 2024-04-26T20:51:22.744 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:50:19.993731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:50:29.964420+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:50:30.920082+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T20:50:34.650207+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T20:51:22.744 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:51:22.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.849+0000 7fd2d8914640 1 Processor -- start 2024-04-26T20:51:22.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.849+0000 7fd2d8914640 1 -- start start 2024-04-26T20:51:22.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.850+0000 7fd2d8914640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd2d4160910 0x7fd2d40d1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:22.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.850+0000 7fd2d8914640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2d4161bf0 con 0x7fd2d40b3c60 2024-04-26T20:51:22.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.850+0000 7fd2d8914640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2d40cda60 con 0x7fd2d40d3970 2024-04-26T20:51:22.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.850+0000 7fd2d8914640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7fd2d40d02f0 con 0x7fd2d4160910 2024-04-26T20:51:22.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.850+0000 7fd2d259c640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd2d4160910 0x7fd2d40d1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:22.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.850+0000 7fd2cbfff640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fd2d40b3c60 0x7fd2d4160dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:34546/0 (socket says 172.21.15.73:34546) 2024-04-26T20:51:22.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.850+0000 7fd2cbfff640 1 -- 172.21.15.73:0/2848471330 learned_addr learned my addr 172.21.15.73:0/2848471330 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:22.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.850+0000 7fd2d259c640 1 -- 172.21.15.73:0/2848471330 >> v1:172.21.15.120:6789/0 conn(0x7fd2d40d3970 legacy=0x7fd2d41614e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:22.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.850+0000 7fd2d259c640 1 -- 172.21.15.73:0/2848471330 >> v1:172.21.15.73:6789/0 conn(0x7fd2d40b3c60 legacy=0x7fd2d4160dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:22.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.850+0000 7fd2d259c640 1 -- 172.21.15.73:0/2848471330 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2d40d0010 con 0x7fd2d4160910 2024-04-26T20:51:22.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.850+0000 7fd2d1d9b640 1 -- 172.21.15.73:0/2848471330 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4261641298 0 0) 0x7fd2d4161bf0 con 0x7fd2d40b3c60 2024-04-26T20:51:22.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.851+0000 7fd2d259c640 1 --2- 172.21.15.73:0/2848471330 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd2d4160910 0x7fd2d40d1430 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd2c40097a0 tx=0x7fd2c402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80c9ef468fd28d87 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:22.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.851+0000 7fd2d1d9b640 1 -- 172.21.15.73:0/2848471330 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c4037070 con 0x7fd2d4160910 2024-04-26T20:51:22.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.851+0000 7fd2d1d9b640 1 -- 172.21.15.73:0/2848471330 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2c4032440 con 0x7fd2d4160910 2024-04-26T20:51:22.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.851+0000 7fd2d1d9b640 1 -- 172.21.15.73:0/2848471330 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c402ae60 con 0x7fd2d4160910 2024-04-26T20:51:22.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.852+0000 7fd2d8914640 1 -- 172.21.15.73:0/2848471330 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd2d4160910 msgr2=0x7fd2d40d1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:22.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.852+0000 7fd2d8914640 1 --2- 172.21.15.73:0/2848471330 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd2d4160910 0x7fd2d40d1430 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd2c40097a0 tx=0x7fd2c402a110 comp rx=0 tx=0).stop 2024-04-26T20:51:22.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.852+0000 7fd2d8914640 1 -- 172.21.15.73:0/2848471330 shutdown_connections 2024-04-26T20:51:22.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.852+0000 7fd2d8914640 1 --2- 172.21.15.73:0/2848471330 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd2d4160910 0x7fd2d40d1430 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:22.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.852+0000 7fd2d8914640 1 -- 172.21.15.73:0/2848471330 >> 172.21.15.73:0/2848471330 conn(0x7fd2d40d2e50 msgr2=0x7fd2d40d3260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:22.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.852+0000 7fd2d8914640 1 -- 172.21.15.73:0/2848471330 shutdown_connections 2024-04-26T20:51:22.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.852+0000 7fd2d8914640 1 -- 172.21.15.73:0/2848471330 wait complete. 2024-04-26T20:51:22.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.853+0000 7fd2d8914640 1 Processor -- start 2024-04-26T20:51:22.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.853+0000 7fd2d8914640 1 -- start start 2024-04-26T20:51:22.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.854+0000 7fd2d8914640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd2d40b3c60 0x7fd2d415f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:22.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.854+0000 7fd2d8914640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd2d40d3970 0x7fd2d40c9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:22.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.854+0000 7fd2d8914640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd2d4160910 0x7fd2d40c78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:22.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.854+0000 7fd2d8914640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d40d0ea0 con 0x7fd2d40b3c60 2024-04-26T20:51:22.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.854+0000 7fd2d8914640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d40d1180 con 0x7fd2d40d3970 2024-04-26T20:51:22.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.854+0000 7fd2d8914640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fd2d40b4260 con 0x7fd2d4160910 2024-04-26T20:51:22.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.854+0000 7fd2d259c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd2d40b3c60 0x7fd2d415f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:22.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.854+0000 7fd2d259c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd2d40b3c60 0x7fd2d415f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59886/0 (socket says 172.21.15.73:59886) 2024-04-26T20:51:22.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.854+0000 7fd2d259c640 1 -- 172.21.15.73:0/1576006007 learned_addr learned my addr 172.21.15.73:0/1576006007 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:22.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.854+0000 7fd2cbfff640 1 --2- 172.21.15.73:0/1576006007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd2d40d3970 0x7fd2d40c9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:22.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.854+0000 7fd2d2d9d640 1 --2- 172.21.15.73:0/1576006007 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd2d4160910 0x7fd2d40c78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:22.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.854+0000 7fd2d259c640 1 -- 172.21.15.73:0/1576006007 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd2d4160910 msgr2=0x7fd2d40c78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:22.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.854+0000 7fd2d259c640 1 --2- 172.21.15.73:0/1576006007 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd2d4160910 0x7fd2d40c78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:22.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.854+0000 7fd2d259c640 1 -- 172.21.15.73:0/1576006007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd2d40d3970 msgr2=0x7fd2d40c9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:22.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.854+0000 7fd2d259c640 1 --2- 172.21.15.73:0/1576006007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd2d40d3970 0x7fd2d40c9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:22.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.854+0000 7fd2d259c640 1 -- 172.21.15.73:0/1576006007 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2d40c9860 con 0x7fd2d40b3c60 2024-04-26T20:51:22.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.855+0000 7fd2d259c640 1 --2- 172.21.15.73:0/1576006007 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd2d40b3c60 0x7fd2d415f310 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd2c40098c0 tx=0x7fd2c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:22.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.855+0000 7fd2caffd640 1 -- 172.21.15.73:0/1576006007 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c4037040 con 0x7fd2d40b3c60 2024-04-26T20:51:22.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.855+0000 7fd2caffd640 1 -- 172.21.15.73:0/1576006007 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2c4042070 con 0x7fd2d40b3c60 2024-04-26T20:51:22.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.855+0000 7fd2caffd640 1 -- 172.21.15.73:0/1576006007 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c4030c70 con 0x7fd2d40b3c60 2024-04-26T20:51:22.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.855+0000 7fd2d8914640 1 -- 172.21.15.73:0/1576006007 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2d40c8280 con 0x7fd2d40b3c60 2024-04-26T20:51:22.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.855+0000 7fd2d8914640 1 -- 172.21.15.73:0/1576006007 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2d40c8840 con 0x7fd2d40b3c60 2024-04-26T20:51:22.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.856+0000 7fd2d8914640 1 -- 172.21.15.73:0/1576006007 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2d40694f0 con 0x7fd2d40b3c60 2024-04-26T20:51:22.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.856+0000 7fd2caffd640 1 -- 172.21.15.73:0/1576006007 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd2c4047460 con 0x7fd2d40b3c60 2024-04-26T20:51:22.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.857+0000 7fd2caffd640 1 --2- 172.21.15.73:0/1576006007 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd2b406c8f0 0x7fd2b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:22.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.857+0000 7fd2cbfff640 1 --2- 172.21.15.73:0/1576006007 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd2b406c8f0 0x7fd2b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:22.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.857+0000 7fd2caffd640 1 -- 172.21.15.73:0/1576006007 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fd2c40aca10 con 0x7fd2d40b3c60 2024-04-26T20:51:22.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.857+0000 7fd2cbfff640 1 --2- 172.21.15.73:0/1576006007 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd2b406c8f0 0x7fd2b406edb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd2d40c76d0 tx=0x7fd2bc007e40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:22.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:22.861+0000 7fd2caffd640 1 -- 172.21.15.73:0/1576006007 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd2c407a480 con 0x7fd2d40b3c60 2024-04-26T20:51:23.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.042+0000 7fd2d8914640 1 -- 172.21.15.73:0/1576006007 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd2d40d1db0 con 0x7fd2d40b3c60 2024-04-26T20:51:23.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.043+0000 7fd2caffd640 1 -- 172.21.15.73:0/1576006007 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7fd2c407e780 con 0x7fd2d40b3c60 2024-04-26T20:51:23.044 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-26T20:51:23.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.045+0000 7fd2d8914640 1 -- 172.21.15.73:0/1576006007 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd2b406c8f0 msgr2=0x7fd2b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:23.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.045+0000 7fd2d8914640 1 --2- 172.21.15.73:0/1576006007 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd2b406c8f0 0x7fd2b406edb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd2d40c76d0 tx=0x7fd2bc007e40 comp rx=0 tx=0).stop 2024-04-26T20:51:23.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.045+0000 7fd2d8914640 1 -- 172.21.15.73:0/1576006007 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd2d40b3c60 msgr2=0x7fd2d415f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:23.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.045+0000 7fd2d8914640 1 --2- 172.21.15.73:0/1576006007 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd2d40b3c60 0x7fd2d415f310 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd2c40098c0 tx=0x7fd2c40046c0 comp rx=0 tx=0).stop 2024-04-26T20:51:23.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.045+0000 7fd2d8914640 1 -- 172.21.15.73:0/1576006007 shutdown_connections 2024-04-26T20:51:23.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.045+0000 7fd2d8914640 1 --2- 172.21.15.73:0/1576006007 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd2b406c8f0 0x7fd2b406edb0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:23.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.045+0000 7fd2d8914640 1 --2- 172.21.15.73:0/1576006007 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd2d4160910 0x7fd2d40c78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:23.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.045+0000 7fd2d8914640 1 --2- 172.21.15.73:0/1576006007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd2d40d3970 0x7fd2d40c9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:23.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.045+0000 7fd2d8914640 1 --2- 172.21.15.73:0/1576006007 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd2d40b3c60 0x7fd2d415f310 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:23.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.045+0000 7fd2d8914640 1 -- 172.21.15.73:0/1576006007 >> 172.21.15.73:0/1576006007 conn(0x7fd2d40d2e50 msgr2=0x7fd2d40717e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:23.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.045+0000 7fd2d8914640 1 -- 172.21.15.73:0/1576006007 shutdown_connections 2024-04-26T20:51:23.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.045+0000 7fd2d8914640 1 -- 172.21.15.73:0/1576006007 wait complete. 2024-04-26T20:51:23.056 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:51:23.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.157+0000 7f9261ec8640 1 Processor -- start 2024-04-26T20:51:23.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.157+0000 7f9261ec8640 1 -- start start 2024-04-26T20:51:23.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.157+0000 7f9261ec8640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f925c0d1240 0x7f925c0d1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:23.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.157+0000 7f9261ec8640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f925c0b3610 0x7f925c0cff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:23.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.157+0000 7f9261ec8640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f925c0d1b60 con 0x7f925c0b4360 2024-04-26T20:51:23.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.157+0000 7f9261ec8640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f925c0cf210 con 0x7f925c0d1240 2024-04-26T20:51:23.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.157+0000 7f9261ec8640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f925c0ce5e0 con 0x7f925c0b3610 2024-04-26T20:51:23.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.157+0000 7f925affd640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f925c0b4360 0x7f925c0cf870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:34552/0 (socket says 172.21.15.73:34552) 2024-04-26T20:51:23.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.158+0000 7f925affd640 1 -- 172.21.15.73:0/3457203103 learned_addr learned my addr 172.21.15.73:0/3457203103 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:23.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.158+0000 7f925b7fe640 1 --2- 172.21.15.73:0/3457203103 >> v2:172.21.15.120:3300/0 conn(0x7f925c0d1240 0x7f925c0d1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:23.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.158+0000 7f925bfff640 1 --2- 172.21.15.73:0/3457203103 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f925c0b3610 0x7f925c0cff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:23.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.158+0000 7f925a7fc640 1 -- 172.21.15.73:0/3457203103 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1847840364 0 0) 0x7f925c0d1b60 con 0x7f925c0b4360 2024-04-26T20:51:23.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.158+0000 7f925a7fc640 1 -- 172.21.15.73:0/3457203103 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9244003520 con 0x7f925c0b4360 2024-04-26T20:51:23.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.158+0000 7f925bfff640 1 -- 172.21.15.73:0/3457203103 >> v2:172.21.15.120:3300/0 conn(0x7f925c0d1240 msgr2=0x7f925c0d1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:23.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.158+0000 7f925bfff640 1 --2- 172.21.15.73:0/3457203103 >> v2:172.21.15.120:3300/0 conn(0x7f925c0d1240 0x7f925c0d1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:23.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.158+0000 7f925bfff640 1 -- 172.21.15.73:0/3457203103 >> v1:172.21.15.73:6789/0 conn(0x7f925c0b4360 legacy=0x7f925c0cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:23.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.158+0000 7f925bfff640 1 -- 172.21.15.73:0/3457203103 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f925c0ce8c0 con 0x7f925c0b3610 2024-04-26T20:51:23.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.158+0000 7f925b7fe640 1 --2- 172.21.15.73:0/3457203103 >> v2:172.21.15.120:3300/0 conn(0x7f925c0d1240 0x7f925c0d1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:51:23.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.159+0000 7f925bfff640 1 --2- 172.21.15.73:0/3457203103 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f925c0b3610 0x7f925c0cff80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9254009d90 tx=0x7f925402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebb937abeb0adfa6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:23.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.159+0000 7f925a7fc640 1 -- 172.21.15.73:0/3457203103 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9254007740 con 0x7f925c0b3610 2024-04-26T20:51:23.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.159+0000 7f925a7fc640 1 -- 172.21.15.73:0/3457203103 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92540078e0 con 0x7f925c0b3610 2024-04-26T20:51:23.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.159+0000 7f925a7fc640 1 -- 172.21.15.73:0/3457203103 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9254007cb0 con 0x7f925c0b3610 2024-04-26T20:51:23.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.159+0000 7f9261ec8640 1 -- 172.21.15.73:0/3457203103 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f925c0b3610 msgr2=0x7f925c0cff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:23.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.159+0000 7f9261ec8640 1 --2- 172.21.15.73:0/3457203103 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f925c0b3610 0x7f925c0cff80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9254009d90 tx=0x7f925402a660 comp rx=0 tx=0).stop 2024-04-26T20:51:23.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.159+0000 7f9261ec8640 1 -- 172.21.15.73:0/3457203103 shutdown_connections 2024-04-26T20:51:23.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.159+0000 7f9261ec8640 1 --2- 172.21.15.73:0/3457203103 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f925c0b3610 0x7f925c0cff80 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:23.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.159+0000 7f9261ec8640 1 --2- 172.21.15.73:0/3457203103 >> v2:172.21.15.120:3300/0 conn(0x7f925c0d1240 0x7f925c0d1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:23.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.159+0000 7f9261ec8640 1 -- 172.21.15.73:0/3457203103 >> 172.21.15.73:0/3457203103 conn(0x7f925c0d2e60 msgr2=0x7f925c0d3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:23.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.159+0000 7f9261ec8640 1 -- 172.21.15.73:0/3457203103 shutdown_connections 2024-04-26T20:51:23.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.159+0000 7f9261ec8640 1 -- 172.21.15.73:0/3457203103 wait complete. 2024-04-26T20:51:23.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.161+0000 7f9261ec8640 1 Processor -- start 2024-04-26T20:51:23.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.161+0000 7f9261ec8640 1 -- start start 2024-04-26T20:51:23.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.161+0000 7f9261ec8640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f925c0b3610 0x7f925c153c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:23.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.161+0000 7f9261ec8640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f925c0d1240 0x7f925c1f7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:23.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.161+0000 7f9261ec8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f925c0cc7e0 0x7f925c0c7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:23.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.161+0000 7f9261ec8640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f925c0c6610 con 0x7f925c0d1240 2024-04-26T20:51:23.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.161+0000 7f9261ec8640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f925c0cf460 con 0x7f925c0cc7e0 2024-04-26T20:51:23.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.161+0000 7f9261ec8640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f925c0c6790 con 0x7f925c0b3610 2024-04-26T20:51:23.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.162+0000 7f925affd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f925c0d1240 0x7f925c1f7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:23.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.162+0000 7f925affd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f925c0d1240 0x7f925c1f7050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59894/0 (socket says 172.21.15.73:59894) 2024-04-26T20:51:23.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.162+0000 7f925affd640 1 -- 172.21.15.73:0/3202201059 learned_addr learned my addr 172.21.15.73:0/3202201059 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:23.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.162+0000 7f925bfff640 1 --2- 172.21.15.73:0/3202201059 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f925c0cc7e0 0x7f925c0c7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:23.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.162+0000 7f925b7fe640 1 --2- 172.21.15.73:0/3202201059 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f925c0b3610 0x7f925c153c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:23.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.162+0000 7f925affd640 1 -- 172.21.15.73:0/3202201059 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f925c0b3610 msgr2=0x7f925c153c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:23.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.162+0000 7f925affd640 1 --2- 172.21.15.73:0/3202201059 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f925c0b3610 0x7f925c153c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:23.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.162+0000 7f925affd640 1 -- 172.21.15.73:0/3202201059 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f925c0cc7e0 msgr2=0x7f925c0c7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:23.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.162+0000 7f925affd640 1 --2- 172.21.15.73:0/3202201059 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f925c0cc7e0 0x7f925c0c7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:23.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.162+0000 7f925affd640 1 -- 172.21.15.73:0/3202201059 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f925c0c6a70 con 0x7f925c0d1240 2024-04-26T20:51:23.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.162+0000 7f925affd640 1 --2- 172.21.15.73:0/3202201059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f925c0d1240 0x7f925c1f7050 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f924c00c430 tx=0x7f924c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:23.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.162+0000 7f9243fff640 1 -- 172.21.15.73:0/3202201059 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924c017070 con 0x7f925c0d1240 2024-04-26T20:51:23.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.162+0000 7f9243fff640 1 -- 172.21.15.73:0/3202201059 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f924c005ce0 con 0x7f925c0d1240 2024-04-26T20:51:23.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.162+0000 7f9261ec8640 1 -- 172.21.15.73:0/3202201059 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f925c0c6d50 con 0x7f925c0d1240 2024-04-26T20:51:23.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.162+0000 7f9243fff640 1 -- 172.21.15.73:0/3202201059 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924c0072f0 con 0x7f925c0d1240 2024-04-26T20:51:23.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.162+0000 7f9261ec8640 1 -- 172.21.15.73:0/3202201059 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f925c0c7310 con 0x7f925c0d1240 2024-04-26T20:51:23.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.163+0000 7f9261ec8640 1 -- 172.21.15.73:0/3202201059 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f925c0694f0 con 0x7f925c0d1240 2024-04-26T20:51:23.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.164+0000 7f9243fff640 1 -- 172.21.15.73:0/3202201059 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f924c01a070 con 0x7f925c0d1240 2024-04-26T20:51:23.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.164+0000 7f9243fff640 1 --2- 172.21.15.73:0/3202201059 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f924806c920 0x7f924806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:23.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.164+0000 7f9243fff640 1 -- 172.21.15.73:0/3202201059 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f924c08b2a0 con 0x7f925c0d1240 2024-04-26T20:51:23.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.164+0000 7f925b7fe640 1 --2- 172.21.15.73:0/3202201059 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f924806c920 0x7f924806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:23.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.165+0000 7f925b7fe640 1 --2- 172.21.15.73:0/3202201059 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f924806c920 0x7f924806ede0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9250009560 tx=0x7f9250009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:23.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.169+0000 7f9243fff640 1 -- 172.21.15.73:0/3202201059 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f924c058d10 con 0x7f925c0d1240 2024-04-26T20:51:23.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.351+0000 7f9261ec8640 1 -- 172.21.15.73:0/3202201059 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f925c0d0110 con 0x7f925c0d1240 2024-04-26T20:51:23.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.352+0000 7f9243fff640 1 -- 172.21.15.73:0/3202201059 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f924c05d010 con 0x7f925c0d1240 2024-04-26T20:51:23.353 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-26T20:51:23.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.354+0000 7f9261ec8640 1 -- 172.21.15.73:0/3202201059 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f924806c920 msgr2=0x7f924806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:23.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.354+0000 7f9261ec8640 1 --2- 172.21.15.73:0/3202201059 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f924806c920 0x7f924806ede0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9250009560 tx=0x7f9250009190 comp rx=0 tx=0).stop 2024-04-26T20:51:23.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.354+0000 7f9261ec8640 1 -- 172.21.15.73:0/3202201059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f925c0d1240 msgr2=0x7f925c1f7050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:23.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.354+0000 7f9261ec8640 1 --2- 172.21.15.73:0/3202201059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f925c0d1240 0x7f925c1f7050 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f924c00c430 tx=0x7f924c00c8f0 comp rx=0 tx=0).stop 2024-04-26T20:51:23.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.354+0000 7f9261ec8640 1 -- 172.21.15.73:0/3202201059 shutdown_connections 2024-04-26T20:51:23.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.354+0000 7f9261ec8640 1 --2- 172.21.15.73:0/3202201059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f925c0d1240 0x7f925c1f7050 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:23.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.354+0000 7f9261ec8640 1 --2- 172.21.15.73:0/3202201059 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f924806c920 0x7f924806ede0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:23.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.354+0000 7f9261ec8640 1 --2- 172.21.15.73:0/3202201059 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f925c0cc7e0 0x7f925c0c7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:23.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.354+0000 7f9261ec8640 1 --2- 172.21.15.73:0/3202201059 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f925c0b3610 0x7f925c153c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:23.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.354+0000 7f9261ec8640 1 -- 172.21.15.73:0/3202201059 >> 172.21.15.73:0/3202201059 conn(0x7f925c0d2e60 msgr2=0x7f925c0b1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:23.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.354+0000 7f9261ec8640 1 -- 172.21.15.73:0/3202201059 shutdown_connections 2024-04-26T20:51:23.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.354+0000 7f9261ec8640 1 -- 172.21.15.73:0/3202201059 wait complete. 2024-04-26T20:51:23.365 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:51:23.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.464+0000 7f887bc45640 1 Processor -- start 2024-04-26T20:51:23.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.464+0000 7f887bc45640 1 -- start start 2024-04-26T20:51:23.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.465+0000 7f887bc45640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8874156460 0x7f88740cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:23.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.465+0000 7f887bc45640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f88740cba10 0x7f887406f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:23.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.465+0000 7f887bc45640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8874156840 con 0x7f887406fcd0 2024-04-26T20:51:23.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.465+0000 7f887bc45640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f88740cbf20 con 0x7f8874156460 2024-04-26T20:51:23.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.465+0000 7f887bc45640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f88740cc200 con 0x7f88740cba10 2024-04-26T20:51:23.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.465+0000 7f88799e1640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8874156460 0x7f88740cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:23.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.465+0000 7f887a1e2640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f887406fcd0 0x7f887415f980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:34558/0 (socket says 172.21.15.73:34558) 2024-04-26T20:51:23.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.465+0000 7f887a1e2640 1 -- 172.21.15.73:0/2003201350 learned_addr learned my addr 172.21.15.73:0/2003201350 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:23.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.465+0000 7f88791e0640 1 --2- 172.21.15.73:0/2003201350 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f88740cba10 0x7f887406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:23.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.465+0000 7f88799e1640 1 -- 172.21.15.73:0/2003201350 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f88740cba10 msgr2=0x7f887406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:23.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.465+0000 7f88799e1640 1 --2- 172.21.15.73:0/2003201350 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f88740cba10 0x7f887406f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:23.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.465+0000 7f88799e1640 1 -- 172.21.15.73:0/2003201350 >> v1:172.21.15.73:6789/0 conn(0x7f887406fcd0 legacy=0x7f887415f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:23.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.465+0000 7f88799e1640 1 -- 172.21.15.73:0/2003201350 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88740cc4e0 con 0x7f8874156460 2024-04-26T20:51:23.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.465+0000 7f88789df640 1 -- 172.21.15.73:0/2003201350 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1554541773 0 0) 0x7f8874156840 con 0x7f887406fcd0 2024-04-26T20:51:23.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.466+0000 7f88799e1640 1 --2- 172.21.15.73:0/2003201350 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8874156460 0x7f88740cb4d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f887000bfa0 tx=0x7f887002c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa898f4f301b95f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:23.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.466+0000 7f88789df640 1 -- 172.21.15.73:0/2003201350 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8870039070 con 0x7f8874156460 2024-04-26T20:51:23.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.466+0000 7f88789df640 1 -- 172.21.15.73:0/2003201350 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8870009540 con 0x7f8874156460 2024-04-26T20:51:23.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.466+0000 7f88789df640 1 -- 172.21.15.73:0/2003201350 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8870009910 con 0x7f8874156460 2024-04-26T20:51:23.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.467+0000 7f887bc45640 1 -- 172.21.15.73:0/2003201350 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8874156460 msgr2=0x7f88740cb4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:23.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.467+0000 7f887bc45640 1 --2- 172.21.15.73:0/2003201350 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8874156460 0x7f88740cb4d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f887000bfa0 tx=0x7f887002c860 comp rx=0 tx=0).stop 2024-04-26T20:51:23.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.467+0000 7f887bc45640 1 -- 172.21.15.73:0/2003201350 shutdown_connections 2024-04-26T20:51:23.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.467+0000 7f887bc45640 1 --2- 172.21.15.73:0/2003201350 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f88740cba10 0x7f887406f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:23.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.467+0000 7f887bc45640 1 --2- 172.21.15.73:0/2003201350 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8874156460 0x7f88740cb4d0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:23.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.467+0000 7f887bc45640 1 -- 172.21.15.73:0/2003201350 >> 172.21.15.73:0/2003201350 conn(0x7f88740d0be0 msgr2=0x7f88740d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:23.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.467+0000 7f887bc45640 1 -- 172.21.15.73:0/2003201350 shutdown_connections 2024-04-26T20:51:23.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.467+0000 7f887bc45640 1 -- 172.21.15.73:0/2003201350 wait complete. 2024-04-26T20:51:23.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.468+0000 7f887bc45640 1 Processor -- start 2024-04-26T20:51:23.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.468+0000 7f887bc45640 1 -- start start 2024-04-26T20:51:23.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.469+0000 7f887bc45640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f887406fcd0 0x7f887414e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:23.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.469+0000 7f887bc45640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f88740cba10 0x7f88740c4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:23.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.469+0000 7f887bc45640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8874156460 0x7f88740b5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:23.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.469+0000 7f887bc45640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f887406e260 con 0x7f8874156460 2024-04-26T20:51:23.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.469+0000 7f887bc45640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8874072700 con 0x7f88740cba10 2024-04-26T20:51:23.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.469+0000 7f887bc45640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f887406e3e0 con 0x7f887406fcd0 2024-04-26T20:51:23.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.469+0000 7f887a1e2640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8874156460 0x7f88740b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:23.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.469+0000 7f88799e1640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f887406fcd0 0x7f887414e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:23.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.469+0000 7f887a1e2640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8874156460 0x7f88740b5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59904/0 (socket says 172.21.15.73:59904) 2024-04-26T20:51:23.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.469+0000 7f887a1e2640 1 -- 172.21.15.73:0/2001932152 learned_addr learned my addr 172.21.15.73:0/2001932152 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:23.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.469+0000 7f88791e0640 1 --2- 172.21.15.73:0/2001932152 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f88740cba10 0x7f88740c4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:23.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.469+0000 7f887a1e2640 1 -- 172.21.15.73:0/2001932152 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f887406fcd0 msgr2=0x7f887414e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:23.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.469+0000 7f887a1e2640 1 --2- 172.21.15.73:0/2001932152 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f887406fcd0 0x7f887414e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:23.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.469+0000 7f887a1e2640 1 -- 172.21.15.73:0/2001932152 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f88740cba10 msgr2=0x7f88740c4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:23.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.469+0000 7f887a1e2640 1 --2- 172.21.15.73:0/2001932152 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f88740cba10 0x7f88740c4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:23.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.469+0000 7f887a1e2640 1 -- 172.21.15.73:0/2001932152 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f887414e690 con 0x7f8874156460 2024-04-26T20:51:23.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.469+0000 7f88799e1640 1 --2- 172.21.15.73:0/2001932152 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f887406fcd0 0x7f887414e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:51:23.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.469+0000 7f887a1e2640 1 --2- 172.21.15.73:0/2001932152 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8874156460 0x7f88740b5400 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f886c00c450 tx=0x7f886c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:23.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.470+0000 7f886a7fc640 1 -- 172.21.15.73:0/2001932152 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f886c018070 con 0x7f8874156460 2024-04-26T20:51:23.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.470+0000 7f886a7fc640 1 -- 172.21.15.73:0/2001932152 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f886c007500 con 0x7f8874156460 2024-04-26T20:51:23.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.470+0000 7f886a7fc640 1 -- 172.21.15.73:0/2001932152 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f886c004a20 con 0x7f8874156460 2024-04-26T20:51:23.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.470+0000 7f887bc45640 1 -- 172.21.15.73:0/2001932152 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88740c5160 con 0x7f8874156460 2024-04-26T20:51:23.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.470+0000 7f887bc45640 1 -- 172.21.15.73:0/2001932152 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88740c5720 con 0x7f8874156460 2024-04-26T20:51:23.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.471+0000 7f887bc45640 1 -- 172.21.15.73:0/2001932152 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88740694f0 con 0x7f8874156460 2024-04-26T20:51:23.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.471+0000 7f886a7fc640 1 -- 172.21.15.73:0/2001932152 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f886c01b070 con 0x7f8874156460 2024-04-26T20:51:23.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.471+0000 7f886a7fc640 1 --2- 172.21.15.73:0/2001932152 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f885806c8f0 0x7f885806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:23.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.472+0000 7f886a7fc640 1 -- 172.21.15.73:0/2001932152 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f886c08b100 con 0x7f8874156460 2024-04-26T20:51:23.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.472+0000 7f88799e1640 1 --2- 172.21.15.73:0/2001932152 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f885806c8f0 0x7f885806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:23.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.472+0000 7f88799e1640 1 --2- 172.21.15.73:0/2001932152 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f885806c8f0 0x7f885806edb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8870036000 tx=0x7f887000b000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:23.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.476+0000 7f886a7fc640 1 -- 172.21.15.73:0/2001932152 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f887414e690 con 0x7f8874156460 2024-04-26T20:51:23.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.649+0000 7f887bc45640 1 -- 172.21.15.73:0/2001932152 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f88740b21f0 con 0x7f8874156460 2024-04-26T20:51:23.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.649+0000 7f886a7fc640 1 -- 172.21.15.73:0/2001932152 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f886c058ae0 con 0x7f8874156460 2024-04-26T20:51:23.650 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-26T20:51:23.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.651+0000 7f887bc45640 1 -- 172.21.15.73:0/2001932152 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f885806c8f0 msgr2=0x7f885806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:23.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.651+0000 7f887bc45640 1 --2- 172.21.15.73:0/2001932152 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f885806c8f0 0x7f885806edb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8870036000 tx=0x7f887000b000 comp rx=0 tx=0).stop 2024-04-26T20:51:23.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.651+0000 7f887bc45640 1 -- 172.21.15.73:0/2001932152 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8874156460 msgr2=0x7f88740b5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:23.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.651+0000 7f887bc45640 1 --2- 172.21.15.73:0/2001932152 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8874156460 0x7f88740b5400 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f886c00c450 tx=0x7f886c00c910 comp rx=0 tx=0).stop 2024-04-26T20:51:23.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.651+0000 7f887bc45640 1 -- 172.21.15.73:0/2001932152 shutdown_connections 2024-04-26T20:51:23.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.651+0000 7f887bc45640 1 --2- 172.21.15.73:0/2001932152 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8874156460 0x7f88740b5400 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:23.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.651+0000 7f887bc45640 1 --2- 172.21.15.73:0/2001932152 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f885806c8f0 0x7f885806edb0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:23.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.651+0000 7f887bc45640 1 --2- 172.21.15.73:0/2001932152 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f88740cba10 0x7f88740c4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:23.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.651+0000 7f887bc45640 1 --2- 172.21.15.73:0/2001932152 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f887406fcd0 0x7f887414e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:23.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.651+0000 7f887bc45640 1 -- 172.21.15.73:0/2001932152 >> 172.21.15.73:0/2001932152 conn(0x7f88740d0be0 msgr2=0x7f88740b0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:23.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.651+0000 7f887bc45640 1 -- 172.21.15.73:0/2001932152 shutdown_connections 2024-04-26T20:51:23.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.651+0000 7f887bc45640 1 -- 172.21.15.73:0/2001932152 wait complete. 2024-04-26T20:51:23.663 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T20:51:23.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.761+0000 7f46b59af640 1 Processor -- start 2024-04-26T20:51:23.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.761+0000 7f46b59af640 1 -- start start 2024-04-26T20:51:23.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.762+0000 7f46b59af640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46b00b40d0 0x7f46b01625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:23.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.762+0000 7f46b59af640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f46b0162b30 0x7f46b00b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:23.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.762+0000 7f46b59af640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46b00d0a10 con 0x7f46b00d1a00 2024-04-26T20:51:23.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.762+0000 7f46b59af640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f46b00d0cf0 con 0x7f46b0162b30 2024-04-26T20:51:23.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.762+0000 7f46b59af640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f46b00d0fd0 con 0x7f46b00b40d0 2024-04-26T20:51:23.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.762+0000 7f46af7fe640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f46b0162b30 0x7f46b00b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:23.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.762+0000 7f46af7fe640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f46b0162b30 0x7f46b00b4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59912/0 (socket says 172.21.15.73:59912) 2024-04-26T20:51:23.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.762+0000 7f46af7fe640 1 -- 172.21.15.73:0/198521343 learned_addr learned my addr 172.21.15.73:0/198521343 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:23.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.762+0000 7f46af7fe640 1 -- 172.21.15.73:0/198521343 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46b00b40d0 msgr2=0x7f46b01625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:23.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.762+0000 7f46af7fe640 1 --2- 172.21.15.73:0/198521343 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46b00b40d0 0x7f46b01625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:23.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.762+0000 7f46af7fe640 1 -- 172.21.15.73:0/198521343 >> v1:172.21.15.120:6789/0 conn(0x7f46b00d1a00 legacy=0x7f46b00d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:23.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.762+0000 7f46af7fe640 1 -- 172.21.15.73:0/198521343 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46b0162fa0 con 0x7f46b0162b30 2024-04-26T20:51:23.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.763+0000 7f46af7fe640 1 --2- 172.21.15.73:0/198521343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46b0162b30 0x7f46b00b4b00 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f46a8009870 tx=0x7f46a802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a387fa545a6a0d36 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:23.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.763+0000 7f46adffb640 1 -- 172.21.15.73:0/198521343 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a8037070 con 0x7f46b0162b30 2024-04-26T20:51:23.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.763+0000 7f46adffb640 1 -- 172.21.15.73:0/198521343 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46a8032440 con 0x7f46b0162b30 2024-04-26T20:51:23.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.763+0000 7f46adffb640 1 -- 172.21.15.73:0/198521343 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a8032810 con 0x7f46b0162b30 2024-04-26T20:51:23.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.764+0000 7f46b59af640 1 -- 172.21.15.73:0/198521343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46b0162b30 msgr2=0x7f46b00b4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:23.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.764+0000 7f46b59af640 1 --2- 172.21.15.73:0/198521343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46b0162b30 0x7f46b00b4b00 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f46a8009870 tx=0x7f46a802a180 comp rx=0 tx=0).stop 2024-04-26T20:51:23.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.764+0000 7f46b59af640 1 -- 172.21.15.73:0/198521343 shutdown_connections 2024-04-26T20:51:23.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.764+0000 7f46b59af640 1 --2- 172.21.15.73:0/198521343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46b0162b30 0x7f46b00b4b00 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:23.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.764+0000 7f46b59af640 1 --2- 172.21.15.73:0/198521343 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46b00b40d0 0x7f46b01625f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:23.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.764+0000 7f46b59af640 1 -- 172.21.15.73:0/198521343 >> 172.21.15.73:0/198521343 conn(0x7f46b00d4fd0 msgr2=0x7f46b00d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:23.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.764+0000 7f46b59af640 1 -- 172.21.15.73:0/198521343 shutdown_connections 2024-04-26T20:51:23.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.764+0000 7f46b59af640 1 -- 172.21.15.73:0/198521343 wait complete. 2024-04-26T20:51:23.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.765+0000 7f46b59af640 1 Processor -- start 2024-04-26T20:51:23.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.765+0000 7f46b59af640 1 -- start start 2024-04-26T20:51:23.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.766+0000 7f46b59af640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46b00b40d0 0x7f46b00cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:23.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.766+0000 7f46b59af640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f46b00d1a00 0x7f46b00ba840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:23.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.766+0000 7f46b59af640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46b0162b30 0x7f46b00cd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:23.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.766+0000 7f46b59af640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f46b00d2670 con 0x7f46b00b40d0 2024-04-26T20:51:23.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.766+0000 7f46b59af640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f46b00d24f0 con 0x7f46b00d1a00 2024-04-26T20:51:23.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.766+0000 7f46b59af640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f46b00b5040 con 0x7f46b0162b30 2024-04-26T20:51:23.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.766+0000 7f46aeffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46b00b40d0 0x7f46b00cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:23.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.766+0000 7f46aeffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46b00b40d0 0x7f46b00cb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59926/0 (socket says 172.21.15.73:59926) 2024-04-26T20:51:23.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.766+0000 7f46aeffd640 1 -- 172.21.15.73:0/1635590536 learned_addr learned my addr 172.21.15.73:0/1635590536 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:23.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.766+0000 7f46ae7fc640 1 --2- 172.21.15.73:0/1635590536 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f46b00d1a00 0x7f46b00ba840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:23.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.766+0000 7f46af7fe640 1 --2- 172.21.15.73:0/1635590536 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46b0162b30 0x7f46b00cd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:23.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.766+0000 7f46aeffd640 1 -- 172.21.15.73:0/1635590536 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46b0162b30 msgr2=0x7f46b00cd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:23.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.766+0000 7f46aeffd640 1 --2- 172.21.15.73:0/1635590536 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46b0162b30 0x7f46b00cd0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:23.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.766+0000 7f46aeffd640 1 -- 172.21.15.73:0/1635590536 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f46b00d1a00 msgr2=0x7f46b00ba840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:23.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.766+0000 7f46aeffd640 1 --2- 172.21.15.73:0/1635590536 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f46b00d1a00 0x7f46b00ba840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:23.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.766+0000 7f46aeffd640 1 -- 172.21.15.73:0/1635590536 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46b0154f40 con 0x7f46b00b40d0 2024-04-26T20:51:23.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.766+0000 7f46aeffd640 1 --2- 172.21.15.73:0/1635590536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46b00b40d0 0x7f46b00cb5a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f46a000c7d0 tx=0x7f46a000cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:23.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.767+0000 7f4697fff640 1 -- 172.21.15.73:0/1635590536 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a0017070 con 0x7f46b00b40d0 2024-04-26T20:51:23.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.767+0000 7f4697fff640 1 -- 172.21.15.73:0/1635590536 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46a0003da0 con 0x7f46b00b40d0 2024-04-26T20:51:23.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.767+0000 7f46b59af640 1 -- 172.21.15.73:0/1635590536 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46b00cadc0 con 0x7f46b00b40d0 2024-04-26T20:51:23.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.767+0000 7f4697fff640 1 -- 172.21.15.73:0/1635590536 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a00052b0 con 0x7f46b00b40d0 2024-04-26T20:51:23.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.767+0000 7f46b59af640 1 -- 172.21.15.73:0/1635590536 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46b00c8a70 con 0x7f46b00b40d0 2024-04-26T20:51:23.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.768+0000 7f46b59af640 1 -- 172.21.15.73:0/1635590536 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46b00c8c70 con 0x7f46b00b40d0 2024-04-26T20:51:23.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.768+0000 7f4697fff640 1 -- 172.21.15.73:0/1635590536 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f46a001a070 con 0x7f46b00b40d0 2024-04-26T20:51:23.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.769+0000 7f4697fff640 1 --2- 172.21.15.73:0/1635590536 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f469c06c920 0x7f469c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:23.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.769+0000 7f46ae7fc640 1 --2- 172.21.15.73:0/1635590536 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f469c06c920 0x7f469c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:23.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.769+0000 7f4697fff640 1 -- 172.21.15.73:0/1635590536 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f46a008b4d0 con 0x7f46b00b40d0 2024-04-26T20:51:23.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.769+0000 7f46ae7fc640 1 --2- 172.21.15.73:0/1635590536 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f469c06c920 0x7f469c06ede0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f46b0069e20 tx=0x7f46a4009110 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:23.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.773+0000 7f4697fff640 1 -- 172.21.15.73:0/1635590536 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f46b00c8c70 con 0x7f46b00b40d0 2024-04-26T20:51:23.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.950+0000 7f46b59af640 1 -- 172.21.15.73:0/1635590536 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f46b00694f0 con 0x7f46b00b40d0 2024-04-26T20:51:23.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.950+0000 7f4697fff640 1 -- 172.21.15.73:0/1635590536 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f46a000ce20 con 0x7f46b00b40d0 2024-04-26T20:51:23.951 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-26T20:51:23.953 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.952+0000 7f46b59af640 1 -- 172.21.15.73:0/1635590536 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f469c06c920 msgr2=0x7f469c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:23.953 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.952+0000 7f46b59af640 1 --2- 172.21.15.73:0/1635590536 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f469c06c920 0x7f469c06ede0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f46b0069e20 tx=0x7f46a4009110 comp rx=0 tx=0).stop 2024-04-26T20:51:23.953 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.952+0000 7f46b59af640 1 -- 172.21.15.73:0/1635590536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46b00b40d0 msgr2=0x7f46b00cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:23.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.952+0000 7f46b59af640 1 --2- 172.21.15.73:0/1635590536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46b00b40d0 0x7f46b00cb5a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f46a000c7d0 tx=0x7f46a000cbd0 comp rx=0 tx=0).stop 2024-04-26T20:51:23.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.952+0000 7f46b59af640 1 -- 172.21.15.73:0/1635590536 shutdown_connections 2024-04-26T20:51:23.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.952+0000 7f46b59af640 1 --2- 172.21.15.73:0/1635590536 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f469c06c920 0x7f469c06ede0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:23.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.952+0000 7f46b59af640 1 --2- 172.21.15.73:0/1635590536 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46b0162b30 0x7f46b00cd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:23.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.952+0000 7f46b59af640 1 --2- 172.21.15.73:0/1635590536 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f46b00d1a00 0x7f46b00ba840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:23.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.952+0000 7f46b59af640 1 --2- 172.21.15.73:0/1635590536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46b00b40d0 0x7f46b00cb5a0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:23.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.952+0000 7f46b59af640 1 -- 172.21.15.73:0/1635590536 >> 172.21.15.73:0/1635590536 conn(0x7f46b00d4fd0 msgr2=0x7f46b00d1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:23.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.952+0000 7f46b59af640 1 -- 172.21.15.73:0/1635590536 shutdown_connections 2024-04-26T20:51:23.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:23.952+0000 7f46b59af640 1 -- 172.21.15.73:0/1635590536 wait complete. 2024-04-26T20:51:23.964 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T20:51:24.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.060+0000 7f0e1579d640 1 Processor -- start 2024-04-26T20:51:24.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.060+0000 7f0e1579d640 1 -- start start 2024-04-26T20:51:24.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.061+0000 7f0e1579d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0e1015f5a0 0x7f0e100d5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:24.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.061+0000 7f0e1579d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f0e100d2830 0x7f0e10162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:24.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.061+0000 7f0e1579d640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e100d1a70 con 0x7f0e100d3c20 2024-04-26T20:51:24.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.061+0000 7f0e1579d640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f0e10163d30 con 0x7f0e100d2830 2024-04-26T20:51:24.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.061+0000 7f0e1579d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f0e100d1d50 con 0x7f0e1015f5a0 2024-04-26T20:51:24.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.061+0000 7f0e0f7fe640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f0e100d3c20 0x7f0e1015f990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:34568/0 (socket says 172.21.15.73:34568) 2024-04-26T20:51:24.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.061+0000 7f0e0f7fe640 1 -- 172.21.15.73:0/4167990534 learned_addr learned my addr 172.21.15.73:0/4167990534 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:24.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.062+0000 7f0e0e7fc640 1 --2- 172.21.15.73:0/4167990534 >> v2:172.21.15.120:3300/0 conn(0x7f0e100d2830 0x7f0e10162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:24.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.062+0000 7f0e0effd640 1 --2- 172.21.15.73:0/4167990534 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0e1015f5a0 0x7f0e100d5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:24.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.062+0000 7f0e0dffb640 1 -- 172.21.15.73:0/4167990534 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2302170546 0 0) 0x7f0e100d1a70 con 0x7f0e100d3c20 2024-04-26T20:51:24.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.062+0000 7f0e0dffb640 1 -- 172.21.15.73:0/4167990534 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0df8003520 con 0x7f0e100d3c20 2024-04-26T20:51:24.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.062+0000 7f0e0e7fc640 1 -- 172.21.15.73:0/4167990534 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0e1015f5a0 msgr2=0x7f0e100d5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:24.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.062+0000 7f0e0e7fc640 1 --2- 172.21.15.73:0/4167990534 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0e1015f5a0 0x7f0e100d5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:24.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.062+0000 7f0e0e7fc640 1 -- 172.21.15.73:0/4167990534 >> v1:172.21.15.73:6789/0 conn(0x7f0e100d3c20 legacy=0x7f0e1015f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:24.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.062+0000 7f0e0e7fc640 1 -- 172.21.15.73:0/4167990534 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e100d2030 con 0x7f0e100d2830 2024-04-26T20:51:24.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.063+0000 7f0e0e7fc640 1 --2- 172.21.15.73:0/4167990534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e100d2830 0x7f0e10162f00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0e040097a0 tx=0x7f0e0402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29d27a82cb88df6b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:24.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.063+0000 7f0e0dffb640 1 -- 172.21.15.73:0/4167990534 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e04037070 con 0x7f0e100d2830 2024-04-26T20:51:24.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.063+0000 7f0e0dffb640 1 -- 172.21.15.73:0/4167990534 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e0402ae50 con 0x7f0e100d2830 2024-04-26T20:51:24.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.063+0000 7f0e0dffb640 1 -- 172.21.15.73:0/4167990534 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e04032660 con 0x7f0e100d2830 2024-04-26T20:51:24.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.063+0000 7f0e1579d640 1 -- 172.21.15.73:0/4167990534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e100d2830 msgr2=0x7f0e10162f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:24.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.063+0000 7f0e1579d640 1 --2- 172.21.15.73:0/4167990534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e100d2830 0x7f0e10162f00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0e040097a0 tx=0x7f0e0402a100 comp rx=0 tx=0).stop 2024-04-26T20:51:24.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.063+0000 7f0e1579d640 1 -- 172.21.15.73:0/4167990534 shutdown_connections 2024-04-26T20:51:24.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.063+0000 7f0e1579d640 1 --2- 172.21.15.73:0/4167990534 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e100d2830 0x7f0e10162f00 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:24.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.063+0000 7f0e1579d640 1 --2- 172.21.15.73:0/4167990534 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0e1015f5a0 0x7f0e100d5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:24.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.063+0000 7f0e1579d640 1 -- 172.21.15.73:0/4167990534 >> 172.21.15.73:0/4167990534 conn(0x7f0e100d5190 msgr2=0x7f0e100d55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:24.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.063+0000 7f0e1579d640 1 -- 172.21.15.73:0/4167990534 shutdown_connections 2024-04-26T20:51:24.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.063+0000 7f0e1579d640 1 -- 172.21.15.73:0/4167990534 wait complete. 2024-04-26T20:51:24.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.065+0000 7f0e1579d640 1 Processor -- start 2024-04-26T20:51:24.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.065+0000 7f0e1579d640 1 -- start start 2024-04-26T20:51:24.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.065+0000 7f0e1579d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e100d2830 0x7f0e10154500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:24.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.065+0000 7f0e1579d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e100d3c20 0x7f0e100cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:24.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.065+0000 7f0e1579d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0e10154ca0 0x7f0e100ce290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:24.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.065+0000 7f0e1579d640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0e10163440 con 0x7f0e100d2830 2024-04-26T20:51:24.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.065+0000 7f0e1579d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0e100d4190 con 0x7f0e100d3c20 2024-04-26T20:51:24.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.065+0000 7f0e1579d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f0e101635c0 con 0x7f0e10154ca0 2024-04-26T20:51:24.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.065+0000 7f0e0effd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e100d2830 0x7f0e10154500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:24.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.065+0000 7f0e0effd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e100d2830 0x7f0e10154500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59936/0 (socket says 172.21.15.73:59936) 2024-04-26T20:51:24.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.065+0000 7f0e0effd640 1 -- 172.21.15.73:0/176314053 learned_addr learned my addr 172.21.15.73:0/176314053 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:24.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.066+0000 7f0e0e7fc640 1 --2- 172.21.15.73:0/176314053 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e100d3c20 0x7f0e100cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:24.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.066+0000 7f0e0f7fe640 1 --2- 172.21.15.73:0/176314053 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0e10154ca0 0x7f0e100ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:24.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.066+0000 7f0e0effd640 1 -- 172.21.15.73:0/176314053 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0e10154ca0 msgr2=0x7f0e100ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:24.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.066+0000 7f0e0effd640 1 --2- 172.21.15.73:0/176314053 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0e10154ca0 0x7f0e100ce290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:24.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.066+0000 7f0e0effd640 1 -- 172.21.15.73:0/176314053 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e100d3c20 msgr2=0x7f0e100cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:24.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.066+0000 7f0e0effd640 1 --2- 172.21.15.73:0/176314053 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e100d3c20 0x7f0e100cdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:24.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.066+0000 7f0e0effd640 1 -- 172.21.15.73:0/176314053 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e100c8eb0 con 0x7f0e100d2830 2024-04-26T20:51:24.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.066+0000 7f0e0f7fe640 1 --2- 172.21.15.73:0/176314053 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0e10154ca0 0x7f0e100ce290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:24.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.066+0000 7f0e0effd640 1 --2- 172.21.15.73:0/176314053 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e100d2830 0x7f0e10154500 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0e0000efc0 tx=0x7f0e0000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:24.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.066+0000 7f0df77fe640 1 -- 172.21.15.73:0/176314053 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e00018020 con 0x7f0e100d2830 2024-04-26T20:51:24.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.066+0000 7f0df77fe640 1 -- 172.21.15.73:0/176314053 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e00014480 con 0x7f0e100d2830 2024-04-26T20:51:24.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.066+0000 7f0df77fe640 1 -- 172.21.15.73:0/176314053 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e00014850 con 0x7f0e100d2830 2024-04-26T20:51:24.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.066+0000 7f0e1579d640 1 -- 172.21.15.73:0/176314053 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e100c9190 con 0x7f0e100d2830 2024-04-26T20:51:24.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.066+0000 7f0e1579d640 1 -- 172.21.15.73:0/176314053 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e100c9c70 con 0x7f0e100d2830 2024-04-26T20:51:24.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.067+0000 7f0e1579d640 1 -- 172.21.15.73:0/176314053 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e100694f0 con 0x7f0e100d2830 2024-04-26T20:51:24.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.067+0000 7f0df77fe640 1 -- 172.21.15.73:0/176314053 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0e00005d10 con 0x7f0e100d2830 2024-04-26T20:51:24.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.068+0000 7f0df77fe640 1 --2- 172.21.15.73:0/176314053 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f0dfc06c8f0 0x7f0dfc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:24.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.068+0000 7f0df77fe640 1 -- 172.21.15.73:0/176314053 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f0e0008ccf0 con 0x7f0e100d2830 2024-04-26T20:51:24.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.068+0000 7f0e0e7fc640 1 --2- 172.21.15.73:0/176314053 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f0dfc06c8f0 0x7f0dfc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:24.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.068+0000 7f0e0e7fc640 1 --2- 172.21.15.73:0/176314053 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f0dfc06c8f0 0x7f0dfc06edb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0e04034000 tx=0x7f0e0402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:24.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.072+0000 7f0df77fe640 1 -- 172.21.15.73:0/176314053 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0e00091070 con 0x7f0e100d2830 2024-04-26T20:51:24.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.260+0000 7f0e1579d640 1 -- 172.21.15.73:0/176314053 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0e10068d30 con 0x7f0e100d2830 2024-04-26T20:51:24.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.260+0000 7f0df77fe640 1 -- 172.21.15.73:0/176314053 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f0e00022030 con 0x7f0e100d2830 2024-04-26T20:51:24.262 INFO:teuthology.orchestra.run.smithi073.stderr:listed 0 entries 2024-04-26T20:51:24.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.262+0000 7f0e1579d640 1 -- 172.21.15.73:0/176314053 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f0dfc06c8f0 msgr2=0x7f0dfc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:24.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.262+0000 7f0e1579d640 1 --2- 172.21.15.73:0/176314053 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f0dfc06c8f0 0x7f0dfc06edb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0e04034000 tx=0x7f0e0402e000 comp rx=0 tx=0).stop 2024-04-26T20:51:24.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.262+0000 7f0e1579d640 1 -- 172.21.15.73:0/176314053 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e100d2830 msgr2=0x7f0e10154500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:24.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.262+0000 7f0e1579d640 1 --2- 172.21.15.73:0/176314053 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e100d2830 0x7f0e10154500 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0e0000efc0 tx=0x7f0e0000c490 comp rx=0 tx=0).stop 2024-04-26T20:51:24.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.262+0000 7f0e1579d640 1 -- 172.21.15.73:0/176314053 shutdown_connections 2024-04-26T20:51:24.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.262+0000 7f0e1579d640 1 --2- 172.21.15.73:0/176314053 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f0dfc06c8f0 0x7f0dfc06edb0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:24.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.262+0000 7f0e1579d640 1 --2- 172.21.15.73:0/176314053 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0e10154ca0 0x7f0e100ce290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:24.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.262+0000 7f0e1579d640 1 --2- 172.21.15.73:0/176314053 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0e100d3c20 0x7f0e100cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:24.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.262+0000 7f0e1579d640 1 --2- 172.21.15.73:0/176314053 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e100d2830 0x7f0e10154500 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:24.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.262+0000 7f0e1579d640 1 -- 172.21.15.73:0/176314053 >> 172.21.15.73:0/176314053 conn(0x7f0e100d5190 msgr2=0x7f0e100d0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:24.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.262+0000 7f0e1579d640 1 -- 172.21.15.73:0/176314053 shutdown_connections 2024-04-26T20:51:24.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.262+0000 7f0e1579d640 1 -- 172.21.15.73:0/176314053 wait complete. 2024-04-26T20:51:24.274 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T20:51:24.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:24.274 DEBUG:teuthology.orchestra.run.smithi073:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-26T20:51:24.299 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi073.front.sepia.ceph.com 2024-04-26T20:51:24.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:24.299 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T20:51:24.354 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-26T20:51:24.355 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:51:24.371 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-26T20:51:24.371 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:51:24.427 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-26T20:51:24.427 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:51:24.483 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:51:24.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.580+0000 7f1b7ecea640 1 Processor -- start 2024-04-26T20:51:24.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.580+0000 7f1b7ecea640 1 -- start start 2024-04-26T20:51:24.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.580+0000 7f1b7ecea640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f1b780b5c10 0x7f1b780cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:24.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.580+0000 7f1b7ecea640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1b780d04a0 0x7f1b780cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:24.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.580+0000 7f1b7ecea640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b780d2100 con 0x7f1b780b88a0 2024-04-26T20:51:24.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.580+0000 7f1b7ecea640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f1b780d2370 con 0x7f1b780d04a0 2024-04-26T20:51:24.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.580+0000 7f1b7ecea640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f1b780b8c80 con 0x7f1b780b5c10 2024-04-26T20:51:24.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.581+0000 7f1b7d287640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1b780d04a0 0x7f1b780cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:24.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.581+0000 7f1b7ca86640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f1b780b88a0 0x7f1b780d19f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:34572/0 (socket says 172.21.15.73:34572) 2024-04-26T20:51:24.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.581+0000 7f1b7ca86640 1 -- 172.21.15.73:0/4250839388 learned_addr learned my addr 172.21.15.73:0/4250839388 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:24.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.581+0000 7f1b7d287640 1 -- 172.21.15.73:0/4250839388 >> v2:172.21.15.120:3300/0 conn(0x7f1b780b5c10 msgr2=0x7f1b780cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:24.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.581+0000 7f1b7d287640 1 --2- 172.21.15.73:0/4250839388 >> v2:172.21.15.120:3300/0 conn(0x7f1b780b5c10 0x7f1b780cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:24.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.581+0000 7f1b7d287640 1 -- 172.21.15.73:0/4250839388 >> v1:172.21.15.73:6789/0 conn(0x7f1b780b88a0 legacy=0x7f1b780d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:24.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.581+0000 7f1b777fe640 1 -- 172.21.15.73:0/4250839388 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2511805563 0 0) 0x7f1b780d2100 con 0x7f1b780b88a0 2024-04-26T20:51:24.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.581+0000 7f1b7d287640 1 -- 172.21.15.73:0/4250839388 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b780cd5b0 con 0x7f1b780d04a0 2024-04-26T20:51:24.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.582+0000 7f1b7d287640 1 --2- 172.21.15.73:0/4250839388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b780d04a0 0x7f1b780cfb80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1b78073720 tx=0x7f1b6002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=831caf1f93ea0fa3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:24.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.582+0000 7f1b777fe640 1 -- 172.21.15.73:0/4250839388 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b60037070 con 0x7f1b780d04a0 2024-04-26T20:51:24.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.582+0000 7f1b777fe640 1 -- 172.21.15.73:0/4250839388 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b60032440 con 0x7f1b780d04a0 2024-04-26T20:51:24.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.582+0000 7f1b777fe640 1 -- 172.21.15.73:0/4250839388 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b60032810 con 0x7f1b780d04a0 2024-04-26T20:51:24.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.582+0000 7f1b7ecea640 1 -- 172.21.15.73:0/4250839388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b780d04a0 msgr2=0x7f1b780cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:24.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.582+0000 7f1b7ecea640 1 --2- 172.21.15.73:0/4250839388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b780d04a0 0x7f1b780cfb80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1b78073720 tx=0x7f1b6002a1a0 comp rx=0 tx=0).stop 2024-04-26T20:51:24.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.582+0000 7f1b7ecea640 1 -- 172.21.15.73:0/4250839388 shutdown_connections 2024-04-26T20:51:24.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.582+0000 7f1b7ecea640 1 --2- 172.21.15.73:0/4250839388 >> v2:172.21.15.120:3300/0 conn(0x7f1b780b5c10 0x7f1b780cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:24.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.582+0000 7f1b7ecea640 1 --2- 172.21.15.73:0/4250839388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b780d04a0 0x7f1b780cfb80 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:24.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.582+0000 7f1b7ecea640 1 -- 172.21.15.73:0/4250839388 >> 172.21.15.73:0/4250839388 conn(0x7f1b780d2dc0 msgr2=0x7f1b780d31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:24.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.582+0000 7f1b7ecea640 1 -- 172.21.15.73:0/4250839388 shutdown_connections 2024-04-26T20:51:24.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.582+0000 7f1b7ecea640 1 -- 172.21.15.73:0/4250839388 wait complete. 2024-04-26T20:51:24.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.583+0000 7f1b7ecea640 1 Processor -- start 2024-04-26T20:51:24.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.583+0000 7f1b7ecea640 1 -- start start 2024-04-26T20:51:24.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.584+0000 7f1b7ecea640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b780b5c10 0x7f1b78160a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:24.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.584+0000 7f1b7ecea640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b780b88a0 0x7f1b781f4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:24.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.584+0000 7f1b7ecea640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1b780d04a0 0x7f1b780b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:24.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.584+0000 7f1b7ecea640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1b780cde50 con 0x7f1b780b88a0 2024-04-26T20:51:24.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.584+0000 7f1b7ecea640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1b780cdb70 con 0x7f1b780b5c10 2024-04-26T20:51:24.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.584+0000 7f1b7ecea640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f1b780ce370 con 0x7f1b780d04a0 2024-04-26T20:51:24.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.584+0000 7f1b77fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b780b88a0 0x7f1b781f4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:24.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.584+0000 7f1b77fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b780b88a0 0x7f1b781f4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59944/0 (socket says 172.21.15.73:59944) 2024-04-26T20:51:24.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.584+0000 7f1b77fff640 1 -- 172.21.15.73:0/123307051 learned_addr learned my addr 172.21.15.73:0/123307051 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:24.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.584+0000 7f1b7ca86640 1 --2- 172.21.15.73:0/123307051 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b780b5c10 0x7f1b78160a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:24.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.584+0000 7f1b7d287640 1 --2- 172.21.15.73:0/123307051 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1b780d04a0 0x7f1b780b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:24.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.585+0000 7f1b77fff640 1 -- 172.21.15.73:0/123307051 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1b780d04a0 msgr2=0x7f1b780b7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:24.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.585+0000 7f1b77fff640 1 --2- 172.21.15.73:0/123307051 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1b780d04a0 0x7f1b780b7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:24.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.585+0000 7f1b77fff640 1 -- 172.21.15.73:0/123307051 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b780b5c10 msgr2=0x7f1b78160a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:24.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.585+0000 7f1b77fff640 1 --2- 172.21.15.73:0/123307051 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b780b5c10 0x7f1b78160a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:24.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.585+0000 7f1b77fff640 1 -- 172.21.15.73:0/123307051 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b780cb5a0 con 0x7f1b780b88a0 2024-04-26T20:51:24.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.585+0000 7f1b77fff640 1 --2- 172.21.15.73:0/123307051 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b780b88a0 0x7f1b781f4870 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1b6800b4a0 tx=0x7f1b6800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:24.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.585+0000 7f1b757fa640 1 -- 172.21.15.73:0/123307051 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b68014070 con 0x7f1b780b88a0 2024-04-26T20:51:24.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.585+0000 7f1b757fa640 1 -- 172.21.15.73:0/123307051 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b68010610 con 0x7f1b780b88a0 2024-04-26T20:51:24.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.585+0000 7f1b7ecea640 1 -- 172.21.15.73:0/123307051 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b780cb880 con 0x7f1b780b88a0 2024-04-26T20:51:24.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.585+0000 7f1b757fa640 1 -- 172.21.15.73:0/123307051 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b68010a00 con 0x7f1b780b88a0 2024-04-26T20:51:24.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.585+0000 7f1b7ecea640 1 -- 172.21.15.73:0/123307051 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b780cbe40 con 0x7f1b780b88a0 2024-04-26T20:51:24.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.586+0000 7f1b7ecea640 1 -- 172.21.15.73:0/123307051 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b780cc040 con 0x7f1b780b88a0 2024-04-26T20:51:24.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.586+0000 7f1b757fa640 1 -- 172.21.15.73:0/123307051 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1b68017070 con 0x7f1b780b88a0 2024-04-26T20:51:24.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.587+0000 7f1b757fa640 1 --2- 172.21.15.73:0/123307051 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1b5806c8f0 0x7f1b5806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:24.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.587+0000 7f1b757fa640 1 -- 172.21.15.73:0/123307051 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f1b6808b8b0 con 0x7f1b780b88a0 2024-04-26T20:51:24.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.587+0000 7f1b7ca86640 1 --2- 172.21.15.73:0/123307051 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1b5806c8f0 0x7f1b5806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:24.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.587+0000 7f1b7ca86640 1 --2- 172.21.15.73:0/123307051 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1b5806c8f0 0x7f1b5806edb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1b700012e0 tx=0x7f1b70001700 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:24.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.591+0000 7f1b757fa640 1 -- 172.21.15.73:0/123307051 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1b780cc040 con 0x7f1b780b88a0 2024-04-26T20:51:24.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.771+0000 7f1b7ecea640 1 -- 172.21.15.73:0/123307051 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1b7806cea0 con 0x7f1b780b88a0 2024-04-26T20:51:24.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.772+0000 7f1b757fa640 1 -- 172.21.15.73:0/123307051 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18902 (secure 0 0 0) 0x7f1b68059320 con 0x7f1b780b88a0 2024-04-26T20:51:24.773 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:51:24.774 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":27,"fsid":"fbd3f548-b547-4713-856b-09f98f5d1764","created":"2024-04-26T20:50:07.876786+0000","modified":"2024-04-26T20:51:13.783133+0000","last_up_change":"2024-04-26T20:50:19.082741+0000","last_in_change":"2024-04-26T20:50:10.018400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:50:19.993731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:50:29.964420+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:50:30.920082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:50:34.650207+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"72d6f4f3-214a-4392-a02f-c33cbcab3c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6809","nonce":3461705124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6811","nonce":3461705124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6815","nonce":3461705124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6813","nonce":3461705124}]},"public_addr":"172.21.15.73:6809/3461705124","cluster_addr":"172.21.15.73:6811/3461705124","heartbeat_back_addr":"172.21.15.73:6815/3461705124","heartbeat_front_addr":"172.21.15.73:6813/3461705124","state":["exists","up"]},{"osd":1,"uuid":"786bf8ca-6ddc-4b3c-be6c-9152398bd5fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6817","nonce":2827619403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6819","nonce":2827619403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6825","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6827","nonce":2827619403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6821","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6823","nonce":2827619403}]},"public_addr":"172.21.15.73:6817/2827619403","cluster_addr":"172.21.15.73:6819/2827619403","heartbeat_back_addr":"172.21.15.73:6827/2827619403","heartbeat_front_addr":"172.21.15.73:6823/2827619403","state":["exists","up"]},{"osd":2,"uuid":"90963b1a-5437-4ef8-ad8b-6b74c534eeb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6801","nonce":2542681064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6803","nonce":2542681064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6808","nonce":2542681064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6805","nonce":2542681064}]},"public_addr":"172.21.15.73:6801/2542681064","cluster_addr":"172.21.15.73:6803/2542681064","heartbeat_back_addr":"172.21.15.73:6808/2542681064","heartbeat_front_addr":"172.21.15.73:6805/2542681064","state":["exists","up"]},{"osd":3,"uuid":"bd3d2148-6e38-41b4-a453-fe6e3b8b4ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6822","nonce":1604042718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6826","nonce":1604042718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6831","nonce":1604042718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6829","nonce":1604042718}]},"public_addr":"172.21.15.73:6822/1604042718","cluster_addr":"172.21.15.73:6826/1604042718","heartbeat_back_addr":"172.21.15.73:6831/1604042718","heartbeat_front_addr":"172.21.15.73:6829/1604042718","state":["exists","up"]},{"osd":4,"uuid":"e8668dc8-a23f-46d2-8311-cb882da08e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6817","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6819","nonce":3591055561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6824","nonce":3591055561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6831","nonce":3591055561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6828","nonce":3591055561}]},"public_addr":"172.21.15.120:6819/3591055561","cluster_addr":"172.21.15.120:6824/3591055561","heartbeat_back_addr":"172.21.15.120:6831/3591055561","heartbeat_front_addr":"172.21.15.120:6828/3591055561","state":["exists","up"]},{"osd":5,"uuid":"cbac0bdb-96bd-4852-bcce-e564495c3fcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6809","nonce":819450433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6811","nonce":819450433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6815","nonce":819450433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6813","nonce":819450433}]},"public_addr":"172.21.15.120:6809/819450433","cluster_addr":"172.21.15.120:6811/819450433","heartbeat_back_addr":"172.21.15.120:6815/819450433","heartbeat_front_addr":"172.21.15.120:6813/819450433","state":["exists","up"]},{"osd":6,"uuid":"1884b2a8-569f-4272-96a9-f25a09364dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6801","nonce":3946210727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6803","nonce":3946210727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6807","nonce":3946210727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6805","nonce":3946210727}]},"public_addr":"172.21.15.120:6801/3946210727","cluster_addr":"172.21.15.120:6803/3946210727","heartbeat_back_addr":"172.21.15.120:6807/3946210727","heartbeat_front_addr":"172.21.15.120:6805/3946210727","state":["exists","up"]},{"osd":7,"uuid":"c0c3bf6d-a4c5-4f1e-aa2d-330951830666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6818","nonce":2915574674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6821","nonce":2915574674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6827","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6829","nonce":2915574674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6823","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6825","nonce":2915574674}]},"public_addr":"172.21.15.120:6818/2915574674","cluster_addr":"172.21.15.120:6821/2915574674","heartbeat_back_addr":"172.21.15.120:6829/2915574674","heartbeat_front_addr":"172.21.15.120:6825/2915574674","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.218216+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.208710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.112705+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.182789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.164324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.174212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.225396+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.261559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:51:24.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.774+0000 7f1b7ecea640 1 -- 172.21.15.73:0/123307051 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1b5806c8f0 msgr2=0x7f1b5806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:24.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.774+0000 7f1b7ecea640 1 --2- 172.21.15.73:0/123307051 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1b5806c8f0 0x7f1b5806edb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1b700012e0 tx=0x7f1b70001700 comp rx=0 tx=0).stop 2024-04-26T20:51:24.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.774+0000 7f1b7ecea640 1 -- 172.21.15.73:0/123307051 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b780b88a0 msgr2=0x7f1b781f4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:24.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.774+0000 7f1b7ecea640 1 --2- 172.21.15.73:0/123307051 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b780b88a0 0x7f1b781f4870 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1b6800b4a0 tx=0x7f1b6800b960 comp rx=0 tx=0).stop 2024-04-26T20:51:24.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.774+0000 7f1b7ecea640 1 -- 172.21.15.73:0/123307051 shutdown_connections 2024-04-26T20:51:24.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.774+0000 7f1b7ecea640 1 --2- 172.21.15.73:0/123307051 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b780b88a0 0x7f1b781f4870 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:24.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.774+0000 7f1b7ecea640 1 --2- 172.21.15.73:0/123307051 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1b5806c8f0 0x7f1b5806edb0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:24.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.774+0000 7f1b7ecea640 1 --2- 172.21.15.73:0/123307051 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1b780d04a0 0x7f1b780b7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:24.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.774+0000 7f1b7ecea640 1 --2- 172.21.15.73:0/123307051 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1b780b5c10 0x7f1b78160a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:24.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.774+0000 7f1b7ecea640 1 -- 172.21.15.73:0/123307051 >> 172.21.15.73:0/123307051 conn(0x7f1b780d2dc0 msgr2=0x7f1b780cf640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:24.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.774+0000 7f1b7ecea640 1 -- 172.21.15.73:0/123307051 shutdown_connections 2024-04-26T20:51:24.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.774+0000 7f1b7ecea640 1 -- 172.21.15.73:0/123307051 wait complete. 2024-04-26T20:51:24.786 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:50:19.993731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:50:29.964420+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:50:30.920082+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T20:50:34.650207+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T20:51:24.787 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:51:24.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.886+0000 7f9e60d4e640 1 Processor -- start 2024-04-26T20:51:24.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.886+0000 7f9e60d4e640 1 -- start start 2024-04-26T20:51:24.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.887+0000 7f9e60d4e640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f9e5c072fb0 0x7f9e5c0b56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:24.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.887+0000 7f9e60d4e640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e5c0b5e30 con 0x7f9e5c073390 2024-04-26T20:51:24.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.887+0000 7f9e60d4e640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e5c0b6110 con 0x7f9e5c0b06f0 2024-04-26T20:51:24.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.887+0000 7f9e60d4e640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f9e5c0b0b80 con 0x7f9e5c072fb0 2024-04-26T20:51:24.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.887+0000 7f9e53fff640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f9e5c073390 0x7f9e5c06f740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:34574/0 (socket says 172.21.15.73:34574) 2024-04-26T20:51:24.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.887+0000 7f9e53fff640 1 -- 172.21.15.73:0/388557602 learned_addr learned my addr 172.21.15.73:0/388557602 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:24.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.887+0000 7f9e59d9b640 1 -- 172.21.15.73:0/388557602 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3468520451 0 0) 0x7f9e5c0b5e30 con 0x7f9e5c073390 2024-04-26T20:51:24.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.887+0000 7f9e5a59c640 1 --2- 172.21.15.73:0/388557602 >> v2:172.21.15.120:3300/0 conn(0x7f9e5c072fb0 0x7f9e5c0b56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:24.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.887+0000 7f9e59d9b640 1 -- 172.21.15.73:0/388557602 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9e48003520 con 0x7f9e5c073390 2024-04-26T20:51:24.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.887+0000 7f9e59d9b640 1 -- 172.21.15.73:0/388557602 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4138645153 0 0) 0x7f9e48003520 con 0x7f9e5c073390 2024-04-26T20:51:24.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.887+0000 7f9e59d9b640 1 -- 172.21.15.73:0/388557602 >> v2:172.21.15.120:3300/0 conn(0x7f9e5c072fb0 msgr2=0x7f9e5c0b56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:24.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.887+0000 7f9e59d9b640 1 --2- 172.21.15.73:0/388557602 >> v2:172.21.15.120:3300/0 conn(0x7f9e5c072fb0 0x7f9e5c0b56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:24.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.887+0000 7f9e59d9b640 1 -- 172.21.15.73:0/388557602 >> v1:172.21.15.120:6789/0 conn(0x7f9e5c0b06f0 legacy=0x7f9e5c06fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:24.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.887+0000 7f9e59d9b640 1 -- 172.21.15.73:0/388557602 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e5c0b63f0 con 0x7f9e5c073390 2024-04-26T20:51:24.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.887+0000 7f9e59d9b640 1 -- 172.21.15.73:0/388557602 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7f9e44001670 con 0x7f9e5c073390 2024-04-26T20:51:24.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.887+0000 7f9e59d9b640 1 -- 172.21.15.73:0/388557602 >> v1:172.21.15.73:6789/0 conn(0x7f9e5c073390 legacy=0x7f9e5c06f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:24.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.887+0000 7f9e59d9b640 1 --2- 172.21.15.73:0/388557602 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9e480039b0 0x7f9e48003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:24.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.887+0000 7f9e59d9b640 1 --2- 172.21.15.73:0/388557602 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9e48004800 0x7f9e48004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:24.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.887+0000 7f9e59d9b640 1 --2- 172.21.15.73:0/388557602 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9e480051c0 0x7f9e4800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:24.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.887+0000 7f9e59d9b640 1 -- 172.21.15.73:0/388557602 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5c0b0b80 con 0x7f9e48004800 2024-04-26T20:51:24.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.887+0000 7f9e59d9b640 1 -- 172.21.15.73:0/388557602 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4800c060 con 0x7f9e480051c0 2024-04-26T20:51:24.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.887+0000 7f9e59d9b640 1 -- 172.21.15.73:0/388557602 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f9e4800c260 con 0x7f9e480039b0 2024-04-26T20:51:24.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.887+0000 7f9e53fff640 1 --2- 172.21.15.73:0/388557602 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9e48004800 0x7f9e48004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:24.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.888+0000 7f9e5ad9d640 1 --2- 172.21.15.73:0/388557602 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9e480051c0 0x7f9e4800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:24.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.888+0000 7f9e5a59c640 1 --2- 172.21.15.73:0/388557602 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9e480039b0 0x7f9e48003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:24.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.888+0000 7f9e53fff640 1 -- 172.21.15.73:0/388557602 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9e480039b0 msgr2=0x7f9e48003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:24.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.888+0000 7f9e53fff640 1 --2- 172.21.15.73:0/388557602 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9e480039b0 0x7f9e48003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:24.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.888+0000 7f9e53fff640 1 -- 172.21.15.73:0/388557602 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9e480051c0 msgr2=0x7f9e4800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:24.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.888+0000 7f9e53fff640 1 --2- 172.21.15.73:0/388557602 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9e480051c0 0x7f9e4800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:24.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.888+0000 7f9e53fff640 1 -- 172.21.15.73:0/388557602 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e48003520 con 0x7f9e48004800 2024-04-26T20:51:24.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.888+0000 7f9e5ad9d640 1 -- 172.21.15.73:0/388557602 reap_dead start 2024-04-26T20:51:24.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.888+0000 7f9e53fff640 1 --2- 172.21.15.73:0/388557602 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9e48004800 0x7f9e48004c80 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9e4400c210 tx=0x7f9e48005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf3ff19d81224164 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:24.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.889+0000 7f9e59d9b640 1 -- 172.21.15.73:0/388557602 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e44007d30 con 0x7f9e48004800 2024-04-26T20:51:24.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.889+0000 7f9e59d9b640 1 -- 172.21.15.73:0/388557602 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e44005ad0 con 0x7f9e48004800 2024-04-26T20:51:24.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.889+0000 7f9e59d9b640 1 -- 172.21.15.73:0/388557602 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e44007070 con 0x7f9e48004800 2024-04-26T20:51:24.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.889+0000 7f9e60d4e640 1 -- 172.21.15.73:0/388557602 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9e48004800 msgr2=0x7f9e48004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:24.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.889+0000 7f9e60d4e640 1 --2- 172.21.15.73:0/388557602 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9e48004800 0x7f9e48004c80 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9e4400c210 tx=0x7f9e48005830 comp rx=0 tx=0).stop 2024-04-26T20:51:24.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.889+0000 7f9e60d4e640 1 -- 172.21.15.73:0/388557602 shutdown_connections 2024-04-26T20:51:24.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.889+0000 7f9e60d4e640 1 --2- 172.21.15.73:0/388557602 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9e48004800 0x7f9e48004c80 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:24.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.889+0000 7f9e60d4e640 1 -- 172.21.15.73:0/388557602 >> 172.21.15.73:0/388557602 conn(0x7f9e5c0b39b0 msgr2=0x7f9e5c0b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:24.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.889+0000 7f9e60d4e640 1 -- 172.21.15.73:0/388557602 shutdown_connections 2024-04-26T20:51:24.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.889+0000 7f9e60d4e640 1 -- 172.21.15.73:0/388557602 wait complete. 2024-04-26T20:51:24.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.890+0000 7f9e60d4e640 1 Processor -- start 2024-04-26T20:51:24.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.890+0000 7f9e60d4e640 1 -- start start 2024-04-26T20:51:24.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.891+0000 7f9e60d4e640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9e48004800 0x7f9e5c1488f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:24.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.891+0000 7f9e60d4e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9e5c15c520 0x7f9e5c15c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:24.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.891+0000 7f9e60d4e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9e5c159900 0x7f9e5c159db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:24.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.891+0000 7f9e60d4e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5c15c160 con 0x7f9e5c15c520 2024-04-26T20:51:24.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.891+0000 7f9e60d4e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5c06f2b0 con 0x7f9e5c159900 2024-04-26T20:51:24.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.891+0000 7f9e60d4e640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f9e5c15c2e0 con 0x7f9e48004800 2024-04-26T20:51:24.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.891+0000 7f9e53fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9e5c15c520 0x7f9e5c15c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:24.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.891+0000 7f9e53fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9e5c15c520 0x7f9e5c15c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59960/0 (socket says 172.21.15.73:59960) 2024-04-26T20:51:24.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.891+0000 7f9e53fff640 1 -- 172.21.15.73:0/3257534180 learned_addr learned my addr 172.21.15.73:0/3257534180 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:24.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.891+0000 7f9e5a59c640 1 --2- 172.21.15.73:0/3257534180 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9e48004800 0x7f9e5c1488f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:24.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.891+0000 7f9e5ad9d640 1 --2- 172.21.15.73:0/3257534180 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9e5c159900 0x7f9e5c159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:24.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.892+0000 7f9e53fff640 1 -- 172.21.15.73:0/3257534180 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9e48004800 msgr2=0x7f9e5c1488f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:24.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.892+0000 7f9e53fff640 1 --2- 172.21.15.73:0/3257534180 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9e48004800 0x7f9e5c1488f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:24.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.892+0000 7f9e53fff640 1 -- 172.21.15.73:0/3257534180 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9e5c159900 msgr2=0x7f9e5c159db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:24.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.892+0000 7f9e53fff640 1 --2- 172.21.15.73:0/3257534180 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9e5c159900 0x7f9e5c159db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:24.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.892+0000 7f9e53fff640 1 -- 172.21.15.73:0/3257534180 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e5c15a630 con 0x7f9e5c15c520 2024-04-26T20:51:24.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.892+0000 7f9e53fff640 1 --2- 172.21.15.73:0/3257534180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9e5c15c520 0x7f9e5c15c9a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9e44001970 tx=0x7f9e44001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:24.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.892+0000 7f9e52ffd640 1 -- 172.21.15.73:0/3257534180 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e44041070 con 0x7f9e5c15c520 2024-04-26T20:51:24.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.892+0000 7f9e52ffd640 1 -- 172.21.15.73:0/3257534180 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e44000ed0 con 0x7f9e5c15c520 2024-04-26T20:51:24.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.892+0000 7f9e52ffd640 1 -- 172.21.15.73:0/3257534180 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e44007290 con 0x7f9e5c15c520 2024-04-26T20:51:24.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.892+0000 7f9e60d4e640 1 -- 172.21.15.73:0/3257534180 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e5c15a910 con 0x7f9e5c15c520 2024-04-26T20:51:24.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.892+0000 7f9e60d4e640 1 -- 172.21.15.73:0/3257534180 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e5c15aed0 con 0x7f9e5c15c520 2024-04-26T20:51:24.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.893+0000 7f9e60d4e640 1 -- 172.21.15.73:0/3257534180 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e5c0694f0 con 0x7f9e5c15c520 2024-04-26T20:51:24.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.893+0000 7f9e52ffd640 1 -- 172.21.15.73:0/3257534180 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9e44044070 con 0x7f9e5c15c520 2024-04-26T20:51:24.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.894+0000 7f9e52ffd640 1 --2- 172.21.15.73:0/3257534180 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f9e3c06c920 0x7f9e3c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:24.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.894+0000 7f9e52ffd640 1 -- 172.21.15.73:0/3257534180 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f9e440ac460 con 0x7f9e5c15c520 2024-04-26T20:51:24.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.894+0000 7f9e5a59c640 1 --2- 172.21.15.73:0/3257534180 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f9e3c06c920 0x7f9e3c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:24.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.894+0000 7f9e5a59c640 1 --2- 172.21.15.73:0/3257534180 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f9e3c06c920 0x7f9e3c06ede0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c009c40 tx=0x7f9e4c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:24.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:24.898+0000 7f9e52ffd640 1 -- 172.21.15.73:0/3257534180 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9e4407a520 con 0x7f9e5c15c520 2024-04-26T20:51:25.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.073+0000 7f9e60d4e640 1 -- 172.21.15.73:0/3257534180 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9e5c154f80 con 0x7f9e5c15c520 2024-04-26T20:51:25.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.073+0000 7f9e52ffd640 1 -- 172.21.15.73:0/3257534180 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f9e44079e20 con 0x7f9e5c15c520 2024-04-26T20:51:25.074 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-26T20:51:25.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.075+0000 7f9e60d4e640 1 -- 172.21.15.73:0/3257534180 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f9e3c06c920 msgr2=0x7f9e3c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:25.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.075+0000 7f9e60d4e640 1 --2- 172.21.15.73:0/3257534180 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f9e3c06c920 0x7f9e3c06ede0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c009c40 tx=0x7f9e4c009190 comp rx=0 tx=0).stop 2024-04-26T20:51:25.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.075+0000 7f9e60d4e640 1 -- 172.21.15.73:0/3257534180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9e5c15c520 msgr2=0x7f9e5c15c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:25.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.075+0000 7f9e60d4e640 1 --2- 172.21.15.73:0/3257534180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9e5c15c520 0x7f9e5c15c9a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9e44001970 tx=0x7f9e44001a40 comp rx=0 tx=0).stop 2024-04-26T20:51:25.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.075+0000 7f9e60d4e640 1 -- 172.21.15.73:0/3257534180 shutdown_connections 2024-04-26T20:51:25.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.075+0000 7f9e60d4e640 1 --2- 172.21.15.73:0/3257534180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9e5c15c520 0x7f9e5c15c9a0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:25.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.075+0000 7f9e60d4e640 1 --2- 172.21.15.73:0/3257534180 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f9e3c06c920 0x7f9e3c06ede0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:25.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.075+0000 7f9e60d4e640 1 --2- 172.21.15.73:0/3257534180 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9e5c159900 0x7f9e5c159db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:25.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.075+0000 7f9e60d4e640 1 --2- 172.21.15.73:0/3257534180 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9e48004800 0x7f9e5c1488f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:25.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.075+0000 7f9e60d4e640 1 -- 172.21.15.73:0/3257534180 >> 172.21.15.73:0/3257534180 conn(0x7f9e5c0b39b0 msgr2=0x7f9e5c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:25.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.075+0000 7f9e60d4e640 1 -- 172.21.15.73:0/3257534180 shutdown_connections 2024-04-26T20:51:25.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.075+0000 7f9e60d4e640 1 -- 172.21.15.73:0/3257534180 wait complete. 2024-04-26T20:51:25.087 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:51:25.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.185+0000 7f120daa2640 1 Processor -- start 2024-04-26T20:51:25.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.185+0000 7f120daa2640 1 -- start start 2024-04-26T20:51:25.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.186+0000 7f120daa2640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f120815d250 0x7f12080d0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:25.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.186+0000 7f120daa2640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f12080d39f0 0x7f12080d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:25.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.186+0000 7f120daa2640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12080701e0 con 0x7f12080d1550 2024-04-26T20:51:25.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.186+0000 7f120daa2640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f120815d630 con 0x7f120815d250 2024-04-26T20:51:25.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.186+0000 7f120daa2640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f12080704e0 con 0x7f12080d39f0 2024-04-26T20:51:25.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.186+0000 7f12077fe640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f120815d250 0x7f12080d0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:25.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.186+0000 7f1207fff640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f12080d1550 0x7f12080cf750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:34590/0 (socket says 172.21.15.73:34590) 2024-04-26T20:51:25.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.186+0000 7f1207fff640 1 -- 172.21.15.73:0/4064293035 learned_addr learned my addr 172.21.15.73:0/4064293035 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:25.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.186+0000 7f12067fc640 1 -- 172.21.15.73:0/4064293035 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2655801552 0 0) 0x7f12080701e0 con 0x7f12080d1550 2024-04-26T20:51:25.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.186+0000 7f12067fc640 1 -- 172.21.15.73:0/4064293035 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f11f4003520 con 0x7f12080d1550 2024-04-26T20:51:25.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.186+0000 7f1206ffd640 1 --2- 172.21.15.73:0/4064293035 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f12080d39f0 0x7f12080d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:25.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.186+0000 7f12077fe640 1 -- 172.21.15.73:0/4064293035 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f12080d39f0 msgr2=0x7f12080d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:25.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.186+0000 7f12077fe640 1 --2- 172.21.15.73:0/4064293035 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f12080d39f0 0x7f12080d1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:25.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.186+0000 7f12077fe640 1 -- 172.21.15.73:0/4064293035 >> v1:172.21.15.73:6789/0 conn(0x7f12080d1550 legacy=0x7f12080cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:25.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.186+0000 7f12077fe640 1 -- 172.21.15.73:0/4064293035 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12080ce370 con 0x7f120815d250 2024-04-26T20:51:25.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.186+0000 7f1206ffd640 1 --2- 172.21.15.73:0/4064293035 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f12080d39f0 0x7f12080d1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:25.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.187+0000 7f12077fe640 1 --2- 172.21.15.73:0/4064293035 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f120815d250 0x7f12080d0950 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f11f00097a0 tx=0x7f11f002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=920a9a55240c7cea server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:25.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.187+0000 7f12067fc640 1 -- 172.21.15.73:0/4064293035 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f0037070 con 0x7f120815d250 2024-04-26T20:51:25.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.187+0000 7f12067fc640 1 -- 172.21.15.73:0/4064293035 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11f002ae50 con 0x7f120815d250 2024-04-26T20:51:25.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.187+0000 7f12067fc640 1 -- 172.21.15.73:0/4064293035 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f0032660 con 0x7f120815d250 2024-04-26T20:51:25.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.187+0000 7f120daa2640 1 -- 172.21.15.73:0/4064293035 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f120815d250 msgr2=0x7f12080d0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:25.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.187+0000 7f120daa2640 1 --2- 172.21.15.73:0/4064293035 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f120815d250 0x7f12080d0950 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f11f00097a0 tx=0x7f11f002a100 comp rx=0 tx=0).stop 2024-04-26T20:51:25.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.188+0000 7f120daa2640 1 -- 172.21.15.73:0/4064293035 shutdown_connections 2024-04-26T20:51:25.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.188+0000 7f120daa2640 1 --2- 172.21.15.73:0/4064293035 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f12080d39f0 0x7f12080d1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:25.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.188+0000 7f120daa2640 1 --2- 172.21.15.73:0/4064293035 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f120815d250 0x7f12080d0950 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:25.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.188+0000 7f120daa2640 1 -- 172.21.15.73:0/4064293035 >> 172.21.15.73:0/4064293035 conn(0x7f12080d2d80 msgr2=0x7f12080d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:25.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.188+0000 7f120daa2640 1 -- 172.21.15.73:0/4064293035 shutdown_connections 2024-04-26T20:51:25.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.188+0000 7f120daa2640 1 -- 172.21.15.73:0/4064293035 wait complete. 2024-04-26T20:51:25.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.189+0000 7f120daa2640 1 Processor -- start 2024-04-26T20:51:25.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.189+0000 7f120daa2640 1 -- start start 2024-04-26T20:51:25.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.190+0000 7f120daa2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f12080d1550 0x7f12080afab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:25.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.190+0000 7f120daa2640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f12080d39f0 0x7f120815a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:25.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.190+0000 7f120daa2640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f120815d250 0x7f12080b5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:25.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.190+0000 7f120daa2640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f12080cebb0 con 0x7f120815d250 2024-04-26T20:51:25.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.190+0000 7f120daa2640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f12080ce8d0 con 0x7f12080d1550 2024-04-26T20:51:25.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.190+0000 7f120daa2640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f1208071980 con 0x7f12080d39f0 2024-04-26T20:51:25.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.190+0000 7f1207fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f120815d250 0x7f12080b5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:25.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.190+0000 7f1207fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f120815d250 0x7f12080b5f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59978/0 (socket says 172.21.15.73:59978) 2024-04-26T20:51:25.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.190+0000 7f1207fff640 1 -- 172.21.15.73:0/1921022733 learned_addr learned my addr 172.21.15.73:0/1921022733 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:25.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.190+0000 7f12077fe640 1 --2- 172.21.15.73:0/1921022733 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f12080d1550 0x7f12080afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:25.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.190+0000 7f1206ffd640 1 --2- 172.21.15.73:0/1921022733 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f12080d39f0 0x7f120815a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:25.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.190+0000 7f1207fff640 1 -- 172.21.15.73:0/1921022733 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f12080d39f0 msgr2=0x7f120815a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:25.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.190+0000 7f1207fff640 1 --2- 172.21.15.73:0/1921022733 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f12080d39f0 0x7f120815a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:25.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.190+0000 7f1207fff640 1 -- 172.21.15.73:0/1921022733 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f12080d1550 msgr2=0x7f12080afab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:25.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.190+0000 7f1207fff640 1 --2- 172.21.15.73:0/1921022733 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f12080d1550 0x7f12080afab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:25.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.190+0000 7f1207fff640 1 -- 172.21.15.73:0/1921022733 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12081f2ca0 con 0x7f120815d250 2024-04-26T20:51:25.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.190+0000 7f1207fff640 1 --2- 172.21.15.73:0/1921022733 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f120815d250 0x7f12080b5f50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f120000c430 tx=0x7f120000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:25.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.190+0000 7f11fffff640 1 -- 172.21.15.73:0/1921022733 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1200017070 con 0x7f120815d250 2024-04-26T20:51:25.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.190+0000 7f11fffff640 1 -- 172.21.15.73:0/1921022733 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1200005ce0 con 0x7f120815d250 2024-04-26T20:51:25.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.190+0000 7f120daa2640 1 -- 172.21.15.73:0/1921022733 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12081f2f80 con 0x7f120815d250 2024-04-26T20:51:25.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.190+0000 7f11fffff640 1 -- 172.21.15.73:0/1921022733 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1200007310 con 0x7f120815d250 2024-04-26T20:51:25.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.190+0000 7f120daa2640 1 -- 172.21.15.73:0/1921022733 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12081f3540 con 0x7f120815d250 2024-04-26T20:51:25.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.191+0000 7f120daa2640 1 -- 172.21.15.73:0/1921022733 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12081f3780 con 0x7f120815d250 2024-04-26T20:51:25.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.192+0000 7f11fffff640 1 -- 172.21.15.73:0/1921022733 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f120001a070 con 0x7f120815d250 2024-04-26T20:51:25.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.192+0000 7f11fffff640 1 --2- 172.21.15.73:0/1921022733 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f11e806c8f0 0x7f11e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:25.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.192+0000 7f11fffff640 1 -- 172.21.15.73:0/1921022733 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f120008b360 con 0x7f120815d250 2024-04-26T20:51:25.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.192+0000 7f12077fe640 1 --2- 172.21.15.73:0/1921022733 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f11e806c8f0 0x7f11e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:25.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.193+0000 7f12077fe640 1 --2- 172.21.15.73:0/1921022733 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f11e806c8f0 0x7f11e806edb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f11f0034000 tx=0x7f11f002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:25.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.197+0000 7f11fffff640 1 -- 172.21.15.73:0/1921022733 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1200059420 con 0x7f120815d250 2024-04-26T20:51:25.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.372+0000 7f120daa2640 1 -- 172.21.15.73:0/1921022733 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f12080694f0 con 0x7f120815d250 2024-04-26T20:51:25.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.372+0000 7f11fffff640 1 -- 172.21.15.73:0/1921022733 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f1200058d20 con 0x7f120815d250 2024-04-26T20:51:25.373 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-26T20:51:25.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.374+0000 7f120daa2640 1 -- 172.21.15.73:0/1921022733 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f11e806c8f0 msgr2=0x7f11e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:25.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.374+0000 7f120daa2640 1 --2- 172.21.15.73:0/1921022733 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f11e806c8f0 0x7f11e806edb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f11f0034000 tx=0x7f11f002e000 comp rx=0 tx=0).stop 2024-04-26T20:51:25.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.374+0000 7f120daa2640 1 -- 172.21.15.73:0/1921022733 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f120815d250 msgr2=0x7f12080b5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:25.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.374+0000 7f120daa2640 1 --2- 172.21.15.73:0/1921022733 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f120815d250 0x7f12080b5f50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f120000c430 tx=0x7f120000c8f0 comp rx=0 tx=0).stop 2024-04-26T20:51:25.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.374+0000 7f120daa2640 1 -- 172.21.15.73:0/1921022733 shutdown_connections 2024-04-26T20:51:25.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.374+0000 7f120daa2640 1 --2- 172.21.15.73:0/1921022733 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f120815d250 0x7f12080b5f50 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:25.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.374+0000 7f120daa2640 1 --2- 172.21.15.73:0/1921022733 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f11e806c8f0 0x7f11e806edb0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:25.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.374+0000 7f120daa2640 1 --2- 172.21.15.73:0/1921022733 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f12080d39f0 0x7f120815a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:25.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.374+0000 7f120daa2640 1 --2- 172.21.15.73:0/1921022733 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f12080d1550 0x7f12080afab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:25.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.374+0000 7f120daa2640 1 -- 172.21.15.73:0/1921022733 >> 172.21.15.73:0/1921022733 conn(0x7f12080d2d80 msgr2=0x7f12080cf5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:25.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.374+0000 7f120daa2640 1 -- 172.21.15.73:0/1921022733 shutdown_connections 2024-04-26T20:51:25.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.374+0000 7f120daa2640 1 -- 172.21.15.73:0/1921022733 wait complete. 2024-04-26T20:51:25.386 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:51:25.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.484+0000 7fdafd3c1640 1 Processor -- start 2024-04-26T20:51:25.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.484+0000 7fdafd3c1640 1 -- start start 2024-04-26T20:51:25.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.485+0000 7fdafd3c1640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fdaf80d1a00 0x7fdaf80d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:25.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.485+0000 7fdafd3c1640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdaf80b40d0 0x7fdaf80b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:25.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.485+0000 7fdafd3c1640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdaf80d0a10 con 0x7fdaf80d2320 2024-04-26T20:51:25.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.485+0000 7fdafd3c1640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fdaf80d0cf0 con 0x7fdaf80d1a00 2024-04-26T20:51:25.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.485+0000 7fdafd3c1640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fdaf80d0fd0 con 0x7fdaf80b40d0 2024-04-26T20:51:25.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.485+0000 7fdaeffff640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fdaf80d2320 0x7fdaf81625f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:34606/0 (socket says 172.21.15.73:34606) 2024-04-26T20:51:25.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.485+0000 7fdaeffff640 1 -- 172.21.15.73:0/2783700432 learned_addr learned my addr 172.21.15.73:0/2783700432 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:25.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.485+0000 7fdaf67fc640 1 -- 172.21.15.73:0/2783700432 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3339487723 0 0) 0x7fdaf80d0a10 con 0x7fdaf80d2320 2024-04-26T20:51:25.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.485+0000 7fdaf6ffd640 1 --2- 172.21.15.73:0/2783700432 >> v2:172.21.15.120:3300/0 conn(0x7fdaf80d1a00 0x7fdaf80d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:25.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.485+0000 7fdaf77fe640 1 --2- 172.21.15.73:0/2783700432 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdaf80b40d0 0x7fdaf80b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:25.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.485+0000 7fdaf67fc640 1 -- 172.21.15.73:0/2783700432 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdae4003520 con 0x7fdaf80d2320 2024-04-26T20:51:25.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.485+0000 7fdaf67fc640 1 -- 172.21.15.73:0/2783700432 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 600109666 0 0) 0x7fdae4003520 con 0x7fdaf80d2320 2024-04-26T20:51:25.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.485+0000 7fdaf67fc640 1 -- 172.21.15.73:0/2783700432 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdaf80b40d0 msgr2=0x7fdaf80b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:25.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.485+0000 7fdaf67fc640 1 --2- 172.21.15.73:0/2783700432 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdaf80b40d0 0x7fdaf80b4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:25.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.485+0000 7fdaf67fc640 1 -- 172.21.15.73:0/2783700432 >> v2:172.21.15.120:3300/0 conn(0x7fdaf80d1a00 msgr2=0x7fdaf80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:25.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.485+0000 7fdaf67fc640 1 --2- 172.21.15.73:0/2783700432 >> v2:172.21.15.120:3300/0 conn(0x7fdaf80d1a00 0x7fdaf80d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:25.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.485+0000 7fdaf67fc640 1 -- 172.21.15.73:0/2783700432 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf80d40a0 con 0x7fdaf80d2320 2024-04-26T20:51:25.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.485+0000 7fdaf67fc640 1 -- 172.21.15.73:0/2783700432 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7fdae0001670 con 0x7fdaf80d2320 2024-04-26T20:51:25.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.485+0000 7fdaf67fc640 1 -- 172.21.15.73:0/2783700432 >> v1:172.21.15.73:6789/0 conn(0x7fdaf80d2320 legacy=0x7fdaf81625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:25.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.485+0000 7fdaf67fc640 1 --2- 172.21.15.73:0/2783700432 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdae40039b0 0x7fdae4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:25.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.485+0000 7fdaf67fc640 1 --2- 172.21.15.73:0/2783700432 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdae4004800 0x7fdae4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:25.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.485+0000 7fdaf67fc640 1 --2- 172.21.15.73:0/2783700432 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdaf80b40d0 0x7fdae40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:25.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.485+0000 7fdaf67fc640 1 -- 172.21.15.73:0/2783700432 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf80d0cf0 con 0x7fdae40039b0 2024-04-26T20:51:25.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.485+0000 7fdaf67fc640 1 -- 172.21.15.73:0/2783700432 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf80d0fd0 con 0x7fdae4004800 2024-04-26T20:51:25.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.485+0000 7fdaf67fc640 1 -- 172.21.15.73:0/2783700432 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fdae400ba70 con 0x7fdaf80b40d0 2024-04-26T20:51:25.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.485+0000 7fdaf6ffd640 1 --2- 172.21.15.73:0/2783700432 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdae40039b0 0x7fdae4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:25.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.486+0000 7fdaeffff640 1 --2- 172.21.15.73:0/2783700432 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdae4004800 0x7fdae4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:25.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.486+0000 7fdaf77fe640 1 --2- 172.21.15.73:0/2783700432 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdaf80b40d0 0x7fdae40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:25.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.486+0000 7fdaf6ffd640 1 -- 172.21.15.73:0/2783700432 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdaf80b40d0 msgr2=0x7fdae40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:25.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.486+0000 7fdaf6ffd640 1 --2- 172.21.15.73:0/2783700432 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdaf80b40d0 0x7fdae40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:25.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.486+0000 7fdaf6ffd640 1 -- 172.21.15.73:0/2783700432 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdae4004800 msgr2=0x7fdae4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:25.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.486+0000 7fdaf6ffd640 1 --2- 172.21.15.73:0/2783700432 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdae4004800 0x7fdae4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:25.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.486+0000 7fdaf6ffd640 1 -- 172.21.15.73:0/2783700432 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdae4003520 con 0x7fdae40039b0 2024-04-26T20:51:25.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.486+0000 7fdaeffff640 1 --2- 172.21.15.73:0/2783700432 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdae4004800 0x7fdae4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:25.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.486+0000 7fdaf6ffd640 1 --2- 172.21.15.73:0/2783700432 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdae40039b0 0x7fdae4003dd0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fdae8009ac0 tx=0x7fdae4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c286e998c48be79b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:25.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.487+0000 7fdaf67fc640 1 -- 172.21.15.73:0/2783700432 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae8036070 con 0x7fdae40039b0 2024-04-26T20:51:25.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.487+0000 7fdaf67fc640 1 -- 172.21.15.73:0/2783700432 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdae8031400 con 0x7fdae40039b0 2024-04-26T20:51:25.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.487+0000 7fdaf67fc640 1 -- 172.21.15.73:0/2783700432 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae8009690 con 0x7fdae40039b0 2024-04-26T20:51:25.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.487+0000 7fdafd3c1640 1 -- 172.21.15.73:0/2783700432 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdae40039b0 msgr2=0x7fdae4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:25.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.487+0000 7fdafd3c1640 1 --2- 172.21.15.73:0/2783700432 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdae40039b0 0x7fdae4003dd0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fdae8009ac0 tx=0x7fdae4005830 comp rx=0 tx=0).stop 2024-04-26T20:51:25.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.487+0000 7fdaf77fe640 1 -- 172.21.15.73:0/2783700432 reap_dead start 2024-04-26T20:51:25.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.487+0000 7fdafd3c1640 1 -- 172.21.15.73:0/2783700432 shutdown_connections 2024-04-26T20:51:25.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.487+0000 7fdafd3c1640 1 -- 172.21.15.73:0/2783700432 >> 172.21.15.73:0/2783700432 conn(0x7fdaf80d4fd0 msgr2=0x7fdaf80d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:25.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.487+0000 7fdafd3c1640 1 -- 172.21.15.73:0/2783700432 shutdown_connections 2024-04-26T20:51:25.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.487+0000 7fdafd3c1640 1 -- 172.21.15.73:0/2783700432 wait complete. 2024-04-26T20:51:25.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.488+0000 7fdafd3c1640 1 Processor -- start 2024-04-26T20:51:25.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.488+0000 7fdafd3c1640 1 -- start start 2024-04-26T20:51:25.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.489+0000 7fdafd3c1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdae40039b0 0x7fdaf80c86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:25.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.489+0000 7fdafd3c1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdaf8155240 0x7fdaf80b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:25.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.489+0000 7fdafd3c1640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdaf81556c0 0x7fdaf80b9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:25.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.489+0000 7fdafd3c1640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf80b4b00 con 0x7fdaf8155240 2024-04-26T20:51:25.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.489+0000 7fdafd3c1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf8162f50 con 0x7fdae40039b0 2024-04-26T20:51:25.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.489+0000 7fdafd3c1640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fdaf80b4c80 con 0x7fdaf81556c0 2024-04-26T20:51:25.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.489+0000 7fdaeffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdaf8155240 0x7fdaf80b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:25.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.489+0000 7fdaeffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdaf8155240 0x7fdaf80b8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59988/0 (socket says 172.21.15.73:59988) 2024-04-26T20:51:25.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.489+0000 7fdaeffff640 1 -- 172.21.15.73:0/3230147499 learned_addr learned my addr 172.21.15.73:0/3230147499 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:25.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.489+0000 7fdaf6ffd640 1 --2- 172.21.15.73:0/3230147499 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdae40039b0 0x7fdaf80c86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:25.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.489+0000 7fdaf77fe640 1 --2- 172.21.15.73:0/3230147499 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdaf81556c0 0x7fdaf80b9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:25.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.489+0000 7fdaeffff640 1 -- 172.21.15.73:0/3230147499 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdaf81556c0 msgr2=0x7fdaf80b9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:25.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.489+0000 7fdaeffff640 1 --2- 172.21.15.73:0/3230147499 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdaf81556c0 0x7fdaf80b9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:25.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.489+0000 7fdaeffff640 1 -- 172.21.15.73:0/3230147499 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdae40039b0 msgr2=0x7fdaf80c86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:25.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.489+0000 7fdaeffff640 1 --2- 172.21.15.73:0/3230147499 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdae40039b0 0x7fdaf80c86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:25.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.489+0000 7fdaeffff640 1 -- 172.21.15.73:0/3230147499 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf80cee90 con 0x7fdaf8155240 2024-04-26T20:51:25.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.489+0000 7fdaf6ffd640 1 --2- 172.21.15.73:0/3230147499 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdae40039b0 0x7fdaf80c86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:25.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.490+0000 7fdaeffff640 1 --2- 172.21.15.73:0/3230147499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdaf8155240 0x7fdaf80b8c40 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fdae000e780 tx=0x7fdae000ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:25.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.490+0000 7fdaef7fe640 1 -- 172.21.15.73:0/3230147499 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae000ee90 con 0x7fdaf8155240 2024-04-26T20:51:25.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.490+0000 7fdaef7fe640 1 -- 172.21.15.73:0/3230147499 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdae0018070 con 0x7fdaf8155240 2024-04-26T20:51:25.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.490+0000 7fdafd3c1640 1 -- 172.21.15.73:0/3230147499 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaf80cf170 con 0x7fdaf8155240 2024-04-26T20:51:25.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.490+0000 7fdaef7fe640 1 -- 172.21.15.73:0/3230147499 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae000caf0 con 0x7fdaf8155240 2024-04-26T20:51:25.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.490+0000 7fdafd3c1640 1 -- 172.21.15.73:0/3230147499 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaf80b96c0 con 0x7fdaf8155240 2024-04-26T20:51:25.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.491+0000 7fdafd3c1640 1 -- 172.21.15.73:0/3230147499 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaf80694f0 con 0x7fdaf8155240 2024-04-26T20:51:25.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.491+0000 7fdaef7fe640 1 -- 172.21.15.73:0/3230147499 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdae000cc90 con 0x7fdaf8155240 2024-04-26T20:51:25.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.492+0000 7fdaef7fe640 1 --2- 172.21.15.73:0/3230147499 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fdad806cac0 0x7fdad806ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:25.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.492+0000 7fdaef7fe640 1 -- 172.21.15.73:0/3230147499 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fdae008bd40 con 0x7fdaf8155240 2024-04-26T20:51:25.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.492+0000 7fdaf6ffd640 1 --2- 172.21.15.73:0/3230147499 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fdad806cac0 0x7fdad806ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:25.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.492+0000 7fdaf6ffd640 1 --2- 172.21.15.73:0/3230147499 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fdad806cac0 0x7fdad806ef80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdae8033000 tx=0x7fdae8008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:25.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.496+0000 7fdaef7fe640 1 -- 172.21.15.73:0/3230147499 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdae0059e00 con 0x7fdaf8155240 2024-04-26T20:51:25.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.671+0000 7fdafd3c1640 1 -- 172.21.15.73:0/3230147499 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdaf8161e80 con 0x7fdaf8155240 2024-04-26T20:51:25.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.672+0000 7fdaef7fe640 1 -- 172.21.15.73:0/3230147499 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7fdae0059700 con 0x7fdaf8155240 2024-04-26T20:51:25.673 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-26T20:51:25.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.673+0000 7fdafd3c1640 1 -- 172.21.15.73:0/3230147499 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fdad806cac0 msgr2=0x7fdad806ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:25.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.673+0000 7fdafd3c1640 1 --2- 172.21.15.73:0/3230147499 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fdad806cac0 0x7fdad806ef80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdae8033000 tx=0x7fdae8008030 comp rx=0 tx=0).stop 2024-04-26T20:51:25.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.673+0000 7fdafd3c1640 1 -- 172.21.15.73:0/3230147499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdaf8155240 msgr2=0x7fdaf80b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:25.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.673+0000 7fdafd3c1640 1 --2- 172.21.15.73:0/3230147499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdaf8155240 0x7fdaf80b8c40 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fdae000e780 tx=0x7fdae000ec40 comp rx=0 tx=0).stop 2024-04-26T20:51:25.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.673+0000 7fdafd3c1640 1 -- 172.21.15.73:0/3230147499 shutdown_connections 2024-04-26T20:51:25.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.673+0000 7fdafd3c1640 1 --2- 172.21.15.73:0/3230147499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdaf8155240 0x7fdaf80b8c40 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:25.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.673+0000 7fdafd3c1640 1 --2- 172.21.15.73:0/3230147499 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fdad806cac0 0x7fdad806ef80 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:25.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.673+0000 7fdafd3c1640 1 --2- 172.21.15.73:0/3230147499 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdaf81556c0 0x7fdaf80b9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:25.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.673+0000 7fdafd3c1640 1 --2- 172.21.15.73:0/3230147499 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdae40039b0 0x7fdaf80c86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:25.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.673+0000 7fdafd3c1640 1 -- 172.21.15.73:0/3230147499 >> 172.21.15.73:0/3230147499 conn(0x7fdaf80d4fd0 msgr2=0x7fdaf80d3770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:25.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.674+0000 7fdafd3c1640 1 -- 172.21.15.73:0/3230147499 shutdown_connections 2024-04-26T20:51:25.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.674+0000 7fdafd3c1640 1 -- 172.21.15.73:0/3230147499 wait complete. 2024-04-26T20:51:25.685 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T20:51:25.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.784+0000 7f6b2f7af640 1 Processor -- start 2024-04-26T20:51:25.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.785+0000 7f6b2f7af640 1 -- start start 2024-04-26T20:51:25.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.785+0000 7f6b2f7af640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6b280b40d0 0x7f6b281625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:25.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.785+0000 7f6b2f7af640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6b28162b30 0x7f6b280b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:25.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.785+0000 7f6b2f7af640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b280d0a10 con 0x7f6b280d1a00 2024-04-26T20:51:25.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.785+0000 7f6b2f7af640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f6b280d0cf0 con 0x7f6b28162b30 2024-04-26T20:51:25.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.785+0000 7f6b2f7af640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f6b280d0fd0 con 0x7f6b280b40d0 2024-04-26T20:51:25.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.785+0000 7f6b2dd4c640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6b28162b30 0x7f6b280b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:25.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.785+0000 7f6b2dd4c640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6b28162b30 0x7f6b280b4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59998/0 (socket says 172.21.15.73:59998) 2024-04-26T20:51:25.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.786+0000 7f6b2dd4c640 1 -- 172.21.15.73:0/3673768109 learned_addr learned my addr 172.21.15.73:0/3673768109 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:25.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.786+0000 7f6b2cd4a640 1 --2- 172.21.15.73:0/3673768109 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6b280b40d0 0x7f6b281625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:25.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.786+0000 7f6b2dd4c640 1 -- 172.21.15.73:0/3673768109 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6b280b40d0 msgr2=0x7f6b281625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:25.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.786+0000 7f6b2dd4c640 1 --2- 172.21.15.73:0/3673768109 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6b280b40d0 0x7f6b281625f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:25.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.786+0000 7f6b2dd4c640 1 -- 172.21.15.73:0/3673768109 >> v1:172.21.15.120:6789/0 conn(0x7f6b280d1a00 legacy=0x7f6b280d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:25.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.786+0000 7f6b2dd4c640 1 -- 172.21.15.73:0/3673768109 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b28162fa0 con 0x7f6b28162b30 2024-04-26T20:51:25.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.786+0000 7f6b2cd4a640 1 --2- 172.21.15.73:0/3673768109 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6b280b40d0 0x7f6b281625f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:25.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.786+0000 7f6b2dd4c640 1 --2- 172.21.15.73:0/3673768109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b28162b30 0x7f6b280b4b00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6b20009870 tx=0x7f6b2002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbf697b0a4ea8c39 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:25.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.787+0000 7f6b1ffff640 1 -- 172.21.15.73:0/3673768109 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b20037070 con 0x7f6b28162b30 2024-04-26T20:51:25.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.787+0000 7f6b1ffff640 1 -- 172.21.15.73:0/3673768109 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b20032440 con 0x7f6b28162b30 2024-04-26T20:51:25.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.787+0000 7f6b1ffff640 1 -- 172.21.15.73:0/3673768109 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b20032810 con 0x7f6b28162b30 2024-04-26T20:51:25.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.787+0000 7f6b2f7af640 1 -- 172.21.15.73:0/3673768109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b28162b30 msgr2=0x7f6b280b4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:25.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.787+0000 7f6b2f7af640 1 --2- 172.21.15.73:0/3673768109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b28162b30 0x7f6b280b4b00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6b20009870 tx=0x7f6b2002a1c0 comp rx=0 tx=0).stop 2024-04-26T20:51:25.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.787+0000 7f6b2f7af640 1 -- 172.21.15.73:0/3673768109 shutdown_connections 2024-04-26T20:51:25.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.787+0000 7f6b2f7af640 1 --2- 172.21.15.73:0/3673768109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b28162b30 0x7f6b280b4b00 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:25.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.787+0000 7f6b2f7af640 1 --2- 172.21.15.73:0/3673768109 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6b280b40d0 0x7f6b281625f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:25.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.787+0000 7f6b2f7af640 1 -- 172.21.15.73:0/3673768109 >> 172.21.15.73:0/3673768109 conn(0x7f6b280d4fd0 msgr2=0x7f6b280d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:25.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.787+0000 7f6b2f7af640 1 -- 172.21.15.73:0/3673768109 shutdown_connections 2024-04-26T20:51:25.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.787+0000 7f6b2f7af640 1 -- 172.21.15.73:0/3673768109 wait complete. 2024-04-26T20:51:25.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.788+0000 7f6b2f7af640 1 Processor -- start 2024-04-26T20:51:25.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.788+0000 7f6b2f7af640 1 -- start start 2024-04-26T20:51:25.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.789+0000 7f6b2f7af640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b280b40d0 0x7f6b280cd420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:25.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.789+0000 7f6b2f7af640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6b280d1a00 0x7f6b281545f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:25.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.789+0000 7f6b2f7af640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b28162b30 0x7f6b280ba520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:25.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.789+0000 7f6b2f7af640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6b280d2670 con 0x7f6b28162b30 2024-04-26T20:51:25.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.789+0000 7f6b2f7af640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6b280d24f0 con 0x7f6b280b40d0 2024-04-26T20:51:25.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.789+0000 7f6b2f7af640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f6b280b5040 con 0x7f6b280d1a00 2024-04-26T20:51:25.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.789+0000 7f6b2dd4c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b28162b30 0x7f6b280ba520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:25.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.789+0000 7f6b2dd4c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b28162b30 0x7f6b280ba520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60006/0 (socket says 172.21.15.73:60006) 2024-04-26T20:51:25.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.789+0000 7f6b2dd4c640 1 -- 172.21.15.73:0/3014996145 learned_addr learned my addr 172.21.15.73:0/3014996145 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:25.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.790+0000 7f6b2cd4a640 1 --2- 172.21.15.73:0/3014996145 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6b280d1a00 0x7f6b281545f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:25.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.790+0000 7f6b2dd4c640 1 -- 172.21.15.73:0/3014996145 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6b280d1a00 msgr2=0x7f6b281545f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:25.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.790+0000 7f6b2dd4c640 1 --2- 172.21.15.73:0/3014996145 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6b280d1a00 0x7f6b281545f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:25.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.790+0000 7f6b2dd4c640 1 -- 172.21.15.73:0/3014996145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b280b40d0 msgr2=0x7f6b280cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:25.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.790+0000 7f6b2dd4c640 1 --2- 172.21.15.73:0/3014996145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b280b40d0 0x7f6b280cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:25.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.790+0000 7f6b2dd4c640 1 -- 172.21.15.73:0/3014996145 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b280cc0a0 con 0x7f6b28162b30 2024-04-26T20:51:25.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.790+0000 7f6b2dd4c640 1 --2- 172.21.15.73:0/3014996145 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b28162b30 0x7f6b280ba520 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6b20009870 tx=0x7f6b200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:25.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.790+0000 7f6b1dffb640 1 -- 172.21.15.73:0/3014996145 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b20037040 con 0x7f6b28162b30 2024-04-26T20:51:25.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.790+0000 7f6b1dffb640 1 -- 172.21.15.73:0/3014996145 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b20042070 con 0x7f6b28162b30 2024-04-26T20:51:25.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.790+0000 7f6b1dffb640 1 -- 172.21.15.73:0/3014996145 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b20030920 con 0x7f6b28162b30 2024-04-26T20:51:25.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.790+0000 7f6b2f7af640 1 -- 172.21.15.73:0/3014996145 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b280cc380 con 0x7f6b28162b30 2024-04-26T20:51:25.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.790+0000 7f6b2f7af640 1 -- 172.21.15.73:0/3014996145 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b280cc940 con 0x7f6b28162b30 2024-04-26T20:51:25.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.791+0000 7f6b2f7af640 1 -- 172.21.15.73:0/3014996145 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b280694f0 con 0x7f6b28162b30 2024-04-26T20:51:25.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.791+0000 7f6b1dffb640 1 -- 172.21.15.73:0/3014996145 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6b2002ad10 con 0x7f6b28162b30 2024-04-26T20:51:25.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.792+0000 7f6b1dffb640 1 --2- 172.21.15.73:0/3014996145 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f6b0c06c8f0 0x7f6b0c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:25.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.792+0000 7f6b1dffb640 1 -- 172.21.15.73:0/3014996145 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f6b200ac4a0 con 0x7f6b28162b30 2024-04-26T20:51:25.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.792+0000 7f6b2d54b640 1 --2- 172.21.15.73:0/3014996145 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f6b0c06c8f0 0x7f6b0c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:25.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.792+0000 7f6b2d54b640 1 --2- 172.21.15.73:0/3014996145 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f6b0c06c8f0 0x7f6b0c06edb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6b240012e0 tx=0x7f6b24007d50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:25.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.796+0000 7f6b1dffb640 1 -- 172.21.15.73:0/3014996145 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6b20079f10 con 0x7f6b28162b30 2024-04-26T20:51:25.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.973+0000 7f6b2f7af640 1 -- 172.21.15.73:0/3014996145 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f6b280b3ab0 con 0x7f6b28162b30 2024-04-26T20:51:25.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.973+0000 7f6b1dffb640 1 -- 172.21.15.73:0/3014996145 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f6b2007e210 con 0x7f6b28162b30 2024-04-26T20:51:25.975 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-26T20:51:25.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.975+0000 7f6b2f7af640 1 -- 172.21.15.73:0/3014996145 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f6b0c06c8f0 msgr2=0x7f6b0c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:25.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.975+0000 7f6b2f7af640 1 --2- 172.21.15.73:0/3014996145 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f6b0c06c8f0 0x7f6b0c06edb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6b240012e0 tx=0x7f6b24007d50 comp rx=0 tx=0).stop 2024-04-26T20:51:25.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.975+0000 7f6b2f7af640 1 -- 172.21.15.73:0/3014996145 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b28162b30 msgr2=0x7f6b280ba520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:25.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.975+0000 7f6b2f7af640 1 --2- 172.21.15.73:0/3014996145 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b28162b30 0x7f6b280ba520 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6b20009870 tx=0x7f6b200046c0 comp rx=0 tx=0).stop 2024-04-26T20:51:25.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.975+0000 7f6b2f7af640 1 -- 172.21.15.73:0/3014996145 shutdown_connections 2024-04-26T20:51:25.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.975+0000 7f6b2f7af640 1 --2- 172.21.15.73:0/3014996145 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b28162b30 0x7f6b280ba520 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:25.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.975+0000 7f6b2f7af640 1 --2- 172.21.15.73:0/3014996145 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f6b0c06c8f0 0x7f6b0c06edb0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:25.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.975+0000 7f6b2f7af640 1 --2- 172.21.15.73:0/3014996145 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6b280d1a00 0x7f6b281545f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:25.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.975+0000 7f6b2f7af640 1 --2- 172.21.15.73:0/3014996145 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b280b40d0 0x7f6b280cd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:25.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.975+0000 7f6b2f7af640 1 -- 172.21.15.73:0/3014996145 >> 172.21.15.73:0/3014996145 conn(0x7f6b280d4fd0 msgr2=0x7f6b280d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:25.978 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.976+0000 7f6b2f7af640 1 -- 172.21.15.73:0/3014996145 shutdown_connections 2024-04-26T20:51:25.978 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:25.976+0000 7f6b2f7af640 1 -- 172.21.15.73:0/3014996145 wait complete. 2024-04-26T20:51:25.987 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T20:51:26.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.083+0000 7fa4bed5f640 1 Processor -- start 2024-04-26T20:51:26.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.083+0000 7fa4bed5f640 1 -- start start 2024-04-26T20:51:26.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.084+0000 7fa4bed5f640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa4b8162880 0x7fa4b80d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:26.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.084+0000 7fa4bed5f640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa4b80b5f20 0x7fa4b80d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:26.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.084+0000 7fa4bed5f640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4b80b7e50 con 0x7fa4b80d39f0 2024-04-26T20:51:26.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.084+0000 7fa4bed5f640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7fa4b80d0580 con 0x7fa4b8162880 2024-04-26T20:51:26.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.084+0000 7fa4bed5f640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fa4b80d1800 con 0x7fa4b80b5f20 2024-04-26T20:51:26.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.084+0000 7fa4bcafb640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa4b8162880 0x7fa4b80d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:26.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.084+0000 7fa4bcafb640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa4b8162880 0x7fa4b80d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60014/0 (socket says 172.21.15.73:60014) 2024-04-26T20:51:26.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.084+0000 7fa4bcafb640 1 -- 172.21.15.73:0/4095321115 learned_addr learned my addr 172.21.15.73:0/4095321115 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:26.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.085+0000 7fa4bcafb640 1 -- 172.21.15.73:0/4095321115 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa4b80b5f20 msgr2=0x7fa4b80d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:26.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.085+0000 7fa4bcafb640 1 --2- 172.21.15.73:0/4095321115 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa4b80b5f20 0x7fa4b80d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:26.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.085+0000 7fa4bcafb640 1 -- 172.21.15.73:0/4095321115 >> v1:172.21.15.120:6789/0 conn(0x7fa4b80d39f0 legacy=0x7fa4b80b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:26.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.085+0000 7fa4bcafb640 1 -- 172.21.15.73:0/4095321115 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b80d07d0 con 0x7fa4b8162880 2024-04-26T20:51:26.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.085+0000 7fa4bcafb640 1 --2- 172.21.15.73:0/4095321115 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4b8162880 0x7fa4b80d2cb0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8009870 tx=0x7fa4a802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f878086b3f20715 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:26.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.085+0000 7fa4b77fe640 1 -- 172.21.15.73:0/4095321115 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a8037070 con 0x7fa4b8162880 2024-04-26T20:51:26.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.085+0000 7fa4b77fe640 1 -- 172.21.15.73:0/4095321115 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4a8032440 con 0x7fa4b8162880 2024-04-26T20:51:26.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.085+0000 7fa4b77fe640 1 -- 172.21.15.73:0/4095321115 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a8032810 con 0x7fa4b8162880 2024-04-26T20:51:26.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.086+0000 7fa4bed5f640 1 -- 172.21.15.73:0/4095321115 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4b8162880 msgr2=0x7fa4b80d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:26.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.086+0000 7fa4bed5f640 1 --2- 172.21.15.73:0/4095321115 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4b8162880 0x7fa4b80d2cb0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8009870 tx=0x7fa4a802a210 comp rx=0 tx=0).stop 2024-04-26T20:51:26.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.086+0000 7fa4bed5f640 1 -- 172.21.15.73:0/4095321115 shutdown_connections 2024-04-26T20:51:26.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.086+0000 7fa4bed5f640 1 --2- 172.21.15.73:0/4095321115 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa4b80b5f20 0x7fa4b80d34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:26.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.086+0000 7fa4bed5f640 1 --2- 172.21.15.73:0/4095321115 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4b8162880 0x7fa4b80d2cb0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:26.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.086+0000 7fa4bed5f640 1 -- 172.21.15.73:0/4095321115 >> 172.21.15.73:0/4095321115 conn(0x7fa4b80d4f60 msgr2=0x7fa4b80d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:26.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.086+0000 7fa4bed5f640 1 -- 172.21.15.73:0/4095321115 shutdown_connections 2024-04-26T20:51:26.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.086+0000 7fa4bed5f640 1 -- 172.21.15.73:0/4095321115 wait complete. 2024-04-26T20:51:26.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.087+0000 7fa4bed5f640 1 Processor -- start 2024-04-26T20:51:26.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.087+0000 7fa4bed5f640 1 -- start start 2024-04-26T20:51:26.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.088+0000 7fa4bed5f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa4b80b5f20 0x7fa4b815da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:26.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.088+0000 7fa4bed5f640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa4b80d39f0 0x7fa4b80b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:26.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.088+0000 7fa4bed5f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4b8162880 0x7fa4b80b3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:26.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.088+0000 7fa4bed5f640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b8162c60 con 0x7fa4b8162880 2024-04-26T20:51:26.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.088+0000 7fa4bed5f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b80d0d90 con 0x7fa4b80b5f20 2024-04-26T20:51:26.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.088+0000 7fa4bed5f640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fa4b8162de0 con 0x7fa4b80d39f0 2024-04-26T20:51:26.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.088+0000 7fa4bd2fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4b8162880 0x7fa4b80b3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:26.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.088+0000 7fa4bd2fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4b8162880 0x7fa4b80b3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60022/0 (socket says 172.21.15.73:60022) 2024-04-26T20:51:26.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.088+0000 7fa4bd2fc640 1 -- 172.21.15.73:0/3369995497 learned_addr learned my addr 172.21.15.73:0/3369995497 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:26.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.088+0000 7fa4bcafb640 1 --2- 172.21.15.73:0/3369995497 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa4b80b5f20 0x7fa4b815da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:26.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.088+0000 7fa4b7fff640 1 --2- 172.21.15.73:0/3369995497 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa4b80d39f0 0x7fa4b80b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:26.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.088+0000 7fa4bd2fc640 1 -- 172.21.15.73:0/3369995497 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa4b80d39f0 msgr2=0x7fa4b80b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:26.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.088+0000 7fa4bd2fc640 1 --2- 172.21.15.73:0/3369995497 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa4b80d39f0 0x7fa4b80b28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:26.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.088+0000 7fa4bd2fc640 1 -- 172.21.15.73:0/3369995497 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa4b80b5f20 msgr2=0x7fa4b815da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:26.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.088+0000 7fa4bd2fc640 1 --2- 172.21.15.73:0/3369995497 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa4b80b5f20 0x7fa4b815da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:26.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.088+0000 7fa4bd2fc640 1 -- 172.21.15.73:0/3369995497 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b80b43e0 con 0x7fa4b8162880 2024-04-26T20:51:26.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.089+0000 7fa4bd2fc640 1 --2- 172.21.15.73:0/3369995497 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4b8162880 0x7fa4b80b3230 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa4b000c780 tx=0x7fa4b000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:26.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.089+0000 7fa4b57fa640 1 -- 172.21.15.73:0/3369995497 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b0017070 con 0x7fa4b8162880 2024-04-26T20:51:26.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.089+0000 7fa4b57fa640 1 -- 172.21.15.73:0/3369995497 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4b0003da0 con 0x7fa4b8162880 2024-04-26T20:51:26.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.089+0000 7fa4bed5f640 1 -- 172.21.15.73:0/3369995497 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4b80ce5f0 con 0x7fa4b8162880 2024-04-26T20:51:26.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.089+0000 7fa4b57fa640 1 -- 172.21.15.73:0/3369995497 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b00052b0 con 0x7fa4b8162880 2024-04-26T20:51:26.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.089+0000 7fa4bed5f640 1 -- 172.21.15.73:0/3369995497 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4b80b4100 con 0x7fa4b8162880 2024-04-26T20:51:26.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.090+0000 7fa4bed5f640 1 -- 172.21.15.73:0/3369995497 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4b80694f0 con 0x7fa4b8162880 2024-04-26T20:51:26.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.090+0000 7fa4b57fa640 1 -- 172.21.15.73:0/3369995497 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa4b001a070 con 0x7fa4b8162880 2024-04-26T20:51:26.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.090+0000 7fa4b57fa640 1 --2- 172.21.15.73:0/3369995497 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa4a406c920 0x7fa4a406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:26.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.091+0000 7fa4b57fa640 1 -- 172.21.15.73:0/3369995497 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fa4b008b440 con 0x7fa4b8162880 2024-04-26T20:51:26.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.091+0000 7fa4bcafb640 1 --2- 172.21.15.73:0/3369995497 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa4a406c920 0x7fa4a406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:26.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.091+0000 7fa4bcafb640 1 --2- 172.21.15.73:0/3369995497 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa4a406c920 0x7fa4a406ede0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8034000 tx=0x7fa4a802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:26.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.095+0000 7fa4b57fa640 1 -- 172.21.15.73:0/3369995497 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa4b0058eb0 con 0x7fa4b8162880 2024-04-26T20:51:26.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.281+0000 7fa4bed5f640 1 -- 172.21.15.73:0/3369995497 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa4b8075320 con 0x7fa4b8162880 2024-04-26T20:51:26.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.281+0000 7fa4b57fa640 1 -- 172.21.15.73:0/3369995497 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7fa4b005d1b0 con 0x7fa4b8162880 2024-04-26T20:51:26.282 INFO:teuthology.orchestra.run.smithi073.stderr:listed 0 entries 2024-04-26T20:51:26.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.283+0000 7fa4bed5f640 1 -- 172.21.15.73:0/3369995497 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa4a406c920 msgr2=0x7fa4a406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:26.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.283+0000 7fa4bed5f640 1 --2- 172.21.15.73:0/3369995497 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa4a406c920 0x7fa4a406ede0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8034000 tx=0x7fa4a802e000 comp rx=0 tx=0).stop 2024-04-26T20:51:26.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.283+0000 7fa4bed5f640 1 -- 172.21.15.73:0/3369995497 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4b8162880 msgr2=0x7fa4b80b3230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:26.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.283+0000 7fa4bed5f640 1 --2- 172.21.15.73:0/3369995497 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4b8162880 0x7fa4b80b3230 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa4b000c780 tx=0x7fa4b000cc40 comp rx=0 tx=0).stop 2024-04-26T20:51:26.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.283+0000 7fa4bed5f640 1 -- 172.21.15.73:0/3369995497 shutdown_connections 2024-04-26T20:51:26.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.283+0000 7fa4bed5f640 1 --2- 172.21.15.73:0/3369995497 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4b8162880 0x7fa4b80b3230 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:26.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.283+0000 7fa4bed5f640 1 --2- 172.21.15.73:0/3369995497 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa4a406c920 0x7fa4a406ede0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:26.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.283+0000 7fa4bed5f640 1 --2- 172.21.15.73:0/3369995497 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa4b80d39f0 0x7fa4b80b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:26.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.283+0000 7fa4bed5f640 1 --2- 172.21.15.73:0/3369995497 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa4b80b5f20 0x7fa4b815da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:26.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.283+0000 7fa4bed5f640 1 -- 172.21.15.73:0/3369995497 >> 172.21.15.73:0/3369995497 conn(0x7fa4b80d4f60 msgr2=0x7fa4b8160b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:26.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.283+0000 7fa4bed5f640 1 -- 172.21.15.73:0/3369995497 shutdown_connections 2024-04-26T20:51:26.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.283+0000 7fa4bed5f640 1 -- 172.21.15.73:0/3369995497 wait complete. 2024-04-26T20:51:26.295 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-26T20:51:26.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:26.295 DEBUG:teuthology.orchestra.run.smithi120:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-04-26T20:51:26.327 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi120.front.sepia.ceph.com 2024-04-26T20:51:26.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:26.327 DEBUG:teuthology.orchestra.run.smithi120:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-26T20:51:26.391 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-26T20:51:26.391 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:51:26.407 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-26T20:51:26.407 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:51:26.463 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-26T20:51:26.463 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:51:26.519 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:51:26.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.654+0000 7f1adb646640 1 Processor -- start 2024-04-26T20:51:26.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.655+0000 7f1adb646640 1 -- start start 2024-04-26T20:51:26.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.655+0000 7f1adb646640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1ad415f3a0 0x7f1ad40d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:26.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.655+0000 7f1adb646640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f1ad40d1770 0x7f1ad40d1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:26.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.655+0000 7f1adb646640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ad40d2590 con 0x7f1ad40d2160 2024-04-26T20:51:26.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.655+0000 7f1adb646640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f1ad40d0e20 con 0x7f1ad415f3a0 2024-04-26T20:51:26.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.655+0000 7f1adb646640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f1ad40b1d70 con 0x7f1ad40d1770 2024-04-26T20:51:26.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.655+0000 7f1ad93e2640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1ad415f3a0 0x7f1ad40d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:26.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.656+0000 7f1ad93e2640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1ad415f3a0 0x7f1ad40d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60030/0 (socket says 172.21.15.73:60030) 2024-04-26T20:51:26.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.656+0000 7f1ad93e2640 1 -- 172.21.15.73:0/3204575205 learned_addr learned my addr 172.21.15.73:0/3204575205 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:26.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.656+0000 7f1ad8be1640 1 --2- 172.21.15.73:0/3204575205 >> v2:172.21.15.120:3300/0 conn(0x7f1ad40d1770 0x7f1ad40d1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:26.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.656+0000 7f1ad93e2640 1 -- 172.21.15.73:0/3204575205 >> v2:172.21.15.120:3300/0 conn(0x7f1ad40d1770 msgr2=0x7f1ad40d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:26.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.656+0000 7f1ad93e2640 1 --2- 172.21.15.73:0/3204575205 >> v2:172.21.15.120:3300/0 conn(0x7f1ad40d1770 0x7f1ad40d1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:26.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.656+0000 7f1ad93e2640 1 -- 172.21.15.73:0/3204575205 >> v1:172.21.15.120:6789/0 conn(0x7f1ad40d2160 legacy=0x7f1ad40d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:26.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.656+0000 7f1ad93e2640 1 -- 172.21.15.73:0/3204575205 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ad40b2050 con 0x7f1ad415f3a0 2024-04-26T20:51:26.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.656+0000 7f1ad93e2640 1 --2- 172.21.15.73:0/3204575205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ad415f3a0 0x7f1ad40d06f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f1ad40b7b80 tx=0x7f1ad002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbb147fe39197ce8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:26.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.657+0000 7f1acbfff640 1 -- 172.21.15.73:0/3204575205 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad0037070 con 0x7f1ad415f3a0 2024-04-26T20:51:26.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.657+0000 7f1acbfff640 1 -- 172.21.15.73:0/3204575205 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ad0032440 con 0x7f1ad415f3a0 2024-04-26T20:51:26.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.657+0000 7f1acbfff640 1 -- 172.21.15.73:0/3204575205 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad0032810 con 0x7f1ad415f3a0 2024-04-26T20:51:26.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.657+0000 7f1adb646640 1 -- 172.21.15.73:0/3204575205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ad415f3a0 msgr2=0x7f1ad40d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:26.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.657+0000 7f1adb646640 1 --2- 172.21.15.73:0/3204575205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ad415f3a0 0x7f1ad40d06f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f1ad40b7b80 tx=0x7f1ad002a270 comp rx=0 tx=0).stop 2024-04-26T20:51:26.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.657+0000 7f1adb646640 1 -- 172.21.15.73:0/3204575205 shutdown_connections 2024-04-26T20:51:26.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.657+0000 7f1adb646640 1 --2- 172.21.15.73:0/3204575205 >> v2:172.21.15.120:3300/0 conn(0x7f1ad40d1770 0x7f1ad40d1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:26.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.657+0000 7f1adb646640 1 --2- 172.21.15.73:0/3204575205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ad415f3a0 0x7f1ad40d06f0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:26.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.657+0000 7f1adb646640 1 -- 172.21.15.73:0/3204575205 >> 172.21.15.73:0/3204575205 conn(0x7f1ad40d4f90 msgr2=0x7f1ad40d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:26.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.657+0000 7f1adb646640 1 -- 172.21.15.73:0/3204575205 shutdown_connections 2024-04-26T20:51:26.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.657+0000 7f1adb646640 1 -- 172.21.15.73:0/3204575205 wait complete. 2024-04-26T20:51:26.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.658+0000 7f1adb646640 1 Processor -- start 2024-04-26T20:51:26.660 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.659+0000 7f1adb646640 1 -- start start 2024-04-26T20:51:26.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.659+0000 7f1adb646640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ad40d1770 0x7f1ad41534a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:26.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.659+0000 7f1adb646640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1ad40d2160 0x7f1ad40b8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:26.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.659+0000 7f1adb646640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ad415f3a0 0x7f1ad40b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:26.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.659+0000 7f1adb646640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad415f900 con 0x7f1ad415f3a0 2024-04-26T20:51:26.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.659+0000 7f1adb646640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad415f780 con 0x7f1ad40d1770 2024-04-26T20:51:26.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.659+0000 7f1adb646640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f1ad40b3860 con 0x7f1ad40d2160 2024-04-26T20:51:26.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.659+0000 7f1ad9be3640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ad415f3a0 0x7f1ad40b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:26.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.659+0000 7f1ad9be3640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ad415f3a0 0x7f1ad40b4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60044/0 (socket says 172.21.15.73:60044) 2024-04-26T20:51:26.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.659+0000 7f1ad9be3640 1 -- 172.21.15.73:0/1586148017 learned_addr learned my addr 172.21.15.73:0/1586148017 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:26.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.660+0000 7f1ad9be3640 1 -- 172.21.15.73:0/1586148017 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1ad40d2160 msgr2=0x7f1ad40b8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:26.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.660+0000 7f1ad8be1640 1 --2- 172.21.15.73:0/1586148017 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1ad40d2160 0x7f1ad40b8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:26.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.660+0000 7f1ad9be3640 1 --2- 172.21.15.73:0/1586148017 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1ad40d2160 0x7f1ad40b8790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:26.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.660+0000 7f1ad93e2640 1 --2- 172.21.15.73:0/1586148017 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ad40d1770 0x7f1ad41534a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:26.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.660+0000 7f1ad9be3640 1 -- 172.21.15.73:0/1586148017 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ad40d1770 msgr2=0x7f1ad41534a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:26.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.660+0000 7f1ad9be3640 1 --2- 172.21.15.73:0/1586148017 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ad40d1770 0x7f1ad41534a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:26.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.660+0000 7f1ad9be3640 1 -- 172.21.15.73:0/1586148017 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ad40b8cd0 con 0x7f1ad415f3a0 2024-04-26T20:51:26.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.660+0000 7f1ad9be3640 1 --2- 172.21.15.73:0/1586148017 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ad415f3a0 0x7f1ad40b4e50 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f1acc00c7d0 tx=0x7f1acc00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:26.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.660+0000 7f1ac9ffb640 1 -- 172.21.15.73:0/1586148017 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1acc003b70 con 0x7f1ad415f3a0 2024-04-26T20:51:26.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.660+0000 7f1ac9ffb640 1 -- 172.21.15.73:0/1586148017 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1acc003d30 con 0x7f1ad415f3a0 2024-04-26T20:51:26.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.660+0000 7f1adb646640 1 -- 172.21.15.73:0/1586148017 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ad4160a50 con 0x7f1ad415f3a0 2024-04-26T20:51:26.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.660+0000 7f1ac9ffb640 1 -- 172.21.15.73:0/1586148017 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1acc00aa90 con 0x7f1ad415f3a0 2024-04-26T20:51:26.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.660+0000 7f1adb646640 1 -- 172.21.15.73:0/1586148017 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ad40b7cb0 con 0x7f1ad415f3a0 2024-04-26T20:51:26.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.661+0000 7f1adb646640 1 -- 172.21.15.73:0/1586148017 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ad40694f0 con 0x7f1ad415f3a0 2024-04-26T20:51:26.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.661+0000 7f1ac9ffb640 1 -- 172.21.15.73:0/1586148017 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1acc00ac30 con 0x7f1ad415f3a0 2024-04-26T20:51:26.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.662+0000 7f1ac9ffb640 1 --2- 172.21.15.73:0/1586148017 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1ab806c8f0 0x7f1ab806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:26.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.662+0000 7f1ac9ffb640 1 -- 172.21.15.73:0/1586148017 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f1acc08be20 con 0x7f1ad415f3a0 2024-04-26T20:51:26.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.662+0000 7f1ad93e2640 1 --2- 172.21.15.73:0/1586148017 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1ab806c8f0 0x7f1ab806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:26.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.662+0000 7f1ad93e2640 1 --2- 172.21.15.73:0/1586148017 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1ab806c8f0 0x7f1ab806edb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0034000 tx=0x7f1ad002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:26.667 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.666+0000 7f1ac9ffb640 1 -- 172.21.15.73:0/1586148017 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1acc059ee0 con 0x7f1ad415f3a0 2024-04-26T20:51:26.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.843+0000 7f1adb646640 1 -- 172.21.15.73:0/1586148017 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1ad40b65b0 con 0x7f1ad415f3a0 2024-04-26T20:51:26.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.844+0000 7f1ac9ffb640 1 -- 172.21.15.73:0/1586148017 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18902 (secure 0 0 0) 0x7f1acc0597e0 con 0x7f1ad415f3a0 2024-04-26T20:51:26.845 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:51:26.845 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":27,"fsid":"fbd3f548-b547-4713-856b-09f98f5d1764","created":"2024-04-26T20:50:07.876786+0000","modified":"2024-04-26T20:51:13.783133+0000","last_up_change":"2024-04-26T20:50:19.082741+0000","last_in_change":"2024-04-26T20:50:10.018400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:50:19.993731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:50:29.964420+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:50:30.920082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:50:34.650207+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"72d6f4f3-214a-4392-a02f-c33cbcab3c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6809","nonce":3461705124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6811","nonce":3461705124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6815","nonce":3461705124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6813","nonce":3461705124}]},"public_addr":"172.21.15.73:6809/3461705124","cluster_addr":"172.21.15.73:6811/3461705124","heartbeat_back_addr":"172.21.15.73:6815/3461705124","heartbeat_front_addr":"172.21.15.73:6813/3461705124","state":["exists","up"]},{"osd":1,"uuid":"786bf8ca-6ddc-4b3c-be6c-9152398bd5fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6817","nonce":2827619403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6819","nonce":2827619403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6825","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6827","nonce":2827619403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6821","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6823","nonce":2827619403}]},"public_addr":"172.21.15.73:6817/2827619403","cluster_addr":"172.21.15.73:6819/2827619403","heartbeat_back_addr":"172.21.15.73:6827/2827619403","heartbeat_front_addr":"172.21.15.73:6823/2827619403","state":["exists","up"]},{"osd":2,"uuid":"90963b1a-5437-4ef8-ad8b-6b74c534eeb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6801","nonce":2542681064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6803","nonce":2542681064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6808","nonce":2542681064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6805","nonce":2542681064}]},"public_addr":"172.21.15.73:6801/2542681064","cluster_addr":"172.21.15.73:6803/2542681064","heartbeat_back_addr":"172.21.15.73:6808/2542681064","heartbeat_front_addr":"172.21.15.73:6805/2542681064","state":["exists","up"]},{"osd":3,"uuid":"bd3d2148-6e38-41b4-a453-fe6e3b8b4ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6822","nonce":1604042718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6826","nonce":1604042718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6831","nonce":1604042718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6829","nonce":1604042718}]},"public_addr":"172.21.15.73:6822/1604042718","cluster_addr":"172.21.15.73:6826/1604042718","heartbeat_back_addr":"172.21.15.73:6831/1604042718","heartbeat_front_addr":"172.21.15.73:6829/1604042718","state":["exists","up"]},{"osd":4,"uuid":"e8668dc8-a23f-46d2-8311-cb882da08e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6817","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6819","nonce":3591055561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6824","nonce":3591055561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6831","nonce":3591055561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6828","nonce":3591055561}]},"public_addr":"172.21.15.120:6819/3591055561","cluster_addr":"172.21.15.120:6824/3591055561","heartbeat_back_addr":"172.21.15.120:6831/3591055561","heartbeat_front_addr":"172.21.15.120:6828/3591055561","state":["exists","up"]},{"osd":5,"uuid":"cbac0bdb-96bd-4852-bcce-e564495c3fcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6809","nonce":819450433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6811","nonce":819450433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6815","nonce":819450433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6813","nonce":819450433}]},"public_addr":"172.21.15.120:6809/819450433","cluster_addr":"172.21.15.120:6811/819450433","heartbeat_back_addr":"172.21.15.120:6815/819450433","heartbeat_front_addr":"172.21.15.120:6813/819450433","state":["exists","up"]},{"osd":6,"uuid":"1884b2a8-569f-4272-96a9-f25a09364dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6801","nonce":3946210727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6803","nonce":3946210727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6807","nonce":3946210727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6805","nonce":3946210727}]},"public_addr":"172.21.15.120:6801/3946210727","cluster_addr":"172.21.15.120:6803/3946210727","heartbeat_back_addr":"172.21.15.120:6807/3946210727","heartbeat_front_addr":"172.21.15.120:6805/3946210727","state":["exists","up"]},{"osd":7,"uuid":"c0c3bf6d-a4c5-4f1e-aa2d-330951830666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6818","nonce":2915574674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6821","nonce":2915574674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6827","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6829","nonce":2915574674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6823","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6825","nonce":2915574674}]},"public_addr":"172.21.15.120:6818/2915574674","cluster_addr":"172.21.15.120:6821/2915574674","heartbeat_back_addr":"172.21.15.120:6829/2915574674","heartbeat_front_addr":"172.21.15.120:6825/2915574674","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.218216+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.208710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.112705+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.182789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.164324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.174212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.225396+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.261559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:51:26.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.846+0000 7f1adb646640 1 -- 172.21.15.73:0/1586148017 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1ab806c8f0 msgr2=0x7f1ab806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:26.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.846+0000 7f1adb646640 1 --2- 172.21.15.73:0/1586148017 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1ab806c8f0 0x7f1ab806edb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0034000 tx=0x7f1ad002e000 comp rx=0 tx=0).stop 2024-04-26T20:51:26.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.846+0000 7f1adb646640 1 -- 172.21.15.73:0/1586148017 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ad415f3a0 msgr2=0x7f1ad40b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:26.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.846+0000 7f1adb646640 1 --2- 172.21.15.73:0/1586148017 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ad415f3a0 0x7f1ad40b4e50 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f1acc00c7d0 tx=0x7f1acc00cc90 comp rx=0 tx=0).stop 2024-04-26T20:51:26.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.846+0000 7f1adb646640 1 -- 172.21.15.73:0/1586148017 shutdown_connections 2024-04-26T20:51:26.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.846+0000 7f1adb646640 1 --2- 172.21.15.73:0/1586148017 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ad415f3a0 0x7f1ad40b4e50 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:26.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.846+0000 7f1adb646640 1 --2- 172.21.15.73:0/1586148017 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1ab806c8f0 0x7f1ab806edb0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:26.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.846+0000 7f1adb646640 1 --2- 172.21.15.73:0/1586148017 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1ad40d2160 0x7f1ad40b8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:26.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.846+0000 7f1adb646640 1 --2- 172.21.15.73:0/1586148017 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ad40d1770 0x7f1ad41534a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:26.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.846+0000 7f1adb646640 1 -- 172.21.15.73:0/1586148017 >> 172.21.15.73:0/1586148017 conn(0x7f1ad40d4f90 msgr2=0x7f1ad40b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:26.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.846+0000 7f1adb646640 1 -- 172.21.15.73:0/1586148017 shutdown_connections 2024-04-26T20:51:26.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.846+0000 7f1adb646640 1 -- 172.21.15.73:0/1586148017 wait complete. 2024-04-26T20:51:26.858 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:50:19.993731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:50:29.964420+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:50:30.920082+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T20:50:34.650207+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T20:51:26.859 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:51:26.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.957+0000 7fe12a740640 1 Processor -- start 2024-04-26T20:51:26.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.957+0000 7fe12a740640 1 -- start start 2024-04-26T20:51:26.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.958+0000 7fe12a740640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe124071260 0x7fe12415e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:26.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.958+0000 7fe12a740640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1240d2390 con 0x7fe1240cfde0 2024-04-26T20:51:26.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.958+0000 7fe12a740640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe12406d3f0 con 0x7fe124070350 2024-04-26T20:51:26.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.958+0000 7fe12a740640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7fe1240afab0 con 0x7fe124071260 2024-04-26T20:51:26.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.958+0000 7fe128cdd640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe124071260 0x7fe12415e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:26.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.958+0000 7fe123fff640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fe1240cfde0 0x7fe1240d1b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:34610/0 (socket says 172.21.15.73:34610) 2024-04-26T20:51:26.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.958+0000 7fe123fff640 1 -- 172.21.15.73:0/3832659950 learned_addr learned my addr 172.21.15.73:0/3832659950 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:26.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.958+0000 7fe128cdd640 1 -- 172.21.15.73:0/3832659950 >> v1:172.21.15.120:6789/0 conn(0x7fe124070350 legacy=0x7fe124070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:26.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.958+0000 7fe128cdd640 1 -- 172.21.15.73:0/3832659950 >> v1:172.21.15.73:6789/0 conn(0x7fe1240cfde0 legacy=0x7fe1240d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:26.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.958+0000 7fe128cdd640 1 -- 172.21.15.73:0/3832659950 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe12406d6d0 con 0x7fe124071260 2024-04-26T20:51:26.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.959+0000 7fe128cdd640 1 --2- 172.21.15.73:0/3832659950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe124071260 0x7fe12415e9f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe11c009d90 tx=0x7fe11c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3abbea71234a98db server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:26.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.959+0000 7fe122ffd640 1 -- 172.21.15.73:0/3832659950 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe11c007740 con 0x7fe124071260 2024-04-26T20:51:26.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.959+0000 7fe122ffd640 1 -- 172.21.15.73:0/3832659950 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe11c0078e0 con 0x7fe124071260 2024-04-26T20:51:26.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.959+0000 7fe122ffd640 1 -- 172.21.15.73:0/3832659950 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe11c007cb0 con 0x7fe124071260 2024-04-26T20:51:26.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.959+0000 7fe12a740640 1 -- 172.21.15.73:0/3832659950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe124071260 msgr2=0x7fe12415e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:26.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.959+0000 7fe12a740640 1 --2- 172.21.15.73:0/3832659950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe124071260 0x7fe12415e9f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe11c009d90 tx=0x7fe11c02a660 comp rx=0 tx=0).stop 2024-04-26T20:51:26.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.959+0000 7fe12a740640 1 -- 172.21.15.73:0/3832659950 shutdown_connections 2024-04-26T20:51:26.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.959+0000 7fe12a740640 1 --2- 172.21.15.73:0/3832659950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe124071260 0x7fe12415e9f0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:26.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.959+0000 7fe12a740640 1 -- 172.21.15.73:0/3832659950 >> 172.21.15.73:0/3832659950 conn(0x7fe1240d2d80 msgr2=0x7fe1240d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:26.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.960+0000 7fe12a740640 1 -- 172.21.15.73:0/3832659950 shutdown_connections 2024-04-26T20:51:26.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.960+0000 7fe12a740640 1 -- 172.21.15.73:0/3832659950 wait complete. 2024-04-26T20:51:26.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.961+0000 7fe12a740640 1 Processor -- start 2024-04-26T20:51:26.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.961+0000 7fe12a740640 1 -- start start 2024-04-26T20:51:26.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.961+0000 7fe12a740640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe124070350 0x7fe124147d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:26.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.961+0000 7fe12a740640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe124071260 0x7fe1240c7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:26.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.961+0000 7fe12a740640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe1240cfde0 0x7fe124157d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:26.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.961+0000 7fe12a740640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe12406f1c0 con 0x7fe124071260 2024-04-26T20:51:26.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.961+0000 7fe12a740640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe12406dd20 con 0x7fe1240cfde0 2024-04-26T20:51:26.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.961+0000 7fe12a740640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fe12406f340 con 0x7fe124070350 2024-04-26T20:51:26.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.962+0000 7fe1237fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe124071260 0x7fe1240c7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:26.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.962+0000 7fe1237fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe124071260 0x7fe1240c7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60054/0 (socket says 172.21.15.73:60054) 2024-04-26T20:51:26.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.962+0000 7fe1237fe640 1 -- 172.21.15.73:0/2254149393 learned_addr learned my addr 172.21.15.73:0/2254149393 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:26.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.962+0000 7fe128cdd640 1 --2- 172.21.15.73:0/2254149393 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe1240cfde0 0x7fe124157d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:26.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.962+0000 7fe123fff640 1 --2- 172.21.15.73:0/2254149393 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe124070350 0x7fe124147d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:26.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.962+0000 7fe1237fe640 1 -- 172.21.15.73:0/2254149393 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe124070350 msgr2=0x7fe124147d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:26.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.962+0000 7fe1237fe640 1 --2- 172.21.15.73:0/2254149393 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe124070350 0x7fe124147d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:26.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.962+0000 7fe1237fe640 1 -- 172.21.15.73:0/2254149393 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe1240cfde0 msgr2=0x7fe124157d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:26.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.962+0000 7fe1237fe640 1 --2- 172.21.15.73:0/2254149393 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe1240cfde0 0x7fe124157d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:26.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.962+0000 7fe1237fe640 1 -- 172.21.15.73:0/2254149393 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1240c8290 con 0x7fe124071260 2024-04-26T20:51:26.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.962+0000 7fe1237fe640 1 --2- 172.21.15.73:0/2254149393 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe124071260 0x7fe1240c7840 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe11800c3c0 tx=0x7fe11800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:26.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.962+0000 7fe120ff9640 1 -- 172.21.15.73:0/2254149393 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1180052d0 con 0x7fe124071260 2024-04-26T20:51:26.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.962+0000 7fe120ff9640 1 -- 172.21.15.73:0/2254149393 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe118005490 con 0x7fe124071260 2024-04-26T20:51:26.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.962+0000 7fe120ff9640 1 -- 172.21.15.73:0/2254149393 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe118007260 con 0x7fe124071260 2024-04-26T20:51:26.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.962+0000 7fe12a740640 1 -- 172.21.15.73:0/2254149393 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1240c8570 con 0x7fe124071260 2024-04-26T20:51:26.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.962+0000 7fe12a740640 1 -- 172.21.15.73:0/2254149393 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1240c8b30 con 0x7fe124071260 2024-04-26T20:51:26.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.963+0000 7fe12a740640 1 -- 172.21.15.73:0/2254149393 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1240694f0 con 0x7fe124071260 2024-04-26T20:51:26.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.964+0000 7fe120ff9640 1 -- 172.21.15.73:0/2254149393 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe118007400 con 0x7fe124071260 2024-04-26T20:51:26.967 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.964+0000 7fe120ff9640 1 --2- 172.21.15.73:0/2254149393 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe11006c8f0 0x7fe11006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:26.967 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.964+0000 7fe120ff9640 1 -- 172.21.15.73:0/2254149393 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fe11808b340 con 0x7fe124071260 2024-04-26T20:51:26.967 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.964+0000 7fe123fff640 1 --2- 172.21.15.73:0/2254149393 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe11006c8f0 0x7fe11006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:26.967 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.964+0000 7fe123fff640 1 --2- 172.21.15.73:0/2254149393 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe11006c8f0 0x7fe11006edb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe114005880 tx=0x7fe114009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:26.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:26.968+0000 7fe120ff9640 1 -- 172.21.15.73:0/2254149393 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe118058d20 con 0x7fe124071260 2024-04-26T20:51:27.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.141+0000 7fe12a740640 1 -- 172.21.15.73:0/2254149393 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe12415c820 con 0x7fe124071260 2024-04-26T20:51:27.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.142+0000 7fe120ff9640 1 -- 172.21.15.73:0/2254149393 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7fe11805f020 con 0x7fe124071260 2024-04-26T20:51:27.143 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-26T20:51:27.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.144+0000 7fe12a740640 1 -- 172.21.15.73:0/2254149393 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe11006c8f0 msgr2=0x7fe11006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:27.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.144+0000 7fe12a740640 1 --2- 172.21.15.73:0/2254149393 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe11006c8f0 0x7fe11006edb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe114005880 tx=0x7fe114009040 comp rx=0 tx=0).stop 2024-04-26T20:51:27.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.144+0000 7fe12a740640 1 -- 172.21.15.73:0/2254149393 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe124071260 msgr2=0x7fe1240c7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:27.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.144+0000 7fe12a740640 1 --2- 172.21.15.73:0/2254149393 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe124071260 0x7fe1240c7840 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe11800c3c0 tx=0x7fe11800c880 comp rx=0 tx=0).stop 2024-04-26T20:51:27.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.144+0000 7fe12a740640 1 -- 172.21.15.73:0/2254149393 shutdown_connections 2024-04-26T20:51:27.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.144+0000 7fe12a740640 1 --2- 172.21.15.73:0/2254149393 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe124071260 0x7fe1240c7840 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:27.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.144+0000 7fe12a740640 1 --2- 172.21.15.73:0/2254149393 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe11006c8f0 0x7fe11006edb0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:27.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.144+0000 7fe12a740640 1 --2- 172.21.15.73:0/2254149393 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe1240cfde0 0x7fe124157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:27.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.144+0000 7fe12a740640 1 --2- 172.21.15.73:0/2254149393 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe124070350 0x7fe124147d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:27.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.144+0000 7fe12a740640 1 -- 172.21.15.73:0/2254149393 >> 172.21.15.73:0/2254149393 conn(0x7fe1240d2d80 msgr2=0x7fe124070b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:27.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.144+0000 7fe12a740640 1 -- 172.21.15.73:0/2254149393 shutdown_connections 2024-04-26T20:51:27.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.144+0000 7fe12a740640 1 -- 172.21.15.73:0/2254149393 wait complete. 2024-04-26T20:51:27.155 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:51:27.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.254+0000 7f5c66124640 1 Processor -- start 2024-04-26T20:51:27.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.254+0000 7f5c66124640 1 -- start start 2024-04-26T20:51:27.256 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.255+0000 7f5c66124640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5c600cc5b0 0x7f5c60072240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:27.256 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.255+0000 7f5c66124640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c6006e240 con 0x7f5c60071e60 2024-04-26T20:51:27.256 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.255+0000 7f5c66124640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c6006fc30 con 0x7f5c60072780 2024-04-26T20:51:27.256 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.255+0000 7f5c66124640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f5c600701f0 con 0x7f5c600cc5b0 2024-04-26T20:51:27.256 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.255+0000 7f5c5effd640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5c600cc5b0 0x7f5c60072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:27.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.255+0000 7f5c5f7fe640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f5c60071e60 0x7f5c6015df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:34618/0 (socket says 172.21.15.73:34618) 2024-04-26T20:51:27.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.255+0000 7f5c5f7fe640 1 -- 172.21.15.73:0/3511877435 learned_addr learned my addr 172.21.15.73:0/3511877435 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:27.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.255+0000 7f5c5effd640 1 -- 172.21.15.73:0/3511877435 >> v1:172.21.15.120:6789/0 conn(0x7f5c60072780 legacy=0x7f5c60166b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:27.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.255+0000 7f5c5effd640 1 -- 172.21.15.73:0/3511877435 >> v1:172.21.15.73:6789/0 conn(0x7f5c60071e60 legacy=0x7f5c6015df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:27.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.255+0000 7f5c5effd640 1 -- 172.21.15.73:0/3511877435 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c6006ff10 con 0x7f5c600cc5b0 2024-04-26T20:51:27.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.255+0000 7f5c5e7fc640 1 -- 172.21.15.73:0/3511877435 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1399081238 0 0) 0x7f5c6006e240 con 0x7f5c60071e60 2024-04-26T20:51:27.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.256+0000 7f5c5effd640 1 --2- 172.21.15.73:0/3511877435 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c600cc5b0 0x7f5c60072240 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5c540097a0 tx=0x7f5c5402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91f38f70d2d0aaeb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:27.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.256+0000 7f5c5e7fc640 1 -- 172.21.15.73:0/3511877435 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c54037070 con 0x7f5c600cc5b0 2024-04-26T20:51:27.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.256+0000 7f5c5e7fc640 1 -- 172.21.15.73:0/3511877435 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c54032440 con 0x7f5c600cc5b0 2024-04-26T20:51:27.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.256+0000 7f5c5e7fc640 1 -- 172.21.15.73:0/3511877435 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c54032810 con 0x7f5c600cc5b0 2024-04-26T20:51:27.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.256+0000 7f5c66124640 1 -- 172.21.15.73:0/3511877435 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c600cc5b0 msgr2=0x7f5c60072240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:27.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.256+0000 7f5c66124640 1 --2- 172.21.15.73:0/3511877435 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c600cc5b0 0x7f5c60072240 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5c540097a0 tx=0x7f5c5402a1a0 comp rx=0 tx=0).stop 2024-04-26T20:51:27.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.256+0000 7f5c66124640 1 -- 172.21.15.73:0/3511877435 shutdown_connections 2024-04-26T20:51:27.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.256+0000 7f5c66124640 1 --2- 172.21.15.73:0/3511877435 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c600cc5b0 0x7f5c60072240 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:27.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.256+0000 7f5c66124640 1 -- 172.21.15.73:0/3511877435 >> 172.21.15.73:0/3511877435 conn(0x7f5c600d0be0 msgr2=0x7f5c600d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:27.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.257+0000 7f5c66124640 1 -- 172.21.15.73:0/3511877435 shutdown_connections 2024-04-26T20:51:27.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.257+0000 7f5c66124640 1 -- 172.21.15.73:0/3511877435 wait complete. 2024-04-26T20:51:27.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.258+0000 7f5c66124640 1 Processor -- start 2024-04-26T20:51:27.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.258+0000 7f5c66124640 1 -- start start 2024-04-26T20:51:27.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.258+0000 7f5c66124640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5c60071e60 0x7f5c600c4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:27.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.258+0000 7f5c66124640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c60072780 0x7f5c600c6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:27.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.258+0000 7f5c66124640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c600cc5b0 0x7f5c600c6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:27.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.258+0000 7f5c66124640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5c6006f8e0 con 0x7f5c600cc5b0 2024-04-26T20:51:27.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.258+0000 7f5c66124640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5c60072c10 con 0x7f5c60072780 2024-04-26T20:51:27.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.258+0000 7f5c66124640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f5c6006ddf0 con 0x7f5c60071e60 2024-04-26T20:51:27.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.259+0000 7f5c5ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c600cc5b0 0x7f5c600c6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:27.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.259+0000 7f5c5ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c600cc5b0 0x7f5c600c6850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60070/0 (socket says 172.21.15.73:60070) 2024-04-26T20:51:27.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.259+0000 7f5c5ffff640 1 -- 172.21.15.73:0/3731895555 learned_addr learned my addr 172.21.15.73:0/3731895555 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:27.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.259+0000 7f5c5effd640 1 --2- 172.21.15.73:0/3731895555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c60072780 0x7f5c600c6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:27.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.259+0000 7f5c5f7fe640 1 --2- 172.21.15.73:0/3731895555 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5c60071e60 0x7f5c600c4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:27.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.259+0000 7f5c5ffff640 1 -- 172.21.15.73:0/3731895555 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5c60071e60 msgr2=0x7f5c600c4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:27.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.259+0000 7f5c5ffff640 1 --2- 172.21.15.73:0/3731895555 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5c60071e60 0x7f5c600c4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:27.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.259+0000 7f5c5ffff640 1 -- 172.21.15.73:0/3731895555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c60072780 msgr2=0x7f5c600c6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:27.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.259+0000 7f5c5ffff640 1 --2- 172.21.15.73:0/3731895555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c60072780 0x7f5c600c6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:27.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.259+0000 7f5c5ffff640 1 -- 172.21.15.73:0/3731895555 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c600c9f20 con 0x7f5c600cc5b0 2024-04-26T20:51:27.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.259+0000 7f5c5ffff640 1 --2- 172.21.15.73:0/3731895555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c600cc5b0 0x7f5c600c6850 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5c5800c7a0 tx=0x7f5c5800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:27.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.259+0000 7f5c47fff640 1 -- 172.21.15.73:0/3731895555 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c58003b70 con 0x7f5c600cc5b0 2024-04-26T20:51:27.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.259+0000 7f5c47fff640 1 -- 172.21.15.73:0/3731895555 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c58003d30 con 0x7f5c600cc5b0 2024-04-26T20:51:27.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.259+0000 7f5c66124640 1 -- 172.21.15.73:0/3731895555 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c600b5480 con 0x7f5c600cc5b0 2024-04-26T20:51:27.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.259+0000 7f5c47fff640 1 -- 172.21.15.73:0/3731895555 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5800aa90 con 0x7f5c600cc5b0 2024-04-26T20:51:27.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.259+0000 7f5c66124640 1 -- 172.21.15.73:0/3731895555 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c600b5a40 con 0x7f5c600cc5b0 2024-04-26T20:51:27.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.260+0000 7f5c66124640 1 -- 172.21.15.73:0/3731895555 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c600694f0 con 0x7f5c600cc5b0 2024-04-26T20:51:27.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.261+0000 7f5c47fff640 1 -- 172.21.15.73:0/3731895555 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5c5800ac30 con 0x7f5c600cc5b0 2024-04-26T20:51:27.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.261+0000 7f5c47fff640 1 --2- 172.21.15.73:0/3731895555 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f5c4c06c8f0 0x7f5c4c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:27.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.261+0000 7f5c5f7fe640 1 --2- 172.21.15.73:0/3731895555 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f5c4c06c8f0 0x7f5c4c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:27.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.261+0000 7f5c47fff640 1 -- 172.21.15.73:0/3731895555 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f5c5808bcc0 con 0x7f5c600cc5b0 2024-04-26T20:51:27.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.262+0000 7f5c5f7fe640 1 --2- 172.21.15.73:0/3731895555 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f5c4c06c8f0 0x7f5c4c06edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5c50000f60 tx=0x7f5c50007e40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:27.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.265+0000 7f5c47fff640 1 -- 172.21.15.73:0/3731895555 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5c600c9f20 con 0x7f5c600cc5b0 2024-04-26T20:51:27.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.443+0000 7f5c66124640 1 -- 172.21.15.73:0/3731895555 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5c60157eb0 con 0x7f5c600cc5b0 2024-04-26T20:51:27.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.443+0000 7f5c47fff640 1 -- 172.21.15.73:0/3731895555 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f5c58059730 con 0x7f5c600cc5b0 2024-04-26T20:51:27.444 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-26T20:51:27.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.445+0000 7f5c66124640 1 -- 172.21.15.73:0/3731895555 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f5c4c06c8f0 msgr2=0x7f5c4c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:27.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.445+0000 7f5c66124640 1 --2- 172.21.15.73:0/3731895555 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f5c4c06c8f0 0x7f5c4c06edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5c50000f60 tx=0x7f5c50007e40 comp rx=0 tx=0).stop 2024-04-26T20:51:27.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.445+0000 7f5c66124640 1 -- 172.21.15.73:0/3731895555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c600cc5b0 msgr2=0x7f5c600c6850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:27.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.445+0000 7f5c66124640 1 --2- 172.21.15.73:0/3731895555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c600cc5b0 0x7f5c600c6850 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5c5800c7a0 tx=0x7f5c5800cc60 comp rx=0 tx=0).stop 2024-04-26T20:51:27.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.445+0000 7f5c66124640 1 -- 172.21.15.73:0/3731895555 shutdown_connections 2024-04-26T20:51:27.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.445+0000 7f5c66124640 1 --2- 172.21.15.73:0/3731895555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c600cc5b0 0x7f5c600c6850 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:27.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.445+0000 7f5c66124640 1 --2- 172.21.15.73:0/3731895555 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f5c4c06c8f0 0x7f5c4c06edb0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:27.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.445+0000 7f5c66124640 1 --2- 172.21.15.73:0/3731895555 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5c60072780 0x7f5c600c6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:27.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.445+0000 7f5c66124640 1 --2- 172.21.15.73:0/3731895555 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f5c60071e60 0x7f5c600c4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:27.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.445+0000 7f5c66124640 1 -- 172.21.15.73:0/3731895555 >> 172.21.15.73:0/3731895555 conn(0x7f5c600d0be0 msgr2=0x7f5c6015f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:27.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.445+0000 7f5c66124640 1 -- 172.21.15.73:0/3731895555 shutdown_connections 2024-04-26T20:51:27.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.445+0000 7f5c66124640 1 -- 172.21.15.73:0/3731895555 wait complete. 2024-04-26T20:51:27.457 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:51:27.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.555+0000 7f697a8b1640 1 Processor -- start 2024-04-26T20:51:27.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.555+0000 7f697a8b1640 1 -- start start 2024-04-26T20:51:27.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.556+0000 7f697a8b1640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f69740d3bc0 0x7f69740d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:27.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.556+0000 7f697a8b1640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69740d1040 con 0x7f697415f550 2024-04-26T20:51:27.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.556+0000 7f697a8b1640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69740c9680 con 0x7f697415f170 2024-04-26T20:51:27.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.556+0000 7f697a8b1640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f69740c9430 con 0x7f69740d3bc0 2024-04-26T20:51:27.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.556+0000 7f69737fe640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f697415f550 0x7f69740ca440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:34630/0 (socket says 172.21.15.73:34630) 2024-04-26T20:51:27.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.556+0000 7f69737fe640 1 -- 172.21.15.73:0/3942356921 learned_addr learned my addr 172.21.15.73:0/3942356921 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:27.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.557+0000 7f6978e4e640 1 --2- 172.21.15.73:0/3942356921 >> v2:172.21.15.120:3300/0 conn(0x7f69740d3bc0 0x7f69740d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:27.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.557+0000 7f6972ffd640 1 -- 172.21.15.73:0/3942356921 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1543411664 0 0) 0x7f6968001110 con 0x7f697415f550 2024-04-26T20:51:27.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.557+0000 7f6972ffd640 1 -- 172.21.15.73:0/3942356921 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f695c003520 con 0x7f697415f550 2024-04-26T20:51:27.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.557+0000 7f6972ffd640 1 -- 172.21.15.73:0/3942356921 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2885516853 0 0) 0x7f695c003520 con 0x7f697415f550 2024-04-26T20:51:27.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.557+0000 7f6972ffd640 1 -- 172.21.15.73:0/3942356921 >> v2:172.21.15.120:3300/0 conn(0x7f69740d3bc0 msgr2=0x7f69740d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:27.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.557+0000 7f6972ffd640 1 --2- 172.21.15.73:0/3942356921 >> v2:172.21.15.120:3300/0 conn(0x7f69740d3bc0 0x7f69740d5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:27.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.557+0000 7f6972ffd640 1 -- 172.21.15.73:0/3942356921 >> v1:172.21.15.120:6789/0 conn(0x7f697415f170 legacy=0x7f69740d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:27.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.557+0000 7f6972ffd640 1 -- 172.21.15.73:0/3942356921 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69740c9960 con 0x7f697415f550 2024-04-26T20:51:27.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.557+0000 7f6972ffd640 1 -- 172.21.15.73:0/3942356921 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7f6968001840 con 0x7f697415f550 2024-04-26T20:51:27.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.557+0000 7f6972ffd640 1 -- 172.21.15.73:0/3942356921 >> v1:172.21.15.73:6789/0 conn(0x7f697415f550 legacy=0x7f69740ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:27.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.557+0000 7f6972ffd640 1 --2- 172.21.15.73:0/3942356921 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f695c0039b0 0x7f695c003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:27.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.557+0000 7f6972ffd640 1 --2- 172.21.15.73:0/3942356921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f695c004800 0x7f695c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:27.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.557+0000 7f6972ffd640 1 --2- 172.21.15.73:0/3942356921 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f695c0051c0 0x7f695c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:27.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.557+0000 7f6972ffd640 1 -- 172.21.15.73:0/3942356921 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f69740c9430 con 0x7f695c004800 2024-04-26T20:51:27.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.557+0000 7f6972ffd640 1 -- 172.21.15.73:0/3942356921 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f695c00c090 con 0x7f695c0051c0 2024-04-26T20:51:27.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.557+0000 7f6972ffd640 1 -- 172.21.15.73:0/3942356921 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f695c00c290 con 0x7f695c0039b0 2024-04-26T20:51:27.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.557+0000 7f69737fe640 1 --2- 172.21.15.73:0/3942356921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f695c004800 0x7f695c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:27.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.557+0000 7f6978e4e640 1 --2- 172.21.15.73:0/3942356921 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f695c0051c0 0x7f695c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:27.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.557+0000 7f6973fff640 1 --2- 172.21.15.73:0/3942356921 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f695c0039b0 0x7f695c003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:27.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.557+0000 7f69737fe640 1 -- 172.21.15.73:0/3942356921 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f695c0039b0 msgr2=0x7f695c003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:27.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.557+0000 7f69737fe640 1 --2- 172.21.15.73:0/3942356921 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f695c0039b0 0x7f695c003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:27.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.557+0000 7f69737fe640 1 -- 172.21.15.73:0/3942356921 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f695c0051c0 msgr2=0x7f695c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:27.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.557+0000 7f69737fe640 1 --2- 172.21.15.73:0/3942356921 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f695c0051c0 0x7f695c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:27.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.557+0000 7f69737fe640 1 -- 172.21.15.73:0/3942356921 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f695c003520 con 0x7f695c004800 2024-04-26T20:51:27.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.557+0000 7f6978e4e640 1 -- 172.21.15.73:0/3942356921 reap_dead start 2024-04-26T20:51:27.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.557+0000 7f6973fff640 1 --2- 172.21.15.73:0/3942356921 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f695c0039b0 0x7f695c003df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:27.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.558+0000 7f69737fe640 1 --2- 172.21.15.73:0/3942356921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f695c004800 0x7f695c004c80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f69740c8990 tx=0x7f695c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=711e0b3b12702c54 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:27.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.558+0000 7f6972ffd640 1 -- 172.21.15.73:0/3942356921 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f696800c0e0 con 0x7f695c004800 2024-04-26T20:51:27.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.558+0000 7f6972ffd640 1 -- 172.21.15.73:0/3942356921 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6968005af0 con 0x7f695c004800 2024-04-26T20:51:27.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.558+0000 7f6972ffd640 1 -- 172.21.15.73:0/3942356921 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6968007070 con 0x7f695c004800 2024-04-26T20:51:27.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.558+0000 7f697a8b1640 1 -- 172.21.15.73:0/3942356921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f695c004800 msgr2=0x7f695c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:27.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.559+0000 7f697a8b1640 1 --2- 172.21.15.73:0/3942356921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f695c004800 0x7f695c004c80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f69740c8990 tx=0x7f695c005830 comp rx=0 tx=0).stop 2024-04-26T20:51:27.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.559+0000 7f697a8b1640 1 -- 172.21.15.73:0/3942356921 shutdown_connections 2024-04-26T20:51:27.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.559+0000 7f697a8b1640 1 --2- 172.21.15.73:0/3942356921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f695c004800 0x7f695c004c80 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:27.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.559+0000 7f697a8b1640 1 -- 172.21.15.73:0/3942356921 >> 172.21.15.73:0/3942356921 conn(0x7f69740d4f60 msgr2=0x7f69740d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:27.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.559+0000 7f697a8b1640 1 -- 172.21.15.73:0/3942356921 shutdown_connections 2024-04-26T20:51:27.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.559+0000 7f697a8b1640 1 -- 172.21.15.73:0/3942356921 wait complete. 2024-04-26T20:51:27.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.560+0000 7f697a8b1640 1 Processor -- start 2024-04-26T20:51:27.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.560+0000 7f697a8b1640 1 -- start start 2024-04-26T20:51:27.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.561+0000 7f697a8b1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f695c004800 0x7f697409ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:27.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.561+0000 7f697a8b1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69740ba6a0 0x7f6974150280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:27.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.561+0000 7f697a8b1640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f69741f8430 0x7f69741f88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:27.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.561+0000 7f697a8b1640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f69740d5c00 con 0x7f69740ba6a0 2024-04-26T20:51:27.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.561+0000 7f697a8b1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f69740d5a80 con 0x7f695c004800 2024-04-26T20:51:27.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.561+0000 7f697a8b1640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f69740d5d80 con 0x7f69741f8430 2024-04-26T20:51:27.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.561+0000 7f69737fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69740ba6a0 0x7f6974150280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:27.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.561+0000 7f69737fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69740ba6a0 0x7f6974150280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60080/0 (socket says 172.21.15.73:60080) 2024-04-26T20:51:27.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.561+0000 7f69737fe640 1 -- 172.21.15.73:0/977495008 learned_addr learned my addr 172.21.15.73:0/977495008 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:27.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.561+0000 7f6978e4e640 1 --2- 172.21.15.73:0/977495008 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f69741f8430 0x7f69741f88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:27.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.561+0000 7f6973fff640 1 --2- 172.21.15.73:0/977495008 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f695c004800 0x7f697409ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:27.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.561+0000 7f69737fe640 1 -- 172.21.15.73:0/977495008 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f69741f8430 msgr2=0x7f69741f88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:27.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.561+0000 7f69737fe640 1 --2- 172.21.15.73:0/977495008 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f69741f8430 0x7f69741f88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:27.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.561+0000 7f69737fe640 1 -- 172.21.15.73:0/977495008 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f695c004800 msgr2=0x7f697409ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:27.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.561+0000 7f69737fe640 1 --2- 172.21.15.73:0/977495008 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f695c004800 0x7f697409ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:27.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.561+0000 7f69737fe640 1 -- 172.21.15.73:0/977495008 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69740b7eb0 con 0x7f69740ba6a0 2024-04-26T20:51:27.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.561+0000 7f69737fe640 1 --2- 172.21.15.73:0/977495008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69740ba6a0 0x7f6974150280 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f696800c580 tx=0x7f69740c9850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:27.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.561+0000 7f6970ff9640 1 -- 172.21.15.73:0/977495008 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6968007270 con 0x7f69740ba6a0 2024-04-26T20:51:27.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.561+0000 7f6970ff9640 1 -- 172.21.15.73:0/977495008 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6968007410 con 0x7f69740ba6a0 2024-04-26T20:51:27.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.561+0000 7f697a8b1640 1 -- 172.21.15.73:0/977495008 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6974159aa0 con 0x7f69740ba6a0 2024-04-26T20:51:27.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.561+0000 7f6970ff9640 1 -- 172.21.15.73:0/977495008 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69680077c0 con 0x7f69740ba6a0 2024-04-26T20:51:27.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.561+0000 7f697a8b1640 1 -- 172.21.15.73:0/977495008 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69740b8430 con 0x7f69740ba6a0 2024-04-26T20:51:27.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.562+0000 7f697a8b1640 1 -- 172.21.15.73:0/977495008 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69740694f0 con 0x7f69740ba6a0 2024-04-26T20:51:27.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.563+0000 7f6970ff9640 1 -- 172.21.15.73:0/977495008 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f696803c430 con 0x7f69740ba6a0 2024-04-26T20:51:27.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.563+0000 7f6970ff9640 1 --2- 172.21.15.73:0/977495008 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f696006ca90 0x7f696006ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:27.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.563+0000 7f6973fff640 1 --2- 172.21.15.73:0/977495008 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f696006ca90 0x7f696006ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:27.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.564+0000 7f6970ff9640 1 -- 172.21.15.73:0/977495008 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f69680aca50 con 0x7f69740ba6a0 2024-04-26T20:51:27.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.564+0000 7f6973fff640 1 --2- 172.21.15.73:0/977495008 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f696006ca90 0x7f696006ef50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6964000c00 tx=0x7f6964009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:27.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.568+0000 7f6970ff9640 1 -- 172.21.15.73:0/977495008 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6968049030 con 0x7f69740ba6a0 2024-04-26T20:51:27.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.746+0000 7f697a8b1640 1 -- 172.21.15.73:0/977495008 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6974074590 con 0x7f69740ba6a0 2024-04-26T20:51:27.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.746+0000 7f6970ff9640 1 -- 172.21.15.73:0/977495008 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f6968040020 con 0x7f69740ba6a0 2024-04-26T20:51:27.747 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-26T20:51:27.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.748+0000 7f697a8b1640 1 -- 172.21.15.73:0/977495008 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f696006ca90 msgr2=0x7f696006ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:27.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.748+0000 7f697a8b1640 1 --2- 172.21.15.73:0/977495008 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f696006ca90 0x7f696006ef50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6964000c00 tx=0x7f6964009040 comp rx=0 tx=0).stop 2024-04-26T20:51:27.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.748+0000 7f697a8b1640 1 -- 172.21.15.73:0/977495008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69740ba6a0 msgr2=0x7f6974150280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:27.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.748+0000 7f697a8b1640 1 --2- 172.21.15.73:0/977495008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69740ba6a0 0x7f6974150280 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f696800c580 tx=0x7f69740c9850 comp rx=0 tx=0).stop 2024-04-26T20:51:27.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.748+0000 7f697a8b1640 1 -- 172.21.15.73:0/977495008 shutdown_connections 2024-04-26T20:51:27.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.748+0000 7f697a8b1640 1 --2- 172.21.15.73:0/977495008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69740ba6a0 0x7f6974150280 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:27.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.748+0000 7f697a8b1640 1 --2- 172.21.15.73:0/977495008 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f696006ca90 0x7f696006ef50 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:27.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.748+0000 7f697a8b1640 1 --2- 172.21.15.73:0/977495008 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f69741f8430 0x7f69741f88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:27.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.748+0000 7f697a8b1640 1 --2- 172.21.15.73:0/977495008 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f695c004800 0x7f697409ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:27.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.748+0000 7f697a8b1640 1 -- 172.21.15.73:0/977495008 >> 172.21.15.73:0/977495008 conn(0x7f69740d4f60 msgr2=0x7f697406d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:27.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.748+0000 7f697a8b1640 1 -- 172.21.15.73:0/977495008 shutdown_connections 2024-04-26T20:51:27.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.748+0000 7f697a8b1640 1 -- 172.21.15.73:0/977495008 wait complete. 2024-04-26T20:51:27.760 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T20:51:27.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.860+0000 7ff9e5946640 1 Processor -- start 2024-04-26T20:51:27.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.860+0000 7ff9e5946640 1 -- start start 2024-04-26T20:51:27.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.861+0000 7ff9e5946640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff9e00d1a00 0x7ff9e00d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:27.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.861+0000 7ff9e5946640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff9e00b40d0 0x7ff9e00b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:27.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.861+0000 7ff9e5946640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9e00d0a10 con 0x7ff9e00d2320 2024-04-26T20:51:27.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.861+0000 7ff9e5946640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7ff9e00d0cf0 con 0x7ff9e00d1a00 2024-04-26T20:51:27.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.861+0000 7ff9e5946640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7ff9e00d0fd0 con 0x7ff9e00b40d0 2024-04-26T20:51:27.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.861+0000 7ff9deffd640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff9e00d1a00 0x7ff9e00d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:27.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.861+0000 7ff9de7fc640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7ff9e00d2320 0x7ff9e01625f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:34632/0 (socket says 172.21.15.73:34632) 2024-04-26T20:51:27.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.861+0000 7ff9de7fc640 1 -- 172.21.15.73:0/3729145999 learned_addr learned my addr 172.21.15.73:0/3729145999 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:27.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.861+0000 7ff9df7fe640 1 --2- 172.21.15.73:0/3729145999 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff9e00b40d0 0x7ff9e00b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:27.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.861+0000 7ff9deffd640 1 -- 172.21.15.73:0/3729145999 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff9e00b40d0 msgr2=0x7ff9e00b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:27.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.861+0000 7ff9deffd640 1 --2- 172.21.15.73:0/3729145999 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff9e00b40d0 0x7ff9e00b4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:27.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.861+0000 7ff9deffd640 1 -- 172.21.15.73:0/3729145999 >> v1:172.21.15.73:6789/0 conn(0x7ff9e00d2320 legacy=0x7ff9e01625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:27.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.861+0000 7ff9deffd640 1 -- 172.21.15.73:0/3729145999 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9e00d40a0 con 0x7ff9e00d1a00 2024-04-26T20:51:27.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.861+0000 7ff9ddffb640 1 -- 172.21.15.73:0/3729145999 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2599579514 0 0) 0x7ff9e00d0a10 con 0x7ff9e00d2320 2024-04-26T20:51:27.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.861+0000 7ff9df7fe640 1 --2- 172.21.15.73:0/3729145999 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff9e00b40d0 0x7ff9e00b4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:27.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.862+0000 7ff9deffd640 1 --2- 172.21.15.73:0/3729145999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9e00d1a00 0x7ff9e00d1de0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0009d90 tx=0x7ff9d002a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bdd9233afdf29b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:27.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.862+0000 7ff9ddffb640 1 -- 172.21.15.73:0/3729145999 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d0007740 con 0x7ff9e00d1a00 2024-04-26T20:51:27.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.862+0000 7ff9ddffb640 1 -- 172.21.15.73:0/3729145999 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9d00078e0 con 0x7ff9e00d1a00 2024-04-26T20:51:27.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.862+0000 7ff9ddffb640 1 -- 172.21.15.73:0/3729145999 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d0007cb0 con 0x7ff9e00d1a00 2024-04-26T20:51:27.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.862+0000 7ff9e5946640 1 -- 172.21.15.73:0/3729145999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9e00d1a00 msgr2=0x7ff9e00d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:27.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.862+0000 7ff9e5946640 1 --2- 172.21.15.73:0/3729145999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9e00d1a00 0x7ff9e00d1de0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0009d90 tx=0x7ff9d002a530 comp rx=0 tx=0).stop 2024-04-26T20:51:27.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.863+0000 7ff9e5946640 1 -- 172.21.15.73:0/3729145999 shutdown_connections 2024-04-26T20:51:27.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.863+0000 7ff9e5946640 1 --2- 172.21.15.73:0/3729145999 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff9e00b40d0 0x7ff9e00b4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:27.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.863+0000 7ff9e5946640 1 --2- 172.21.15.73:0/3729145999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9e00d1a00 0x7ff9e00d1de0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:27.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.863+0000 7ff9e5946640 1 -- 172.21.15.73:0/3729145999 >> 172.21.15.73:0/3729145999 conn(0x7ff9e00d4fd0 msgr2=0x7ff9e00d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:27.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.863+0000 7ff9e5946640 1 -- 172.21.15.73:0/3729145999 shutdown_connections 2024-04-26T20:51:27.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.863+0000 7ff9e5946640 1 -- 172.21.15.73:0/3729145999 wait complete. 2024-04-26T20:51:27.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.864+0000 7ff9e5946640 1 Processor -- start 2024-04-26T20:51:27.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.864+0000 7ff9e5946640 1 -- start start 2024-04-26T20:51:27.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.865+0000 7ff9e5946640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9e00b40d0 0x7ff9e0151ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:27.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.865+0000 7ff9e5946640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9e00d1a00 0x7ff9e014f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:27.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.865+0000 7ff9e5946640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff9e00d2320 0x7ff9e00cea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:27.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.865+0000 7ff9e5946640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e00b5040 con 0x7ff9e00d1a00 2024-04-26T20:51:27.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.865+0000 7ff9e5946640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e0162f50 con 0x7ff9e00b40d0 2024-04-26T20:51:27.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.865+0000 7ff9e5946640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7ff9e00b51c0 con 0x7ff9e00d2320 2024-04-26T20:51:27.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.865+0000 7ff9de7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9e00d1a00 0x7ff9e014f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:27.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.865+0000 7ff9de7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9e00d1a00 0x7ff9e014f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60094/0 (socket says 172.21.15.73:60094) 2024-04-26T20:51:27.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.865+0000 7ff9de7fc640 1 -- 172.21.15.73:0/1664419050 learned_addr learned my addr 172.21.15.73:0/1664419050 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:27.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.865+0000 7ff9df7fe640 1 --2- 172.21.15.73:0/1664419050 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff9e00d2320 0x7ff9e00cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:27.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.865+0000 7ff9deffd640 1 --2- 172.21.15.73:0/1664419050 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9e00b40d0 0x7ff9e0151ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:27.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.865+0000 7ff9de7fc640 1 -- 172.21.15.73:0/1664419050 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff9e00d2320 msgr2=0x7ff9e00cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:27.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.865+0000 7ff9de7fc640 1 --2- 172.21.15.73:0/1664419050 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff9e00d2320 0x7ff9e00cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:27.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.865+0000 7ff9de7fc640 1 -- 172.21.15.73:0/1664419050 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9e00b40d0 msgr2=0x7ff9e0151ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:27.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.865+0000 7ff9de7fc640 1 --2- 172.21.15.73:0/1664419050 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9e00b40d0 0x7ff9e0151ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:27.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.865+0000 7ff9de7fc640 1 -- 172.21.15.73:0/1664419050 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9e00b92b0 con 0x7ff9e00d1a00 2024-04-26T20:51:27.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.865+0000 7ff9de7fc640 1 --2- 172.21.15.73:0/1664419050 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9e00d1a00 0x7ff9e014f3c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff9d400c460 tx=0x7ff9d400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:27.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.865+0000 7ff9c7fff640 1 -- 172.21.15.73:0/1664419050 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d4018070 con 0x7ff9e00d1a00 2024-04-26T20:51:27.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.865+0000 7ff9c7fff640 1 -- 172.21.15.73:0/1664419050 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9d4007500 con 0x7ff9e00d1a00 2024-04-26T20:51:27.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.865+0000 7ff9e5946640 1 -- 172.21.15.73:0/1664419050 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9e00b9530 con 0x7ff9e00d1a00 2024-04-26T20:51:27.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.865+0000 7ff9c7fff640 1 -- 172.21.15.73:0/1664419050 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d4004a20 con 0x7ff9e00d1a00 2024-04-26T20:51:27.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.865+0000 7ff9e5946640 1 -- 172.21.15.73:0/1664419050 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9e00b9af0 con 0x7ff9e00d1a00 2024-04-26T20:51:27.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.866+0000 7ff9e5946640 1 -- 172.21.15.73:0/1664419050 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9e00694f0 con 0x7ff9e00d1a00 2024-04-26T20:51:27.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.867+0000 7ff9c7fff640 1 -- 172.21.15.73:0/1664419050 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff9d401b070 con 0x7ff9e00d1a00 2024-04-26T20:51:27.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.867+0000 7ff9c7fff640 1 --2- 172.21.15.73:0/1664419050 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff9cc06c8f0 0x7ff9cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:27.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.867+0000 7ff9c7fff640 1 -- 172.21.15.73:0/1664419050 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7ff9d408b190 con 0x7ff9e00d1a00 2024-04-26T20:51:27.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.867+0000 7ff9deffd640 1 --2- 172.21.15.73:0/1664419050 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff9cc06c8f0 0x7ff9cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:27.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.868+0000 7ff9deffd640 1 --2- 172.21.15.73:0/1664419050 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff9cc06c8f0 0x7ff9cc06edb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0034000 tx=0x7ff9d002f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:27.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:27.871+0000 7ff9c7fff640 1 -- 172.21.15.73:0/1664419050 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff9e00b92b0 con 0x7ff9e00d1a00 2024-04-26T20:51:28.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.050+0000 7ff9e5946640 1 -- 172.21.15.73:0/1664419050 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7ff9e0162010 con 0x7ff9e00d1a00 2024-04-26T20:51:28.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.051+0000 7ff9c7fff640 1 -- 172.21.15.73:0/1664419050 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7ff9d40591c0 con 0x7ff9e00d1a00 2024-04-26T20:51:28.052 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-26T20:51:28.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.052+0000 7ff9e5946640 1 -- 172.21.15.73:0/1664419050 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff9cc06c8f0 msgr2=0x7ff9cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:28.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.052+0000 7ff9e5946640 1 --2- 172.21.15.73:0/1664419050 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff9cc06c8f0 0x7ff9cc06edb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0034000 tx=0x7ff9d002f000 comp rx=0 tx=0).stop 2024-04-26T20:51:28.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.052+0000 7ff9e5946640 1 -- 172.21.15.73:0/1664419050 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9e00d1a00 msgr2=0x7ff9e014f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:28.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.052+0000 7ff9e5946640 1 --2- 172.21.15.73:0/1664419050 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9e00d1a00 0x7ff9e014f3c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff9d400c460 tx=0x7ff9d400c920 comp rx=0 tx=0).stop 2024-04-26T20:51:28.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.052+0000 7ff9e5946640 1 -- 172.21.15.73:0/1664419050 shutdown_connections 2024-04-26T20:51:28.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.052+0000 7ff9e5946640 1 --2- 172.21.15.73:0/1664419050 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9e00d1a00 0x7ff9e014f3c0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:28.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.052+0000 7ff9e5946640 1 --2- 172.21.15.73:0/1664419050 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff9cc06c8f0 0x7ff9cc06edb0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:28.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.052+0000 7ff9e5946640 1 --2- 172.21.15.73:0/1664419050 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff9e00d2320 0x7ff9e00cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:28.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.052+0000 7ff9e5946640 1 --2- 172.21.15.73:0/1664419050 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9e00b40d0 0x7ff9e0151ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:28.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.053+0000 7ff9e5946640 1 -- 172.21.15.73:0/1664419050 >> 172.21.15.73:0/1664419050 conn(0x7ff9e00d4fd0 msgr2=0x7ff9e0161540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:28.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.053+0000 7ff9e5946640 1 -- 172.21.15.73:0/1664419050 shutdown_connections 2024-04-26T20:51:28.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.053+0000 7ff9e5946640 1 -- 172.21.15.73:0/1664419050 wait complete. 2024-04-26T20:51:28.064 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T20:51:28.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.161+0000 7f88be493640 1 Processor -- start 2024-04-26T20:51:28.162 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.161+0000 7f88be493640 1 -- start start 2024-04-26T20:51:28.163 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.162+0000 7f88be493640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f88b80b0380 0x7f88b80ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:28.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.162+0000 7f88be493640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88b80cfd80 con 0x7f88b806c950 2024-04-26T20:51:28.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.162+0000 7f88be493640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88b80d0020 con 0x7f88b80b10b0 2024-04-26T20:51:28.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.162+0000 7f88be493640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f88b80af8e0 con 0x7f88b80b0380 2024-04-26T20:51:28.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.162+0000 7f88b7fff640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f88b80b0380 0x7f88b80ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:28.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.162+0000 7f88bca30640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f88b806c950 0x7f88b815e970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:34642/0 (socket says 172.21.15.73:34642) 2024-04-26T20:51:28.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.162+0000 7f88bca30640 1 -- 172.21.15.73:0/1438707392 learned_addr learned my addr 172.21.15.73:0/1438707392 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:28.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.162+0000 7f88b7fff640 1 -- 172.21.15.73:0/1438707392 >> v1:172.21.15.120:6789/0 conn(0x7f88b80b10b0 legacy=0x7f88b806c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:28.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.162+0000 7f88b7fff640 1 -- 172.21.15.73:0/1438707392 >> v1:172.21.15.73:6789/0 conn(0x7f88b806c950 legacy=0x7f88b815e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:28.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.162+0000 7f88b7fff640 1 -- 172.21.15.73:0/1438707392 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88b80d0300 con 0x7f88b80b0380 2024-04-26T20:51:28.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.162+0000 7f88b6ffd640 1 -- 172.21.15.73:0/1438707392 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4043285423 0 0) 0x7f88b80cfd80 con 0x7f88b806c950 2024-04-26T20:51:28.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.163+0000 7f88b7fff640 1 --2- 172.21.15.73:0/1438707392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f88b80b0380 0x7f88b80ccd50 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f88a80097a0 tx=0x7f88a802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=765db267a9628bb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:28.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.163+0000 7f88b6ffd640 1 -- 172.21.15.73:0/1438707392 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a8037070 con 0x7f88b80b0380 2024-04-26T20:51:28.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.163+0000 7f88b6ffd640 1 -- 172.21.15.73:0/1438707392 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88a802ae50 con 0x7f88b80b0380 2024-04-26T20:51:28.164 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.163+0000 7f88b6ffd640 1 -- 172.21.15.73:0/1438707392 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a8032660 con 0x7f88b80b0380 2024-04-26T20:51:28.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.163+0000 7f88be493640 1 -- 172.21.15.73:0/1438707392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f88b80b0380 msgr2=0x7f88b80ccd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:28.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.163+0000 7f88be493640 1 --2- 172.21.15.73:0/1438707392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f88b80b0380 0x7f88b80ccd50 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f88a80097a0 tx=0x7f88a802a140 comp rx=0 tx=0).stop 2024-04-26T20:51:28.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.163+0000 7f88be493640 1 -- 172.21.15.73:0/1438707392 shutdown_connections 2024-04-26T20:51:28.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.163+0000 7f88be493640 1 --2- 172.21.15.73:0/1438707392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f88b80b0380 0x7f88b80ccd50 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:28.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.163+0000 7f88be493640 1 -- 172.21.15.73:0/1438707392 >> 172.21.15.73:0/1438707392 conn(0x7f88b80d1cf0 msgr2=0x7f88b80d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:28.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.163+0000 7f88be493640 1 -- 172.21.15.73:0/1438707392 shutdown_connections 2024-04-26T20:51:28.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.163+0000 7f88be493640 1 -- 172.21.15.73:0/1438707392 wait complete. 2024-04-26T20:51:28.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.164+0000 7f88be493640 1 Processor -- start 2024-04-26T20:51:28.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.165+0000 7f88be493640 1 -- start start 2024-04-26T20:51:28.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.165+0000 7f88be493640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f88b806c950 0x7f88b80b7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:28.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.165+0000 7f88be493640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f88b80b0380 0x7f88b81522b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:28.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.165+0000 7f88be493640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f88b80b10b0 0x7f88b81527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:28.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.165+0000 7f88be493640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f88b80b0760 con 0x7f88b80b10b0 2024-04-26T20:51:28.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.165+0000 7f88be493640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f88b80d1980 con 0x7f88b806c950 2024-04-26T20:51:28.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.165+0000 7f88be493640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f88b80d15c0 con 0x7f88b80b0380 2024-04-26T20:51:28.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.166+0000 7f88bca30640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f88b80b10b0 0x7f88b81527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:28.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.166+0000 7f88bca30640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f88b80b10b0 0x7f88b81527f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60106/0 (socket says 172.21.15.73:60106) 2024-04-26T20:51:28.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.166+0000 7f88bca30640 1 -- 172.21.15.73:0/4220504012 learned_addr learned my addr 172.21.15.73:0/4220504012 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:28.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.166+0000 7f88b7fff640 1 --2- 172.21.15.73:0/4220504012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f88b806c950 0x7f88b80b7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:28.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.166+0000 7f88b77fe640 1 --2- 172.21.15.73:0/4220504012 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f88b80b0380 0x7f88b81522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:28.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.166+0000 7f88bca30640 1 -- 172.21.15.73:0/4220504012 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f88b80b0380 msgr2=0x7f88b81522b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:28.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.166+0000 7f88bca30640 1 --2- 172.21.15.73:0/4220504012 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f88b80b0380 0x7f88b81522b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:28.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.166+0000 7f88bca30640 1 -- 172.21.15.73:0/4220504012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f88b806c950 msgr2=0x7f88b80b7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:28.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.166+0000 7f88bca30640 1 --2- 172.21.15.73:0/4220504012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f88b806c950 0x7f88b80b7160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:28.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.166+0000 7f88bca30640 1 -- 172.21.15.73:0/4220504012 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88b8152f90 con 0x7f88b80b10b0 2024-04-26T20:51:28.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.166+0000 7f88bca30640 1 --2- 172.21.15.73:0/4220504012 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f88b80b10b0 0x7f88b81527f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f88b000caf0 tx=0x7f88b0002190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:28.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.166+0000 7f88b4ff9640 1 -- 172.21.15.73:0/4220504012 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b000abd0 con 0x7f88b80b10b0 2024-04-26T20:51:28.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.166+0000 7f88b4ff9640 1 -- 172.21.15.73:0/4220504012 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88b000ad70 con 0x7f88b80b10b0 2024-04-26T20:51:28.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.166+0000 7f88be493640 1 -- 172.21.15.73:0/4220504012 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88b8153230 con 0x7f88b80b10b0 2024-04-26T20:51:28.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.166+0000 7f88b4ff9640 1 -- 172.21.15.73:0/4220504012 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b0012640 con 0x7f88b80b10b0 2024-04-26T20:51:28.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.166+0000 7f88be493640 1 -- 172.21.15.73:0/4220504012 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88b81537f0 con 0x7f88b80b10b0 2024-04-26T20:51:28.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.167+0000 7f88be493640 1 -- 172.21.15.73:0/4220504012 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88b80694f0 con 0x7f88b80b10b0 2024-04-26T20:51:28.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.167+0000 7f88b4ff9640 1 -- 172.21.15.73:0/4220504012 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f88b00127e0 con 0x7f88b80b10b0 2024-04-26T20:51:28.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.168+0000 7f88b4ff9640 1 --2- 172.21.15.73:0/4220504012 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f88a406c8f0 0x7f88a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:28.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.168+0000 7f88b4ff9640 1 -- 172.21.15.73:0/4220504012 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f88b008c560 con 0x7f88b80b10b0 2024-04-26T20:51:28.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.168+0000 7f88b7fff640 1 --2- 172.21.15.73:0/4220504012 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f88a406c8f0 0x7f88a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:28.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.168+0000 7f88b7fff640 1 --2- 172.21.15.73:0/4220504012 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f88a406c8f0 0x7f88a406edb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f88a8034000 tx=0x7f88a802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:28.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.172+0000 7f88b4ff9640 1 -- 172.21.15.73:0/4220504012 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f88b005a510 con 0x7f88b80b10b0 2024-04-26T20:51:28.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.357+0000 7f88be493640 1 -- 172.21.15.73:0/4220504012 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f88b80d2760 con 0x7f88b80b10b0 2024-04-26T20:51:28.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.358+0000 7f88b4ff9640 1 -- 172.21.15.73:0/4220504012 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f88b0059e10 con 0x7f88b80b10b0 2024-04-26T20:51:28.359 INFO:teuthology.orchestra.run.smithi073.stderr:listed 0 entries 2024-04-26T20:51:28.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.360+0000 7f88be493640 1 -- 172.21.15.73:0/4220504012 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f88a406c8f0 msgr2=0x7f88a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:28.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.360+0000 7f88be493640 1 --2- 172.21.15.73:0/4220504012 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f88a406c8f0 0x7f88a406edb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f88a8034000 tx=0x7f88a802e000 comp rx=0 tx=0).stop 2024-04-26T20:51:28.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.360+0000 7f88be493640 1 -- 172.21.15.73:0/4220504012 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f88b80b10b0 msgr2=0x7f88b81527f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:28.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.360+0000 7f88be493640 1 --2- 172.21.15.73:0/4220504012 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f88b80b10b0 0x7f88b81527f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f88b000caf0 tx=0x7f88b0002190 comp rx=0 tx=0).stop 2024-04-26T20:51:28.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.360+0000 7f88be493640 1 -- 172.21.15.73:0/4220504012 shutdown_connections 2024-04-26T20:51:28.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.360+0000 7f88be493640 1 --2- 172.21.15.73:0/4220504012 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f88b80b10b0 0x7f88b81527f0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:28.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.360+0000 7f88be493640 1 --2- 172.21.15.73:0/4220504012 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f88a406c8f0 0x7f88a406edb0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:28.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.360+0000 7f88be493640 1 --2- 172.21.15.73:0/4220504012 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f88b80b0380 0x7f88b81522b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:28.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.360+0000 7f88be493640 1 --2- 172.21.15.73:0/4220504012 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f88b806c950 0x7f88b80b7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:28.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.360+0000 7f88be493640 1 -- 172.21.15.73:0/4220504012 >> 172.21.15.73:0/4220504012 conn(0x7f88b80d1cf0 msgr2=0x7f88b80b1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:28.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.360+0000 7f88be493640 1 -- 172.21.15.73:0/4220504012 shutdown_connections 2024-04-26T20:51:28.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.360+0000 7f88be493640 1 -- 172.21.15.73:0/4220504012 wait complete. 2024-04-26T20:51:28.371 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-04-26T20:51:28.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:28.372 DEBUG:teuthology.orchestra.run.smithi073:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-04-26T20:51:28.396 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi073.front.sepia.ceph.com 2024-04-26T20:51:28.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:28.396 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-26T20:51:28.451 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-26T20:51:28.452 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:51:28.470 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-26T20:51:28.470 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:51:28.528 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-26T20:51:28.528 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:51:28.586 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:51:28.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.684+0000 7f81421da640 1 Processor -- start 2024-04-26T20:51:28.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.684+0000 7f81421da640 1 -- start start 2024-04-26T20:51:28.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.685+0000 7f81421da640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f813c0d0040 0x7f813c161070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:28.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.685+0000 7f81421da640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f813c1615b0 0x7f813c0cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:28.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.685+0000 7f81421da640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f813c0d0580 con 0x7f813c160c90 2024-04-26T20:51:28.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.685+0000 7f81421da640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f813c0d0980 con 0x7f813c1615b0 2024-04-26T20:51:28.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.685+0000 7f81421da640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f813c0d0bd0 con 0x7f813c0d0040 2024-04-26T20:51:28.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.685+0000 7f813b7fe640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f813c160c90 0x7f813c0d16b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:34648/0 (socket says 172.21.15.73:34648) 2024-04-26T20:51:28.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.685+0000 7f813b7fe640 1 -- 172.21.15.73:0/352778161 learned_addr learned my addr 172.21.15.73:0/352778161 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:28.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.686+0000 7f813affd640 1 -- 172.21.15.73:0/352778161 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3939948944 0 0) 0x7f813c0d0580 con 0x7f813c160c90 2024-04-26T20:51:28.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.686+0000 7f813bfff640 1 --2- 172.21.15.73:0/352778161 >> v2:172.21.15.120:3300/0 conn(0x7f813c1615b0 0x7f813c0cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:28.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.686+0000 7f813affd640 1 -- 172.21.15.73:0/352778161 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8128003520 con 0x7f813c160c90 2024-04-26T20:51:28.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.686+0000 7f8132ffd640 1 --2- 172.21.15.73:0/352778161 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f813c0d0040 0x7f813c161070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:28.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.686+0000 7f813affd640 1 -- 172.21.15.73:0/352778161 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3769269176 0 0) 0x7f8128003520 con 0x7f813c160c90 2024-04-26T20:51:28.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.686+0000 7f813affd640 1 -- 172.21.15.73:0/352778161 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f813c0d0040 msgr2=0x7f813c161070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:28.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.686+0000 7f813affd640 1 --2- 172.21.15.73:0/352778161 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f813c0d0040 0x7f813c161070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:28.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.686+0000 7f813affd640 1 -- 172.21.15.73:0/352778161 >> v2:172.21.15.120:3300/0 conn(0x7f813c1615b0 msgr2=0x7f813c0cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:28.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.686+0000 7f813affd640 1 --2- 172.21.15.73:0/352778161 >> v2:172.21.15.120:3300/0 conn(0x7f813c1615b0 0x7f813c0cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:28.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.686+0000 7f813affd640 1 -- 172.21.15.73:0/352778161 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f813c0d0eb0 con 0x7f813c160c90 2024-04-26T20:51:28.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.686+0000 7f813affd640 1 -- 172.21.15.73:0/352778161 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7f812c001670 con 0x7f813c160c90 2024-04-26T20:51:28.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.686+0000 7f813affd640 1 -- 172.21.15.73:0/352778161 >> v1:172.21.15.73:6789/0 conn(0x7f813c160c90 legacy=0x7f813c0d16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:28.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.686+0000 7f813affd640 1 --2- 172.21.15.73:0/352778161 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f81280039b0 0x7f8128003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:28.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.686+0000 7f813affd640 1 --2- 172.21.15.73:0/352778161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8128004800 0x7f8128004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:28.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.686+0000 7f813affd640 1 --2- 172.21.15.73:0/352778161 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f813c0d0040 0x7f81280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:28.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.686+0000 7f813affd640 1 -- 172.21.15.73:0/352778161 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f813c0d0980 con 0x7f8128004800 2024-04-26T20:51:28.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.686+0000 7f813affd640 1 -- 172.21.15.73:0/352778161 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f813c0d0bd0 con 0x7f81280039b0 2024-04-26T20:51:28.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.686+0000 7f813affd640 1 -- 172.21.15.73:0/352778161 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f812800b9c0 con 0x7f813c0d0040 2024-04-26T20:51:28.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.686+0000 7f8132ffd640 1 --2- 172.21.15.73:0/352778161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8128004800 0x7f8128004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:28.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.686+0000 7f813b7fe640 1 --2- 172.21.15.73:0/352778161 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f81280039b0 0x7f8128003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:28.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.686+0000 7f813bfff640 1 --2- 172.21.15.73:0/352778161 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f813c0d0040 0x7f81280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:28.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.686+0000 7f8132ffd640 1 -- 172.21.15.73:0/352778161 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f813c0d0040 msgr2=0x7f81280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:28.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.686+0000 7f8132ffd640 1 --2- 172.21.15.73:0/352778161 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f813c0d0040 0x7f81280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:28.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.686+0000 7f8132ffd640 1 -- 172.21.15.73:0/352778161 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f81280039b0 msgr2=0x7f8128003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:28.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.686+0000 7f8132ffd640 1 --2- 172.21.15.73:0/352778161 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f81280039b0 0x7f8128003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:28.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.686+0000 7f8132ffd640 1 -- 172.21.15.73:0/352778161 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8128003520 con 0x7f8128004800 2024-04-26T20:51:28.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.686+0000 7f813bfff640 1 --2- 172.21.15.73:0/352778161 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f813c0d0040 0x7f81280051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:28.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.687+0000 7f8132ffd640 1 --2- 172.21.15.73:0/352778161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8128004800 0x7f8128004c80 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8124009ac0 tx=0x7f8128005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f11f6273c2779b00 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:28.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.687+0000 7f813affd640 1 -- 172.21.15.73:0/352778161 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8124036070 con 0x7f8128004800 2024-04-26T20:51:28.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.687+0000 7f813affd640 1 -- 172.21.15.73:0/352778161 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8124031400 con 0x7f8128004800 2024-04-26T20:51:28.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.687+0000 7f813affd640 1 -- 172.21.15.73:0/352778161 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8124009690 con 0x7f8128004800 2024-04-26T20:51:28.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.688+0000 7f81421da640 1 -- 172.21.15.73:0/352778161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8128004800 msgr2=0x7f8128004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:28.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.688+0000 7f81421da640 1 --2- 172.21.15.73:0/352778161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8128004800 0x7f8128004c80 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8124009ac0 tx=0x7f8128005830 comp rx=0 tx=0).stop 2024-04-26T20:51:28.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.688+0000 7f813bfff640 1 -- 172.21.15.73:0/352778161 reap_dead start 2024-04-26T20:51:28.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.688+0000 7f81421da640 1 -- 172.21.15.73:0/352778161 shutdown_connections 2024-04-26T20:51:28.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.688+0000 7f81421da640 1 -- 172.21.15.73:0/352778161 >> 172.21.15.73:0/352778161 conn(0x7f813c0d2db0 msgr2=0x7f813c0d31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:28.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.688+0000 7f81421da640 1 -- 172.21.15.73:0/352778161 shutdown_connections 2024-04-26T20:51:28.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.688+0000 7f81421da640 1 -- 172.21.15.73:0/352778161 wait complete. 2024-04-26T20:51:28.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.689+0000 7f81421da640 1 Processor -- start 2024-04-26T20:51:28.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.689+0000 7f81421da640 1 -- start start 2024-04-26T20:51:28.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.690+0000 7f81421da640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8128004800 0x7f813c155370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:28.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.690+0000 7f81421da640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f813c159ff0 0x7f813c15a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:28.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.690+0000 7f81421da640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f813c0afbb0 0x7f813c0b0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:28.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.690+0000 7f81421da640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f813c15e470 con 0x7f813c0afbb0 2024-04-26T20:51:28.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.690+0000 7f81421da640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f813c0b2400 con 0x7f813c159ff0 2024-04-26T20:51:28.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.690+0000 7f81421da640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f813c15e5f0 con 0x7f8128004800 2024-04-26T20:51:28.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.690+0000 7f813bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f813c0afbb0 0x7f813c0b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:28.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.690+0000 7f813bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f813c0afbb0 0x7f813c0b0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60134/0 (socket says 172.21.15.73:60134) 2024-04-26T20:51:28.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.690+0000 7f813bfff640 1 -- 172.21.15.73:0/1538755894 learned_addr learned my addr 172.21.15.73:0/1538755894 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:28.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.690+0000 7f8132ffd640 1 --2- 172.21.15.73:0/1538755894 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f813c159ff0 0x7f813c15a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:28.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.690+0000 7f813b7fe640 1 --2- 172.21.15.73:0/1538755894 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8128004800 0x7f813c155370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:28.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.690+0000 7f813bfff640 1 -- 172.21.15.73:0/1538755894 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8128004800 msgr2=0x7f813c155370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:28.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.690+0000 7f813bfff640 1 --2- 172.21.15.73:0/1538755894 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8128004800 0x7f813c155370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:28.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.690+0000 7f813bfff640 1 -- 172.21.15.73:0/1538755894 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f813c159ff0 msgr2=0x7f813c15a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:28.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.690+0000 7f813bfff640 1 --2- 172.21.15.73:0/1538755894 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f813c159ff0 0x7f813c15a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:28.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.690+0000 7f813bfff640 1 -- 172.21.15.73:0/1538755894 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f813c0b0970 con 0x7f813c0afbb0 2024-04-26T20:51:28.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.690+0000 7f813bfff640 1 --2- 172.21.15.73:0/1538755894 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f813c0afbb0 0x7f813c0b0060 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f813c073a00 tx=0x7f813400f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:28.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.690+0000 7f8138ff9640 1 -- 172.21.15.73:0/1538755894 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8134016620 con 0x7f813c0afbb0 2024-04-26T20:51:28.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.690+0000 7f8138ff9640 1 -- 172.21.15.73:0/1538755894 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f813401b070 con 0x7f813c0afbb0 2024-04-26T20:51:28.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.690+0000 7f81421da640 1 -- 172.21.15.73:0/1538755894 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f813c0b0c50 con 0x7f813c0afbb0 2024-04-26T20:51:28.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.690+0000 7f8138ff9640 1 -- 172.21.15.73:0/1538755894 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81340169e0 con 0x7f813c0afbb0 2024-04-26T20:51:28.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.690+0000 7f81421da640 1 -- 172.21.15.73:0/1538755894 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f813c0b1210 con 0x7f813c0afbb0 2024-04-26T20:51:28.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.691+0000 7f81421da640 1 -- 172.21.15.73:0/1538755894 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f813c0694f0 con 0x7f813c0afbb0 2024-04-26T20:51:28.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.692+0000 7f8138ff9640 1 -- 172.21.15.73:0/1538755894 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8134005d10 con 0x7f813c0afbb0 2024-04-26T20:51:28.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.692+0000 7f8138ff9640 1 --2- 172.21.15.73:0/1538755894 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f811c06c920 0x7f811c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:28.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.692+0000 7f8138ff9640 1 -- 172.21.15.73:0/1538755894 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f813408c7e0 con 0x7f813c0afbb0 2024-04-26T20:51:28.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.692+0000 7f813b7fe640 1 --2- 172.21.15.73:0/1538755894 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f811c06c920 0x7f811c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:28.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.693+0000 7f813b7fe640 1 --2- 172.21.15.73:0/1538755894 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f811c06c920 0x7f811c06ede0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f813c15ee70 tx=0x7f812c0078f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:28.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.696+0000 7f8138ff9640 1 -- 172.21.15.73:0/1538755894 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f813405a1c0 con 0x7f813c0afbb0 2024-04-26T20:51:28.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.877+0000 7f81421da640 1 -- 172.21.15.73:0/1538755894 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f813c074380 con 0x7f813c0afbb0 2024-04-26T20:51:28.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.879+0000 7f8138ff9640 1 -- 172.21.15.73:0/1538755894 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18902 (secure 0 0 0) 0x7f813405e4c0 con 0x7f813c0afbb0 2024-04-26T20:51:28.880 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:51:28.880 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":27,"fsid":"fbd3f548-b547-4713-856b-09f98f5d1764","created":"2024-04-26T20:50:07.876786+0000","modified":"2024-04-26T20:51:13.783133+0000","last_up_change":"2024-04-26T20:50:19.082741+0000","last_in_change":"2024-04-26T20:50:10.018400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:50:19.993731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:50:29.964420+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:50:30.920082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:50:34.650207+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"72d6f4f3-214a-4392-a02f-c33cbcab3c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6809","nonce":3461705124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6811","nonce":3461705124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6815","nonce":3461705124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6813","nonce":3461705124}]},"public_addr":"172.21.15.73:6809/3461705124","cluster_addr":"172.21.15.73:6811/3461705124","heartbeat_back_addr":"172.21.15.73:6815/3461705124","heartbeat_front_addr":"172.21.15.73:6813/3461705124","state":["exists","up"]},{"osd":1,"uuid":"786bf8ca-6ddc-4b3c-be6c-9152398bd5fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6817","nonce":2827619403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6819","nonce":2827619403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6825","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6827","nonce":2827619403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6821","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6823","nonce":2827619403}]},"public_addr":"172.21.15.73:6817/2827619403","cluster_addr":"172.21.15.73:6819/2827619403","heartbeat_back_addr":"172.21.15.73:6827/2827619403","heartbeat_front_addr":"172.21.15.73:6823/2827619403","state":["exists","up"]},{"osd":2,"uuid":"90963b1a-5437-4ef8-ad8b-6b74c534eeb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6801","nonce":2542681064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6803","nonce":2542681064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6808","nonce":2542681064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6805","nonce":2542681064}]},"public_addr":"172.21.15.73:6801/2542681064","cluster_addr":"172.21.15.73:6803/2542681064","heartbeat_back_addr":"172.21.15.73:6808/2542681064","heartbeat_front_addr":"172.21.15.73:6805/2542681064","state":["exists","up"]},{"osd":3,"uuid":"bd3d2148-6e38-41b4-a453-fe6e3b8b4ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6822","nonce":1604042718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6826","nonce":1604042718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6831","nonce":1604042718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6829","nonce":1604042718}]},"public_addr":"172.21.15.73:6822/1604042718","cluster_addr":"172.21.15.73:6826/1604042718","heartbeat_back_addr":"172.21.15.73:6831/1604042718","heartbeat_front_addr":"172.21.15.73:6829/1604042718","state":["exists","up"]},{"osd":4,"uuid":"e8668dc8-a23f-46d2-8311-cb882da08e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6817","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6819","nonce":3591055561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6824","nonce":3591055561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6831","nonce":3591055561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6828","nonce":3591055561}]},"public_addr":"172.21.15.120:6819/3591055561","cluster_addr":"172.21.15.120:6824/3591055561","heartbeat_back_addr":"172.21.15.120:6831/3591055561","heartbeat_front_addr":"172.21.15.120:6828/3591055561","state":["exists","up"]},{"osd":5,"uuid":"cbac0bdb-96bd-4852-bcce-e564495c3fcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6809","nonce":819450433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6811","nonce":819450433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6815","nonce":819450433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6813","nonce":819450433}]},"public_addr":"172.21.15.120:6809/819450433","cluster_addr":"172.21.15.120:6811/819450433","heartbeat_back_addr":"172.21.15.120:6815/819450433","heartbeat_front_addr":"172.21.15.120:6813/819450433","state":["exists","up"]},{"osd":6,"uuid":"1884b2a8-569f-4272-96a9-f25a09364dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6801","nonce":3946210727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6803","nonce":3946210727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6807","nonce":3946210727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6805","nonce":3946210727}]},"public_addr":"172.21.15.120:6801/3946210727","cluster_addr":"172.21.15.120:6803/3946210727","heartbeat_back_addr":"172.21.15.120:6807/3946210727","heartbeat_front_addr":"172.21.15.120:6805/3946210727","state":["exists","up"]},{"osd":7,"uuid":"c0c3bf6d-a4c5-4f1e-aa2d-330951830666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6818","nonce":2915574674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6821","nonce":2915574674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6827","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6829","nonce":2915574674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6823","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6825","nonce":2915574674}]},"public_addr":"172.21.15.120:6818/2915574674","cluster_addr":"172.21.15.120:6821/2915574674","heartbeat_back_addr":"172.21.15.120:6829/2915574674","heartbeat_front_addr":"172.21.15.120:6825/2915574674","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.218216+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.208710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.112705+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.182789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.164324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.174212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.225396+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.261559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:51:28.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.881+0000 7f81421da640 1 -- 172.21.15.73:0/1538755894 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f811c06c920 msgr2=0x7f811c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:28.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.881+0000 7f81421da640 1 --2- 172.21.15.73:0/1538755894 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f811c06c920 0x7f811c06ede0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f813c15ee70 tx=0x7f812c0078f0 comp rx=0 tx=0).stop 2024-04-26T20:51:28.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.881+0000 7f81421da640 1 -- 172.21.15.73:0/1538755894 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f813c0afbb0 msgr2=0x7f813c0b0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:28.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.881+0000 7f81421da640 1 --2- 172.21.15.73:0/1538755894 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f813c0afbb0 0x7f813c0b0060 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f813c073a00 tx=0x7f813400f520 comp rx=0 tx=0).stop 2024-04-26T20:51:28.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.881+0000 7f81421da640 1 -- 172.21.15.73:0/1538755894 shutdown_connections 2024-04-26T20:51:28.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.881+0000 7f81421da640 1 --2- 172.21.15.73:0/1538755894 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f813c0afbb0 0x7f813c0b0060 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:28.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.881+0000 7f81421da640 1 --2- 172.21.15.73:0/1538755894 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f811c06c920 0x7f811c06ede0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:28.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.881+0000 7f81421da640 1 --2- 172.21.15.73:0/1538755894 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f813c159ff0 0x7f813c15a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:28.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.881+0000 7f81421da640 1 --2- 172.21.15.73:0/1538755894 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8128004800 0x7f813c155370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:28.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.881+0000 7f81421da640 1 -- 172.21.15.73:0/1538755894 >> 172.21.15.73:0/1538755894 conn(0x7f813c0d2db0 msgr2=0x7f813c0b4880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:28.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.881+0000 7f81421da640 1 -- 172.21.15.73:0/1538755894 shutdown_connections 2024-04-26T20:51:28.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.881+0000 7f81421da640 1 -- 172.21.15.73:0/1538755894 wait complete. 2024-04-26T20:51:28.893 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:50:19.993731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:50:29.964420+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:50:30.920082+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T20:50:34.650207+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T20:51:28.893 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:51:28.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.992+0000 7f388875f640 1 Processor -- start 2024-04-26T20:51:28.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.992+0000 7f388875f640 1 -- start start 2024-04-26T20:51:29.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.993+0000 7f388875f640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f38800b5b10 0x7f388016a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:29.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.993+0000 7f388875f640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38800d0a10 con 0x7f38800d1a00 2024-04-26T20:51:29.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.993+0000 7f388875f640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38800d0cf0 con 0x7f38800b40d0 2024-04-26T20:51:29.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.993+0000 7f388875f640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f38800d0fd0 con 0x7f38800b5b10 2024-04-26T20:51:29.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.993+0000 7f38864fb640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f38800d1a00 0x7f38800d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:34654/0 (socket says 172.21.15.73:34654) 2024-04-26T20:51:29.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.993+0000 7f38864fb640 1 -- 172.21.15.73:0/2584946204 learned_addr learned my addr 172.21.15.73:0/2584946204 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:29.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.994+0000 7f38854f9640 1 -- 172.21.15.73:0/2584946204 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2678279257 0 0) 0x7f38800d0a10 con 0x7f38800d1a00 2024-04-26T20:51:29.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.994+0000 7f38854f9640 1 -- 172.21.15.73:0/2584946204 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f386c003520 con 0x7f38800d1a00 2024-04-26T20:51:29.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.994+0000 7f3886cfc640 1 --2- 172.21.15.73:0/2584946204 >> v2:172.21.15.120:3300/0 conn(0x7f38800b5b10 0x7f388016a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:29.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.994+0000 7f38854f9640 1 -- 172.21.15.73:0/2584946204 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3009772772 0 0) 0x7f386c003520 con 0x7f38800d1a00 2024-04-26T20:51:29.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.994+0000 7f38854f9640 1 -- 172.21.15.73:0/2584946204 >> v2:172.21.15.120:3300/0 conn(0x7f38800b5b10 msgr2=0x7f388016a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:29.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.994+0000 7f38854f9640 1 --2- 172.21.15.73:0/2584946204 >> v2:172.21.15.120:3300/0 conn(0x7f38800b5b10 0x7f388016a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:29.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.994+0000 7f38854f9640 1 -- 172.21.15.73:0/2584946204 >> v1:172.21.15.120:6789/0 conn(0x7f38800b40d0 legacy=0x7f38801625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:29.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.994+0000 7f38854f9640 1 -- 172.21.15.73:0/2584946204 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38800d40a0 con 0x7f38800d1a00 2024-04-26T20:51:29.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.994+0000 7f38854f9640 1 -- 172.21.15.73:0/2584946204 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7f387c0019f0 con 0x7f38800d1a00 2024-04-26T20:51:29.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.994+0000 7f38854f9640 1 -- 172.21.15.73:0/2584946204 >> v1:172.21.15.73:6789/0 conn(0x7f38800d1a00 legacy=0x7f38800d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:29.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.994+0000 7f38854f9640 1 --2- 172.21.15.73:0/2584946204 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f386c0039b0 0x7f386c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:29.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.994+0000 7f38854f9640 1 --2- 172.21.15.73:0/2584946204 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f386c004800 0x7f386c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:29.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.994+0000 7f38854f9640 1 --2- 172.21.15.73:0/2584946204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f386c0051c0 0x7f386c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:29.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.994+0000 7f38854f9640 1 -- 172.21.15.73:0/2584946204 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f38800d0fd0 con 0x7f386c0051c0 2024-04-26T20:51:29.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.994+0000 7f38854f9640 1 -- 172.21.15.73:0/2584946204 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f386c00c160 con 0x7f386c0039b0 2024-04-26T20:51:29.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.994+0000 7f38854f9640 1 -- 172.21.15.73:0/2584946204 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f386c00c360 con 0x7f386c004800 2024-04-26T20:51:29.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.994+0000 7f3886cfc640 1 --2- 172.21.15.73:0/2584946204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f386c0051c0 0x7f386c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:29.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.994+0000 7f38864fb640 1 --2- 172.21.15.73:0/2584946204 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f386c0039b0 0x7f386c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:29.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.994+0000 7f3885cfa640 1 --2- 172.21.15.73:0/2584946204 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f386c004800 0x7f386c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:29.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.994+0000 7f3886cfc640 1 -- 172.21.15.73:0/2584946204 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f386c004800 msgr2=0x7f386c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:29.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.994+0000 7f3886cfc640 1 --2- 172.21.15.73:0/2584946204 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f386c004800 0x7f386c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:29.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.994+0000 7f3886cfc640 1 -- 172.21.15.73:0/2584946204 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f386c0039b0 msgr2=0x7f386c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:29.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.994+0000 7f3886cfc640 1 --2- 172.21.15.73:0/2584946204 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f386c0039b0 0x7f386c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:29.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.994+0000 7f3886cfc640 1 -- 172.21.15.73:0/2584946204 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f386c003520 con 0x7f386c0051c0 2024-04-26T20:51:29.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.994+0000 7f3885cfa640 1 --2- 172.21.15.73:0/2584946204 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f386c004800 0x7f386c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:29.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.994+0000 7f38864fb640 1 --2- 172.21.15.73:0/2584946204 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f386c0039b0 0x7f386c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:29.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.995+0000 7f3886cfc640 1 -- 172.21.15.73:0/2584946204 reap_dead start 2024-04-26T20:51:29.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.995+0000 7f3886cfc640 1 --2- 172.21.15.73:0/2584946204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f386c0051c0 0x7f386c00b9c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f3878009ac0 tx=0x7f386c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c63ecdb6716f3618 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:29.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.995+0000 7f38854f9640 1 -- 172.21.15.73:0/2584946204 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3878009690 con 0x7f386c0051c0 2024-04-26T20:51:29.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.995+0000 7f38854f9640 1 -- 172.21.15.73:0/2584946204 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3878004020 con 0x7f386c0051c0 2024-04-26T20:51:29.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.995+0000 7f38854f9640 1 -- 172.21.15.73:0/2584946204 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3878004410 con 0x7f386c0051c0 2024-04-26T20:51:29.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.996+0000 7f388875f640 1 -- 172.21.15.73:0/2584946204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f386c0051c0 msgr2=0x7f386c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:29.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.996+0000 7f388875f640 1 --2- 172.21.15.73:0/2584946204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f386c0051c0 0x7f386c00b9c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f3878009ac0 tx=0x7f386c005830 comp rx=0 tx=0).stop 2024-04-26T20:51:29.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.996+0000 7f388875f640 1 -- 172.21.15.73:0/2584946204 shutdown_connections 2024-04-26T20:51:29.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.996+0000 7f388875f640 1 --2- 172.21.15.73:0/2584946204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f386c0051c0 0x7f386c00b9c0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:29.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.996+0000 7f388875f640 1 -- 172.21.15.73:0/2584946204 >> 172.21.15.73:0/2584946204 conn(0x7f38800d4fd0 msgr2=0x7f38800d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:29.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.996+0000 7f388875f640 1 -- 172.21.15.73:0/2584946204 shutdown_connections 2024-04-26T20:51:29.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.996+0000 7f388875f640 1 -- 172.21.15.73:0/2584946204 wait complete. 2024-04-26T20:51:29.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.997+0000 7f388875f640 1 Processor -- start 2024-04-26T20:51:29.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.997+0000 7f388875f640 1 -- start start 2024-04-26T20:51:29.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.998+0000 7f388875f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f386c0051c0 0x7f38800ba490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:29.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.998+0000 7f388875f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f38800b82b0 0x7f38800b8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:29.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.998+0000 7f388875f640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f388015e2c0 0x7f388015e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:29.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.998+0000 7f388875f640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f38800d2170 con 0x7f386c0051c0 2024-04-26T20:51:29.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.998+0000 7f388875f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f38800d1ff0 con 0x7f38800b82b0 2024-04-26T20:51:29.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.998+0000 7f388875f640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f38800d22f0 con 0x7f388015e2c0 2024-04-26T20:51:29.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.998+0000 7f38864fb640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f386c0051c0 0x7f38800ba490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:29.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.998+0000 7f38864fb640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f386c0051c0 0x7f38800ba490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60164/0 (socket says 172.21.15.73:60164) 2024-04-26T20:51:29.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.998+0000 7f38864fb640 1 -- 172.21.15.73:0/2283703105 learned_addr learned my addr 172.21.15.73:0/2283703105 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:29.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.998+0000 7f3886cfc640 1 --2- 172.21.15.73:0/2283703105 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f388015e2c0 0x7f388015e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:29.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.998+0000 7f3885cfa640 1 --2- 172.21.15.73:0/2283703105 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f38800b82b0 0x7f38800b8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:29.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.998+0000 7f38864fb640 1 -- 172.21.15.73:0/2283703105 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f388015e2c0 msgr2=0x7f388015e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:29.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.998+0000 7f38864fb640 1 --2- 172.21.15.73:0/2283703105 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f388015e2c0 0x7f388015e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:29.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.998+0000 7f38864fb640 1 -- 172.21.15.73:0/2283703105 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f38800b82b0 msgr2=0x7f38800b8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:29.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.998+0000 7f38864fb640 1 --2- 172.21.15.73:0/2283703105 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f38800b82b0 0x7f38800b8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:29.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.998+0000 7f38864fb640 1 -- 172.21.15.73:0/2283703105 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38800ccd00 con 0x7f386c0051c0 2024-04-26T20:51:29.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.998+0000 7f3885cfa640 1 --2- 172.21.15.73:0/2283703105 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f38800b82b0 0x7f38800b8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:29.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.998+0000 7f38864fb640 1 --2- 172.21.15.73:0/2283703105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f386c0051c0 0x7f38800ba490 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f387c0079d0 tx=0x7f387c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:29.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.998+0000 7f3876ffd640 1 -- 172.21.15.73:0/2283703105 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f387c00edf0 con 0x7f386c0051c0 2024-04-26T20:51:29.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.998+0000 7f3876ffd640 1 -- 172.21.15.73:0/2283703105 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f387c018070 con 0x7f386c0051c0 2024-04-26T20:51:29.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.998+0000 7f388875f640 1 -- 172.21.15.73:0/2283703105 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38800ccfe0 con 0x7f386c0051c0 2024-04-26T20:51:29.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.998+0000 7f3876ffd640 1 -- 172.21.15.73:0/2283703105 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f387c013660 con 0x7f386c0051c0 2024-04-26T20:51:29.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:28.998+0000 7f388875f640 1 -- 172.21.15.73:0/2283703105 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38800cd5a0 con 0x7f386c0051c0 2024-04-26T20:51:29.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.000+0000 7f388875f640 1 -- 172.21.15.73:0/2283703105 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38800694f0 con 0x7f386c0051c0 2024-04-26T20:51:29.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.000+0000 7f3876ffd640 1 -- 172.21.15.73:0/2283703105 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f387c013800 con 0x7f386c0051c0 2024-04-26T20:51:29.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.000+0000 7f3876ffd640 1 --2- 172.21.15.73:0/2283703105 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f386406c8f0 0x7f386406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:29.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.000+0000 7f3876ffd640 1 -- 172.21.15.73:0/2283703105 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f387c08c580 con 0x7f386c0051c0 2024-04-26T20:51:29.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.000+0000 7f3885cfa640 1 --2- 172.21.15.73:0/2283703105 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f386406c8f0 0x7f386406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:29.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.001+0000 7f3885cfa640 1 --2- 172.21.15.73:0/2283703105 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f386406c8f0 0x7f386406edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3870000c00 tx=0x7f3870009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:29.007 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.005+0000 7f3876ffd640 1 -- 172.21.15.73:0/2283703105 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f387c05a5b0 con 0x7f386c0051c0 2024-04-26T20:51:29.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.178+0000 7f388875f640 1 -- 172.21.15.73:0/2283703105 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f38800b7bf0 con 0x7f386c0051c0 2024-04-26T20:51:29.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.179+0000 7f3876ffd640 1 -- 172.21.15.73:0/2283703105 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f387c059eb0 con 0x7f386c0051c0 2024-04-26T20:51:29.180 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-26T20:51:29.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.181+0000 7f388875f640 1 -- 172.21.15.73:0/2283703105 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f386406c8f0 msgr2=0x7f386406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:29.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.181+0000 7f388875f640 1 --2- 172.21.15.73:0/2283703105 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f386406c8f0 0x7f386406edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3870000c00 tx=0x7f3870009040 comp rx=0 tx=0).stop 2024-04-26T20:51:29.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.181+0000 7f388875f640 1 -- 172.21.15.73:0/2283703105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f386c0051c0 msgr2=0x7f38800ba490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:29.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.181+0000 7f388875f640 1 --2- 172.21.15.73:0/2283703105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f386c0051c0 0x7f38800ba490 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f387c0079d0 tx=0x7f387c00e530 comp rx=0 tx=0).stop 2024-04-26T20:51:29.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.181+0000 7f388875f640 1 -- 172.21.15.73:0/2283703105 shutdown_connections 2024-04-26T20:51:29.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.181+0000 7f388875f640 1 --2- 172.21.15.73:0/2283703105 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f386406c8f0 0x7f386406edb0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:29.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.181+0000 7f388875f640 1 --2- 172.21.15.73:0/2283703105 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f388015e2c0 0x7f388015e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:29.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.181+0000 7f388875f640 1 --2- 172.21.15.73:0/2283703105 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f38800b82b0 0x7f38800b8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:29.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.181+0000 7f388875f640 1 --2- 172.21.15.73:0/2283703105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f386c0051c0 0x7f38800ba490 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:29.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.181+0000 7f388875f640 1 -- 172.21.15.73:0/2283703105 >> 172.21.15.73:0/2283703105 conn(0x7f38800d4fd0 msgr2=0x7f38801594d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:29.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.181+0000 7f388875f640 1 -- 172.21.15.73:0/2283703105 shutdown_connections 2024-04-26T20:51:29.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.181+0000 7f388875f640 1 -- 172.21.15.73:0/2283703105 wait complete. 2024-04-26T20:51:29.193 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:51:29.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.291+0000 7f9379e6b640 1 Processor -- start 2024-04-26T20:51:29.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.291+0000 7f9379e6b640 1 -- start start 2024-04-26T20:51:29.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.292+0000 7f9379e6b640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f937415ab60 0x7f937406c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:29.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.292+0000 7f9379e6b640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f93741617f0 0x7f937406c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:29.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.292+0000 7f9379e6b640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f937415af40 con 0x7f937415d5f0 2024-04-26T20:51:29.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.292+0000 7f9379e6b640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f937415e320 con 0x7f937415ab60 2024-04-26T20:51:29.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.292+0000 7f9379e6b640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f9374160d10 con 0x7f93741617f0 2024-04-26T20:51:29.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.292+0000 7f93737fe640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f937415ab60 0x7f937406c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:29.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.292+0000 7f93737fe640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f937415ab60 0x7f937406c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60166/0 (socket says 172.21.15.73:60166) 2024-04-26T20:51:29.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.292+0000 7f93737fe640 1 -- 172.21.15.73:0/2286834607 learned_addr learned my addr 172.21.15.73:0/2286834607 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:29.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.292+0000 7f93737fe640 1 -- 172.21.15.73:0/2286834607 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f93741617f0 msgr2=0x7f937406c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:29.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.292+0000 7f9372ffd640 1 --2- 172.21.15.73:0/2286834607 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f93741617f0 0x7f937406c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:29.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.292+0000 7f93737fe640 1 --2- 172.21.15.73:0/2286834607 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f93741617f0 0x7f937406c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:29.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.292+0000 7f93737fe640 1 -- 172.21.15.73:0/2286834607 >> v1:172.21.15.120:6789/0 conn(0x7f937415d5f0 legacy=0x7f937415daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:29.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.292+0000 7f93737fe640 1 -- 172.21.15.73:0/2286834607 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9374160ff0 con 0x7f937415ab60 2024-04-26T20:51:29.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.293+0000 7f93737fe640 1 --2- 172.21.15.73:0/2286834607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f937415ab60 0x7f937406c240 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9364009870 tx=0x7f936402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61a49e97f6bf0ab1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:29.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.293+0000 7f93727fc640 1 -- 172.21.15.73:0/2286834607 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9364037070 con 0x7f937415ab60 2024-04-26T20:51:29.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.293+0000 7f93727fc640 1 -- 172.21.15.73:0/2286834607 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9364032440 con 0x7f937415ab60 2024-04-26T20:51:29.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.293+0000 7f93727fc640 1 -- 172.21.15.73:0/2286834607 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9364032810 con 0x7f937415ab60 2024-04-26T20:51:29.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.294+0000 7f9379e6b640 1 -- 172.21.15.73:0/2286834607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f937415ab60 msgr2=0x7f937406c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:29.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.294+0000 7f9379e6b640 1 --2- 172.21.15.73:0/2286834607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f937415ab60 0x7f937406c240 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9364009870 tx=0x7f936402a210 comp rx=0 tx=0).stop 2024-04-26T20:51:29.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.294+0000 7f9379e6b640 1 -- 172.21.15.73:0/2286834607 shutdown_connections 2024-04-26T20:51:29.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.294+0000 7f9379e6b640 1 --2- 172.21.15.73:0/2286834607 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f93741617f0 0x7f937406c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:29.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.294+0000 7f9379e6b640 1 --2- 172.21.15.73:0/2286834607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f937415ab60 0x7f937406c240 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:29.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.294+0000 7f9379e6b640 1 -- 172.21.15.73:0/2286834607 >> 172.21.15.73:0/2286834607 conn(0x7f93740d1cf0 msgr2=0x7f93740d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:29.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.294+0000 7f9379e6b640 1 -- 172.21.15.73:0/2286834607 shutdown_connections 2024-04-26T20:51:29.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.294+0000 7f9379e6b640 1 -- 172.21.15.73:0/2286834607 wait complete. 2024-04-26T20:51:29.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.295+0000 7f9379e6b640 1 Processor -- start 2024-04-26T20:51:29.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.295+0000 7f9379e6b640 1 -- start start 2024-04-26T20:51:29.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.296+0000 7f9379e6b640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f937415ab60 0x7f93740d0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:29.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.296+0000 7f9379e6b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f937415d5f0 0x7f9374159510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:29.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.296+0000 7f9379e6b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93741617f0 0x7f9374143de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:29.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.296+0000 7f9379e6b640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f93740cc4c0 con 0x7f937415d5f0 2024-04-26T20:51:29.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.296+0000 7f9379e6b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f93740cc340 con 0x7f93741617f0 2024-04-26T20:51:29.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.296+0000 7f9379e6b640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f93740cc640 con 0x7f937415ab60 2024-04-26T20:51:29.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.296+0000 7f9372ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f937415d5f0 0x7f9374159510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:29.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.296+0000 7f9372ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f937415d5f0 0x7f9374159510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60178/0 (socket says 172.21.15.73:60178) 2024-04-26T20:51:29.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.296+0000 7f9372ffd640 1 -- 172.21.15.73:0/3736084532 learned_addr learned my addr 172.21.15.73:0/3736084532 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:29.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.296+0000 7f93737fe640 1 --2- 172.21.15.73:0/3736084532 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f937415ab60 0x7f93740d0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:29.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.296+0000 7f9373fff640 1 --2- 172.21.15.73:0/3736084532 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93741617f0 0x7f9374143de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:29.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.296+0000 7f9372ffd640 1 -- 172.21.15.73:0/3736084532 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f937415ab60 msgr2=0x7f93740d0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:29.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.296+0000 7f9372ffd640 1 --2- 172.21.15.73:0/3736084532 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f937415ab60 0x7f93740d0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:29.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.296+0000 7f9372ffd640 1 -- 172.21.15.73:0/3736084532 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93741617f0 msgr2=0x7f9374143de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:29.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.296+0000 7f9372ffd640 1 --2- 172.21.15.73:0/3736084532 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93741617f0 0x7f9374143de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:29.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.296+0000 7f9372ffd640 1 -- 172.21.15.73:0/3736084532 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93740af7e0 con 0x7f937415d5f0 2024-04-26T20:51:29.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.296+0000 7f9372ffd640 1 --2- 172.21.15.73:0/3736084532 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f937415d5f0 0x7f9374159510 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f936800c7f0 tx=0x7f936800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:29.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.297+0000 7f935bfff640 1 -- 172.21.15.73:0/3736084532 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9368015070 con 0x7f937415d5f0 2024-04-26T20:51:29.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.297+0000 7f935bfff640 1 -- 172.21.15.73:0/3736084532 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9368004220 con 0x7f937415d5f0 2024-04-26T20:51:29.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.297+0000 7f9379e6b640 1 -- 172.21.15.73:0/3736084532 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93740afac0 con 0x7f937415d5f0 2024-04-26T20:51:29.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.297+0000 7f935bfff640 1 -- 172.21.15.73:0/3736084532 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9368007620 con 0x7f937415d5f0 2024-04-26T20:51:29.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.297+0000 7f9379e6b640 1 -- 172.21.15.73:0/3736084532 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93740b0080 con 0x7f937415d5f0 2024-04-26T20:51:29.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.298+0000 7f9379e6b640 1 -- 172.21.15.73:0/3736084532 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93740694f0 con 0x7f937415d5f0 2024-04-26T20:51:29.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.298+0000 7f935bfff640 1 -- 172.21.15.73:0/3736084532 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f93680079d0 con 0x7f937415d5f0 2024-04-26T20:51:29.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.298+0000 7f935bfff640 1 --2- 172.21.15.73:0/3736084532 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f936006c8f0 0x7f936006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:29.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.299+0000 7f935bfff640 1 -- 172.21.15.73:0/3736084532 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f936808c260 con 0x7f937415d5f0 2024-04-26T20:51:29.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.299+0000 7f93737fe640 1 --2- 172.21.15.73:0/3736084532 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f936006c8f0 0x7f936006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:29.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.299+0000 7f93737fe640 1 --2- 172.21.15.73:0/3736084532 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f936006c8f0 0x7f936006edb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9364034000 tx=0x7f936402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:29.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.303+0000 7f935bfff640 1 -- 172.21.15.73:0/3736084532 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9368059c40 con 0x7f937415d5f0 2024-04-26T20:51:29.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.478+0000 7f9379e6b640 1 -- 172.21.15.73:0/3736084532 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9374143b40 con 0x7f937415d5f0 2024-04-26T20:51:29.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.478+0000 7f935bfff640 1 -- 172.21.15.73:0/3736084532 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f936805df40 con 0x7f937415d5f0 2024-04-26T20:51:29.479 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-26T20:51:29.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.480+0000 7f9379e6b640 1 -- 172.21.15.73:0/3736084532 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f936006c8f0 msgr2=0x7f936006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:29.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.480+0000 7f9379e6b640 1 --2- 172.21.15.73:0/3736084532 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f936006c8f0 0x7f936006edb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9364034000 tx=0x7f936402e000 comp rx=0 tx=0).stop 2024-04-26T20:51:29.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.480+0000 7f9379e6b640 1 -- 172.21.15.73:0/3736084532 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f937415d5f0 msgr2=0x7f9374159510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:29.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.480+0000 7f9379e6b640 1 --2- 172.21.15.73:0/3736084532 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f937415d5f0 0x7f9374159510 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f936800c7f0 tx=0x7f936800ccb0 comp rx=0 tx=0).stop 2024-04-26T20:51:29.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.480+0000 7f9379e6b640 1 -- 172.21.15.73:0/3736084532 shutdown_connections 2024-04-26T20:51:29.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.480+0000 7f9379e6b640 1 --2- 172.21.15.73:0/3736084532 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f937415d5f0 0x7f9374159510 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:29.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.480+0000 7f9379e6b640 1 --2- 172.21.15.73:0/3736084532 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f936006c8f0 0x7f936006edb0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:29.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.480+0000 7f9379e6b640 1 --2- 172.21.15.73:0/3736084532 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93741617f0 0x7f9374143de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:29.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.480+0000 7f9379e6b640 1 --2- 172.21.15.73:0/3736084532 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f937415ab60 0x7f93740d0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:29.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.480+0000 7f9379e6b640 1 -- 172.21.15.73:0/3736084532 >> 172.21.15.73:0/3736084532 conn(0x7f93740d1cf0 msgr2=0x7f93740cee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:29.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.480+0000 7f9379e6b640 1 -- 172.21.15.73:0/3736084532 shutdown_connections 2024-04-26T20:51:29.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.480+0000 7f9379e6b640 1 -- 172.21.15.73:0/3736084532 wait complete. 2024-04-26T20:51:29.492 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:51:29.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.592+0000 7f25ad8fc640 1 Processor -- start 2024-04-26T20:51:29.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.592+0000 7f25ad8fc640 1 -- start start 2024-04-26T20:51:29.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.593+0000 7f25ad8fc640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f25a80b40d0 0x7f25a81625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:29.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.593+0000 7f25ad8fc640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25a80d0a10 con 0x7f25a80d1a00 2024-04-26T20:51:29.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.593+0000 7f25ad8fc640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25a80d0cf0 con 0x7f25a8162b30 2024-04-26T20:51:29.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.593+0000 7f25ad8fc640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f25a80d0fd0 con 0x7f25a80b40d0 2024-04-26T20:51:29.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.593+0000 7f25a67fc640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f25a80b40d0 0x7f25a81625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:29.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.593+0000 7f25a6ffd640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f25a80d1a00 0x7f25a80d1de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40680/0 (socket says 172.21.15.73:40680) 2024-04-26T20:51:29.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.593+0000 7f25a6ffd640 1 -- 172.21.15.73:0/2966518517 learned_addr learned my addr 172.21.15.73:0/2966518517 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:29.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.593+0000 7f25a67fc640 1 -- 172.21.15.73:0/2966518517 >> v1:172.21.15.120:6789/0 conn(0x7f25a8162b30 legacy=0x7f25a80b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:29.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.593+0000 7f25a67fc640 1 -- 172.21.15.73:0/2966518517 >> v1:172.21.15.73:6789/0 conn(0x7f25a80d1a00 legacy=0x7f25a80d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:29.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.593+0000 7f25a5ffb640 1 -- 172.21.15.73:0/2966518517 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1312851956 0 0) 0x7f25a80d0a10 con 0x7f25a80d1a00 2024-04-26T20:51:29.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.593+0000 7f25a67fc640 1 -- 172.21.15.73:0/2966518517 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25a8162f80 con 0x7f25a80b40d0 2024-04-26T20:51:29.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.594+0000 7f25a67fc640 1 --2- 172.21.15.73:0/2966518517 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25a80b40d0 0x7f25a81625f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f259c0097a0 tx=0x7f259c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13f44c9947d106fe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:29.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.594+0000 7f25a5ffb640 1 -- 172.21.15.73:0/2966518517 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f259c037070 con 0x7f25a80b40d0 2024-04-26T20:51:29.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.594+0000 7f25a5ffb640 1 -- 172.21.15.73:0/2966518517 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f259c02ae50 con 0x7f25a80b40d0 2024-04-26T20:51:29.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.594+0000 7f25a5ffb640 1 -- 172.21.15.73:0/2966518517 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f259c032660 con 0x7f25a80b40d0 2024-04-26T20:51:29.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.595+0000 7f25ad8fc640 1 -- 172.21.15.73:0/2966518517 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25a80b40d0 msgr2=0x7f25a81625f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:29.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.595+0000 7f25ad8fc640 1 --2- 172.21.15.73:0/2966518517 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25a80b40d0 0x7f25a81625f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f259c0097a0 tx=0x7f259c02a140 comp rx=0 tx=0).stop 2024-04-26T20:51:29.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.595+0000 7f25ad8fc640 1 -- 172.21.15.73:0/2966518517 shutdown_connections 2024-04-26T20:51:29.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.595+0000 7f25ad8fc640 1 --2- 172.21.15.73:0/2966518517 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25a80b40d0 0x7f25a81625f0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:29.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.595+0000 7f25ad8fc640 1 -- 172.21.15.73:0/2966518517 >> 172.21.15.73:0/2966518517 conn(0x7f25a80d4fd0 msgr2=0x7f25a80d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:29.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.595+0000 7f25ad8fc640 1 -- 172.21.15.73:0/2966518517 shutdown_connections 2024-04-26T20:51:29.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.595+0000 7f25ad8fc640 1 -- 172.21.15.73:0/2966518517 wait complete. 2024-04-26T20:51:29.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.596+0000 7f25ad8fc640 1 Processor -- start 2024-04-26T20:51:29.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.596+0000 7f25ad8fc640 1 -- start start 2024-04-26T20:51:29.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.597+0000 7f25ad8fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25a80b40d0 0x7f25a8154460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:29.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.597+0000 7f25ad8fc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f25a80d1a00 0x7f25a80b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:29.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.597+0000 7f25ad8fc640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f25a8162b30 0x7f25a80ba170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:29.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.597+0000 7f25ad8fc640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f25a80d2670 con 0x7f25a80b40d0 2024-04-26T20:51:29.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.597+0000 7f25ad8fc640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f25a80d24f0 con 0x7f25a80d1a00 2024-04-26T20:51:29.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.597+0000 7f25ad8fc640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f25a80b5210 con 0x7f25a8162b30 2024-04-26T20:51:29.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.597+0000 7f25a6ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25a80b40d0 0x7f25a8154460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:29.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.597+0000 7f25a6ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25a80b40d0 0x7f25a8154460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49982/0 (socket says 172.21.15.73:49982) 2024-04-26T20:51:29.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.597+0000 7f25a6ffd640 1 -- 172.21.15.73:0/2747198389 learned_addr learned my addr 172.21.15.73:0/2747198389 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:29.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.597+0000 7f25a67fc640 1 --2- 172.21.15.73:0/2747198389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f25a80d1a00 0x7f25a80b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:29.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.597+0000 7f25a77fe640 1 --2- 172.21.15.73:0/2747198389 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f25a8162b30 0x7f25a80ba170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:29.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.597+0000 7f25a6ffd640 1 -- 172.21.15.73:0/2747198389 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f25a8162b30 msgr2=0x7f25a80ba170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:29.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.597+0000 7f25a6ffd640 1 --2- 172.21.15.73:0/2747198389 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f25a8162b30 0x7f25a80ba170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:29.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.597+0000 7f25a6ffd640 1 -- 172.21.15.73:0/2747198389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f25a80d1a00 msgr2=0x7f25a80b61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:29.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.597+0000 7f25a6ffd640 1 --2- 172.21.15.73:0/2747198389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f25a80d1a00 0x7f25a80b61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:29.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.597+0000 7f25a6ffd640 1 -- 172.21.15.73:0/2747198389 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25a81549a0 con 0x7f25a80b40d0 2024-04-26T20:51:29.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.597+0000 7f25a67fc640 1 --2- 172.21.15.73:0/2747198389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f25a80d1a00 0x7f25a80b61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:29.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.597+0000 7f25a6ffd640 1 --2- 172.21.15.73:0/2747198389 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25a80b40d0 0x7f25a8154460 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f259800c8c0 tx=0x7f259800cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:29.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.598+0000 7f258ffff640 1 -- 172.21.15.73:0/2747198389 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2598018070 con 0x7f25a80b40d0 2024-04-26T20:51:29.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.598+0000 7f258ffff640 1 -- 172.21.15.73:0/2747198389 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f259800a640 con 0x7f25a80b40d0 2024-04-26T20:51:29.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.598+0000 7f25ad8fc640 1 -- 172.21.15.73:0/2747198389 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25a80b90d0 con 0x7f25a80b40d0 2024-04-26T20:51:29.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.598+0000 7f258ffff640 1 -- 172.21.15.73:0/2747198389 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f259800aa30 con 0x7f25a80b40d0 2024-04-26T20:51:29.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.598+0000 7f25ad8fc640 1 -- 172.21.15.73:0/2747198389 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25a80b9690 con 0x7f25a80b40d0 2024-04-26T20:51:29.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.599+0000 7f25ad8fc640 1 -- 172.21.15.73:0/2747198389 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25a80d1210 con 0x7f25a80b40d0 2024-04-26T20:51:29.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.599+0000 7f258ffff640 1 -- 172.21.15.73:0/2747198389 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f259801b070 con 0x7f25a80b40d0 2024-04-26T20:51:29.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.599+0000 7f258ffff640 1 --2- 172.21.15.73:0/2747198389 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f259406c8f0 0x7f259406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:29.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.600+0000 7f258ffff640 1 -- 172.21.15.73:0/2747198389 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f259808b510 con 0x7f25a80b40d0 2024-04-26T20:51:29.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.600+0000 7f25a67fc640 1 --2- 172.21.15.73:0/2747198389 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f259406c8f0 0x7f259406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:29.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.600+0000 7f25a67fc640 1 --2- 172.21.15.73:0/2747198389 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f259406c8f0 0x7f259406edb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f259c034000 tx=0x7f259c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:29.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.604+0000 7f258ffff640 1 -- 172.21.15.73:0/2747198389 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f25a80d1210 con 0x7f25a80b40d0 2024-04-26T20:51:29.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.779+0000 7f25ad8fc640 1 -- 172.21.15.73:0/2747198389 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f25a80694f0 con 0x7f25a80b40d0 2024-04-26T20:51:29.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.780+0000 7f258ffff640 1 -- 172.21.15.73:0/2747198389 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f2598058ef0 con 0x7f25a80b40d0 2024-04-26T20:51:29.781 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-26T20:51:29.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.781+0000 7f25ad8fc640 1 -- 172.21.15.73:0/2747198389 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f259406c8f0 msgr2=0x7f259406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:29.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.781+0000 7f25ad8fc640 1 --2- 172.21.15.73:0/2747198389 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f259406c8f0 0x7f259406edb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f259c034000 tx=0x7f259c02e000 comp rx=0 tx=0).stop 2024-04-26T20:51:29.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.781+0000 7f25ad8fc640 1 -- 172.21.15.73:0/2747198389 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25a80b40d0 msgr2=0x7f25a8154460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:29.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.781+0000 7f25ad8fc640 1 --2- 172.21.15.73:0/2747198389 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25a80b40d0 0x7f25a8154460 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f259800c8c0 tx=0x7f259800cd80 comp rx=0 tx=0).stop 2024-04-26T20:51:29.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.781+0000 7f25ad8fc640 1 -- 172.21.15.73:0/2747198389 shutdown_connections 2024-04-26T20:51:29.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.781+0000 7f25ad8fc640 1 --2- 172.21.15.73:0/2747198389 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f259406c8f0 0x7f259406edb0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:29.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.781+0000 7f25ad8fc640 1 --2- 172.21.15.73:0/2747198389 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f25a8162b30 0x7f25a80ba170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:29.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.781+0000 7f25ad8fc640 1 --2- 172.21.15.73:0/2747198389 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f25a80d1a00 0x7f25a80b61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:29.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.781+0000 7f25ad8fc640 1 --2- 172.21.15.73:0/2747198389 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25a80b40d0 0x7f25a8154460 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:29.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.781+0000 7f25ad8fc640 1 -- 172.21.15.73:0/2747198389 >> 172.21.15.73:0/2747198389 conn(0x7f25a80d4fd0 msgr2=0x7f25a80b3910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:29.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.782+0000 7f25ad8fc640 1 -- 172.21.15.73:0/2747198389 shutdown_connections 2024-04-26T20:51:29.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.782+0000 7f25ad8fc640 1 -- 172.21.15.73:0/2747198389 wait complete. 2024-04-26T20:51:29.793 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T20:51:29.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.893+0000 7f339767e640 1 Processor -- start 2024-04-26T20:51:29.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.893+0000 7f339767e640 1 -- start start 2024-04-26T20:51:29.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.894+0000 7f339767e640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f339015a9f0 0x7f33900b4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:29.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.894+0000 7f339767e640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33900ce060 con 0x7f3390161510 2024-04-26T20:51:29.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.894+0000 7f339767e640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33900ce340 con 0x7f33900726e0 2024-04-26T20:51:29.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.894+0000 7f339767e640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f33900ce620 con 0x7f339015a9f0 2024-04-26T20:51:29.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.894+0000 7f3394c19640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f3390161510 0x7f339015dbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40692/0 (socket says 172.21.15.73:40692) 2024-04-26T20:51:29.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.894+0000 7f3394c19640 1 -- 172.21.15.73:0/2355019314 learned_addr learned my addr 172.21.15.73:0/2355019314 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:29.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.894+0000 7f339541a640 1 --2- 172.21.15.73:0/2355019314 >> v2:172.21.15.120:3300/0 conn(0x7f339015a9f0 0x7f33900b4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:29.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.894+0000 7f3387fff640 1 -- 172.21.15.73:0/2355019314 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 215251209 0 0) 0x7f33900ce060 con 0x7f3390161510 2024-04-26T20:51:29.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.894+0000 7f3387fff640 1 -- 172.21.15.73:0/2355019314 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f337c003520 con 0x7f3390161510 2024-04-26T20:51:29.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.894+0000 7f3387fff640 1 -- 172.21.15.73:0/2355019314 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2578245887 0 0) 0x7f337c003520 con 0x7f3390161510 2024-04-26T20:51:29.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.894+0000 7f3387fff640 1 -- 172.21.15.73:0/2355019314 >> v2:172.21.15.120:3300/0 conn(0x7f339015a9f0 msgr2=0x7f33900b4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:29.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.894+0000 7f3387fff640 1 --2- 172.21.15.73:0/2355019314 >> v2:172.21.15.120:3300/0 conn(0x7f339015a9f0 0x7f33900b4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:29.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.894+0000 7f3387fff640 1 -- 172.21.15.73:0/2355019314 >> v1:172.21.15.120:6789/0 conn(0x7f33900726e0 legacy=0x7f339006c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:29.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.894+0000 7f3387fff640 1 -- 172.21.15.73:0/2355019314 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f339015d6b0 con 0x7f3390161510 2024-04-26T20:51:29.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.894+0000 7f3387fff640 1 -- 172.21.15.73:0/2355019314 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7f3380001670 con 0x7f3390161510 2024-04-26T20:51:29.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.894+0000 7f3387fff640 1 -- 172.21.15.73:0/2355019314 >> v1:172.21.15.73:6789/0 conn(0x7f3390161510 legacy=0x7f339015dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:29.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.894+0000 7f3387fff640 1 --2- 172.21.15.73:0/2355019314 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f337c0039b0 0x7f337c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:29.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.894+0000 7f3387fff640 1 --2- 172.21.15.73:0/2355019314 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f337c004800 0x7f337c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:29.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.894+0000 7f3387fff640 1 --2- 172.21.15.73:0/2355019314 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f337c0051c0 0x7f337c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:29.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.894+0000 7f3387fff640 1 -- 172.21.15.73:0/2355019314 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f33900ce620 con 0x7f337c0039b0 2024-04-26T20:51:29.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.894+0000 7f3387fff640 1 -- 172.21.15.73:0/2355019314 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f337c00c060 con 0x7f337c0051c0 2024-04-26T20:51:29.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.894+0000 7f3387fff640 1 -- 172.21.15.73:0/2355019314 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f337c00c260 con 0x7f337c004800 2024-04-26T20:51:29.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.894+0000 7f339541a640 1 --2- 172.21.15.73:0/2355019314 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f337c0039b0 0x7f337c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:29.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.895+0000 7f3395c1b640 1 --2- 172.21.15.73:0/2355019314 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f337c0051c0 0x7f337c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:29.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.895+0000 7f3394c19640 1 --2- 172.21.15.73:0/2355019314 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f337c004800 0x7f337c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:29.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.895+0000 7f339541a640 1 -- 172.21.15.73:0/2355019314 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f337c004800 msgr2=0x7f337c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:29.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.895+0000 7f339541a640 1 --2- 172.21.15.73:0/2355019314 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f337c004800 0x7f337c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:29.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.895+0000 7f339541a640 1 -- 172.21.15.73:0/2355019314 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f337c0051c0 msgr2=0x7f337c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:29.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.895+0000 7f339541a640 1 --2- 172.21.15.73:0/2355019314 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f337c0051c0 0x7f337c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:29.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.895+0000 7f339541a640 1 -- 172.21.15.73:0/2355019314 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f337c003520 con 0x7f337c0039b0 2024-04-26T20:51:29.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.895+0000 7f3395c1b640 1 -- 172.21.15.73:0/2355019314 reap_dead start 2024-04-26T20:51:29.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.895+0000 7f3394c19640 1 --2- 172.21.15.73:0/2355019314 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f337c004800 0x7f337c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:29.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.895+0000 7f339541a640 1 --2- 172.21.15.73:0/2355019314 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f337c0039b0 0x7f337c003dd0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3388009ac0 tx=0x7f337c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c76200e8c8e04b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:29.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.895+0000 7f3387fff640 1 -- 172.21.15.73:0/2355019314 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3388036070 con 0x7f337c0039b0 2024-04-26T20:51:29.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.895+0000 7f3387fff640 1 -- 172.21.15.73:0/2355019314 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3388031400 con 0x7f337c0039b0 2024-04-26T20:51:29.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.895+0000 7f3387fff640 1 -- 172.21.15.73:0/2355019314 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3388009690 con 0x7f337c0039b0 2024-04-26T20:51:29.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.896+0000 7f339767e640 1 -- 172.21.15.73:0/2355019314 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f337c0039b0 msgr2=0x7f337c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:29.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.896+0000 7f339767e640 1 --2- 172.21.15.73:0/2355019314 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f337c0039b0 0x7f337c003dd0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3388009ac0 tx=0x7f337c005830 comp rx=0 tx=0).stop 2024-04-26T20:51:29.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.896+0000 7f339767e640 1 -- 172.21.15.73:0/2355019314 shutdown_connections 2024-04-26T20:51:29.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.896+0000 7f339767e640 1 --2- 172.21.15.73:0/2355019314 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f337c0039b0 0x7f337c003dd0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:29.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.896+0000 7f339767e640 1 -- 172.21.15.73:0/2355019314 >> 172.21.15.73:0/2355019314 conn(0x7f33900b3180 msgr2=0x7f33900b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:29.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.896+0000 7f339767e640 1 -- 172.21.15.73:0/2355019314 shutdown_connections 2024-04-26T20:51:29.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.896+0000 7f339767e640 1 -- 172.21.15.73:0/2355019314 wait complete. 2024-04-26T20:51:29.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.897+0000 7f339767e640 1 Processor -- start 2024-04-26T20:51:29.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.897+0000 7f339767e640 1 -- start start 2024-04-26T20:51:29.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.898+0000 7f339767e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f337c0039b0 0x7f33901482f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:29.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.898+0000 7f339767e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33900c9040 0x7f33900c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:29.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.898+0000 7f339767e640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f33900cb230 0x7f33900cb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:29.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.898+0000 7f339767e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f33900b4480 con 0x7f33900c9040 2024-04-26T20:51:29.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.898+0000 7f339767e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f339015add0 con 0x7f337c0039b0 2024-04-26T20:51:29.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.898+0000 7f339767e640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f33900b4600 con 0x7f33900cb230 2024-04-26T20:51:29.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.898+0000 7f3394c19640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33900c9040 0x7f33900c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:29.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.898+0000 7f3394c19640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33900c9040 0x7f33900c94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50006/0 (socket says 172.21.15.73:50006) 2024-04-26T20:51:29.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.898+0000 7f3394c19640 1 -- 172.21.15.73:0/469950133 learned_addr learned my addr 172.21.15.73:0/469950133 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:29.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.898+0000 7f3395c1b640 1 --2- 172.21.15.73:0/469950133 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f33900cb230 0x7f33900cb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:29.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.898+0000 7f339541a640 1 --2- 172.21.15.73:0/469950133 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f337c0039b0 0x7f33901482f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:29.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.898+0000 7f3394c19640 1 -- 172.21.15.73:0/469950133 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f33900cb230 msgr2=0x7f33900cb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:29.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.898+0000 7f3394c19640 1 --2- 172.21.15.73:0/469950133 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f33900cb230 0x7f33900cb6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:29.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.898+0000 7f3394c19640 1 -- 172.21.15.73:0/469950133 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f337c0039b0 msgr2=0x7f33901482f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:29.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.898+0000 7f3394c19640 1 --2- 172.21.15.73:0/469950133 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f337c0039b0 0x7f33901482f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:29.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.898+0000 7f3394c19640 1 -- 172.21.15.73:0/469950133 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33900c9a00 con 0x7f33900c9040 2024-04-26T20:51:29.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.899+0000 7f3394c19640 1 --2- 172.21.15.73:0/469950133 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33900c9040 0x7f33900c94c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f338000e780 tx=0x7f338000ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:29.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.899+0000 7f3385ffb640 1 -- 172.21.15.73:0/469950133 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338000ee90 con 0x7f33900c9040 2024-04-26T20:51:29.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.899+0000 7f3385ffb640 1 -- 172.21.15.73:0/469950133 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3380018070 con 0x7f33900c9040 2024-04-26T20:51:29.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.899+0000 7f3385ffb640 1 -- 172.21.15.73:0/469950133 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338000ca70 con 0x7f33900c9040 2024-04-26T20:51:29.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.899+0000 7f339767e640 1 -- 172.21.15.73:0/469950133 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33900cbfb0 con 0x7f33900c9040 2024-04-26T20:51:29.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.899+0000 7f339767e640 1 -- 172.21.15.73:0/469950133 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33900cc570 con 0x7f33900c9040 2024-04-26T20:51:29.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.900+0000 7f339767e640 1 -- 172.21.15.73:0/469950133 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3390068fc0 con 0x7f33900c9040 2024-04-26T20:51:29.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.900+0000 7f3385ffb640 1 -- 172.21.15.73:0/469950133 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f338000cc10 con 0x7f33900c9040 2024-04-26T20:51:29.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.900+0000 7f3385ffb640 1 --2- 172.21.15.73:0/469950133 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f337406cac0 0x7f337406ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:29.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.901+0000 7f3385ffb640 1 -- 172.21.15.73:0/469950133 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f338008b900 con 0x7f33900c9040 2024-04-26T20:51:29.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.901+0000 7f339541a640 1 --2- 172.21.15.73:0/469950133 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f337406cac0 0x7f337406ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:29.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.901+0000 7f339541a640 1 --2- 172.21.15.73:0/469950133 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f337406cac0 0x7f337406ef80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3388033000 tx=0x7f3388008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:29.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:29.905+0000 7f3385ffb640 1 -- 172.21.15.73:0/469950133 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f33900c9a00 con 0x7f33900c9040 2024-04-26T20:51:30.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.079+0000 7f339767e640 1 -- 172.21.15.73:0/469950133 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f3390075ef0 con 0x7f33900c9040 2024-04-26T20:51:30.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.079+0000 7f3385ffb640 1 -- 172.21.15.73:0/469950133 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f33800592e0 con 0x7f33900c9040 2024-04-26T20:51:30.081 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-26T20:51:30.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.081+0000 7f339767e640 1 -- 172.21.15.73:0/469950133 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f337406cac0 msgr2=0x7f337406ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:30.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.081+0000 7f339767e640 1 --2- 172.21.15.73:0/469950133 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f337406cac0 0x7f337406ef80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3388033000 tx=0x7f3388008030 comp rx=0 tx=0).stop 2024-04-26T20:51:30.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.081+0000 7f339767e640 1 -- 172.21.15.73:0/469950133 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33900c9040 msgr2=0x7f33900c94c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:30.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.081+0000 7f339767e640 1 --2- 172.21.15.73:0/469950133 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33900c9040 0x7f33900c94c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f338000e780 tx=0x7f338000ec40 comp rx=0 tx=0).stop 2024-04-26T20:51:30.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.081+0000 7f339767e640 1 -- 172.21.15.73:0/469950133 shutdown_connections 2024-04-26T20:51:30.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.081+0000 7f339767e640 1 --2- 172.21.15.73:0/469950133 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33900c9040 0x7f33900c94c0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:30.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.081+0000 7f339767e640 1 --2- 172.21.15.73:0/469950133 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f337406cac0 0x7f337406ef80 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:30.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.081+0000 7f339767e640 1 --2- 172.21.15.73:0/469950133 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f33900cb230 0x7f33900cb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:30.083 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.081+0000 7f339767e640 1 --2- 172.21.15.73:0/469950133 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f337c0039b0 0x7f33901482f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:30.084 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.081+0000 7f339767e640 1 -- 172.21.15.73:0/469950133 >> 172.21.15.73:0/469950133 conn(0x7f33900b3180 msgr2=0x7f33900cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:30.084 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.082+0000 7f339767e640 1 -- 172.21.15.73:0/469950133 shutdown_connections 2024-04-26T20:51:30.084 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.082+0000 7f339767e640 1 -- 172.21.15.73:0/469950133 wait complete. 2024-04-26T20:51:30.093 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T20:51:30.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.190+0000 7f0fa0f51640 1 Processor -- start 2024-04-26T20:51:30.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.190+0000 7f0fa0f51640 1 -- start start 2024-04-26T20:51:30.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.191+0000 7f0fa0f51640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f0f9c15d1e0 0x7f0f9c0d09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:30.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.191+0000 7f0fa0f51640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0f9c0d1a90 0x7f0f9c070ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:30.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.191+0000 7f0fa0f51640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f9c0ce840 con 0x7f0f9c0d0f20 2024-04-26T20:51:30.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.191+0000 7f0fa0f51640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f0f9c0cede0 con 0x7f0f9c0d1a90 2024-04-26T20:51:30.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.191+0000 7f0fa0f51640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f0f9c070650 con 0x7f0f9c15d1e0 2024-04-26T20:51:30.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.191+0000 7f0f9ad9d640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0f9c0d1a90 0x7f0f9c070ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:30.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.191+0000 7f0f99d9b640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f0f9c0d0f20 0x7f0f9c0d1380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40700/0 (socket says 172.21.15.73:40700) 2024-04-26T20:51:30.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.191+0000 7f0f99d9b640 1 -- 172.21.15.73:0/3407189099 learned_addr learned my addr 172.21.15.73:0/3407189099 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:30.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.191+0000 7f0f9ad9d640 1 -- 172.21.15.73:0/3407189099 >> v2:172.21.15.120:3300/0 conn(0x7f0f9c15d1e0 msgr2=0x7f0f9c0d09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:30.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.191+0000 7f0f9ad9d640 1 --2- 172.21.15.73:0/3407189099 >> v2:172.21.15.120:3300/0 conn(0x7f0f9c15d1e0 0x7f0f9c0d09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:30.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.191+0000 7f0f9ad9d640 1 -- 172.21.15.73:0/3407189099 >> v1:172.21.15.73:6789/0 conn(0x7f0f9c0d0f20 legacy=0x7f0f9c0d1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:30.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.191+0000 7f0f9ad9d640 1 -- 172.21.15.73:0/3407189099 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f9c0ceb00 con 0x7f0f9c0d1a90 2024-04-26T20:51:30.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.191+0000 7f0f9959a640 1 -- 172.21.15.73:0/3407189099 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 656968711 0 0) 0x7f0f9c0ce840 con 0x7f0f9c0d0f20 2024-04-26T20:51:30.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.192+0000 7f0f9ad9d640 1 --2- 172.21.15.73:0/3407189099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f9c0d1a90 0x7f0f9c070ae0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c073d90 tx=0x7f0f9402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fa427bee5cafeca server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:30.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.192+0000 7f0f9959a640 1 -- 172.21.15.73:0/3407189099 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f94007740 con 0x7f0f9c0d1a90 2024-04-26T20:51:30.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.192+0000 7f0f9959a640 1 -- 172.21.15.73:0/3407189099 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f940078e0 con 0x7f0f9c0d1a90 2024-04-26T20:51:30.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.192+0000 7f0f9959a640 1 -- 172.21.15.73:0/3407189099 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f94007cb0 con 0x7f0f9c0d1a90 2024-04-26T20:51:30.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.192+0000 7f0fa0f51640 1 -- 172.21.15.73:0/3407189099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f9c0d1a90 msgr2=0x7f0f9c070ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:30.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.192+0000 7f0fa0f51640 1 --2- 172.21.15.73:0/3407189099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f9c0d1a90 0x7f0f9c070ae0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c073d90 tx=0x7f0f9402a680 comp rx=0 tx=0).stop 2024-04-26T20:51:30.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.192+0000 7f0fa0f51640 1 -- 172.21.15.73:0/3407189099 shutdown_connections 2024-04-26T20:51:30.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.192+0000 7f0fa0f51640 1 --2- 172.21.15.73:0/3407189099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f9c0d1a90 0x7f0f9c070ae0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:30.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.192+0000 7f0fa0f51640 1 --2- 172.21.15.73:0/3407189099 >> v2:172.21.15.120:3300/0 conn(0x7f0f9c15d1e0 0x7f0f9c0d09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:30.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.192+0000 7f0fa0f51640 1 -- 172.21.15.73:0/3407189099 >> 172.21.15.73:0/3407189099 conn(0x7f0f9c0d2da0 msgr2=0x7f0f9c0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:30.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.193+0000 7f0fa0f51640 1 -- 172.21.15.73:0/3407189099 shutdown_connections 2024-04-26T20:51:30.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.193+0000 7f0fa0f51640 1 -- 172.21.15.73:0/3407189099 wait complete. 2024-04-26T20:51:30.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.194+0000 7f0fa0f51640 1 Processor -- start 2024-04-26T20:51:30.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.194+0000 7f0fa0f51640 1 -- start start 2024-04-26T20:51:30.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.194+0000 7f0fa0f51640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f9c0d0f20 0x7f0f9c0cc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:30.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.194+0000 7f0fa0f51640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f9c0d1a90 0x7f0f9c0cb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:30.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.194+0000 7f0fa0f51640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0f9c15d1e0 0x7f0f9c14ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:30.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.194+0000 7f0fa0f51640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9c071020 con 0x7f0f9c0d1a90 2024-04-26T20:51:30.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.194+0000 7f0fa0f51640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9c0ce4c0 con 0x7f0f9c0d0f20 2024-04-26T20:51:30.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.194+0000 7f0fa0f51640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f0f9c0711a0 con 0x7f0f9c15d1e0 2024-04-26T20:51:30.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.195+0000 7f0f99d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f9c0d1a90 0x7f0f9c0cb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:30.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.195+0000 7f0f99d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f9c0d1a90 0x7f0f9c0cb6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50016/0 (socket says 172.21.15.73:50016) 2024-04-26T20:51:30.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.195+0000 7f0f99d9b640 1 -- 172.21.15.73:0/4226463392 learned_addr learned my addr 172.21.15.73:0/4226463392 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:30.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.195+0000 7f0f99d9b640 1 -- 172.21.15.73:0/4226463392 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0f9c15d1e0 msgr2=0x7f0f9c14ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:30.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.195+0000 7f0f99d9b640 1 --2- 172.21.15.73:0/4226463392 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0f9c15d1e0 0x7f0f9c14ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:30.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.195+0000 7f0f99d9b640 1 -- 172.21.15.73:0/4226463392 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f9c0d0f20 msgr2=0x7f0f9c0cc6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:30.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.195+0000 7f0f99d9b640 1 --2- 172.21.15.73:0/4226463392 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f9c0d0f20 0x7f0f9c0cc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:30.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.195+0000 7f0f99d9b640 1 -- 172.21.15.73:0/4226463392 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f9c15d660 con 0x7f0f9c0d1a90 2024-04-26T20:51:30.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.195+0000 7f0f99d9b640 1 --2- 172.21.15.73:0/4226463392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f9c0d1a90 0x7f0f9c0cb6d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0f9000c470 tx=0x7f0f9000c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:30.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.195+0000 7f0f8affd640 1 -- 172.21.15.73:0/4226463392 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f90017070 con 0x7f0f9c0d1a90 2024-04-26T20:51:30.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.195+0000 7f0f8affd640 1 -- 172.21.15.73:0/4226463392 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f90005ce0 con 0x7f0f9c0d1a90 2024-04-26T20:51:30.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.195+0000 7f0f8affd640 1 -- 172.21.15.73:0/4226463392 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f900072b0 con 0x7f0f9c0d1a90 2024-04-26T20:51:30.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.195+0000 7f0fa0f51640 1 -- 172.21.15.73:0/4226463392 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f9c15d940 con 0x7f0f9c0d1a90 2024-04-26T20:51:30.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.195+0000 7f0fa0f51640 1 -- 172.21.15.73:0/4226463392 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f9c15e1e0 con 0x7f0f9c0d1a90 2024-04-26T20:51:30.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.196+0000 7f0fa0f51640 1 -- 172.21.15.73:0/4226463392 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f9c15db10 con 0x7f0f9c0d1a90 2024-04-26T20:51:30.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.197+0000 7f0f8affd640 1 -- 172.21.15.73:0/4226463392 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0f9001a070 con 0x7f0f9c0d1a90 2024-04-26T20:51:30.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.197+0000 7f0f8affd640 1 --2- 172.21.15.73:0/4226463392 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f0f7c06c920 0x7f0f7c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:30.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.197+0000 7f0f8affd640 1 -- 172.21.15.73:0/4226463392 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f0f9008b280 con 0x7f0f9c0d1a90 2024-04-26T20:51:30.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.197+0000 7f0f9a59c640 1 --2- 172.21.15.73:0/4226463392 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f0f7c06c920 0x7f0f7c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:30.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.198+0000 7f0f9a59c640 1 --2- 172.21.15.73:0/4226463392 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f0f7c06c920 0x7f0f7c06ede0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c009670 tx=0x7f0f8c006bf0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:30.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.201+0000 7f0f8affd640 1 -- 172.21.15.73:0/4226463392 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0f900592b0 con 0x7f0f9c0d1a90 2024-04-26T20:51:30.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.391+0000 7f0fa0f51640 1 -- 172.21.15.73:0/4226463392 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0f9c15e3e0 con 0x7f0f9c0d1a90 2024-04-26T20:51:30.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.391+0000 7f0f8affd640 1 -- 172.21.15.73:0/4226463392 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f0f9c15e3e0 con 0x7f0f9c0d1a90 2024-04-26T20:51:30.392 INFO:teuthology.orchestra.run.smithi073.stderr:listed 0 entries 2024-04-26T20:51:30.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.393+0000 7f0fa0f51640 1 -- 172.21.15.73:0/4226463392 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f0f7c06c920 msgr2=0x7f0f7c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:30.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.393+0000 7f0fa0f51640 1 --2- 172.21.15.73:0/4226463392 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f0f7c06c920 0x7f0f7c06ede0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c009670 tx=0x7f0f8c006bf0 comp rx=0 tx=0).stop 2024-04-26T20:51:30.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.393+0000 7f0fa0f51640 1 -- 172.21.15.73:0/4226463392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f9c0d1a90 msgr2=0x7f0f9c0cb6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:30.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.393+0000 7f0fa0f51640 1 --2- 172.21.15.73:0/4226463392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f9c0d1a90 0x7f0f9c0cb6d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0f9000c470 tx=0x7f0f9000c930 comp rx=0 tx=0).stop 2024-04-26T20:51:30.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.393+0000 7f0fa0f51640 1 -- 172.21.15.73:0/4226463392 shutdown_connections 2024-04-26T20:51:30.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.393+0000 7f0fa0f51640 1 --2- 172.21.15.73:0/4226463392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f9c0d1a90 0x7f0f9c0cb6d0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:30.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.393+0000 7f0fa0f51640 1 --2- 172.21.15.73:0/4226463392 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f0f7c06c920 0x7f0f7c06ede0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:30.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.393+0000 7f0fa0f51640 1 --2- 172.21.15.73:0/4226463392 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0f9c15d1e0 0x7f0f9c14ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:30.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.393+0000 7f0fa0f51640 1 --2- 172.21.15.73:0/4226463392 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0f9c0d0f20 0x7f0f9c0cc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:30.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.393+0000 7f0fa0f51640 1 -- 172.21.15.73:0/4226463392 >> 172.21.15.73:0/4226463392 conn(0x7f0f9c0d2da0 msgr2=0x7f0f9c071560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:30.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.393+0000 7f0fa0f51640 1 -- 172.21.15.73:0/4226463392 shutdown_connections 2024-04-26T20:51:30.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.393+0000 7f0fa0f51640 1 -- 172.21.15.73:0/4226463392 wait complete. 2024-04-26T20:51:30.405 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-04-26T20:51:30.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:30.405 DEBUG:teuthology.orchestra.run.smithi120:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-04-26T20:51:30.436 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi120.front.sepia.ceph.com 2024-04-26T20:51:30.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:30.436 DEBUG:teuthology.orchestra.run.smithi120:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-26T20:51:30.494 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-26T20:51:30.494 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:51:30.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.590+0000 7f46b8fe6640 1 Processor -- start 2024-04-26T20:51:30.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.590+0000 7f46b8fe6640 1 -- start start 2024-04-26T20:51:30.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.591+0000 7f46b8fe6640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f46b40b0380 0x7f46b40ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:30.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.591+0000 7f46b8fe6640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f46b406c950 0x7f46b415e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:30.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.591+0000 7f46b8fe6640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46b40cfd80 con 0x7f46b40b10b0 2024-04-26T20:51:30.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.591+0000 7f46b8fe6640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f46b40af8e0 con 0x7f46b406c950 2024-04-26T20:51:30.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.591+0000 7f46b8fe6640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f46b40d0020 con 0x7f46b40b0380 2024-04-26T20:51:30.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.591+0000 7f46b2d9d640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f46b406c950 0x7f46b415e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:30.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.591+0000 7f46b2d9d640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f46b406c950 0x7f46b415e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50028/0 (socket says 172.21.15.73:50028) 2024-04-26T20:51:30.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.591+0000 7f46b1d9b640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f46b40b10b0 0x7f46b406c240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40716/0 (socket says 172.21.15.73:40716) 2024-04-26T20:51:30.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.591+0000 7f46b2d9d640 1 -- 172.21.15.73:0/3771171354 learned_addr learned my addr 172.21.15.73:0/3771171354 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:30.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.591+0000 7f46b259c640 1 --2- 172.21.15.73:0/3771171354 >> v2:172.21.15.120:3300/0 conn(0x7f46b40b0380 0x7f46b40ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:30.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.592+0000 7f46b2d9d640 1 -- 172.21.15.73:0/3771171354 >> v2:172.21.15.120:3300/0 conn(0x7f46b40b0380 msgr2=0x7f46b40ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:30.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.592+0000 7f46b2d9d640 1 --2- 172.21.15.73:0/3771171354 >> v2:172.21.15.120:3300/0 conn(0x7f46b40b0380 0x7f46b40ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:30.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.592+0000 7f46b2d9d640 1 -- 172.21.15.73:0/3771171354 >> v1:172.21.15.73:6789/0 conn(0x7f46b40b10b0 legacy=0x7f46b406c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:30.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.592+0000 7f46b159a640 1 -- 172.21.15.73:0/3771171354 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3703517471 0 0) 0x7f46b40cfd80 con 0x7f46b40b10b0 2024-04-26T20:51:30.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.592+0000 7f46b2d9d640 1 -- 172.21.15.73:0/3771171354 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46b40d0300 con 0x7f46b406c950 2024-04-26T20:51:30.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.592+0000 7f46b259c640 1 --2- 172.21.15.73:0/3771171354 >> v2:172.21.15.120:3300/0 conn(0x7f46b40b0380 0x7f46b40ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:30.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.592+0000 7f46b2d9d640 1 --2- 172.21.15.73:0/3771171354 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46b406c950 0x7f46b415e970 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f46b406d290 tx=0x7f46ac02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8512b10d082f02b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:30.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.592+0000 7f46b159a640 1 -- 172.21.15.73:0/3771171354 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac036070 con 0x7f46b406c950 2024-04-26T20:51:30.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.592+0000 7f46b159a640 1 -- 172.21.15.73:0/3771171354 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46ac031600 con 0x7f46b406c950 2024-04-26T20:51:30.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.592+0000 7f46b159a640 1 -- 172.21.15.73:0/3771171354 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac0319d0 con 0x7f46b406c950 2024-04-26T20:51:30.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.593+0000 7f46b8fe6640 1 -- 172.21.15.73:0/3771171354 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46b406c950 msgr2=0x7f46b415e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:30.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.593+0000 7f46b8fe6640 1 --2- 172.21.15.73:0/3771171354 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46b406c950 0x7f46b415e970 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f46b406d290 tx=0x7f46ac02a6a0 comp rx=0 tx=0).stop 2024-04-26T20:51:30.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.593+0000 7f46b8fe6640 1 -- 172.21.15.73:0/3771171354 shutdown_connections 2024-04-26T20:51:30.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.593+0000 7f46b8fe6640 1 --2- 172.21.15.73:0/3771171354 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46b406c950 0x7f46b415e970 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:30.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.593+0000 7f46b8fe6640 1 --2- 172.21.15.73:0/3771171354 >> v2:172.21.15.120:3300/0 conn(0x7f46b40b0380 0x7f46b40ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:30.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.593+0000 7f46b8fe6640 1 -- 172.21.15.73:0/3771171354 >> 172.21.15.73:0/3771171354 conn(0x7f46b40d1cf0 msgr2=0x7f46b40d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:30.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.593+0000 7f46b8fe6640 1 -- 172.21.15.73:0/3771171354 shutdown_connections 2024-04-26T20:51:30.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.593+0000 7f46b8fe6640 1 -- 172.21.15.73:0/3771171354 wait complete. 2024-04-26T20:51:30.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.594+0000 7f46b8fe6640 1 Processor -- start 2024-04-26T20:51:30.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.594+0000 7f46b8fe6640 1 -- start start 2024-04-26T20:51:30.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.595+0000 7f46b8fe6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f46b406c950 0x7f46b40b72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:30.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.595+0000 7f46b8fe6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46b40b0380 0x7f46b41f7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:30.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.595+0000 7f46b8fe6640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46b40b10b0 0x7f46b40c7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:30.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.595+0000 7f46b8fe6640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f46b40b0760 con 0x7f46b40b0380 2024-04-26T20:51:30.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.595+0000 7f46b8fe6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f46b40d1980 con 0x7f46b406c950 2024-04-26T20:51:30.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.595+0000 7f46b8fe6640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f46b40d15c0 con 0x7f46b40b10b0 2024-04-26T20:51:30.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.595+0000 7f46b1d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46b40b0380 0x7f46b41f7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:30.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.595+0000 7f46b1d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46b40b0380 0x7f46b41f7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50044/0 (socket says 172.21.15.73:50044) 2024-04-26T20:51:30.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.595+0000 7f46b1d9b640 1 -- 172.21.15.73:0/2034570487 learned_addr learned my addr 172.21.15.73:0/2034570487 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:30.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.595+0000 7f46b1d9b640 1 -- 172.21.15.73:0/2034570487 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46b40b10b0 msgr2=0x7f46b40c7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:30.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.595+0000 7f46b1d9b640 1 --2- 172.21.15.73:0/2034570487 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46b40b10b0 0x7f46b40c7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:30.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.595+0000 7f46b1d9b640 1 -- 172.21.15.73:0/2034570487 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f46b406c950 msgr2=0x7f46b40b72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:30.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.595+0000 7f46b1d9b640 1 --2- 172.21.15.73:0/2034570487 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f46b406c950 0x7f46b40b72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:30.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.595+0000 7f46b1d9b640 1 -- 172.21.15.73:0/2034570487 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46b41f8490 con 0x7f46b40b0380 2024-04-26T20:51:30.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.596+0000 7f46b1d9b640 1 --2- 172.21.15.73:0/2034570487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46b40b0380 0x7f46b41f7d20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f46a800c410 tx=0x7f46a800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:30.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.596+0000 7f46a2ffd640 1 -- 172.21.15.73:0/2034570487 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a80074e0 con 0x7f46b40b0380 2024-04-26T20:51:30.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.596+0000 7f46a2ffd640 1 -- 172.21.15.73:0/2034570487 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46a8007680 con 0x7f46b40b0380 2024-04-26T20:51:30.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.596+0000 7f46b8fe6640 1 -- 172.21.15.73:0/2034570487 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46b41f8770 con 0x7f46b40b0380 2024-04-26T20:51:30.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.596+0000 7f46a2ffd640 1 -- 172.21.15.73:0/2034570487 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a8004a20 con 0x7f46b40b0380 2024-04-26T20:51:30.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.596+0000 7f46b8fe6640 1 -- 172.21.15.73:0/2034570487 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46b40c8350 con 0x7f46b40b0380 2024-04-26T20:51:30.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.597+0000 7f46b8fe6640 1 -- 172.21.15.73:0/2034570487 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46b40694f0 con 0x7f46b40b0380 2024-04-26T20:51:30.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.597+0000 7f46a2ffd640 1 -- 172.21.15.73:0/2034570487 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f46a801a020 con 0x7f46b40b0380 2024-04-26T20:51:30.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.598+0000 7f46a2ffd640 1 --2- 172.21.15.73:0/2034570487 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f469406c8f0 0x7f469406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:30.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.598+0000 7f46a2ffd640 1 -- 172.21.15.73:0/2034570487 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f46a808b7d0 con 0x7f46b40b0380 2024-04-26T20:51:30.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.598+0000 7f46b259c640 1 --2- 172.21.15.73:0/2034570487 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f469406c8f0 0x7f469406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:30.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.598+0000 7f46b259c640 1 --2- 172.21.15.73:0/2034570487 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f469406c8f0 0x7f469406edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f46a4006fd0 tx=0x7f46a4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:30.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.602+0000 7f46a2ffd640 1 -- 172.21.15.73:0/2034570487 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f46a8059800 con 0x7f46b40b0380 2024-04-26T20:51:30.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.812+0000 7f46b8fe6640 1 -- 172.21.15.73:0/2034570487 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46b406df70 con 0x7f46b40b0380 2024-04-26T20:51:30.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.812+0000 7f46a2ffd640 1 -- 172.21.15.73:0/2034570487 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5180 (secure 0 0 0) 0x7f46a8059100 con 0x7f46b40b0380 2024-04-26T20:51:30.813 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:51:30.813 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/4263167680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6833","nonce":4263167680},{"type":"v1","addr":"172.21.15.120:6835","nonce":4263167680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6834/1614342510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1614342510},{"type":"v1","addr":"172.21.15.120:6834","nonce":1614342510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2146325725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2146325725},{"type":"v1","addr":"172.21.15.73:6835","nonce":2146325725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:50:31.964317+0000","modified":"2024-04-26T20:50:54.049029+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/181202544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":181202544},{"type":"v1","addr":"172.21.15.73:6837","nonce":181202544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4281,"qdb_cluster":[4281]},"id":1}]} 2024-04-26T20:51:30.814 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 8 2024-04-26T20:51:30.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.814+0000 7f46b8fe6640 1 -- 172.21.15.73:0/2034570487 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f469406c8f0 msgr2=0x7f469406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:30.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.814+0000 7f46b8fe6640 1 --2- 172.21.15.73:0/2034570487 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f469406c8f0 0x7f469406edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f46a4006fd0 tx=0x7f46a4008040 comp rx=0 tx=0).stop 2024-04-26T20:51:30.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.814+0000 7f46b8fe6640 1 -- 172.21.15.73:0/2034570487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46b40b0380 msgr2=0x7f46b41f7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:30.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.814+0000 7f46b8fe6640 1 --2- 172.21.15.73:0/2034570487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46b40b0380 0x7f46b41f7d20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f46a800c410 tx=0x7f46a800c8d0 comp rx=0 tx=0).stop 2024-04-26T20:51:30.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.814+0000 7f46b8fe6640 1 -- 172.21.15.73:0/2034570487 shutdown_connections 2024-04-26T20:51:30.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.814+0000 7f46b8fe6640 1 --2- 172.21.15.73:0/2034570487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46b40b0380 0x7f46b41f7d20 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:30.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.814+0000 7f46b8fe6640 1 --2- 172.21.15.73:0/2034570487 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f469406c8f0 0x7f469406edb0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:30.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.814+0000 7f46b8fe6640 1 --2- 172.21.15.73:0/2034570487 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46b40b10b0 0x7f46b40c7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:30.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.814+0000 7f46b8fe6640 1 --2- 172.21.15.73:0/2034570487 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f46b406c950 0x7f46b40b72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:30.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.814+0000 7f46b8fe6640 1 -- 172.21.15.73:0/2034570487 >> 172.21.15.73:0/2034570487 conn(0x7f46b40d1cf0 msgr2=0x7f46b41620a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:30.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.814+0000 7f46b8fe6640 1 -- 172.21.15.73:0/2034570487 shutdown_connections 2024-04-26T20:51:30.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.814+0000 7f46b8fe6640 1 -- 172.21.15.73:0/2034570487 wait complete. 2024-04-26T20:51:30.826 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:51:30.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.923+0000 7f2a0142a640 1 Processor -- start 2024-04-26T20:51:30.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.923+0000 7f2a0142a640 1 -- start start 2024-04-26T20:51:30.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.924+0000 7f2a0142a640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f29fc0d0070 0x7f29fc0d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:30.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.924+0000 7f2a0142a640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f29fc06d560 0x7f29fc06db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:30.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.924+0000 7f2a0142a640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f29fc06e0a0 0x7f29fc162f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:30.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.924+0000 7f2a0142a640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f29fc162d00 con 0x7f29fc06d560 2024-04-26T20:51:30.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.924+0000 7f2a0142a640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f29fc0d2d10 con 0x7f29fc0d0070 2024-04-26T20:51:30.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.924+0000 7f2a0142a640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f29fc0d2420 con 0x7f29fc06e0a0 2024-04-26T20:51:30.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.924+0000 7f29fa7fc640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f29fc06d560 0x7f29fc06db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:30.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.924+0000 7f29fa7fc640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f29fc06d560 0x7f29fc06db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50054/0 (socket says 172.21.15.73:50054) 2024-04-26T20:51:30.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.924+0000 7f29fa7fc640 1 -- 172.21.15.73:0/2332639557 learned_addr learned my addr 172.21.15.73:0/2332639557 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:30.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.924+0000 7f29fb7fe640 1 --2- 172.21.15.73:0/2332639557 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f29fc06e0a0 0x7f29fc162f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:30.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.924+0000 7f29faffd640 1 --2- 172.21.15.73:0/2332639557 >> v2:172.21.15.120:3300/0 conn(0x7f29fc0d0070 0x7f29fc0d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:30.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.924+0000 7f29fa7fc640 1 -- 172.21.15.73:0/2332639557 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f29fc06e0a0 msgr2=0x7f29fc162f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:30.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.924+0000 7f29fa7fc640 1 --2- 172.21.15.73:0/2332639557 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f29fc06e0a0 0x7f29fc162f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:30.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.924+0000 7f29fa7fc640 1 -- 172.21.15.73:0/2332639557 >> v2:172.21.15.120:3300/0 conn(0x7f29fc0d0070 msgr2=0x7f29fc0d1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:30.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.924+0000 7f29fa7fc640 1 --2- 172.21.15.73:0/2332639557 >> v2:172.21.15.120:3300/0 conn(0x7f29fc0d0070 0x7f29fc0d1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:30.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.924+0000 7f29fa7fc640 1 -- 172.21.15.73:0/2332639557 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29fc0cd6b0 con 0x7f29fc06d560 2024-04-26T20:51:30.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.925+0000 7f29fa7fc640 1 --2- 172.21.15.73:0/2332639557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29fc06d560 0x7f29fc06db60 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f29fc0731d0 tx=0x7f29f002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cebdf5cff7b5e2ae server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:30.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.925+0000 7f29f9ffb640 1 -- 172.21.15.73:0/2332639557 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f0037070 con 0x7f29fc06d560 2024-04-26T20:51:30.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.925+0000 7f29f9ffb640 1 -- 172.21.15.73:0/2332639557 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29f0032440 con 0x7f29fc06d560 2024-04-26T20:51:30.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.925+0000 7f29f9ffb640 1 -- 172.21.15.73:0/2332639557 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f0032810 con 0x7f29fc06d560 2024-04-26T20:51:30.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.925+0000 7f2a0142a640 1 -- 172.21.15.73:0/2332639557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29fc06d560 msgr2=0x7f29fc06db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:30.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.925+0000 7f2a0142a640 1 --2- 172.21.15.73:0/2332639557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29fc06d560 0x7f29fc06db60 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f29fc0731d0 tx=0x7f29f002a1e0 comp rx=0 tx=0).stop 2024-04-26T20:51:30.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.925+0000 7f2a0142a640 1 -- 172.21.15.73:0/2332639557 shutdown_connections 2024-04-26T20:51:30.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.925+0000 7f2a0142a640 1 --2- 172.21.15.73:0/2332639557 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f29fc06e0a0 0x7f29fc162f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:30.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.925+0000 7f2a0142a640 1 --2- 172.21.15.73:0/2332639557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29fc06d560 0x7f29fc06db60 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:30.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.925+0000 7f2a0142a640 1 --2- 172.21.15.73:0/2332639557 >> v2:172.21.15.120:3300/0 conn(0x7f29fc0d0070 0x7f29fc0d1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:30.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.925+0000 7f2a0142a640 1 -- 172.21.15.73:0/2332639557 >> 172.21.15.73:0/2332639557 conn(0x7f29fc0d2ec0 msgr2=0x7f29fc0d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:30.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.926+0000 7f2a0142a640 1 -- 172.21.15.73:0/2332639557 shutdown_connections 2024-04-26T20:51:30.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.926+0000 7f2a0142a640 1 -- 172.21.15.73:0/2332639557 wait complete. 2024-04-26T20:51:30.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.927+0000 7f2a0142a640 1 Processor -- start 2024-04-26T20:51:30.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.927+0000 7f2a0142a640 1 -- start start 2024-04-26T20:51:30.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.928+0000 7f2a0142a640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f29fc06d560 0x7f29fc15f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:30.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.928+0000 7f2a0142a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29fc06e0a0 0x7f29fc1eb140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:30.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.928+0000 7f2a0142a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29fc0d0070 0x7f29fc0c7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:30.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.928+0000 7f2a0142a640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f29fc163450 con 0x7f29fc0d0070 2024-04-26T20:51:30.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.928+0000 7f2a0142a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f29fc0cdc70 con 0x7f29fc06e0a0 2024-04-26T20:51:30.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.928+0000 7f2a0142a640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f29fc1635d0 con 0x7f29fc06d560 2024-04-26T20:51:30.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.928+0000 7f29fb7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29fc0d0070 0x7f29fc0c7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:30.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.928+0000 7f29fb7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29fc0d0070 0x7f29fc0c7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50068/0 (socket says 172.21.15.73:50068) 2024-04-26T20:51:30.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.928+0000 7f29fb7fe640 1 -- 172.21.15.73:0/1069946459 learned_addr learned my addr 172.21.15.73:0/1069946459 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:30.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.928+0000 7f29faffd640 1 --2- 172.21.15.73:0/1069946459 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f29fc06d560 0x7f29fc15f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:30.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.928+0000 7f29fa7fc640 1 --2- 172.21.15.73:0/1069946459 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29fc06e0a0 0x7f29fc1eb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:30.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.928+0000 7f29fb7fe640 1 -- 172.21.15.73:0/1069946459 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f29fc06d560 msgr2=0x7f29fc15f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:30.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.928+0000 7f29fb7fe640 1 --2- 172.21.15.73:0/1069946459 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f29fc06d560 0x7f29fc15f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:30.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.928+0000 7f29fb7fe640 1 -- 172.21.15.73:0/1069946459 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29fc06e0a0 msgr2=0x7f29fc1eb140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:30.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.928+0000 7f29fb7fe640 1 --2- 172.21.15.73:0/1069946459 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29fc06e0a0 0x7f29fc1eb140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:30.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.928+0000 7f29fb7fe640 1 -- 172.21.15.73:0/1069946459 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29fc160770 con 0x7f29fc0d0070 2024-04-26T20:51:30.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.928+0000 7f29fb7fe640 1 --2- 172.21.15.73:0/1069946459 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29fc0d0070 0x7f29fc0c7910 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f29f4007e70 tx=0x7f29f400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:30.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.928+0000 7f29e37fe640 1 -- 172.21.15.73:0/1069946459 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f4014620 con 0x7f29fc0d0070 2024-04-26T20:51:30.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.928+0000 7f29e37fe640 1 -- 172.21.15.73:0/1069946459 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29f4019070 con 0x7f29fc0d0070 2024-04-26T20:51:30.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.928+0000 7f29e37fe640 1 -- 172.21.15.73:0/1069946459 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f4014a20 con 0x7f29fc0d0070 2024-04-26T20:51:30.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.928+0000 7f2a0142a640 1 -- 172.21.15.73:0/1069946459 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29fc1ea0a0 con 0x7f29fc0d0070 2024-04-26T20:51:30.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.928+0000 7f2a0142a640 1 -- 172.21.15.73:0/1069946459 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29fc1ea660 con 0x7f29fc0d0070 2024-04-26T20:51:30.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.929+0000 7f2a0142a640 1 -- 172.21.15.73:0/1069946459 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29fc1eae30 con 0x7f29fc0d0070 2024-04-26T20:51:30.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.930+0000 7f29e37fe640 1 -- 172.21.15.73:0/1069946459 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f29f4014bc0 con 0x7f29fc0d0070 2024-04-26T20:51:30.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.930+0000 7f29e37fe640 1 --2- 172.21.15.73:0/1069946459 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f29e806c8f0 0x7f29e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:30.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.930+0000 7f29e37fe640 1 -- 172.21.15.73:0/1069946459 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f29f408c810 con 0x7f29fc0d0070 2024-04-26T20:51:30.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.930+0000 7f29faffd640 1 --2- 172.21.15.73:0/1069946459 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f29e806c8f0 0x7f29e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:30.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.931+0000 7f29faffd640 1 --2- 172.21.15.73:0/1069946459 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f29e806c8f0 0x7f29e806edb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f29ec006fd0 tx=0x7f29ec008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:30.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:30.934+0000 7f29e37fe640 1 -- 172.21.15.73:0/1069946459 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f29fc160770 con 0x7f29fc0d0070 2024-04-26T20:51:31.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.110+0000 7f2a0142a640 1 -- 172.21.15.73:0/1069946459 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f29fc163bc0 con 0x7f29fc0d0070 2024-04-26T20:51:31.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.111+0000 7f29e37fe640 1 -- 172.21.15.73:0/1069946459 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18902 (secure 0 0 0) 0x7f29f4011260 con 0x7f29fc0d0070 2024-04-26T20:51:31.112 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:51:31.112 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":27,"fsid":"fbd3f548-b547-4713-856b-09f98f5d1764","created":"2024-04-26T20:50:07.876786+0000","modified":"2024-04-26T20:51:13.783133+0000","last_up_change":"2024-04-26T20:50:19.082741+0000","last_in_change":"2024-04-26T20:50:10.018400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:50:19.993731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:50:29.964420+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:50:30.920082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:50:34.650207+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"72d6f4f3-214a-4392-a02f-c33cbcab3c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6809","nonce":3461705124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6811","nonce":3461705124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6815","nonce":3461705124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6813","nonce":3461705124}]},"public_addr":"172.21.15.73:6809/3461705124","cluster_addr":"172.21.15.73:6811/3461705124","heartbeat_back_addr":"172.21.15.73:6815/3461705124","heartbeat_front_addr":"172.21.15.73:6813/3461705124","state":["exists","up"]},{"osd":1,"uuid":"786bf8ca-6ddc-4b3c-be6c-9152398bd5fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6817","nonce":2827619403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6819","nonce":2827619403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6825","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6827","nonce":2827619403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6821","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6823","nonce":2827619403}]},"public_addr":"172.21.15.73:6817/2827619403","cluster_addr":"172.21.15.73:6819/2827619403","heartbeat_back_addr":"172.21.15.73:6827/2827619403","heartbeat_front_addr":"172.21.15.73:6823/2827619403","state":["exists","up"]},{"osd":2,"uuid":"90963b1a-5437-4ef8-ad8b-6b74c534eeb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6801","nonce":2542681064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6803","nonce":2542681064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6808","nonce":2542681064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6805","nonce":2542681064}]},"public_addr":"172.21.15.73:6801/2542681064","cluster_addr":"172.21.15.73:6803/2542681064","heartbeat_back_addr":"172.21.15.73:6808/2542681064","heartbeat_front_addr":"172.21.15.73:6805/2542681064","state":["exists","up"]},{"osd":3,"uuid":"bd3d2148-6e38-41b4-a453-fe6e3b8b4ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6822","nonce":1604042718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6826","nonce":1604042718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6831","nonce":1604042718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6829","nonce":1604042718}]},"public_addr":"172.21.15.73:6822/1604042718","cluster_addr":"172.21.15.73:6826/1604042718","heartbeat_back_addr":"172.21.15.73:6831/1604042718","heartbeat_front_addr":"172.21.15.73:6829/1604042718","state":["exists","up"]},{"osd":4,"uuid":"e8668dc8-a23f-46d2-8311-cb882da08e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6817","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6819","nonce":3591055561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6824","nonce":3591055561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6831","nonce":3591055561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6828","nonce":3591055561}]},"public_addr":"172.21.15.120:6819/3591055561","cluster_addr":"172.21.15.120:6824/3591055561","heartbeat_back_addr":"172.21.15.120:6831/3591055561","heartbeat_front_addr":"172.21.15.120:6828/3591055561","state":["exists","up"]},{"osd":5,"uuid":"cbac0bdb-96bd-4852-bcce-e564495c3fcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6809","nonce":819450433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6811","nonce":819450433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6815","nonce":819450433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6813","nonce":819450433}]},"public_addr":"172.21.15.120:6809/819450433","cluster_addr":"172.21.15.120:6811/819450433","heartbeat_back_addr":"172.21.15.120:6815/819450433","heartbeat_front_addr":"172.21.15.120:6813/819450433","state":["exists","up"]},{"osd":6,"uuid":"1884b2a8-569f-4272-96a9-f25a09364dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6801","nonce":3946210727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6803","nonce":3946210727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6807","nonce":3946210727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6805","nonce":3946210727}]},"public_addr":"172.21.15.120:6801/3946210727","cluster_addr":"172.21.15.120:6803/3946210727","heartbeat_back_addr":"172.21.15.120:6807/3946210727","heartbeat_front_addr":"172.21.15.120:6805/3946210727","state":["exists","up"]},{"osd":7,"uuid":"c0c3bf6d-a4c5-4f1e-aa2d-330951830666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6818","nonce":2915574674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6821","nonce":2915574674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6827","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6829","nonce":2915574674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6823","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6825","nonce":2915574674}]},"public_addr":"172.21.15.120:6818/2915574674","cluster_addr":"172.21.15.120:6821/2915574674","heartbeat_back_addr":"172.21.15.120:6829/2915574674","heartbeat_front_addr":"172.21.15.120:6825/2915574674","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.218216+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.208710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.112705+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.182789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.164324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.174212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.225396+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.261559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:51:31.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.113+0000 7f2a0142a640 1 -- 172.21.15.73:0/1069946459 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f29e806c8f0 msgr2=0x7f29e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:31.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.113+0000 7f2a0142a640 1 --2- 172.21.15.73:0/1069946459 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f29e806c8f0 0x7f29e806edb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f29ec006fd0 tx=0x7f29ec008040 comp rx=0 tx=0).stop 2024-04-26T20:51:31.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.113+0000 7f2a0142a640 1 -- 172.21.15.73:0/1069946459 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29fc0d0070 msgr2=0x7f29fc0c7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:31.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.113+0000 7f2a0142a640 1 --2- 172.21.15.73:0/1069946459 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29fc0d0070 0x7f29fc0c7910 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f29f4007e70 tx=0x7f29f400e4b0 comp rx=0 tx=0).stop 2024-04-26T20:51:31.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.113+0000 7f2a0142a640 1 -- 172.21.15.73:0/1069946459 shutdown_connections 2024-04-26T20:51:31.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.113+0000 7f2a0142a640 1 --2- 172.21.15.73:0/1069946459 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29fc0d0070 0x7f29fc0c7910 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:31.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.113+0000 7f2a0142a640 1 --2- 172.21.15.73:0/1069946459 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f29e806c8f0 0x7f29e806edb0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:31.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.113+0000 7f2a0142a640 1 --2- 172.21.15.73:0/1069946459 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29fc06e0a0 0x7f29fc1eb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:31.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.113+0000 7f2a0142a640 1 --2- 172.21.15.73:0/1069946459 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f29fc06d560 0x7f29fc15f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:31.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.113+0000 7f2a0142a640 1 -- 172.21.15.73:0/1069946459 >> 172.21.15.73:0/1069946459 conn(0x7f29fc0d2ec0 msgr2=0x7f29fc0cccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:31.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.113+0000 7f2a0142a640 1 -- 172.21.15.73:0/1069946459 shutdown_connections 2024-04-26T20:51:31.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.113+0000 7f2a0142a640 1 -- 172.21.15.73:0/1069946459 wait complete. 2024-04-26T20:51:31.125 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:50:19.993731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:50:29.964420+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:50:30.920082+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T20:50:34.650207+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T20:51:31.125 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:51:31.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.223+0000 7fabf70b0640 1 Processor -- start 2024-04-26T20:51:31.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.224+0000 7fabf70b0640 1 -- start start 2024-04-26T20:51:31.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.224+0000 7fabf70b0640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fabf00b56c0 0x7fabf00afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:31.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.224+0000 7fabf70b0640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabf00b5e30 con 0x7fabf0072fb0 2024-04-26T20:51:31.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.224+0000 7fabf70b0640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabf00b6110 con 0x7fabf0070750 2024-04-26T20:51:31.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.224+0000 7fabf70b0640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fabf006f2b0 con 0x7fabf00b56c0 2024-04-26T20:51:31.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.225+0000 7fabf4e4c640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fabf0072fb0 0x7fabf0073390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40728/0 (socket says 172.21.15.73:40728) 2024-04-26T20:51:31.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.225+0000 7fabf4e4c640 1 -- 172.21.15.73:0/4217822681 learned_addr learned my addr 172.21.15.73:0/4217822681 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:31.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.225+0000 7fabf564d640 1 --2- 172.21.15.73:0/4217822681 >> v2:172.21.15.120:3300/0 conn(0x7fabf00b56c0 0x7fabf00afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:31.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.225+0000 7fabef7fe640 1 -- 172.21.15.73:0/4217822681 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 92947224 0 0) 0x7fabf00b5e30 con 0x7fabf0072fb0 2024-04-26T20:51:31.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.225+0000 7fabef7fe640 1 -- 172.21.15.73:0/4217822681 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fabd8003520 con 0x7fabf0072fb0 2024-04-26T20:51:31.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.225+0000 7fabef7fe640 1 -- 172.21.15.73:0/4217822681 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1821144551 0 0) 0x7fabd8003520 con 0x7fabf0072fb0 2024-04-26T20:51:31.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.225+0000 7fabf564d640 1 -- 172.21.15.73:0/4217822681 >> v1:172.21.15.120:6789/0 conn(0x7fabf0070750 legacy=0x7fabf00af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:31.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.225+0000 7fabf564d640 1 -- 172.21.15.73:0/4217822681 >> v1:172.21.15.73:6789/0 conn(0x7fabf0072fb0 legacy=0x7fabf0073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:31.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.225+0000 7fabf564d640 1 -- 172.21.15.73:0/4217822681 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabf00b63f0 con 0x7fabf00b56c0 2024-04-26T20:51:31.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.226+0000 7fabf564d640 1 --2- 172.21.15.73:0/4217822681 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fabf00b56c0 0x7fabf00afdf0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fabe8009d90 tx=0x7fabe802a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d316f1ccc2613546 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:31.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.226+0000 7fabef7fe640 1 -- 172.21.15.73:0/4217822681 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe0001670 con 0x7fabf00b56c0 2024-04-26T20:51:31.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.226+0000 7fabef7fe640 1 -- 172.21.15.73:0/4217822681 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabe8007540 con 0x7fabf00b56c0 2024-04-26T20:51:31.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.226+0000 7fabef7fe640 1 -- 172.21.15.73:0/4217822681 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe80078f0 con 0x7fabf00b56c0 2024-04-26T20:51:31.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.227+0000 7fabf70b0640 1 -- 172.21.15.73:0/4217822681 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fabf00b56c0 msgr2=0x7fabf00afdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:31.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.227+0000 7fabf70b0640 1 --2- 172.21.15.73:0/4217822681 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fabf00b56c0 0x7fabf00afdf0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fabe8009d90 tx=0x7fabe802a680 comp rx=0 tx=0).stop 2024-04-26T20:51:31.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.227+0000 7fabf70b0640 1 -- 172.21.15.73:0/4217822681 shutdown_connections 2024-04-26T20:51:31.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.227+0000 7fabf70b0640 1 --2- 172.21.15.73:0/4217822681 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fabf00b56c0 0x7fabf00afdf0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:31.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.227+0000 7fabf70b0640 1 -- 172.21.15.73:0/4217822681 >> 172.21.15.73:0/4217822681 conn(0x7fabf00b39b0 msgr2=0x7fabf00b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:31.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.227+0000 7fabf70b0640 1 -- 172.21.15.73:0/4217822681 shutdown_connections 2024-04-26T20:51:31.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.227+0000 7fabf70b0640 1 -- 172.21.15.73:0/4217822681 wait complete. 2024-04-26T20:51:31.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.228+0000 7fabf70b0640 1 Processor -- start 2024-04-26T20:51:31.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.228+0000 7fabf70b0640 1 -- start start 2024-04-26T20:51:31.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.229+0000 7fabf70b0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fabf0070750 0x7fabf00cdd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:31.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.229+0000 7fabf70b0640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fabf0072fb0 0x7fabf0159e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:31.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.229+0000 7fabf70b0640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabf00b56c0 0x7fabf015a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:31.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.229+0000 7fabf70b0640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fabf015c2e0 con 0x7fabf00b56c0 2024-04-26T20:51:31.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.229+0000 7fabf70b0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fabf015c160 con 0x7fabf0070750 2024-04-26T20:51:31.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.229+0000 7fabf70b0640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fabf00b0330 con 0x7fabf0072fb0 2024-04-26T20:51:31.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.229+0000 7fabf564d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabf00b56c0 0x7fabf015a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:31.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.229+0000 7fabf564d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabf00b56c0 0x7fabf015a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50070/0 (socket says 172.21.15.73:50070) 2024-04-26T20:51:31.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.229+0000 7fabf564d640 1 -- 172.21.15.73:0/834557649 learned_addr learned my addr 172.21.15.73:0/834557649 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:31.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.229+0000 7fabf4e4c640 1 --2- 172.21.15.73:0/834557649 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fabf0070750 0x7fabf00cdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:31.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.229+0000 7fabf564d640 1 -- 172.21.15.73:0/834557649 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fabf0072fb0 msgr2=0x7fabf0159e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:31.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.229+0000 7fabeffff640 1 --2- 172.21.15.73:0/834557649 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fabf0072fb0 0x7fabf0159e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:31.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.229+0000 7fabf564d640 1 --2- 172.21.15.73:0/834557649 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fabf0072fb0 0x7fabf0159e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:31.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.229+0000 7fabf564d640 1 -- 172.21.15.73:0/834557649 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fabf0070750 msgr2=0x7fabf00cdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:31.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.229+0000 7fabf564d640 1 --2- 172.21.15.73:0/834557649 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fabf0070750 0x7fabf00cdd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:31.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.229+0000 7fabf564d640 1 -- 172.21.15.73:0/834557649 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabf014dd70 con 0x7fabf00b56c0 2024-04-26T20:51:31.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.229+0000 7fabeffff640 1 --2- 172.21.15.73:0/834557649 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fabf0072fb0 0x7fabf0159e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:31.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.229+0000 7fabf564d640 1 --2- 172.21.15.73:0/834557649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabf00b56c0 0x7fabf015a370 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fabe8009eb0 tx=0x7fabe8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:31.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.229+0000 7fabed7fa640 1 -- 172.21.15.73:0/834557649 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe8040070 con 0x7fabf00b56c0 2024-04-26T20:51:31.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.229+0000 7fabed7fa640 1 -- 172.21.15.73:0/834557649 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabe8031c40 con 0x7fabf00b56c0 2024-04-26T20:51:31.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.229+0000 7fabf70b0640 1 -- 172.21.15.73:0/834557649 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabf014bd60 con 0x7fabf00b56c0 2024-04-26T20:51:31.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.229+0000 7fabed7fa640 1 -- 172.21.15.73:0/834557649 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe8009450 con 0x7fabf00b56c0 2024-04-26T20:51:31.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.229+0000 7fabf70b0640 1 -- 172.21.15.73:0/834557649 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabf014c320 con 0x7fabf00b56c0 2024-04-26T20:51:31.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.230+0000 7fabf70b0640 1 -- 172.21.15.73:0/834557649 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabf014df40 con 0x7fabf00b56c0 2024-04-26T20:51:31.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.231+0000 7fabed7fa640 1 -- 172.21.15.73:0/834557649 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fabe8039040 con 0x7fabf00b56c0 2024-04-26T20:51:31.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.231+0000 7fabed7fa640 1 --2- 172.21.15.73:0/834557649 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fabdc06c920 0x7fabdc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:31.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.231+0000 7fabed7fa640 1 -- 172.21.15.73:0/834557649 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fabe80ad3f0 con 0x7fabf00b56c0 2024-04-26T20:51:31.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.231+0000 7fabf4e4c640 1 --2- 172.21.15.73:0/834557649 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fabdc06c920 0x7fabdc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:31.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.232+0000 7fabf4e4c640 1 --2- 172.21.15.73:0/834557649 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fabdc06c920 0x7fabdc06ede0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fabe0005880 tx=0x7fabe0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:31.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.236+0000 7fabed7fa640 1 -- 172.21.15.73:0/834557649 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fabe807b420 con 0x7fabf00b56c0 2024-04-26T20:51:31.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.410+0000 7fabf70b0640 1 -- 172.21.15.73:0/834557649 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fabf00694f0 con 0x7fabf00b56c0 2024-04-26T20:51:31.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.410+0000 7fabed7fa640 1 -- 172.21.15.73:0/834557649 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7fabe807ad20 con 0x7fabf00b56c0 2024-04-26T20:51:31.412 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-26T20:51:31.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.412+0000 7fabf70b0640 1 -- 172.21.15.73:0/834557649 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fabdc06c920 msgr2=0x7fabdc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:31.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.412+0000 7fabf70b0640 1 --2- 172.21.15.73:0/834557649 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fabdc06c920 0x7fabdc06ede0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fabe0005880 tx=0x7fabe0009040 comp rx=0 tx=0).stop 2024-04-26T20:51:31.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.412+0000 7fabf70b0640 1 -- 172.21.15.73:0/834557649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabf00b56c0 msgr2=0x7fabf015a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:31.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.412+0000 7fabf70b0640 1 --2- 172.21.15.73:0/834557649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabf00b56c0 0x7fabf015a370 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fabe8009eb0 tx=0x7fabe8004a40 comp rx=0 tx=0).stop 2024-04-26T20:51:31.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.412+0000 7fabf70b0640 1 -- 172.21.15.73:0/834557649 shutdown_connections 2024-04-26T20:51:31.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.412+0000 7fabf70b0640 1 --2- 172.21.15.73:0/834557649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabf00b56c0 0x7fabf015a370 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:31.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.412+0000 7fabf70b0640 1 --2- 172.21.15.73:0/834557649 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fabdc06c920 0x7fabdc06ede0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:31.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.412+0000 7fabf70b0640 1 --2- 172.21.15.73:0/834557649 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fabf0072fb0 0x7fabf0159e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:31.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.412+0000 7fabf70b0640 1 --2- 172.21.15.73:0/834557649 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fabf0070750 0x7fabf00cdd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:31.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.412+0000 7fabf70b0640 1 -- 172.21.15.73:0/834557649 >> 172.21.15.73:0/834557649 conn(0x7fabf00b39b0 msgr2=0x7fabf00af6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:31.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.413+0000 7fabf70b0640 1 -- 172.21.15.73:0/834557649 shutdown_connections 2024-04-26T20:51:31.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.413+0000 7fabf70b0640 1 -- 172.21.15.73:0/834557649 wait complete. 2024-04-26T20:51:31.424 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:51:31.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.523+0000 7f9ba470a640 1 Processor -- start 2024-04-26T20:51:31.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.523+0000 7f9ba470a640 1 -- start start 2024-04-26T20:51:31.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.524+0000 7f9ba470a640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9b9c070350 0x7f9b9c070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:31.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.524+0000 7f9ba470a640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9b9c071090 0x7f9b9c15e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:31.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.524+0000 7f9ba470a640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b9c0d2390 con 0x7f9b9c0cfde0 2024-04-26T20:51:31.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.524+0000 7f9ba470a640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f9b9c071500 con 0x7f9b9c070350 2024-04-26T20:51:31.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.524+0000 7f9ba470a640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f9b9c0afab0 con 0x7f9b9c071090 2024-04-26T20:51:31.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.524+0000 7f9ba1ca5640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9b9c070350 0x7f9b9c070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:31.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.524+0000 7f9ba1ca5640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9b9c070350 0x7f9b9c070b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50080/0 (socket says 172.21.15.73:50080) 2024-04-26T20:51:31.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.524+0000 7f9ba1ca5640 1 -- 172.21.15.73:0/3791335294 learned_addr learned my addr 172.21.15.73:0/3791335294 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:31.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.524+0000 7f9ba2ca7640 1 --2- 172.21.15.73:0/3791335294 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9b9c071090 0x7f9b9c15e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:31.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.525+0000 7f9ba1ca5640 1 -- 172.21.15.73:0/3791335294 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9b9c071090 msgr2=0x7f9b9c15e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:31.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.525+0000 7f9ba1ca5640 1 --2- 172.21.15.73:0/3791335294 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9b9c071090 0x7f9b9c15e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:31.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.525+0000 7f9ba1ca5640 1 -- 172.21.15.73:0/3791335294 >> v1:172.21.15.120:6789/0 conn(0x7f9b9c0cfde0 legacy=0x7f9b9c0d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:31.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.525+0000 7f9ba1ca5640 1 -- 172.21.15.73:0/3791335294 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b9c06d3f0 con 0x7f9b9c070350 2024-04-26T20:51:31.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.525+0000 7f9ba1ca5640 1 --2- 172.21.15.73:0/3791335294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b9c070350 0x7f9b9c070b50 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c009870 tx=0x7f9b8c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4d66f89c11346b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:31.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.525+0000 7f9ba14a4640 1 -- 172.21.15.73:0/3791335294 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c037070 con 0x7f9b9c070350 2024-04-26T20:51:31.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.525+0000 7f9ba14a4640 1 -- 172.21.15.73:0/3791335294 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b8c032440 con 0x7f9b9c070350 2024-04-26T20:51:31.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.525+0000 7f9ba14a4640 1 -- 172.21.15.73:0/3791335294 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c032810 con 0x7f9b9c070350 2024-04-26T20:51:31.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.526+0000 7f9ba470a640 1 -- 172.21.15.73:0/3791335294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b9c070350 msgr2=0x7f9b9c070b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:31.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.526+0000 7f9ba470a640 1 --2- 172.21.15.73:0/3791335294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b9c070350 0x7f9b9c070b50 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c009870 tx=0x7f9b8c02a1c0 comp rx=0 tx=0).stop 2024-04-26T20:51:31.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.526+0000 7f9ba470a640 1 -- 172.21.15.73:0/3791335294 shutdown_connections 2024-04-26T20:51:31.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.526+0000 7f9ba470a640 1 --2- 172.21.15.73:0/3791335294 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9b9c071090 0x7f9b9c15e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:31.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.526+0000 7f9ba470a640 1 --2- 172.21.15.73:0/3791335294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b9c070350 0x7f9b9c070b50 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:31.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.526+0000 7f9ba470a640 1 -- 172.21.15.73:0/3791335294 >> 172.21.15.73:0/3791335294 conn(0x7f9b9c0d2d80 msgr2=0x7f9b9c0d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:31.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.526+0000 7f9ba470a640 1 -- 172.21.15.73:0/3791335294 shutdown_connections 2024-04-26T20:51:31.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.526+0000 7f9ba470a640 1 -- 172.21.15.73:0/3791335294 wait complete. 2024-04-26T20:51:31.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.527+0000 7f9ba470a640 1 Processor -- start 2024-04-26T20:51:31.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.527+0000 7f9ba470a640 1 -- start start 2024-04-26T20:51:31.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.528+0000 7f9ba470a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9b9c070350 0x7f9b9c148ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:31.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.528+0000 7f9ba470a640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9b9c071090 0x7f9b9c149220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:31.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.528+0000 7f9ba470a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b9c0cfde0 0x7f9b9c1f8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:31.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.528+0000 7f9ba470a640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9b9c06d9b0 con 0x7f9b9c0cfde0 2024-04-26T20:51:31.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.528+0000 7f9ba470a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9b9c06dd20 con 0x7f9b9c070350 2024-04-26T20:51:31.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.528+0000 7f9ba470a640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f9b9c15ef30 con 0x7f9b9c071090 2024-04-26T20:51:31.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.528+0000 7f9ba2ca7640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b9c0cfde0 0x7f9b9c1f8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:31.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.528+0000 7f9ba2ca7640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b9c0cfde0 0x7f9b9c1f8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50096/0 (socket says 172.21.15.73:50096) 2024-04-26T20:51:31.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.528+0000 7f9ba2ca7640 1 -- 172.21.15.73:0/178313679 learned_addr learned my addr 172.21.15.73:0/178313679 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:31.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.528+0000 7f9ba1ca5640 1 --2- 172.21.15.73:0/178313679 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9b9c071090 0x7f9b9c149220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:31.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.528+0000 7f9ba24a6640 1 --2- 172.21.15.73:0/178313679 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9b9c070350 0x7f9b9c148ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:31.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.528+0000 7f9ba2ca7640 1 -- 172.21.15.73:0/178313679 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9b9c071090 msgr2=0x7f9b9c149220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:31.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.528+0000 7f9ba2ca7640 1 --2- 172.21.15.73:0/178313679 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9b9c071090 0x7f9b9c149220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:31.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.528+0000 7f9ba2ca7640 1 -- 172.21.15.73:0/178313679 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9b9c070350 msgr2=0x7f9b9c148ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:31.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.528+0000 7f9ba2ca7640 1 --2- 172.21.15.73:0/178313679 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9b9c070350 0x7f9b9c148ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:31.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.528+0000 7f9ba2ca7640 1 -- 172.21.15.73:0/178313679 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b9c160210 con 0x7f9b9c0cfde0 2024-04-26T20:51:31.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.528+0000 7f9ba24a6640 1 --2- 172.21.15.73:0/178313679 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9b9c070350 0x7f9b9c148ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:31.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.528+0000 7f9ba2ca7640 1 --2- 172.21.15.73:0/178313679 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b9c0cfde0 0x7f9b9c1f8770 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f9b9400d800 tx=0x7f9b9400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:31.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.529+0000 7f9b92ffd640 1 -- 172.21.15.73:0/178313679 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b9400b6c0 con 0x7f9b9c0cfde0 2024-04-26T20:51:31.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.529+0000 7f9b92ffd640 1 -- 172.21.15.73:0/178313679 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b94019070 con 0x7f9b9c0cfde0 2024-04-26T20:51:31.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.529+0000 7f9ba470a640 1 -- 172.21.15.73:0/178313679 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b9c1604f0 con 0x7f9b9c0cfde0 2024-04-26T20:51:31.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.529+0000 7f9b92ffd640 1 -- 172.21.15.73:0/178313679 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b9400bac0 con 0x7f9b9c0cfde0 2024-04-26T20:51:31.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.529+0000 7f9ba470a640 1 -- 172.21.15.73:0/178313679 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b9c160ab0 con 0x7f9b9c0cfde0 2024-04-26T20:51:31.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.530+0000 7f9ba470a640 1 -- 172.21.15.73:0/178313679 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b9c0694f0 con 0x7f9b9c0cfde0 2024-04-26T20:51:31.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.530+0000 7f9b92ffd640 1 -- 172.21.15.73:0/178313679 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9b9400bc60 con 0x7f9b9c0cfde0 2024-04-26T20:51:31.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.530+0000 7f9b92ffd640 1 --2- 172.21.15.73:0/178313679 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f9b8006c8f0 0x7f9b8006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:31.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.530+0000 7f9b92ffd640 1 -- 172.21.15.73:0/178313679 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f9b9408c3b0 con 0x7f9b9c0cfde0 2024-04-26T20:51:31.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.530+0000 7f9ba24a6640 1 --2- 172.21.15.73:0/178313679 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f9b8006c8f0 0x7f9b8006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:31.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.531+0000 7f9ba24a6640 1 --2- 172.21.15.73:0/178313679 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f9b8006c8f0 0x7f9b8006edb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9b98001d60 tx=0x7f9b98009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:31.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.535+0000 7f9b92ffd640 1 -- 172.21.15.73:0/178313679 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9b9405a360 con 0x7f9b9c0cfde0 2024-04-26T20:51:31.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.710+0000 7f9ba470a640 1 -- 172.21.15.73:0/178313679 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9b9c159650 con 0x7f9b9c0cfde0 2024-04-26T20:51:31.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.711+0000 7f9b92ffd640 1 -- 172.21.15.73:0/178313679 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f9b94059c60 con 0x7f9b9c0cfde0 2024-04-26T20:51:31.712 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-26T20:51:31.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.713+0000 7f9ba470a640 1 -- 172.21.15.73:0/178313679 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f9b8006c8f0 msgr2=0x7f9b8006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:31.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.713+0000 7f9ba470a640 1 --2- 172.21.15.73:0/178313679 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f9b8006c8f0 0x7f9b8006edb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9b98001d60 tx=0x7f9b98009040 comp rx=0 tx=0).stop 2024-04-26T20:51:31.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.713+0000 7f9ba470a640 1 -- 172.21.15.73:0/178313679 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b9c0cfde0 msgr2=0x7f9b9c1f8770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:31.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.713+0000 7f9ba470a640 1 --2- 172.21.15.73:0/178313679 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b9c0cfde0 0x7f9b9c1f8770 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f9b9400d800 tx=0x7f9b9400dcc0 comp rx=0 tx=0).stop 2024-04-26T20:51:31.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.713+0000 7f9ba470a640 1 -- 172.21.15.73:0/178313679 shutdown_connections 2024-04-26T20:51:31.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.713+0000 7f9ba470a640 1 --2- 172.21.15.73:0/178313679 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b9c0cfde0 0x7f9b9c1f8770 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:31.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.713+0000 7f9ba470a640 1 --2- 172.21.15.73:0/178313679 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f9b8006c8f0 0x7f9b8006edb0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:31.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.713+0000 7f9ba470a640 1 --2- 172.21.15.73:0/178313679 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9b9c071090 0x7f9b9c149220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:31.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.713+0000 7f9ba470a640 1 --2- 172.21.15.73:0/178313679 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9b9c070350 0x7f9b9c148ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:31.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.713+0000 7f9ba470a640 1 -- 172.21.15.73:0/178313679 >> 172.21.15.73:0/178313679 conn(0x7f9b9c0d2d80 msgr2=0x7f9b9c06e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:31.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.713+0000 7f9ba470a640 1 -- 172.21.15.73:0/178313679 shutdown_connections 2024-04-26T20:51:31.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.713+0000 7f9ba470a640 1 -- 172.21.15.73:0/178313679 wait complete. 2024-04-26T20:51:31.724 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:51:31.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.823+0000 7f539a80f640 1 Processor -- start 2024-04-26T20:51:31.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.823+0000 7f539a80f640 1 -- start start 2024-04-26T20:51:31.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.823+0000 7f539a80f640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f53940cb7e0 0x7f5394162e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:31.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.823+0000 7f539a80f640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f53941633c0 0x7f53940726c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:31.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.823+0000 7f539a80f640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f539406f5a0 con 0x7f539415f290 2024-04-26T20:51:31.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.823+0000 7f539a80f640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f539406f350 con 0x7f53941633c0 2024-04-26T20:51:31.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.823+0000 7f539a80f640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f53940cbc20 con 0x7f53940cb7e0 2024-04-26T20:51:31.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.824+0000 7f5393fff640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f539415f290 0x7f53940cd060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40740/0 (socket says 172.21.15.73:40740) 2024-04-26T20:51:31.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.824+0000 7f5393fff640 1 -- 172.21.15.73:0/4196472473 learned_addr learned my addr 172.21.15.73:0/4196472473 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:31.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.824+0000 7f5398dac640 1 --2- 172.21.15.73:0/4196472473 >> v2:172.21.15.120:3300/0 conn(0x7f53941633c0 0x7f53940726c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:31.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.824+0000 7f53937fe640 1 --2- 172.21.15.73:0/4196472473 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f53940cb7e0 0x7f5394162e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:31.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.824+0000 7f5392ffd640 1 -- 172.21.15.73:0/4196472473 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4148416925 0 0) 0x7f539406f5a0 con 0x7f539415f290 2024-04-26T20:51:31.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.824+0000 7f5392ffd640 1 -- 172.21.15.73:0/4196472473 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f537c003520 con 0x7f539415f290 2024-04-26T20:51:31.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.824+0000 7f5398dac640 1 -- 172.21.15.73:0/4196472473 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f53940cb7e0 msgr2=0x7f5394162e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:31.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.824+0000 7f5398dac640 1 --2- 172.21.15.73:0/4196472473 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f53940cb7e0 0x7f5394162e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:31.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.824+0000 7f5398dac640 1 -- 172.21.15.73:0/4196472473 >> v1:172.21.15.73:6789/0 conn(0x7f539415f290 legacy=0x7f53940cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:31.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.824+0000 7f5398dac640 1 -- 172.21.15.73:0/4196472473 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f539406f880 con 0x7f53941633c0 2024-04-26T20:51:31.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.825+0000 7f5398dac640 1 --2- 172.21.15.73:0/4196472473 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53941633c0 0x7f53940726c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f538c009d90 tx=0x7f538c02a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e61a4175f332ec3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:31.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.825+0000 7f5392ffd640 1 -- 172.21.15.73:0/4196472473 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538c007740 con 0x7f53941633c0 2024-04-26T20:51:31.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.825+0000 7f5392ffd640 1 -- 172.21.15.73:0/4196472473 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f538c0078e0 con 0x7f53941633c0 2024-04-26T20:51:31.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.825+0000 7f5392ffd640 1 -- 172.21.15.73:0/4196472473 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538c007c90 con 0x7f53941633c0 2024-04-26T20:51:31.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.825+0000 7f539a80f640 1 -- 172.21.15.73:0/4196472473 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53941633c0 msgr2=0x7f53940726c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:31.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.826+0000 7f539a80f640 1 --2- 172.21.15.73:0/4196472473 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53941633c0 0x7f53940726c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f538c009d90 tx=0x7f538c02a6f0 comp rx=0 tx=0).stop 2024-04-26T20:51:31.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.826+0000 7f539a80f640 1 -- 172.21.15.73:0/4196472473 shutdown_connections 2024-04-26T20:51:31.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.826+0000 7f539a80f640 1 --2- 172.21.15.73:0/4196472473 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53941633c0 0x7f53940726c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:31.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.826+0000 7f539a80f640 1 --2- 172.21.15.73:0/4196472473 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f53940cb7e0 0x7f5394162e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:31.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.826+0000 7f539a80f640 1 -- 172.21.15.73:0/4196472473 >> 172.21.15.73:0/4196472473 conn(0x7f53940d0fd0 msgr2=0x7f53940d13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:31.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.826+0000 7f539a80f640 1 -- 172.21.15.73:0/4196472473 shutdown_connections 2024-04-26T20:51:31.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.826+0000 7f539a80f640 1 -- 172.21.15.73:0/4196472473 wait complete. 2024-04-26T20:51:31.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.827+0000 7f539a80f640 1 Processor -- start 2024-04-26T20:51:31.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.827+0000 7f539a80f640 1 -- start start 2024-04-26T20:51:31.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.828+0000 7f539a80f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53940cb7e0 0x7f539409c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:31.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.828+0000 7f539a80f640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f539415f290 0x7f5394150330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:31.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.828+0000 7f539a80f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53941633c0 0x7f539409df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:31.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.828+0000 7f539a80f640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f539415f670 con 0x7f53940cb7e0 2024-04-26T20:51:31.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.828+0000 7f539a80f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f5394072c00 con 0x7f53941633c0 2024-04-26T20:51:31.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.828+0000 7f539a80f640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f5394150c80 con 0x7f539415f290 2024-04-26T20:51:31.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.828+0000 7f5393fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53940cb7e0 0x7f539409c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:31.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.828+0000 7f5393fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53940cb7e0 0x7f539409c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50108/0 (socket says 172.21.15.73:50108) 2024-04-26T20:51:31.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.828+0000 7f5393fff640 1 -- 172.21.15.73:0/1620309970 learned_addr learned my addr 172.21.15.73:0/1620309970 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:31.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.828+0000 7f53937fe640 1 --2- 172.21.15.73:0/1620309970 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f539415f290 0x7f5394150330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:31.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.828+0000 7f5398dac640 1 --2- 172.21.15.73:0/1620309970 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53941633c0 0x7f539409df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:31.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.828+0000 7f5393fff640 1 -- 172.21.15.73:0/1620309970 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f539415f290 msgr2=0x7f5394150330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:31.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.828+0000 7f5393fff640 1 --2- 172.21.15.73:0/1620309970 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f539415f290 0x7f5394150330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:31.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.828+0000 7f5393fff640 1 -- 172.21.15.73:0/1620309970 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53941633c0 msgr2=0x7f539409df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:31.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.828+0000 7f5393fff640 1 --2- 172.21.15.73:0/1620309970 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53941633c0 0x7f539409df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:31.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.828+0000 7f5393fff640 1 -- 172.21.15.73:0/1620309970 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53940b48d0 con 0x7f53940cb7e0 2024-04-26T20:51:31.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.828+0000 7f53937fe640 1 --2- 172.21.15.73:0/1620309970 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f539415f290 0x7f5394150330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:31.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.828+0000 7f5398dac640 1 --2- 172.21.15.73:0/1620309970 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53941633c0 0x7f539409df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:31.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.828+0000 7f5393fff640 1 --2- 172.21.15.73:0/1620309970 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53940cb7e0 0x7f539409c3f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f538400c430 tx=0x7f538400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:31.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.828+0000 7f5390ff9640 1 -- 172.21.15.73:0/1620309970 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5384017070 con 0x7f53940cb7e0 2024-04-26T20:51:31.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.828+0000 7f5390ff9640 1 -- 172.21.15.73:0/1620309970 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5384005ce0 con 0x7f53940cb7e0 2024-04-26T20:51:31.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.828+0000 7f5390ff9640 1 -- 172.21.15.73:0/1620309970 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53840072f0 con 0x7f53940cb7e0 2024-04-26T20:51:31.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.828+0000 7f539a80f640 1 -- 172.21.15.73:0/1620309970 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53940b4bb0 con 0x7f53940cb7e0 2024-04-26T20:51:31.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.828+0000 7f539a80f640 1 -- 172.21.15.73:0/1620309970 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53940b5170 con 0x7f53940cb7e0 2024-04-26T20:51:31.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.829+0000 7f539a80f640 1 -- 172.21.15.73:0/1620309970 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53940694f0 con 0x7f53940cb7e0 2024-04-26T20:51:31.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.830+0000 7f5390ff9640 1 -- 172.21.15.73:0/1620309970 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f538401a070 con 0x7f53940cb7e0 2024-04-26T20:51:31.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.830+0000 7f5390ff9640 1 --2- 172.21.15.73:0/1620309970 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f538006c8f0 0x7f538006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:31.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.830+0000 7f5390ff9640 1 -- 172.21.15.73:0/1620309970 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f538408b3a0 con 0x7f53940cb7e0 2024-04-26T20:51:31.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.830+0000 7f53937fe640 1 --2- 172.21.15.73:0/1620309970 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f538006c8f0 0x7f538006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:31.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.831+0000 7f53937fe640 1 --2- 172.21.15.73:0/1620309970 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f538006c8f0 0x7f538006edb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f539409c170 tx=0x7f5388008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:31.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:31.834+0000 7f5390ff9640 1 -- 172.21.15.73:0/1620309970 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5384059460 con 0x7f53940cb7e0 2024-04-26T20:51:32.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.010+0000 7f539a80f640 1 -- 172.21.15.73:0/1620309970 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f53940d0db0 con 0x7f53940cb7e0 2024-04-26T20:51:32.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.010+0000 7f5390ff9640 1 -- 172.21.15.73:0/1620309970 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f5384058d60 con 0x7f53940cb7e0 2024-04-26T20:51:32.012 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-26T20:51:32.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.012+0000 7f539a80f640 1 -- 172.21.15.73:0/1620309970 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f538006c8f0 msgr2=0x7f538006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:32.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.012+0000 7f539a80f640 1 --2- 172.21.15.73:0/1620309970 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f538006c8f0 0x7f538006edb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f539409c170 tx=0x7f5388008040 comp rx=0 tx=0).stop 2024-04-26T20:51:32.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.012+0000 7f539a80f640 1 -- 172.21.15.73:0/1620309970 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53940cb7e0 msgr2=0x7f539409c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:32.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.012+0000 7f539a80f640 1 --2- 172.21.15.73:0/1620309970 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53940cb7e0 0x7f539409c3f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f538400c430 tx=0x7f538400c8f0 comp rx=0 tx=0).stop 2024-04-26T20:51:32.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.012+0000 7f539a80f640 1 -- 172.21.15.73:0/1620309970 shutdown_connections 2024-04-26T20:51:32.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.012+0000 7f539a80f640 1 --2- 172.21.15.73:0/1620309970 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f538006c8f0 0x7f538006edb0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:32.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.012+0000 7f539a80f640 1 --2- 172.21.15.73:0/1620309970 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f53941633c0 0x7f539409df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:32.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.012+0000 7f539a80f640 1 --2- 172.21.15.73:0/1620309970 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f539415f290 0x7f5394150330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:32.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.012+0000 7f539a80f640 1 --2- 172.21.15.73:0/1620309970 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53940cb7e0 0x7f539409c3f0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:32.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.012+0000 7f539a80f640 1 -- 172.21.15.73:0/1620309970 >> 172.21.15.73:0/1620309970 conn(0x7f53940d0fd0 msgr2=0x7f5394162b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:32.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.012+0000 7f539a80f640 1 -- 172.21.15.73:0/1620309970 shutdown_connections 2024-04-26T20:51:32.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.012+0000 7f539a80f640 1 -- 172.21.15.73:0/1620309970 wait complete. 2024-04-26T20:51:32.024 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T20:51:32.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.122+0000 7efde5b19640 1 Processor -- start 2024-04-26T20:51:32.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.122+0000 7efde5b19640 1 -- start start 2024-04-26T20:51:32.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.123+0000 7efde5b19640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7efde015a9f0 0x7efde00b4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:32.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.123+0000 7efde5b19640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efde00ce060 con 0x7efde00726e0 2024-04-26T20:51:32.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.123+0000 7efde5b19640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efde00ce340 con 0x7efde0161510 2024-04-26T20:51:32.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.123+0000 7efde5b19640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7efde00ce620 con 0x7efde015a9f0 2024-04-26T20:51:32.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.123+0000 7efddffff640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7efde00726e0 0x7efde006c240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40744/0 (socket says 172.21.15.73:40744) 2024-04-26T20:51:32.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.123+0000 7efddffff640 1 -- 172.21.15.73:0/4098545430 learned_addr learned my addr 172.21.15.73:0/4098545430 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:32.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.123+0000 7efdde7fc640 1 -- 172.21.15.73:0/4098545430 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 943930883 0 0) 0x7efde00ce060 con 0x7efde00726e0 2024-04-26T20:51:32.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.123+0000 7efdde7fc640 1 -- 172.21.15.73:0/4098545430 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efdc8003520 con 0x7efde00726e0 2024-04-26T20:51:32.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.123+0000 7efddf7fe640 1 --2- 172.21.15.73:0/4098545430 >> v2:172.21.15.120:3300/0 conn(0x7efde015a9f0 0x7efde00b4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:32.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.123+0000 7efdde7fc640 1 -- 172.21.15.73:0/4098545430 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 729943484 0 0) 0x7efdc8003520 con 0x7efde00726e0 2024-04-26T20:51:32.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.123+0000 7efdde7fc640 1 -- 172.21.15.73:0/4098545430 >> v2:172.21.15.120:3300/0 conn(0x7efde015a9f0 msgr2=0x7efde00b4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:32.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.123+0000 7efdde7fc640 1 --2- 172.21.15.73:0/4098545430 >> v2:172.21.15.120:3300/0 conn(0x7efde015a9f0 0x7efde00b4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:32.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.123+0000 7efdde7fc640 1 -- 172.21.15.73:0/4098545430 >> v1:172.21.15.120:6789/0 conn(0x7efde0161510 legacy=0x7efde015dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:32.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.123+0000 7efdde7fc640 1 -- 172.21.15.73:0/4098545430 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efde015d6b0 con 0x7efde00726e0 2024-04-26T20:51:32.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.123+0000 7efdde7fc640 1 -- 172.21.15.73:0/4098545430 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7efdd80019f0 con 0x7efde00726e0 2024-04-26T20:51:32.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.124+0000 7efdde7fc640 1 -- 172.21.15.73:0/4098545430 >> v1:172.21.15.73:6789/0 conn(0x7efde00726e0 legacy=0x7efde006c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:32.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.124+0000 7efdde7fc640 1 --2- 172.21.15.73:0/4098545430 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efdc80039b0 0x7efdc8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:32.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.124+0000 7efdde7fc640 1 --2- 172.21.15.73:0/4098545430 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efdc8004800 0x7efdc8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:32.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.124+0000 7efdde7fc640 1 --2- 172.21.15.73:0/4098545430 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efdc80051c0 0x7efdc800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:32.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.124+0000 7efdde7fc640 1 -- 172.21.15.73:0/4098545430 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efde00ce620 con 0x7efdc80039b0 2024-04-26T20:51:32.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.124+0000 7efdde7fc640 1 -- 172.21.15.73:0/4098545430 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efdc800c060 con 0x7efdc80051c0 2024-04-26T20:51:32.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.124+0000 7efdde7fc640 1 -- 172.21.15.73:0/4098545430 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7efdc800c260 con 0x7efdc8004800 2024-04-26T20:51:32.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.124+0000 7efddf7fe640 1 --2- 172.21.15.73:0/4098545430 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efdc80039b0 0x7efdc8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:32.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.124+0000 7efddffff640 1 --2- 172.21.15.73:0/4098545430 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efdc80051c0 0x7efdc800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:32.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.124+0000 7efddeffd640 1 --2- 172.21.15.73:0/4098545430 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efdc8004800 0x7efdc8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:32.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.124+0000 7efddf7fe640 1 -- 172.21.15.73:0/4098545430 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efdc8004800 msgr2=0x7efdc8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:32.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.124+0000 7efddf7fe640 1 --2- 172.21.15.73:0/4098545430 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efdc8004800 0x7efdc8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:32.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.124+0000 7efddf7fe640 1 -- 172.21.15.73:0/4098545430 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efdc80051c0 msgr2=0x7efdc800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:32.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.124+0000 7efddf7fe640 1 --2- 172.21.15.73:0/4098545430 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efdc80051c0 0x7efdc800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:32.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.124+0000 7efddffff640 1 -- 172.21.15.73:0/4098545430 reap_dead start 2024-04-26T20:51:32.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.124+0000 7efddf7fe640 1 -- 172.21.15.73:0/4098545430 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdc8003520 con 0x7efdc80039b0 2024-04-26T20:51:32.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.124+0000 7efddeffd640 1 --2- 172.21.15.73:0/4098545430 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efdc8004800 0x7efdc8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:32.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.125+0000 7efddf7fe640 1 --2- 172.21.15.73:0/4098545430 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efdc80039b0 0x7efdc8003dd0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7efdd0009ac0 tx=0x7efdc8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21e2f69cefce04cb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:32.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.125+0000 7efdde7fc640 1 -- 172.21.15.73:0/4098545430 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd0036070 con 0x7efdc80039b0 2024-04-26T20:51:32.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.125+0000 7efdde7fc640 1 -- 172.21.15.73:0/4098545430 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdd0031400 con 0x7efdc80039b0 2024-04-26T20:51:32.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.125+0000 7efdde7fc640 1 -- 172.21.15.73:0/4098545430 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd0009690 con 0x7efdc80039b0 2024-04-26T20:51:32.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.125+0000 7efde5b19640 1 -- 172.21.15.73:0/4098545430 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efdc80039b0 msgr2=0x7efdc8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:32.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.125+0000 7efde5b19640 1 --2- 172.21.15.73:0/4098545430 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efdc80039b0 0x7efdc8003dd0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7efdd0009ac0 tx=0x7efdc8005830 comp rx=0 tx=0).stop 2024-04-26T20:51:32.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.125+0000 7efde5b19640 1 -- 172.21.15.73:0/4098545430 shutdown_connections 2024-04-26T20:51:32.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.125+0000 7efde5b19640 1 --2- 172.21.15.73:0/4098545430 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efdc80039b0 0x7efdc8003dd0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:32.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.125+0000 7efde5b19640 1 -- 172.21.15.73:0/4098545430 >> 172.21.15.73:0/4098545430 conn(0x7efde00b3180 msgr2=0x7efde00b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:32.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.125+0000 7efde5b19640 1 -- 172.21.15.73:0/4098545430 shutdown_connections 2024-04-26T20:51:32.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.125+0000 7efde5b19640 1 -- 172.21.15.73:0/4098545430 wait complete. 2024-04-26T20:51:32.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.126+0000 7efde5b19640 1 Processor -- start 2024-04-26T20:51:32.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.127+0000 7efde5b19640 1 -- start start 2024-04-26T20:51:32.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.127+0000 7efde5b19640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efdc80039b0 0x7efde00cc840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:32.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.127+0000 7efde5b19640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efde01f4e60 0x7efde0156020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:32.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.127+0000 7efde5b19640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efde00ca800 0x7efde015eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:32.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.127+0000 7efde5b19640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efde00b4480 con 0x7efde00ca800 2024-04-26T20:51:32.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.127+0000 7efde5b19640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efde015add0 con 0x7efdc80039b0 2024-04-26T20:51:32.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.127+0000 7efde5b19640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7efde00b4600 con 0x7efde01f4e60 2024-04-26T20:51:32.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.127+0000 7efddffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efde00ca800 0x7efde015eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:32.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.127+0000 7efddffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efde00ca800 0x7efde015eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50126/0 (socket says 172.21.15.73:50126) 2024-04-26T20:51:32.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.127+0000 7efddffff640 1 -- 172.21.15.73:0/2359799215 learned_addr learned my addr 172.21.15.73:0/2359799215 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:32.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.128+0000 7efddf7fe640 1 --2- 172.21.15.73:0/2359799215 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efdc80039b0 0x7efde00cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:32.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.128+0000 7efddeffd640 1 --2- 172.21.15.73:0/2359799215 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efde01f4e60 0x7efde0156020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:32.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.128+0000 7efddffff640 1 -- 172.21.15.73:0/2359799215 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efde01f4e60 msgr2=0x7efde0156020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:32.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.128+0000 7efddffff640 1 --2- 172.21.15.73:0/2359799215 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efde01f4e60 0x7efde0156020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:32.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.128+0000 7efddffff640 1 -- 172.21.15.73:0/2359799215 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efdc80039b0 msgr2=0x7efde00cc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:32.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.128+0000 7efddffff640 1 --2- 172.21.15.73:0/2359799215 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efdc80039b0 0x7efde00cc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:32.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.128+0000 7efddffff640 1 -- 172.21.15.73:0/2359799215 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efde01f5360 con 0x7efde00ca800 2024-04-26T20:51:32.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.128+0000 7efddf7fe640 1 --2- 172.21.15.73:0/2359799215 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efdc80039b0 0x7efde00cc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:32.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.128+0000 7efddeffd640 1 --2- 172.21.15.73:0/2359799215 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efde01f4e60 0x7efde0156020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:32.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.128+0000 7efddffff640 1 --2- 172.21.15.73:0/2359799215 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efde00ca800 0x7efde015eb20 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7efde00d1450 tx=0x7efdd8007b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:32.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.128+0000 7efdc7fff640 1 -- 172.21.15.73:0/2359799215 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd800ea40 con 0x7efde00ca800 2024-04-26T20:51:32.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.128+0000 7efdc7fff640 1 -- 172.21.15.73:0/2359799215 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdd800ec20 con 0x7efde00ca800 2024-04-26T20:51:32.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.128+0000 7efde5b19640 1 -- 172.21.15.73:0/2359799215 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efde01f5640 con 0x7efde00ca800 2024-04-26T20:51:32.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.128+0000 7efdc7fff640 1 -- 172.21.15.73:0/2359799215 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd8014430 con 0x7efde00ca800 2024-04-26T20:51:32.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.128+0000 7efde5b19640 1 -- 172.21.15.73:0/2359799215 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efde01f5c00 con 0x7efde00ca800 2024-04-26T20:51:32.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.129+0000 7efde5b19640 1 -- 172.21.15.73:0/2359799215 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efde00694f0 con 0x7efde00ca800 2024-04-26T20:51:32.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.129+0000 7efdc7fff640 1 -- 172.21.15.73:0/2359799215 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efdd80145d0 con 0x7efde00ca800 2024-04-26T20:51:32.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.130+0000 7efdc7fff640 1 --2- 172.21.15.73:0/2359799215 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7efdcc06c920 0x7efdcc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:32.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.130+0000 7efddf7fe640 1 --2- 172.21.15.73:0/2359799215 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7efdcc06c920 0x7efdcc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:32.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.130+0000 7efdc7fff640 1 -- 172.21.15.73:0/2359799215 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7efdd808bb80 con 0x7efde00ca800 2024-04-26T20:51:32.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.130+0000 7efddf7fe640 1 --2- 172.21.15.73:0/2359799215 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7efdcc06c920 0x7efdcc06ede0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7efdd0033000 tx=0x7efdd0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:32.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.134+0000 7efdc7fff640 1 -- 172.21.15.73:0/2359799215 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efdd8059c40 con 0x7efde00ca800 2024-04-26T20:51:32.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.312+0000 7efde5b19640 1 -- 172.21.15.73:0/2359799215 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7efde00736f0 con 0x7efde00ca800 2024-04-26T20:51:32.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.312+0000 7efdc7fff640 1 -- 172.21.15.73:0/2359799215 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7efdd8059540 con 0x7efde00ca800 2024-04-26T20:51:32.313 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-26T20:51:32.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.314+0000 7efde5b19640 1 -- 172.21.15.73:0/2359799215 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7efdcc06c920 msgr2=0x7efdcc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:32.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.314+0000 7efde5b19640 1 --2- 172.21.15.73:0/2359799215 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7efdcc06c920 0x7efdcc06ede0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7efdd0033000 tx=0x7efdd0008030 comp rx=0 tx=0).stop 2024-04-26T20:51:32.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.314+0000 7efde5b19640 1 -- 172.21.15.73:0/2359799215 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efde00ca800 msgr2=0x7efde015eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:32.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.314+0000 7efde5b19640 1 --2- 172.21.15.73:0/2359799215 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efde00ca800 0x7efde015eb20 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7efde00d1450 tx=0x7efdd8007b70 comp rx=0 tx=0).stop 2024-04-26T20:51:32.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.314+0000 7efde5b19640 1 -- 172.21.15.73:0/2359799215 shutdown_connections 2024-04-26T20:51:32.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.314+0000 7efde5b19640 1 --2- 172.21.15.73:0/2359799215 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efde00ca800 0x7efde015eb20 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:32.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.314+0000 7efde5b19640 1 --2- 172.21.15.73:0/2359799215 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7efdcc06c920 0x7efdcc06ede0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:32.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.314+0000 7efde5b19640 1 --2- 172.21.15.73:0/2359799215 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efde01f4e60 0x7efde0156020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:32.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.314+0000 7efde5b19640 1 --2- 172.21.15.73:0/2359799215 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efdc80039b0 0x7efde00cc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:32.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.314+0000 7efde5b19640 1 -- 172.21.15.73:0/2359799215 >> 172.21.15.73:0/2359799215 conn(0x7efde00b3180 msgr2=0x7efde00cd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:32.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.314+0000 7efde5b19640 1 -- 172.21.15.73:0/2359799215 shutdown_connections 2024-04-26T20:51:32.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.314+0000 7efde5b19640 1 -- 172.21.15.73:0/2359799215 wait complete. 2024-04-26T20:51:32.326 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:51:32.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.421+0000 7fb53ce9b640 1 Processor -- start 2024-04-26T20:51:32.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.421+0000 7fb53ce9b640 1 -- start start 2024-04-26T20:51:32.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.422+0000 7fb53ce9b640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fb5380b0380 0x7fb5380ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:32.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.422+0000 7fb53ce9b640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb5380b10b0 0x7fb53806c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:32.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.422+0000 7fb53ce9b640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb53806c780 0x7fb538072f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:32.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.422+0000 7fb53ce9b640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7fb5380cfd80 con 0x7fb53806c780 2024-04-26T20:51:32.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.422+0000 7fb53ce9b640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fb5380d0020 con 0x7fb5380b0380 2024-04-26T20:51:32.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.422+0000 7fb53ce9b640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fb5380d0300 con 0x7fb5380b10b0 2024-04-26T20:51:32.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.422+0000 7fb536d9d640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb53806c780 0x7fb538072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:32.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.422+0000 7fb536d9d640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb53806c780 0x7fb538072f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50142/0 (socket says 172.21.15.73:50142) 2024-04-26T20:51:32.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.422+0000 7fb536d9d640 1 -- 172.21.15.73:0/1037484692 learned_addr learned my addr 172.21.15.73:0/1037484692 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:32.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.422+0000 7fb53659c640 1 --2- 172.21.15.73:0/1037484692 >> v2:172.21.15.120:3300/0 conn(0x7fb5380b0380 0x7fb5380ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:32.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.422+0000 7fb535d9b640 1 --2- 172.21.15.73:0/1037484692 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb5380b10b0 0x7fb53806c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:32.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.423+0000 7fb536d9d640 1 -- 172.21.15.73:0/1037484692 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb5380b10b0 msgr2=0x7fb53806c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:32.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.423+0000 7fb536d9d640 1 --2- 172.21.15.73:0/1037484692 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb5380b10b0 0x7fb53806c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:32.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.423+0000 7fb536d9d640 1 -- 172.21.15.73:0/1037484692 >> v2:172.21.15.120:3300/0 conn(0x7fb5380b0380 msgr2=0x7fb5380ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:32.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.423+0000 7fb536d9d640 1 --2- 172.21.15.73:0/1037484692 >> v2:172.21.15.120:3300/0 conn(0x7fb5380b0380 0x7fb5380ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:32.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.423+0000 7fb536d9d640 1 -- 172.21.15.73:0/1037484692 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5380d05e0 con 0x7fb53806c780 2024-04-26T20:51:32.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.423+0000 7fb536d9d640 1 --2- 172.21.15.73:0/1037484692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb53806c780 0x7fb538072f70 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb53806d290 tx=0x7fb52802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ef2a14e3aaf030 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:32.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.423+0000 7fb53559a640 1 -- 172.21.15.73:0/1037484692 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb528037070 con 0x7fb53806c780 2024-04-26T20:51:32.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.423+0000 7fb53559a640 1 -- 172.21.15.73:0/1037484692 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb528032440 con 0x7fb53806c780 2024-04-26T20:51:32.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.423+0000 7fb53559a640 1 -- 172.21.15.73:0/1037484692 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb528032810 con 0x7fb53806c780 2024-04-26T20:51:32.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.424+0000 7fb53ce9b640 1 -- 172.21.15.73:0/1037484692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb53806c780 msgr2=0x7fb538072f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:32.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.424+0000 7fb53ce9b640 1 --2- 172.21.15.73:0/1037484692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb53806c780 0x7fb538072f70 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb53806d290 tx=0x7fb52802a140 comp rx=0 tx=0).stop 2024-04-26T20:51:32.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.424+0000 7fb53ce9b640 1 -- 172.21.15.73:0/1037484692 shutdown_connections 2024-04-26T20:51:32.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.424+0000 7fb53ce9b640 1 --2- 172.21.15.73:0/1037484692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb53806c780 0x7fb538072f70 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:32.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.424+0000 7fb53ce9b640 1 --2- 172.21.15.73:0/1037484692 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb5380b10b0 0x7fb53806c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:32.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.424+0000 7fb53ce9b640 1 --2- 172.21.15.73:0/1037484692 >> v2:172.21.15.120:3300/0 conn(0x7fb5380b0380 0x7fb5380ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:32.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.424+0000 7fb53ce9b640 1 -- 172.21.15.73:0/1037484692 >> 172.21.15.73:0/1037484692 conn(0x7fb5380d1cf0 msgr2=0x7fb5380d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:32.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.424+0000 7fb53ce9b640 1 -- 172.21.15.73:0/1037484692 shutdown_connections 2024-04-26T20:51:32.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.424+0000 7fb53ce9b640 1 -- 172.21.15.73:0/1037484692 wait complete. 2024-04-26T20:51:32.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.425+0000 7fb53ce9b640 1 Processor -- start 2024-04-26T20:51:32.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.425+0000 7fb53ce9b640 1 -- start start 2024-04-26T20:51:32.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.426+0000 7fb53ce9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb53806c780 0x7fb538161610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:32.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.426+0000 7fb53ce9b640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb5380b0380 0x7fb5380c7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:32.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.426+0000 7fb53ce9b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb5380b10b0 0x7fb5380c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:32.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.426+0000 7fb53ce9b640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb5380d15c0 con 0x7fb53806c780 2024-04-26T20:51:32.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.426+0000 7fb53ce9b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb5380b0760 con 0x7fb5380b10b0 2024-04-26T20:51:32.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.426+0000 7fb53ce9b640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fb5380d1740 con 0x7fb5380b0380 2024-04-26T20:51:32.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.426+0000 7fb53659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb53806c780 0x7fb538161610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:32.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.426+0000 7fb53659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb53806c780 0x7fb538161610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50158/0 (socket says 172.21.15.73:50158) 2024-04-26T20:51:32.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.426+0000 7fb53659c640 1 -- 172.21.15.73:0/1512384355 learned_addr learned my addr 172.21.15.73:0/1512384355 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:32.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.426+0000 7fb536d9d640 1 --2- 172.21.15.73:0/1512384355 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb5380b10b0 0x7fb5380c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:32.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.426+0000 7fb535d9b640 1 --2- 172.21.15.73:0/1512384355 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb5380b0380 0x7fb5380c7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:32.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.426+0000 7fb53659c640 1 -- 172.21.15.73:0/1512384355 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb5380b0380 msgr2=0x7fb5380c7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:32.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.426+0000 7fb53659c640 1 --2- 172.21.15.73:0/1512384355 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb5380b0380 0x7fb5380c7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:32.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.426+0000 7fb53659c640 1 -- 172.21.15.73:0/1512384355 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb5380b10b0 msgr2=0x7fb5380c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:32.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.426+0000 7fb53659c640 1 --2- 172.21.15.73:0/1512384355 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb5380b10b0 0x7fb5380c8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:32.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.427+0000 7fb53659c640 1 -- 172.21.15.73:0/1512384355 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5380c57f0 con 0x7fb53806c780 2024-04-26T20:51:32.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.427+0000 7fb53659c640 1 --2- 172.21.15.73:0/1512384355 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb53806c780 0x7fb538161610 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb530007e60 tx=0x7fb53000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:32.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.427+0000 7fb52effd640 1 -- 172.21.15.73:0/1512384355 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb530014620 con 0x7fb53806c780 2024-04-26T20:51:32.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.427+0000 7fb52effd640 1 -- 172.21.15.73:0/1512384355 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb530019070 con 0x7fb53806c780 2024-04-26T20:51:32.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.427+0000 7fb52effd640 1 -- 172.21.15.73:0/1512384355 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5300149e0 con 0x7fb53806c780 2024-04-26T20:51:32.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.427+0000 7fb53ce9b640 1 -- 172.21.15.73:0/1512384355 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5380c5ab0 con 0x7fb53806c780 2024-04-26T20:51:32.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.427+0000 7fb53ce9b640 1 -- 172.21.15.73:0/1512384355 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5380c6070 con 0x7fb53806c780 2024-04-26T20:51:32.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.428+0000 7fb53ce9b640 1 -- 172.21.15.73:0/1512384355 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5380d0b00 con 0x7fb53806c780 2024-04-26T20:51:32.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.428+0000 7fb52effd640 1 -- 172.21.15.73:0/1512384355 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb530014b80 con 0x7fb53806c780 2024-04-26T20:51:32.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.428+0000 7fb52effd640 1 --2- 172.21.15.73:0/1512384355 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb51806c8f0 0x7fb51806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:32.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.429+0000 7fb52effd640 1 -- 172.21.15.73:0/1512384355 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fb53008c7a0 con 0x7fb53806c780 2024-04-26T20:51:32.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.429+0000 7fb535d9b640 1 --2- 172.21.15.73:0/1512384355 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb51806c8f0 0x7fb51806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:32.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.429+0000 7fb535d9b640 1 --2- 172.21.15.73:0/1512384355 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb51806c8f0 0x7fb51806edb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb5380c8d70 tx=0x7fb520008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:32.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.433+0000 7fb52effd640 1 -- 172.21.15.73:0/1512384355 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb5380d0b00 con 0x7fb53806c780 2024-04-26T20:51:32.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.646+0000 7fb53ce9b640 1 -- 172.21.15.73:0/1512384355 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb5380b3310 con 0x7fb53806c780 2024-04-26T20:51:32.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.647+0000 7fb52effd640 1 -- 172.21.15.73:0/1512384355 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5180 (secure 0 0 0) 0x7fb53005a7d0 con 0x7fb53806c780 2024-04-26T20:51:32.648 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:51:32.649 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/4263167680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6833","nonce":4263167680},{"type":"v1","addr":"172.21.15.120:6835","nonce":4263167680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6834/1614342510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1614342510},{"type":"v1","addr":"172.21.15.120:6834","nonce":1614342510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2146325725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2146325725},{"type":"v1","addr":"172.21.15.73:6835","nonce":2146325725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:50:31.964317+0000","modified":"2024-04-26T20:50:54.049029+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/181202544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":181202544},{"type":"v1","addr":"172.21.15.73:6837","nonce":181202544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4281,"qdb_cluster":[4281]},"id":1}]} 2024-04-26T20:51:32.650 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 8 2024-04-26T20:51:32.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.648+0000 7fb53ce9b640 1 -- 172.21.15.73:0/1512384355 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb51806c8f0 msgr2=0x7fb51806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:32.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.648+0000 7fb53ce9b640 1 --2- 172.21.15.73:0/1512384355 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb51806c8f0 0x7fb51806edb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb5380c8d70 tx=0x7fb520008040 comp rx=0 tx=0).stop 2024-04-26T20:51:32.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.648+0000 7fb53ce9b640 1 -- 172.21.15.73:0/1512384355 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb53806c780 msgr2=0x7fb538161610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:32.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.648+0000 7fb53ce9b640 1 --2- 172.21.15.73:0/1512384355 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb53806c780 0x7fb538161610 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb530007e60 tx=0x7fb53000e4b0 comp rx=0 tx=0).stop 2024-04-26T20:51:32.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.649+0000 7fb53ce9b640 1 -- 172.21.15.73:0/1512384355 shutdown_connections 2024-04-26T20:51:32.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.649+0000 7fb53ce9b640 1 --2- 172.21.15.73:0/1512384355 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb51806c8f0 0x7fb51806edb0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:32.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.649+0000 7fb53ce9b640 1 --2- 172.21.15.73:0/1512384355 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb5380b10b0 0x7fb5380c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:32.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.649+0000 7fb53ce9b640 1 --2- 172.21.15.73:0/1512384355 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb5380b0380 0x7fb5380c7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:32.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.649+0000 7fb53ce9b640 1 --2- 172.21.15.73:0/1512384355 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb53806c780 0x7fb538161610 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:32.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.649+0000 7fb53ce9b640 1 -- 172.21.15.73:0/1512384355 >> 172.21.15.73:0/1512384355 conn(0x7fb5380d1cf0 msgr2=0x7fb53806ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:32.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.649+0000 7fb53ce9b640 1 -- 172.21.15.73:0/1512384355 shutdown_connections 2024-04-26T20:51:32.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.649+0000 7fb53ce9b640 1 -- 172.21.15.73:0/1512384355 wait complete. 2024-04-26T20:51:32.660 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:51:32.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.757+0000 7f43d9b1c640 1 Processor -- start 2024-04-26T20:51:32.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.757+0000 7f43d9b1c640 1 -- start start 2024-04-26T20:51:32.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.758+0000 7f43d9b1c640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f43d4162880 0x7f43d40d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:32.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.758+0000 7f43d9b1c640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f43d40d3bc0 0x7f43d40b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:32.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.758+0000 7f43d9b1c640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43d40b7e50 con 0x7f43d40b5f20 2024-04-26T20:51:32.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.758+0000 7f43d9b1c640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f43d40d0580 con 0x7f43d4162880 2024-04-26T20:51:32.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.758+0000 7f43d9b1c640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f43d40d1800 con 0x7f43d40d3bc0 2024-04-26T20:51:32.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.758+0000 7f43d37fe640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f43d4162880 0x7f43d40d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:32.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.758+0000 7f43d3fff640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f43d40d3bc0 0x7f43d40b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:32.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.758+0000 7f43d2ffd640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f43d40b5f20 0x7f43d40d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.73:40380/0 (socket says 172.21.15.73:40380) 2024-04-26T20:51:32.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.758+0000 7f43d2ffd640 1 -- 172.21.15.73:0/2289467852 learned_addr learned my addr 172.21.15.73:0/2289467852 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:32.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.759+0000 7f43d37fe640 1 -- 172.21.15.73:0/2289467852 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f43d40d3bc0 msgr2=0x7f43d40b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:32.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.759+0000 7f43d37fe640 1 --2- 172.21.15.73:0/2289467852 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f43d40d3bc0 0x7f43d40b8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:32.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.759+0000 7f43d37fe640 1 -- 172.21.15.73:0/2289467852 >> v1:172.21.15.120:6789/0 conn(0x7f43d40b5f20 legacy=0x7f43d40d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:32.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.759+0000 7f43d37fe640 1 -- 172.21.15.73:0/2289467852 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43d40d07d0 con 0x7f43d4162880 2024-04-26T20:51:32.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.759+0000 7f43d27fc640 1 -- 172.21.15.73:0/2289467852 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 237733896 0 0) 0x7f43d40b7e50 con 0x7f43d40b5f20 2024-04-26T20:51:32.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.759+0000 7f43d37fe640 1 --2- 172.21.15.73:0/2289467852 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43d4162880 0x7f43d40d2cb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f43c40097a0 tx=0x7f43c402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c07d357fb6b7a9e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:32.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.760+0000 7f43d27fc640 1 -- 172.21.15.73:0/2289467852 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c4037070 con 0x7f43d4162880 2024-04-26T20:51:32.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.760+0000 7f43d27fc640 1 -- 172.21.15.73:0/2289467852 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43c4032440 con 0x7f43d4162880 2024-04-26T20:51:32.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.760+0000 7f43d27fc640 1 -- 172.21.15.73:0/2289467852 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c4032810 con 0x7f43d4162880 2024-04-26T20:51:32.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.760+0000 7f43d9b1c640 1 -- 172.21.15.73:0/2289467852 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43d4162880 msgr2=0x7f43d40d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:32.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.760+0000 7f43d9b1c640 1 --2- 172.21.15.73:0/2289467852 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43d4162880 0x7f43d40d2cb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f43c40097a0 tx=0x7f43c402a180 comp rx=0 tx=0).stop 2024-04-26T20:51:32.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.760+0000 7f43d9b1c640 1 -- 172.21.15.73:0/2289467852 shutdown_connections 2024-04-26T20:51:32.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.760+0000 7f43d9b1c640 1 --2- 172.21.15.73:0/2289467852 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f43d40d3bc0 0x7f43d40b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:32.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.760+0000 7f43d9b1c640 1 --2- 172.21.15.73:0/2289467852 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43d4162880 0x7f43d40d2cb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:32.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.760+0000 7f43d9b1c640 1 -- 172.21.15.73:0/2289467852 >> 172.21.15.73:0/2289467852 conn(0x7f43d40d4f60 msgr2=0x7f43d40d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:32.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.760+0000 7f43d9b1c640 1 -- 172.21.15.73:0/2289467852 shutdown_connections 2024-04-26T20:51:32.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.760+0000 7f43d9b1c640 1 -- 172.21.15.73:0/2289467852 wait complete. 2024-04-26T20:51:32.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.761+0000 7f43d9b1c640 1 Processor -- start 2024-04-26T20:51:32.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.761+0000 7f43d9b1c640 1 -- start start 2024-04-26T20:51:32.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.762+0000 7f43d9b1c640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f43d40b5f20 0x7f43d415d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:32.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.762+0000 7f43d9b1c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43d40d3bc0 0x7f43d40a1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:32.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.762+0000 7f43d9b1c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43d4162880 0x7f43d40a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:32.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.762+0000 7f43d9b1c640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f43d4162c60 con 0x7f43d40d3bc0 2024-04-26T20:51:32.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.762+0000 7f43d9b1c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f43d40d0d90 con 0x7f43d4162880 2024-04-26T20:51:32.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.762+0000 7f43d9b1c640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f43d4162de0 con 0x7f43d40b5f20 2024-04-26T20:51:32.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.762+0000 7f43d2ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43d40d3bc0 0x7f43d40a1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:32.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.762+0000 7f43d2ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43d40d3bc0 0x7f43d40a1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50170/0 (socket says 172.21.15.73:50170) 2024-04-26T20:51:32.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.762+0000 7f43d2ffd640 1 -- 172.21.15.73:0/1452543756 learned_addr learned my addr 172.21.15.73:0/1452543756 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:32.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.763+0000 7f43d37fe640 1 --2- 172.21.15.73:0/1452543756 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f43d40b5f20 0x7f43d415d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:32.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.763+0000 7f43d3fff640 1 --2- 172.21.15.73:0/1452543756 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43d4162880 0x7f43d40a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:32.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.763+0000 7f43d2ffd640 1 -- 172.21.15.73:0/1452543756 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f43d40b5f20 msgr2=0x7f43d415d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:32.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.763+0000 7f43d2ffd640 1 --2- 172.21.15.73:0/1452543756 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f43d40b5f20 0x7f43d415d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:32.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.763+0000 7f43d2ffd640 1 -- 172.21.15.73:0/1452543756 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43d4162880 msgr2=0x7f43d40a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:32.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.763+0000 7f43d2ffd640 1 --2- 172.21.15.73:0/1452543756 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43d4162880 0x7f43d40a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:32.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.763+0000 7f43d2ffd640 1 -- 172.21.15.73:0/1452543756 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43d409f5a0 con 0x7f43d40d3bc0 2024-04-26T20:51:32.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.763+0000 7f43d37fe640 1 --2- 172.21.15.73:0/1452543756 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f43d40b5f20 0x7f43d415d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:32.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.763+0000 7f43d3fff640 1 --2- 172.21.15.73:0/1452543756 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43d4162880 0x7f43d40a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:32.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.763+0000 7f43d2ffd640 1 --2- 172.21.15.73:0/1452543756 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43d40d3bc0 0x7f43d40a1220 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f43c800c3f0 tx=0x7f43c800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:32.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.763+0000 7f43bbfff640 1 -- 172.21.15.73:0/1452543756 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c80074e0 con 0x7f43d40d3bc0 2024-04-26T20:51:32.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.763+0000 7f43bbfff640 1 -- 172.21.15.73:0/1452543756 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43c80047e0 con 0x7f43d40d3bc0 2024-04-26T20:51:32.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.763+0000 7f43d9b1c640 1 -- 172.21.15.73:0/1452543756 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43d40cdd40 con 0x7f43d40d3bc0 2024-04-26T20:51:32.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.763+0000 7f43bbfff640 1 -- 172.21.15.73:0/1452543756 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c8004c10 con 0x7f43d40d3bc0 2024-04-26T20:51:32.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.763+0000 7f43d9b1c640 1 -- 172.21.15.73:0/1452543756 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43d415d820 con 0x7f43d40d3bc0 2024-04-26T20:51:32.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.764+0000 7f43d9b1c640 1 -- 172.21.15.73:0/1452543756 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43d40694f0 con 0x7f43d40d3bc0 2024-04-26T20:51:32.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.764+0000 7f43bbfff640 1 -- 172.21.15.73:0/1452543756 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f43c8011070 con 0x7f43d40d3bc0 2024-04-26T20:51:32.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.765+0000 7f43bbfff640 1 --2- 172.21.15.73:0/1452543756 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f43c006c8f0 0x7f43c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:32.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.765+0000 7f43bbfff640 1 -- 172.21.15.73:0/1452543756 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f43c808bcb0 con 0x7f43d40d3bc0 2024-04-26T20:51:32.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.765+0000 7f43d37fe640 1 --2- 172.21.15.73:0/1452543756 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f43c006c8f0 0x7f43c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:32.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.765+0000 7f43d37fe640 1 --2- 172.21.15.73:0/1452543756 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f43c006c8f0 0x7f43c006edb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f43c4034000 tx=0x7f43c402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:32.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.769+0000 7f43bbfff640 1 -- 172.21.15.73:0/1452543756 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f43d40cdd40 con 0x7f43d40d3bc0 2024-04-26T20:51:32.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.949+0000 7f43d9b1c640 1 -- 172.21.15.73:0/1452543756 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f43d41610f0 con 0x7f43d40d3bc0 2024-04-26T20:51:32.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.950+0000 7f43bbfff640 1 -- 172.21.15.73:0/1452543756 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18902 (secure 0 0 0) 0x7f43c801a2b0 con 0x7f43d40d3bc0 2024-04-26T20:51:32.952 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:51:32.952 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":27,"fsid":"fbd3f548-b547-4713-856b-09f98f5d1764","created":"2024-04-26T20:50:07.876786+0000","modified":"2024-04-26T20:51:13.783133+0000","last_up_change":"2024-04-26T20:50:19.082741+0000","last_in_change":"2024-04-26T20:50:10.018400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:50:19.993731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:50:29.964420+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:50:30.920082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:50:34.650207+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"72d6f4f3-214a-4392-a02f-c33cbcab3c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6809","nonce":3461705124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6811","nonce":3461705124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6815","nonce":3461705124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6813","nonce":3461705124}]},"public_addr":"172.21.15.73:6809/3461705124","cluster_addr":"172.21.15.73:6811/3461705124","heartbeat_back_addr":"172.21.15.73:6815/3461705124","heartbeat_front_addr":"172.21.15.73:6813/3461705124","state":["exists","up"]},{"osd":1,"uuid":"786bf8ca-6ddc-4b3c-be6c-9152398bd5fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6817","nonce":2827619403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6819","nonce":2827619403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6825","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6827","nonce":2827619403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6821","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6823","nonce":2827619403}]},"public_addr":"172.21.15.73:6817/2827619403","cluster_addr":"172.21.15.73:6819/2827619403","heartbeat_back_addr":"172.21.15.73:6827/2827619403","heartbeat_front_addr":"172.21.15.73:6823/2827619403","state":["exists","up"]},{"osd":2,"uuid":"90963b1a-5437-4ef8-ad8b-6b74c534eeb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6801","nonce":2542681064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6803","nonce":2542681064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6808","nonce":2542681064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6805","nonce":2542681064}]},"public_addr":"172.21.15.73:6801/2542681064","cluster_addr":"172.21.15.73:6803/2542681064","heartbeat_back_addr":"172.21.15.73:6808/2542681064","heartbeat_front_addr":"172.21.15.73:6805/2542681064","state":["exists","up"]},{"osd":3,"uuid":"bd3d2148-6e38-41b4-a453-fe6e3b8b4ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6822","nonce":1604042718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6826","nonce":1604042718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6831","nonce":1604042718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6829","nonce":1604042718}]},"public_addr":"172.21.15.73:6822/1604042718","cluster_addr":"172.21.15.73:6826/1604042718","heartbeat_back_addr":"172.21.15.73:6831/1604042718","heartbeat_front_addr":"172.21.15.73:6829/1604042718","state":["exists","up"]},{"osd":4,"uuid":"e8668dc8-a23f-46d2-8311-cb882da08e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6817","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6819","nonce":3591055561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6824","nonce":3591055561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6831","nonce":3591055561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6828","nonce":3591055561}]},"public_addr":"172.21.15.120:6819/3591055561","cluster_addr":"172.21.15.120:6824/3591055561","heartbeat_back_addr":"172.21.15.120:6831/3591055561","heartbeat_front_addr":"172.21.15.120:6828/3591055561","state":["exists","up"]},{"osd":5,"uuid":"cbac0bdb-96bd-4852-bcce-e564495c3fcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6809","nonce":819450433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6811","nonce":819450433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6815","nonce":819450433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6813","nonce":819450433}]},"public_addr":"172.21.15.120:6809/819450433","cluster_addr":"172.21.15.120:6811/819450433","heartbeat_back_addr":"172.21.15.120:6815/819450433","heartbeat_front_addr":"172.21.15.120:6813/819450433","state":["exists","up"]},{"osd":6,"uuid":"1884b2a8-569f-4272-96a9-f25a09364dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6801","nonce":3946210727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6803","nonce":3946210727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6807","nonce":3946210727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6805","nonce":3946210727}]},"public_addr":"172.21.15.120:6801/3946210727","cluster_addr":"172.21.15.120:6803/3946210727","heartbeat_back_addr":"172.21.15.120:6807/3946210727","heartbeat_front_addr":"172.21.15.120:6805/3946210727","state":["exists","up"]},{"osd":7,"uuid":"c0c3bf6d-a4c5-4f1e-aa2d-330951830666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6818","nonce":2915574674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6821","nonce":2915574674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6827","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6829","nonce":2915574674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6823","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6825","nonce":2915574674}]},"public_addr":"172.21.15.120:6818/2915574674","cluster_addr":"172.21.15.120:6821/2915574674","heartbeat_back_addr":"172.21.15.120:6829/2915574674","heartbeat_front_addr":"172.21.15.120:6825/2915574674","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.218216+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.208710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.112705+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.182789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.164324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.174212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.225396+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.261559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:51:32.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.952+0000 7f43d9b1c640 1 -- 172.21.15.73:0/1452543756 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f43c006c8f0 msgr2=0x7f43c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:32.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.952+0000 7f43d9b1c640 1 --2- 172.21.15.73:0/1452543756 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f43c006c8f0 0x7f43c006edb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f43c4034000 tx=0x7f43c402e000 comp rx=0 tx=0).stop 2024-04-26T20:51:32.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.952+0000 7f43d9b1c640 1 -- 172.21.15.73:0/1452543756 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43d40d3bc0 msgr2=0x7f43d40a1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:32.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.952+0000 7f43d9b1c640 1 --2- 172.21.15.73:0/1452543756 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43d40d3bc0 0x7f43d40a1220 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f43c800c3f0 tx=0x7f43c800c8b0 comp rx=0 tx=0).stop 2024-04-26T20:51:32.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.952+0000 7f43d9b1c640 1 -- 172.21.15.73:0/1452543756 shutdown_connections 2024-04-26T20:51:32.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.952+0000 7f43d9b1c640 1 --2- 172.21.15.73:0/1452543756 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43d40d3bc0 0x7f43d40a1220 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:32.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.952+0000 7f43d9b1c640 1 --2- 172.21.15.73:0/1452543756 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f43c006c8f0 0x7f43c006edb0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:32.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.952+0000 7f43d9b1c640 1 --2- 172.21.15.73:0/1452543756 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f43d4162880 0x7f43d40a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:32.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.952+0000 7f43d9b1c640 1 --2- 172.21.15.73:0/1452543756 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f43d40b5f20 0x7f43d415d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:32.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.952+0000 7f43d9b1c640 1 -- 172.21.15.73:0/1452543756 >> 172.21.15.73:0/1452543756 conn(0x7f43d40d4f60 msgr2=0x7f43d4160700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:32.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.953+0000 7f43d9b1c640 1 -- 172.21.15.73:0/1452543756 shutdown_connections 2024-04-26T20:51:32.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:32.953+0000 7f43d9b1c640 1 -- 172.21.15.73:0/1452543756 wait complete. 2024-04-26T20:51:32.965 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T20:51:32.965 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:51:33.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.060+0000 7fb66850a640 1 Processor -- start 2024-04-26T20:51:33.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.060+0000 7fb66850a640 1 -- start start 2024-04-26T20:51:33.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.061+0000 7fb66850a640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb6600b0b70 0x7fb6600b0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:33.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.061+0000 7fb66850a640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fb6600aed90 0x7fb6600af240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:33.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.061+0000 7fb66850a640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6600b1490 con 0x7fb6600b4d40 2024-04-26T20:51:33.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.061+0000 7fb66850a640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fb660073bc0 con 0x7fb6600aed90 2024-04-26T20:51:33.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.061+0000 7fb66850a640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fb660072fb0 con 0x7fb6600b0b70 2024-04-26T20:51:33.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.062+0000 7fb665aa5640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fb6600b4d40 0x7fb6600b6710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40746/0 (socket says 172.21.15.73:40746) 2024-04-26T20:51:33.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.062+0000 7fb665aa5640 1 -- 172.21.15.73:0/14818310 learned_addr learned my addr 172.21.15.73:0/14818310 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:33.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.062+0000 7fb6662a6640 1 --2- 172.21.15.73:0/14818310 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb6600b0b70 0x7fb6600b0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:33.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.062+0000 7fb6652a4640 1 -- 172.21.15.73:0/14818310 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1549096747 0 0) 0x7fb6600b1490 con 0x7fb6600b4d40 2024-04-26T20:51:33.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.062+0000 7fb666aa7640 1 --2- 172.21.15.73:0/14818310 >> v2:172.21.15.120:3300/0 conn(0x7fb6600aed90 0x7fb6600af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:33.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.062+0000 7fb6652a4640 1 -- 172.21.15.73:0/14818310 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb64c003520 con 0x7fb6600b4d40 2024-04-26T20:51:33.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.062+0000 7fb6652a4640 1 -- 172.21.15.73:0/14818310 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 938974230 0 0) 0x7fb64c003520 con 0x7fb6600b4d40 2024-04-26T20:51:33.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.062+0000 7fb6652a4640 1 -- 172.21.15.73:0/14818310 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb6600b0b70 msgr2=0x7fb6600b0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:33.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.062+0000 7fb6652a4640 1 --2- 172.21.15.73:0/14818310 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb6600b0b70 0x7fb6600b0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:33.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.062+0000 7fb6652a4640 1 -- 172.21.15.73:0/14818310 >> v2:172.21.15.120:3300/0 conn(0x7fb6600aed90 msgr2=0x7fb6600af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:33.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.062+0000 7fb6652a4640 1 --2- 172.21.15.73:0/14818310 >> v2:172.21.15.120:3300/0 conn(0x7fb6600aed90 0x7fb6600af240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:33.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.062+0000 7fb6652a4640 1 -- 172.21.15.73:0/14818310 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb660073290 con 0x7fb6600b4d40 2024-04-26T20:51:33.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.062+0000 7fb6652a4640 1 -- 172.21.15.73:0/14818310 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7fb65c001670 con 0x7fb6600b4d40 2024-04-26T20:51:33.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.062+0000 7fb6652a4640 1 -- 172.21.15.73:0/14818310 >> v1:172.21.15.73:6789/0 conn(0x7fb6600b4d40 legacy=0x7fb6600b6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:33.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.062+0000 7fb6652a4640 1 --2- 172.21.15.73:0/14818310 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb6600b0b70 0x7fb64c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:33.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.062+0000 7fb6652a4640 1 --2- 172.21.15.73:0/14818310 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb64c003f30 0x7fb64c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:33.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.062+0000 7fb6652a4640 1 --2- 172.21.15.73:0/14818310 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb64c004d40 0x7fb64c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:33.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.062+0000 7fb6652a4640 1 -- 172.21.15.73:0/14818310 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb660073bc0 con 0x7fb64c004d40 2024-04-26T20:51:33.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.062+0000 7fb6652a4640 1 -- 172.21.15.73:0/14818310 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb660072fb0 con 0x7fb64c003f30 2024-04-26T20:51:33.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.062+0000 7fb6652a4640 1 -- 172.21.15.73:0/14818310 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fb64c00b9c0 con 0x7fb6600b0b70 2024-04-26T20:51:33.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.062+0000 7fb666aa7640 1 --2- 172.21.15.73:0/14818310 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb64c004d40 0x7fb64c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:33.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.062+0000 7fb6662a6640 1 --2- 172.21.15.73:0/14818310 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb6600b0b70 0x7fb64c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:33.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.062+0000 7fb665aa5640 1 --2- 172.21.15.73:0/14818310 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb64c003f30 0x7fb64c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:33.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.062+0000 7fb666aa7640 1 -- 172.21.15.73:0/14818310 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb6600b0b70 msgr2=0x7fb64c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:33.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.062+0000 7fb666aa7640 1 --2- 172.21.15.73:0/14818310 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb6600b0b70 0x7fb64c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:33.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.062+0000 7fb666aa7640 1 -- 172.21.15.73:0/14818310 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb64c003f30 msgr2=0x7fb64c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:33.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.062+0000 7fb666aa7640 1 --2- 172.21.15.73:0/14818310 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb64c003f30 0x7fb64c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:33.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.062+0000 7fb666aa7640 1 -- 172.21.15.73:0/14818310 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb64c003520 con 0x7fb64c004d40 2024-04-26T20:51:33.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.062+0000 7fb665aa5640 1 --2- 172.21.15.73:0/14818310 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb64c003f30 0x7fb64c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:33.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.063+0000 7fb666aa7640 1 --2- 172.21.15.73:0/14818310 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb64c004d40 0x7fb64c0051c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb658009fc0 tx=0x7fb64c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0459c6e732e3362 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:33.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.063+0000 7fb6652a4640 1 -- 172.21.15.73:0/14818310 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb658009c10 con 0x7fb64c004d40 2024-04-26T20:51:33.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.063+0000 7fb6652a4640 1 -- 172.21.15.73:0/14818310 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb658036070 con 0x7fb64c004d40 2024-04-26T20:51:33.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.063+0000 7fb6652a4640 1 -- 172.21.15.73:0/14818310 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb65802d070 con 0x7fb64c004d40 2024-04-26T20:51:33.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.064+0000 7fb66850a640 1 -- 172.21.15.73:0/14818310 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb64c004d40 msgr2=0x7fb64c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:33.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.064+0000 7fb66850a640 1 --2- 172.21.15.73:0/14818310 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb64c004d40 0x7fb64c0051c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb658009fc0 tx=0x7fb64c005830 comp rx=0 tx=0).stop 2024-04-26T20:51:33.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.064+0000 7fb666aa7640 1 -- 172.21.15.73:0/14818310 reap_dead start 2024-04-26T20:51:33.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.064+0000 7fb66850a640 1 -- 172.21.15.73:0/14818310 shutdown_connections 2024-04-26T20:51:33.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.064+0000 7fb66850a640 1 -- 172.21.15.73:0/14818310 >> 172.21.15.73:0/14818310 conn(0x7fb6600b39b0 msgr2=0x7fb6600b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:33.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.064+0000 7fb66850a640 1 -- 172.21.15.73:0/14818310 shutdown_connections 2024-04-26T20:51:33.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.064+0000 7fb66850a640 1 -- 172.21.15.73:0/14818310 wait complete. 2024-04-26T20:51:33.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.065+0000 7fb66850a640 1 Processor -- start 2024-04-26T20:51:33.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.065+0000 7fb66850a640 1 -- start start 2024-04-26T20:51:33.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.066+0000 7fb66850a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb64c004d40 0x7fb660157da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:33.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.066+0000 7fb66850a640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb660147680 0x7fb660070060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:33.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.066+0000 7fb66850a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb660147ae0 0x7fb6600705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:33.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.066+0000 7fb66850a640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb6600b6e20 con 0x7fb660147ae0 2024-04-26T20:51:33.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.066+0000 7fb66850a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb660073850 con 0x7fb64c004d40 2024-04-26T20:51:33.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.066+0000 7fb66850a640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fb660160360 con 0x7fb660147680 2024-04-26T20:51:33.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.066+0000 7fb666aa7640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb660147ae0 0x7fb6600705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:33.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.066+0000 7fb666aa7640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb660147ae0 0x7fb6600705a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50178/0 (socket says 172.21.15.73:50178) 2024-04-26T20:51:33.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.066+0000 7fb666aa7640 1 -- 172.21.15.73:0/2890261013 learned_addr learned my addr 172.21.15.73:0/2890261013 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:33.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.066+0000 7fb665aa5640 1 --2- 172.21.15.73:0/2890261013 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb660147680 0x7fb660070060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:33.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.066+0000 7fb6662a6640 1 --2- 172.21.15.73:0/2890261013 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb64c004d40 0x7fb660157da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:33.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.066+0000 7fb666aa7640 1 -- 172.21.15.73:0/2890261013 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb660147680 msgr2=0x7fb660070060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:33.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.066+0000 7fb666aa7640 1 --2- 172.21.15.73:0/2890261013 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb660147680 0x7fb660070060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:33.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.066+0000 7fb666aa7640 1 -- 172.21.15.73:0/2890261013 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb64c004d40 msgr2=0x7fb660157da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:33.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.066+0000 7fb666aa7640 1 --2- 172.21.15.73:0/2890261013 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb64c004d40 0x7fb660157da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:33.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.066+0000 7fb666aa7640 1 -- 172.21.15.73:0/2890261013 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb660156d00 con 0x7fb660147ae0 2024-04-26T20:51:33.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.066+0000 7fb666aa7640 1 --2- 172.21.15.73:0/2890261013 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb660147ae0 0x7fb6600705a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb658033040 tx=0x7fb658009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:33.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.066+0000 7fb652ffd640 1 -- 172.21.15.73:0/2890261013 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb658009c10 con 0x7fb660147ae0 2024-04-26T20:51:33.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.066+0000 7fb652ffd640 1 -- 172.21.15.73:0/2890261013 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb658003990 con 0x7fb660147ae0 2024-04-26T20:51:33.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.066+0000 7fb652ffd640 1 -- 172.21.15.73:0/2890261013 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb658003d80 con 0x7fb660147ae0 2024-04-26T20:51:33.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.066+0000 7fb66850a640 1 -- 172.21.15.73:0/2890261013 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb660156fe0 con 0x7fb660147ae0 2024-04-26T20:51:33.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.067+0000 7fb66850a640 1 -- 172.21.15.73:0/2890261013 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6601575a0 con 0x7fb660147ae0 2024-04-26T20:51:33.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.067+0000 7fb66850a640 1 -- 172.21.15.73:0/2890261013 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6600694f0 con 0x7fb660147ae0 2024-04-26T20:51:33.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.068+0000 7fb652ffd640 1 -- 172.21.15.73:0/2890261013 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb658005d10 con 0x7fb660147ae0 2024-04-26T20:51:33.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.068+0000 7fb652ffd640 1 --2- 172.21.15.73:0/2890261013 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb64406c8f0 0x7fb64406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:33.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.068+0000 7fb652ffd640 1 -- 172.21.15.73:0/2890261013 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fb6580ad8b0 con 0x7fb660147ae0 2024-04-26T20:51:33.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.068+0000 7fb6662a6640 1 --2- 172.21.15.73:0/2890261013 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb64406c8f0 0x7fb64406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:33.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.069+0000 7fb6662a6640 1 --2- 172.21.15.73:0/2890261013 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb64406c8f0 0x7fb64406edb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb654009b00 tx=0x7fb654008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:33.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.073+0000 7fb652ffd640 1 -- 172.21.15.73:0/2890261013 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb65804a060 con 0x7fb660147ae0 2024-04-26T20:51:33.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.284+0000 7fb66850a640 1 -- 172.21.15.73:0/2890261013 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb660160ac0 con 0x7fb660147ae0 2024-04-26T20:51:33.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.285+0000 7fb652ffd640 1 -- 172.21.15.73:0/2890261013 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5180 (secure 0 0 0) 0x7fb65807b8e0 con 0x7fb660147ae0 2024-04-26T20:51:33.286 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:51:33.286 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/4263167680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6833","nonce":4263167680},{"type":"v1","addr":"172.21.15.120:6835","nonce":4263167680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6834/1614342510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1614342510},{"type":"v1","addr":"172.21.15.120:6834","nonce":1614342510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2146325725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2146325725},{"type":"v1","addr":"172.21.15.73:6835","nonce":2146325725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:50:31.964317+0000","modified":"2024-04-26T20:50:54.049029+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/181202544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":181202544},{"type":"v1","addr":"172.21.15.73:6837","nonce":181202544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4281,"qdb_cluster":[4281]},"id":1}]} 2024-04-26T20:51:33.286 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 8 2024-04-26T20:51:33.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.287+0000 7fb66850a640 1 -- 172.21.15.73:0/2890261013 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb64406c8f0 msgr2=0x7fb64406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:33.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.287+0000 7fb66850a640 1 --2- 172.21.15.73:0/2890261013 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb64406c8f0 0x7fb64406edb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb654009b00 tx=0x7fb654008040 comp rx=0 tx=0).stop 2024-04-26T20:51:33.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.287+0000 7fb66850a640 1 -- 172.21.15.73:0/2890261013 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb660147ae0 msgr2=0x7fb6600705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:33.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.287+0000 7fb66850a640 1 --2- 172.21.15.73:0/2890261013 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb660147ae0 0x7fb6600705a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb658033040 tx=0x7fb658009750 comp rx=0 tx=0).stop 2024-04-26T20:51:33.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.287+0000 7fb66850a640 1 -- 172.21.15.73:0/2890261013 shutdown_connections 2024-04-26T20:51:33.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.287+0000 7fb66850a640 1 --2- 172.21.15.73:0/2890261013 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb660147ae0 0x7fb6600705a0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:33.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.287+0000 7fb66850a640 1 --2- 172.21.15.73:0/2890261013 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb64406c8f0 0x7fb64406edb0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:33.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.287+0000 7fb66850a640 1 --2- 172.21.15.73:0/2890261013 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb660147680 0x7fb660070060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:33.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.287+0000 7fb66850a640 1 --2- 172.21.15.73:0/2890261013 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb64c004d40 0x7fb660157da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:33.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.287+0000 7fb66850a640 1 -- 172.21.15.73:0/2890261013 >> 172.21.15.73:0/2890261013 conn(0x7fb6600b39b0 msgr2=0x7fb6600b66d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:33.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.287+0000 7fb66850a640 1 -- 172.21.15.73:0/2890261013 shutdown_connections 2024-04-26T20:51:33.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.287+0000 7fb66850a640 1 -- 172.21.15.73:0/2890261013 wait complete. 2024-04-26T20:51:33.299 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:51:33.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.397+0000 7f6252978640 1 Processor -- start 2024-04-26T20:51:33.399 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.397+0000 7f6252978640 1 -- start start 2024-04-26T20:51:33.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.398+0000 7f6252978640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f624c15f3a0 0x7f624c0d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:33.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.398+0000 7f6252978640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f624c0d1770 0x7f624c0d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:33.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.398+0000 7f6252978640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f624c0d25a0 con 0x7f624c0d2130 2024-04-26T20:51:33.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.398+0000 7f6252978640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f624c0d0e20 con 0x7f624c0d1770 2024-04-26T20:51:33.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.398+0000 7f6252978640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f624c0b1d70 con 0x7f624c15f3a0 2024-04-26T20:51:33.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.398+0000 7f624b7fe640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f624c0d1770 0x7f624c0d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:33.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.398+0000 7f624b7fe640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f624c0d1770 0x7f624c0d1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50190/0 (socket says 172.21.15.73:50190) 2024-04-26T20:51:33.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.398+0000 7f624b7fe640 1 -- 172.21.15.73:0/93443595 learned_addr learned my addr 172.21.15.73:0/93443595 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:33.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.399+0000 7f624b7fe640 1 -- 172.21.15.73:0/93443595 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f624c15f3a0 msgr2=0x7f624c0d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:33.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.399+0000 7f624b7fe640 1 --2- 172.21.15.73:0/93443595 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f624c15f3a0 0x7f624c0d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:33.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.399+0000 7f624b7fe640 1 -- 172.21.15.73:0/93443595 >> v1:172.21.15.120:6789/0 conn(0x7f624c0d2130 legacy=0x7f624c0d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:33.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.399+0000 7f624b7fe640 1 -- 172.21.15.73:0/93443595 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f624c0b2050 con 0x7f624c0d1770 2024-04-26T20:51:33.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.399+0000 7f624b7fe640 1 --2- 172.21.15.73:0/93443595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f624c0d1770 0x7f624c0d1bf0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f623c009870 tx=0x7f623c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5939ff72f87d125d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:33.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.399+0000 7f624affd640 1 -- 172.21.15.73:0/93443595 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623c037070 con 0x7f624c0d1770 2024-04-26T20:51:33.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.399+0000 7f624affd640 1 -- 172.21.15.73:0/93443595 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f623c032440 con 0x7f624c0d1770 2024-04-26T20:51:33.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.399+0000 7f624affd640 1 -- 172.21.15.73:0/93443595 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623c032810 con 0x7f624c0d1770 2024-04-26T20:51:33.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.400+0000 7f6252978640 1 -- 172.21.15.73:0/93443595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f624c0d1770 msgr2=0x7f624c0d1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:33.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.400+0000 7f6252978640 1 --2- 172.21.15.73:0/93443595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f624c0d1770 0x7f624c0d1bf0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f623c009870 tx=0x7f623c02a210 comp rx=0 tx=0).stop 2024-04-26T20:51:33.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.400+0000 7f6252978640 1 -- 172.21.15.73:0/93443595 shutdown_connections 2024-04-26T20:51:33.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.400+0000 7f6252978640 1 --2- 172.21.15.73:0/93443595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f624c0d1770 0x7f624c0d1bf0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:33.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.400+0000 7f6252978640 1 --2- 172.21.15.73:0/93443595 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f624c15f3a0 0x7f624c0d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:33.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.400+0000 7f6252978640 1 -- 172.21.15.73:0/93443595 >> 172.21.15.73:0/93443595 conn(0x7f624c0d4f90 msgr2=0x7f624c0d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:33.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.400+0000 7f6252978640 1 -- 172.21.15.73:0/93443595 shutdown_connections 2024-04-26T20:51:33.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.400+0000 7f6252978640 1 -- 172.21.15.73:0/93443595 wait complete. 2024-04-26T20:51:33.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.401+0000 7f6252978640 1 Processor -- start 2024-04-26T20:51:33.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.402+0000 7f6252978640 1 -- start start 2024-04-26T20:51:33.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.402+0000 7f6252978640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f624c0d1770 0x7f624c0b98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:33.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.402+0000 7f6252978640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f624c0d2130 0x7f624c158f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:33.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.402+0000 7f6252978640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f624c15f3a0 0x7f624c15c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:33.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.402+0000 7f6252978640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f624c15f900 con 0x7f624c0d2130 2024-04-26T20:51:33.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.402+0000 7f6252978640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f624c15f780 con 0x7f624c15f3a0 2024-04-26T20:51:33.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.402+0000 7f6252978640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f624c0b3860 con 0x7f624c0d1770 2024-04-26T20:51:33.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.402+0000 7f624b7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f624c0d2130 0x7f624c158f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:33.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.402+0000 7f624b7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f624c0d2130 0x7f624c158f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50196/0 (socket says 172.21.15.73:50196) 2024-04-26T20:51:33.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.402+0000 7f624b7fe640 1 -- 172.21.15.73:0/1898736339 learned_addr learned my addr 172.21.15.73:0/1898736339 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:33.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.403+0000 7f6250f15640 1 --2- 172.21.15.73:0/1898736339 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f624c15f3a0 0x7f624c15c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:33.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.403+0000 7f624bfff640 1 --2- 172.21.15.73:0/1898736339 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f624c0d1770 0x7f624c0b98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:33.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.403+0000 7f624b7fe640 1 -- 172.21.15.73:0/1898736339 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f624c0d1770 msgr2=0x7f624c0b98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:33.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.403+0000 7f624b7fe640 1 --2- 172.21.15.73:0/1898736339 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f624c0d1770 0x7f624c0b98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:33.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.403+0000 7f624b7fe640 1 -- 172.21.15.73:0/1898736339 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f624c15f3a0 msgr2=0x7f624c15c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:33.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.403+0000 7f624b7fe640 1 --2- 172.21.15.73:0/1898736339 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f624c15f3a0 0x7f624c15c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:33.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.403+0000 7f624b7fe640 1 -- 172.21.15.73:0/1898736339 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f624c0ba000 con 0x7f624c0d2130 2024-04-26T20:51:33.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.403+0000 7f6250f15640 1 --2- 172.21.15.73:0/1898736339 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f624c15f3a0 0x7f624c15c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:33.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.403+0000 7f624bfff640 1 --2- 172.21.15.73:0/1898736339 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f624c0d1770 0x7f624c0b98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:33.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.403+0000 7f624b7fe640 1 --2- 172.21.15.73:0/1898736339 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f624c0d2130 0x7f624c158f00 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f623c009870 tx=0x7f623c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:33.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.403+0000 7f6248ff9640 1 -- 172.21.15.73:0/1898736339 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623c041070 con 0x7f624c0d2130 2024-04-26T20:51:33.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.403+0000 7f6248ff9640 1 -- 172.21.15.73:0/1898736339 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f623c03c870 con 0x7f624c0d2130 2024-04-26T20:51:33.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.403+0000 7f6252978640 1 -- 172.21.15.73:0/1898736339 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f624c0ba2e0 con 0x7f624c0d2130 2024-04-26T20:51:33.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.403+0000 7f6248ff9640 1 -- 172.21.15.73:0/1898736339 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623c037040 con 0x7f624c0d2130 2024-04-26T20:51:33.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.403+0000 7f6252978640 1 -- 172.21.15.73:0/1898736339 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f624c0b7e60 con 0x7f624c0d2130 2024-04-26T20:51:33.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.404+0000 7f6252978640 1 -- 172.21.15.73:0/1898736339 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f624c0694f0 con 0x7f624c0d2130 2024-04-26T20:51:33.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.405+0000 7f6248ff9640 1 -- 172.21.15.73:0/1898736339 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f623c044070 con 0x7f624c0d2130 2024-04-26T20:51:33.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.405+0000 7f6248ff9640 1 --2- 172.21.15.73:0/1898736339 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f623806c8f0 0x7f623806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:33.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.405+0000 7f6248ff9640 1 -- 172.21.15.73:0/1898736339 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f623c0aca70 con 0x7f624c0d2130 2024-04-26T20:51:33.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.405+0000 7f624bfff640 1 --2- 172.21.15.73:0/1898736339 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f623806c8f0 0x7f623806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:33.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.406+0000 7f624bfff640 1 --2- 172.21.15.73:0/1898736339 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f623806c8f0 0x7f623806edb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f624000a870 tx=0x7f6240008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:33.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.410+0000 7f6248ff9640 1 -- 172.21.15.73:0/1898736339 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f624c0d25a0 con 0x7f624c0d2130 2024-04-26T20:51:33.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.624+0000 7f6252978640 1 -- 172.21.15.73:0/1898736339 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f624c0b6860 con 0x7f624c0d2130 2024-04-26T20:51:33.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.624+0000 7f6248ff9640 1 -- 172.21.15.73:0/1898736339 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5180 (secure 0 0 0) 0x7f623c07ac50 con 0x7f624c0d2130 2024-04-26T20:51:33.626 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:51:33.626 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/4263167680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6833","nonce":4263167680},{"type":"v1","addr":"172.21.15.120:6835","nonce":4263167680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4276,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6834/1614342510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":1614342510},{"type":"v1","addr":"172.21.15.120:6834","nonce":1614342510}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2146325725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2146325725},{"type":"v1","addr":"172.21.15.73:6835","nonce":2146325725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:50:31.964317+0000","modified":"2024-04-26T20:50:54.049029+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/181202544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":181202544},{"type":"v1","addr":"172.21.15.73:6837","nonce":181202544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4281,"qdb_cluster":[4281]},"id":1}]} 2024-04-26T20:51:33.626 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 8 2024-04-26T20:51:33.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.626+0000 7f6252978640 1 -- 172.21.15.73:0/1898736339 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f623806c8f0 msgr2=0x7f623806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:33.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.626+0000 7f6252978640 1 --2- 172.21.15.73:0/1898736339 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f623806c8f0 0x7f623806edb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f624000a870 tx=0x7f6240008040 comp rx=0 tx=0).stop 2024-04-26T20:51:33.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.626+0000 7f6252978640 1 -- 172.21.15.73:0/1898736339 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f624c0d2130 msgr2=0x7f624c158f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:33.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.626+0000 7f6252978640 1 --2- 172.21.15.73:0/1898736339 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f624c0d2130 0x7f624c158f00 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f623c009870 tx=0x7f623c0046c0 comp rx=0 tx=0).stop 2024-04-26T20:51:33.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.626+0000 7f6252978640 1 -- 172.21.15.73:0/1898736339 shutdown_connections 2024-04-26T20:51:33.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.626+0000 7f6252978640 1 --2- 172.21.15.73:0/1898736339 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f624c0d2130 0x7f624c158f00 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:33.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.626+0000 7f6252978640 1 --2- 172.21.15.73:0/1898736339 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f623806c8f0 0x7f623806edb0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:33.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.626+0000 7f6252978640 1 --2- 172.21.15.73:0/1898736339 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f624c15f3a0 0x7f624c15c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:33.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.626+0000 7f6252978640 1 --2- 172.21.15.73:0/1898736339 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f624c0d1770 0x7f624c0b98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:33.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.626+0000 7f6252978640 1 -- 172.21.15.73:0/1898736339 >> 172.21.15.73:0/1898736339 conn(0x7f624c0d4f90 msgr2=0x7f624c06e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:33.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.626+0000 7f6252978640 1 -- 172.21.15.73:0/1898736339 shutdown_connections 2024-04-26T20:51:33.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.626+0000 7f6252978640 1 -- 172.21.15.73:0/1898736339 wait complete. 2024-04-26T20:51:33.638 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:51:33.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.735+0000 7f090f007640 1 Processor -- start 2024-04-26T20:51:33.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.735+0000 7f090f007640 1 -- start start 2024-04-26T20:51:33.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.736+0000 7f090f007640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f09080b2340 0x7f09080cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:33.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.736+0000 7f090f007640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f09080cd930 0x7f09080d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:33.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.736+0000 7f090f007640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09080d10d0 con 0x7f0908162810 2024-04-26T20:51:33.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.736+0000 7f090f007640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f09080d0750 con 0x7f09080cd930 2024-04-26T20:51:33.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.736+0000 7f090f007640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f09080d16a0 con 0x7f09080b2340 2024-04-26T20:51:33.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.736+0000 7f090d5a4640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f09080cd930 0x7f09080d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:33.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.736+0000 7f090cda3640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f0908162810 0x7f0908070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40760/0 (socket says 172.21.15.73:40760) 2024-04-26T20:51:33.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.736+0000 7f090cda3640 1 -- 172.21.15.73:0/3945403038 learned_addr learned my addr 172.21.15.73:0/3945403038 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:33.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.736+0000 7f0907fff640 1 --2- 172.21.15.73:0/3945403038 >> v2:172.21.15.120:3300/0 conn(0x7f09080b2340 0x7f09080cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:33.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.736+0000 7f090d5a4640 1 -- 172.21.15.73:0/3945403038 >> v2:172.21.15.120:3300/0 conn(0x7f09080b2340 msgr2=0x7f09080cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:33.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.736+0000 7f090d5a4640 1 --2- 172.21.15.73:0/3945403038 >> v2:172.21.15.120:3300/0 conn(0x7f09080b2340 0x7f09080cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:33.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.736+0000 7f090d5a4640 1 -- 172.21.15.73:0/3945403038 >> v1:172.21.15.73:6789/0 conn(0x7f0908162810 legacy=0x7f0908070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:33.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.736+0000 7f090d5a4640 1 -- 172.21.15.73:0/3945403038 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09080d1370 con 0x7f09080cd930 2024-04-26T20:51:33.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.737+0000 7f090d5a4640 1 --2- 172.21.15.73:0/3945403038 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09080cd930 0x7f09080d1b30 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f09080cd0a0 tx=0x7f090002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebc8c1b5e32be49a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:33.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.737+0000 7f09077fe640 1 -- 172.21.15.73:0/3945403038 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0900007740 con 0x7f09080cd930 2024-04-26T20:51:33.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.737+0000 7f09077fe640 1 -- 172.21.15.73:0/3945403038 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09000078e0 con 0x7f09080cd930 2024-04-26T20:51:33.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.737+0000 7f09077fe640 1 -- 172.21.15.73:0/3945403038 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0900007cb0 con 0x7f09080cd930 2024-04-26T20:51:33.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.737+0000 7f090f007640 1 -- 172.21.15.73:0/3945403038 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09080cd930 msgr2=0x7f09080d1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:33.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.737+0000 7f090f007640 1 --2- 172.21.15.73:0/3945403038 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09080cd930 0x7f09080d1b30 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f09080cd0a0 tx=0x7f090002a680 comp rx=0 tx=0).stop 2024-04-26T20:51:33.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.737+0000 7f090f007640 1 -- 172.21.15.73:0/3945403038 shutdown_connections 2024-04-26T20:51:33.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.737+0000 7f090f007640 1 --2- 172.21.15.73:0/3945403038 >> v2:172.21.15.120:3300/0 conn(0x7f09080b2340 0x7f09080cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:33.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.737+0000 7f090f007640 1 --2- 172.21.15.73:0/3945403038 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09080cd930 0x7f09080d1b30 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:33.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.737+0000 7f090f007640 1 -- 172.21.15.73:0/3945403038 >> 172.21.15.73:0/3945403038 conn(0x7f09080d2da0 msgr2=0x7f09080d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:33.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.737+0000 7f090f007640 1 -- 172.21.15.73:0/3945403038 shutdown_connections 2024-04-26T20:51:33.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.737+0000 7f090f007640 1 -- 172.21.15.73:0/3945403038 wait complete. 2024-04-26T20:51:33.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.739+0000 7f090f007640 1 Processor -- start 2024-04-26T20:51:33.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.739+0000 7f090f007640 1 -- start start 2024-04-26T20:51:33.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.739+0000 7f090f007640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09080b2340 0x7f09081f4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:33.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.739+0000 7f090f007640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f09080cd930 0x7f0908160f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:33.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.739+0000 7f090f007640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0908162810 0x7f090815e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:33.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.739+0000 7f090f007640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f09080d2070 con 0x7f09080b2340 2024-04-26T20:51:33.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.739+0000 7f090f007640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f09080d0470 con 0x7f09080cd930 2024-04-26T20:51:33.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.739+0000 7f090f007640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f09080d21f0 con 0x7f0908162810 2024-04-26T20:51:33.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.740+0000 7f090cda3640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09080b2340 0x7f09081f4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:33.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.740+0000 7f090cda3640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09080b2340 0x7f09081f4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50216/0 (socket says 172.21.15.73:50216) 2024-04-26T20:51:33.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.740+0000 7f090cda3640 1 -- 172.21.15.73:0/3066640308 learned_addr learned my addr 172.21.15.73:0/3066640308 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:33.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.740+0000 7f090d5a4640 1 --2- 172.21.15.73:0/3066640308 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0908162810 0x7f090815e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:33.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.740+0000 7f0907fff640 1 --2- 172.21.15.73:0/3066640308 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f09080cd930 0x7f0908160f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:33.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.740+0000 7f090cda3640 1 -- 172.21.15.73:0/3066640308 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0908162810 msgr2=0x7f090815e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:33.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.740+0000 7f090cda3640 1 --2- 172.21.15.73:0/3066640308 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0908162810 0x7f090815e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:33.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.740+0000 7f090cda3640 1 -- 172.21.15.73:0/3066640308 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f09080cd930 msgr2=0x7f0908160f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:33.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.740+0000 7f090cda3640 1 --2- 172.21.15.73:0/3066640308 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f09080cd930 0x7f0908160f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:33.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.740+0000 7f090cda3640 1 -- 172.21.15.73:0/3066640308 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0908158020 con 0x7f09080b2340 2024-04-26T20:51:33.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.740+0000 7f090d5a4640 1 --2- 172.21.15.73:0/3066640308 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0908162810 0x7f090815e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:33.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.740+0000 7f090cda3640 1 --2- 172.21.15.73:0/3066640308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09080b2340 0x7f09081f4e10 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f08fc00c3c0 tx=0x7f08fc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:33.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.740+0000 7f09057fa640 1 -- 172.21.15.73:0/3066640308 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08fc017070 con 0x7f09080b2340 2024-04-26T20:51:33.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.740+0000 7f09057fa640 1 -- 172.21.15.73:0/3066640308 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08fc005ce0 con 0x7f09080b2340 2024-04-26T20:51:33.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.740+0000 7f090f007640 1 -- 172.21.15.73:0/3066640308 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0908158300 con 0x7f09080b2340 2024-04-26T20:51:33.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.740+0000 7f09057fa640 1 -- 172.21.15.73:0/3066640308 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08fc0072f0 con 0x7f09080b2340 2024-04-26T20:51:33.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.740+0000 7f090f007640 1 -- 172.21.15.73:0/3066640308 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09081588c0 con 0x7f09080b2340 2024-04-26T20:51:33.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.741+0000 7f090f007640 1 -- 172.21.15.73:0/3066640308 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0908075d40 con 0x7f09080b2340 2024-04-26T20:51:33.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.742+0000 7f09057fa640 1 -- 172.21.15.73:0/3066640308 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f08fc01a070 con 0x7f09080b2340 2024-04-26T20:51:33.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.742+0000 7f09057fa640 1 --2- 172.21.15.73:0/3066640308 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f08ec06c8f0 0x7f08ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:33.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.742+0000 7f09057fa640 1 -- 172.21.15.73:0/3066640308 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f08fc08b310 con 0x7f09080b2340 2024-04-26T20:51:33.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.742+0000 7f0907fff640 1 --2- 172.21.15.73:0/3066640308 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f08ec06c8f0 0x7f08ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:33.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.742+0000 7f0907fff640 1 --2- 172.21.15.73:0/3066640308 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f08ec06c8f0 0x7f08ec06edb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f09080b31c0 tx=0x7f08f4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:33.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.746+0000 7f09057fa640 1 -- 172.21.15.73:0/3066640308 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0908075d40 con 0x7f09080b2340 2024-04-26T20:51:33.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.927+0000 7f090f007640 1 -- 172.21.15.73:0/3066640308 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f09080694f0 con 0x7f09080b2340 2024-04-26T20:51:33.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.928+0000 7f09057fa640 1 -- 172.21.15.73:0/3066640308 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18902 (secure 0 0 0) 0x7f08fc059340 con 0x7f09080b2340 2024-04-26T20:51:33.929 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:51:33.929 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":27,"fsid":"fbd3f548-b547-4713-856b-09f98f5d1764","created":"2024-04-26T20:50:07.876786+0000","modified":"2024-04-26T20:51:13.783133+0000","last_up_change":"2024-04-26T20:50:19.082741+0000","last_in_change":"2024-04-26T20:50:10.018400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:50:19.993731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:50:29.964420+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:50:30.920082+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T20:50:34.650207+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"72d6f4f3-214a-4392-a02f-c33cbcab3c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6809","nonce":3461705124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6811","nonce":3461705124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6815","nonce":3461705124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6813","nonce":3461705124}]},"public_addr":"172.21.15.73:6809/3461705124","cluster_addr":"172.21.15.73:6811/3461705124","heartbeat_back_addr":"172.21.15.73:6815/3461705124","heartbeat_front_addr":"172.21.15.73:6813/3461705124","state":["exists","up"]},{"osd":1,"uuid":"786bf8ca-6ddc-4b3c-be6c-9152398bd5fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6817","nonce":2827619403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6819","nonce":2827619403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6825","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6827","nonce":2827619403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6821","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6823","nonce":2827619403}]},"public_addr":"172.21.15.73:6817/2827619403","cluster_addr":"172.21.15.73:6819/2827619403","heartbeat_back_addr":"172.21.15.73:6827/2827619403","heartbeat_front_addr":"172.21.15.73:6823/2827619403","state":["exists","up"]},{"osd":2,"uuid":"90963b1a-5437-4ef8-ad8b-6b74c534eeb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6801","nonce":2542681064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6803","nonce":2542681064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6808","nonce":2542681064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6805","nonce":2542681064}]},"public_addr":"172.21.15.73:6801/2542681064","cluster_addr":"172.21.15.73:6803/2542681064","heartbeat_back_addr":"172.21.15.73:6808/2542681064","heartbeat_front_addr":"172.21.15.73:6805/2542681064","state":["exists","up"]},{"osd":3,"uuid":"bd3d2148-6e38-41b4-a453-fe6e3b8b4ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6822","nonce":1604042718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6826","nonce":1604042718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6831","nonce":1604042718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6829","nonce":1604042718}]},"public_addr":"172.21.15.73:6822/1604042718","cluster_addr":"172.21.15.73:6826/1604042718","heartbeat_back_addr":"172.21.15.73:6831/1604042718","heartbeat_front_addr":"172.21.15.73:6829/1604042718","state":["exists","up"]},{"osd":4,"uuid":"e8668dc8-a23f-46d2-8311-cb882da08e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6817","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6819","nonce":3591055561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6824","nonce":3591055561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6831","nonce":3591055561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6828","nonce":3591055561}]},"public_addr":"172.21.15.120:6819/3591055561","cluster_addr":"172.21.15.120:6824/3591055561","heartbeat_back_addr":"172.21.15.120:6831/3591055561","heartbeat_front_addr":"172.21.15.120:6828/3591055561","state":["exists","up"]},{"osd":5,"uuid":"cbac0bdb-96bd-4852-bcce-e564495c3fcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6809","nonce":819450433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6811","nonce":819450433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6815","nonce":819450433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6813","nonce":819450433}]},"public_addr":"172.21.15.120:6809/819450433","cluster_addr":"172.21.15.120:6811/819450433","heartbeat_back_addr":"172.21.15.120:6815/819450433","heartbeat_front_addr":"172.21.15.120:6813/819450433","state":["exists","up"]},{"osd":6,"uuid":"1884b2a8-569f-4272-96a9-f25a09364dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6801","nonce":3946210727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6803","nonce":3946210727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6807","nonce":3946210727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6805","nonce":3946210727}]},"public_addr":"172.21.15.120:6801/3946210727","cluster_addr":"172.21.15.120:6803/3946210727","heartbeat_back_addr":"172.21.15.120:6807/3946210727","heartbeat_front_addr":"172.21.15.120:6805/3946210727","state":["exists","up"]},{"osd":7,"uuid":"c0c3bf6d-a4c5-4f1e-aa2d-330951830666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6818","nonce":2915574674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6821","nonce":2915574674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6827","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6829","nonce":2915574674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6823","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6825","nonce":2915574674}]},"public_addr":"172.21.15.120:6818/2915574674","cluster_addr":"172.21.15.120:6821/2915574674","heartbeat_back_addr":"172.21.15.120:6829/2915574674","heartbeat_front_addr":"172.21.15.120:6825/2915574674","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.218216+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.208710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.112705+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.182789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.164324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.174212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.225396+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.261559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:51:33.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.930+0000 7f090f007640 1 -- 172.21.15.73:0/3066640308 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f08ec06c8f0 msgr2=0x7f08ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:33.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.930+0000 7f090f007640 1 --2- 172.21.15.73:0/3066640308 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f08ec06c8f0 0x7f08ec06edb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f09080b31c0 tx=0x7f08f4008040 comp rx=0 tx=0).stop 2024-04-26T20:51:33.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.930+0000 7f090f007640 1 -- 172.21.15.73:0/3066640308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09080b2340 msgr2=0x7f09081f4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:33.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.930+0000 7f090f007640 1 --2- 172.21.15.73:0/3066640308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09080b2340 0x7f09081f4e10 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f08fc00c3c0 tx=0x7f08fc00c880 comp rx=0 tx=0).stop 2024-04-26T20:51:33.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.930+0000 7f090f007640 1 -- 172.21.15.73:0/3066640308 shutdown_connections 2024-04-26T20:51:33.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.930+0000 7f090f007640 1 --2- 172.21.15.73:0/3066640308 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f08ec06c8f0 0x7f08ec06edb0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:33.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.930+0000 7f090f007640 1 --2- 172.21.15.73:0/3066640308 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0908162810 0x7f090815e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:33.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.930+0000 7f090f007640 1 --2- 172.21.15.73:0/3066640308 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f09080cd930 0x7f0908160f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:33.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.930+0000 7f090f007640 1 --2- 172.21.15.73:0/3066640308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09080b2340 0x7f09081f4e10 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:33.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.930+0000 7f090f007640 1 -- 172.21.15.73:0/3066640308 >> 172.21.15.73:0/3066640308 conn(0x7f09080d2da0 msgr2=0x7f09080b4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:33.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.930+0000 7f090f007640 1 -- 172.21.15.73:0/3066640308 shutdown_connections 2024-04-26T20:51:33.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:33.930+0000 7f090f007640 1 -- 172.21.15.73:0/3066640308 wait complete. 2024-04-26T20:51:33.943 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T20:51:34.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.040+0000 7f0ac6409640 1 Processor -- start 2024-04-26T20:51:34.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.040+0000 7f0ac6409640 1 -- start start 2024-04-26T20:51:34.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.041+0000 7f0ac6409640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0ac006c950 0x7f0ac015d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:34.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.041+0000 7f0ac6409640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ac015af40 con 0x7f0ac015ab60 2024-04-26T20:51:34.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.041+0000 7f0ac6409640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ac015e320 con 0x7f0ac01617f0 2024-04-26T20:51:34.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.041+0000 7f0ac6409640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f0ac0160d10 con 0x7f0ac006c950 2024-04-26T20:51:34.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.041+0000 7f0abffff640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f0ac015ab60 0x7f0ac006c240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40770/0 (socket says 172.21.15.73:40770) 2024-04-26T20:51:34.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.041+0000 7f0abffff640 1 -- 172.21.15.73:0/2457237410 learned_addr learned my addr 172.21.15.73:0/2457237410 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:34.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.041+0000 7f0abf7fe640 1 --2- 172.21.15.73:0/2457237410 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0ac006c950 0x7f0ac015d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:34.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.041+0000 7f0abeffd640 1 -- 172.21.15.73:0/2457237410 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 847746456 0 0) 0x7f0ac015af40 con 0x7f0ac015ab60 2024-04-26T20:51:34.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.042+0000 7f0abeffd640 1 -- 172.21.15.73:0/2457237410 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0aa8003520 con 0x7f0ac015ab60 2024-04-26T20:51:34.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.042+0000 7f0abeffd640 1 -- 172.21.15.73:0/2457237410 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4156515337 0 0) 0x7f0ac015e320 con 0x7f0ac01617f0 2024-04-26T20:51:34.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.042+0000 7f0abeffd640 1 -- 172.21.15.73:0/2457237410 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ac015af40 con 0x7f0ac01617f0 2024-04-26T20:51:34.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.042+0000 7f0abeffd640 1 -- 172.21.15.73:0/2457237410 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2656318761 0 0) 0x7f0aa8003520 con 0x7f0ac015ab60 2024-04-26T20:51:34.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.042+0000 7f0abeffd640 1 -- 172.21.15.73:0/2457237410 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0ac006c950 msgr2=0x7f0ac015d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:34.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.042+0000 7f0abeffd640 1 --2- 172.21.15.73:0/2457237410 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0ac006c950 0x7f0ac015d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:34.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.042+0000 7f0abeffd640 1 -- 172.21.15.73:0/2457237410 >> v1:172.21.15.120:6789/0 conn(0x7f0ac01617f0 legacy=0x7f0ac015db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:34.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.042+0000 7f0abeffd640 1 -- 172.21.15.73:0/2457237410 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ac0160ff0 con 0x7f0ac015ab60 2024-04-26T20:51:34.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.042+0000 7f0abeffd640 1 -- 172.21.15.73:0/2457237410 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7f0ab0001670 con 0x7f0ac015ab60 2024-04-26T20:51:34.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.042+0000 7f0abeffd640 1 -- 172.21.15.73:0/2457237410 >> v1:172.21.15.73:6789/0 conn(0x7f0ac015ab60 legacy=0x7f0ac006c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:34.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.042+0000 7f0abeffd640 1 --2- 172.21.15.73:0/2457237410 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0aa8003b70 0x7f0aa8003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:34.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.042+0000 7f0abeffd640 1 --2- 172.21.15.73:0/2457237410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0aa8004800 0x7f0aa8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:34.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.042+0000 7f0abeffd640 1 --2- 172.21.15.73:0/2457237410 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0aa80051c0 0x7f0aa800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:34.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.042+0000 7f0abeffd640 1 -- 172.21.15.73:0/2457237410 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac0160d10 con 0x7f0aa8003b70 2024-04-26T20:51:34.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.042+0000 7f0abeffd640 1 -- 172.21.15.73:0/2457237410 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa800bfb0 con 0x7f0aa8004800 2024-04-26T20:51:34.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.042+0000 7f0abeffd640 1 -- 172.21.15.73:0/2457237410 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f0aa800c1b0 con 0x7f0aa80051c0 2024-04-26T20:51:34.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.042+0000 7f0abf7fe640 1 --2- 172.21.15.73:0/2457237410 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0ac006c950 0x7f0ac015d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:51:34.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.042+0000 7f0abffff640 1 --2- 172.21.15.73:0/2457237410 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0aa8003b70 0x7f0aa8003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:34.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.042+0000 7f0ac49a6640 1 --2- 172.21.15.73:0/2457237410 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0aa80051c0 0x7f0aa800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:34.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.042+0000 7f0abf7fe640 1 --2- 172.21.15.73:0/2457237410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0aa8004800 0x7f0aa8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:34.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.042+0000 7f0abffff640 1 -- 172.21.15.73:0/2457237410 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0aa80051c0 msgr2=0x7f0aa800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:34.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.042+0000 7f0abffff640 1 --2- 172.21.15.73:0/2457237410 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0aa80051c0 0x7f0aa800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:34.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.042+0000 7f0abffff640 1 -- 172.21.15.73:0/2457237410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0aa8004800 msgr2=0x7f0aa8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:34.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.042+0000 7f0abffff640 1 --2- 172.21.15.73:0/2457237410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0aa8004800 0x7f0aa8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:34.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.042+0000 7f0abffff640 1 -- 172.21.15.73:0/2457237410 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aa8003520 con 0x7f0aa8003b70 2024-04-26T20:51:34.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.042+0000 7f0abf7fe640 1 --2- 172.21.15.73:0/2457237410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0aa8004800 0x7f0aa8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:34.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.043+0000 7f0abffff640 1 --2- 172.21.15.73:0/2457237410 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0aa8003b70 0x7f0aa8003f70 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0ab000c160 tx=0x7f0aa8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2fb5fa3de9e5cc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:34.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.043+0000 7f0abeffd640 1 -- 172.21.15.73:0/2457237410 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab0007ca0 con 0x7f0aa8003b70 2024-04-26T20:51:34.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.043+0000 7f0abeffd640 1 -- 172.21.15.73:0/2457237410 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ab0005af0 con 0x7f0aa8003b70 2024-04-26T20:51:34.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.043+0000 7f0abeffd640 1 -- 172.21.15.73:0/2457237410 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab0007070 con 0x7f0aa8003b70 2024-04-26T20:51:34.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.043+0000 7f0ac6409640 1 -- 172.21.15.73:0/2457237410 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0aa8003b70 msgr2=0x7f0aa8003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:34.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.043+0000 7f0ac6409640 1 --2- 172.21.15.73:0/2457237410 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0aa8003b70 0x7f0aa8003f70 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0ab000c160 tx=0x7f0aa8005830 comp rx=0 tx=0).stop 2024-04-26T20:51:34.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.043+0000 7f0ac49a6640 1 -- 172.21.15.73:0/2457237410 reap_dead start 2024-04-26T20:51:34.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.044+0000 7f0ac6409640 1 -- 172.21.15.73:0/2457237410 shutdown_connections 2024-04-26T20:51:34.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.044+0000 7f0ac6409640 1 -- 172.21.15.73:0/2457237410 >> 172.21.15.73:0/2457237410 conn(0x7f0ac00d1cf0 msgr2=0x7f0ac00d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:34.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.044+0000 7f0ac6409640 1 -- 172.21.15.73:0/2457237410 shutdown_connections 2024-04-26T20:51:34.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.044+0000 7f0ac6409640 1 -- 172.21.15.73:0/2457237410 wait complete. 2024-04-26T20:51:34.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.045+0000 7f0ac6409640 1 Processor -- start 2024-04-26T20:51:34.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.045+0000 7f0ac6409640 1 -- start start 2024-04-26T20:51:34.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.046+0000 7f0ac6409640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0aa8003b70 0x7f0ac00d0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:34.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.046+0000 7f0ac6409640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0ac00b5300 0x7f0ac00ae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:34.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.046+0000 7f0ac6409640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ac00b6c80 0x7f0ac00b7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:34.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.046+0000 7f0ac6409640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac015d770 con 0x7f0ac00b6c80 2024-04-26T20:51:34.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.046+0000 7f0ac6409640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac015d5f0 con 0x7f0ac00b5300 2024-04-26T20:51:34.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.046+0000 7f0ac6409640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f0ac015d8f0 con 0x7f0aa8003b70 2024-04-26T20:51:34.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.046+0000 7f0ac49a6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ac00b6c80 0x7f0ac00b7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:34.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.046+0000 7f0ac49a6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ac00b6c80 0x7f0ac00b7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50228/0 (socket says 172.21.15.73:50228) 2024-04-26T20:51:34.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.046+0000 7f0ac49a6640 1 -- 172.21.15.73:0/1851481247 learned_addr learned my addr 172.21.15.73:0/1851481247 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:34.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.046+0000 7f0abf7fe640 1 --2- 172.21.15.73:0/1851481247 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0ac00b5300 0x7f0ac00ae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:34.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.046+0000 7f0abffff640 1 --2- 172.21.15.73:0/1851481247 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0aa8003b70 0x7f0ac00d0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:34.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.046+0000 7f0ac49a6640 1 -- 172.21.15.73:0/1851481247 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0aa8003b70 msgr2=0x7f0ac00d0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:34.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.046+0000 7f0ac49a6640 1 --2- 172.21.15.73:0/1851481247 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0aa8003b70 0x7f0ac00d0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:34.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.046+0000 7f0ac49a6640 1 -- 172.21.15.73:0/1851481247 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0ac00b5300 msgr2=0x7f0ac00ae8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:34.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.046+0000 7f0ac49a6640 1 --2- 172.21.15.73:0/1851481247 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0ac00b5300 0x7f0ac00ae8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:34.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.046+0000 7f0ac49a6640 1 -- 172.21.15.73:0/1851481247 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ac00b5be0 con 0x7f0ac00b6c80 2024-04-26T20:51:34.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.046+0000 7f0abf7fe640 1 --2- 172.21.15.73:0/1851481247 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0ac00b5300 0x7f0ac00ae8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:34.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.046+0000 7f0abffff640 1 --2- 172.21.15.73:0/1851481247 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0aa8003b70 0x7f0ac00d0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:34.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.046+0000 7f0ac49a6640 1 --2- 172.21.15.73:0/1851481247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ac00b6c80 0x7f0ac00b7110 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0161c20 tx=0x7f0ab800ff70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:34.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.046+0000 7f0abcff9640 1 -- 172.21.15.73:0/1851481247 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab8019070 con 0x7f0ac00b6c80 2024-04-26T20:51:34.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.046+0000 7f0abcff9640 1 -- 172.21.15.73:0/1851481247 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ab8006e20 con 0x7f0ac00b6c80 2024-04-26T20:51:34.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.046+0000 7f0abcff9640 1 -- 172.21.15.73:0/1851481247 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab8003bb0 con 0x7f0ac00b6c80 2024-04-26T20:51:34.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.046+0000 7f0ac6409640 1 -- 172.21.15.73:0/1851481247 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ac00b5ec0 con 0x7f0ac00b6c80 2024-04-26T20:51:34.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.046+0000 7f0ac6409640 1 -- 172.21.15.73:0/1851481247 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ac00b6480 con 0x7f0ac00b6c80 2024-04-26T20:51:34.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.047+0000 7f0ac6409640 1 -- 172.21.15.73:0/1851481247 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ac00694f0 con 0x7f0ac00b6c80 2024-04-26T20:51:34.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.048+0000 7f0abcff9640 1 -- 172.21.15.73:0/1851481247 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0ab801c070 con 0x7f0ac00b6c80 2024-04-26T20:51:34.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.048+0000 7f0abcff9640 1 --2- 172.21.15.73:0/1851481247 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f0aac06c8f0 0x7f0aac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:34.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.048+0000 7f0abcff9640 1 -- 172.21.15.73:0/1851481247 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f0ab808b2b0 con 0x7f0ac00b6c80 2024-04-26T20:51:34.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.048+0000 7f0abffff640 1 --2- 172.21.15.73:0/1851481247 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f0aac06c8f0 0x7f0aac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:34.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.048+0000 7f0abffff640 1 --2- 172.21.15.73:0/1851481247 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f0aac06c8f0 0x7f0aac06edb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0007fd0 tx=0x7f0ab0009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:34.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.052+0000 7f0abcff9640 1 -- 172.21.15.73:0/1851481247 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0ab8059370 con 0x7f0ac00b6c80 2024-04-26T20:51:34.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.263+0000 7f0ac6409640 1 -- 172.21.15.73:0/1851481247 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f0ac00ce060 con 0x7f0ac00b6c80 2024-04-26T20:51:34.265 INFO:tasks.ceph.mon.a.smithi073.stderr:2024-04-26T20:51:34.264+0000 7f712aaa2640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T20:51:34.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.266+0000 7f0abcff9640 1 -- 172.21.15.73:0/1851481247 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7f0ab805f030 con 0x7f0ac00b6c80 2024-04-26T20:51:34.267 INFO:teuthology.orchestra.run.smithi073.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T20:51:34.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.268+0000 7f0ac6409640 1 -- 172.21.15.73:0/1851481247 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f0aac06c8f0 msgr2=0x7f0aac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:34.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.268+0000 7f0ac6409640 1 --2- 172.21.15.73:0/1851481247 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f0aac06c8f0 0x7f0aac06edb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0007fd0 tx=0x7f0ab0009030 comp rx=0 tx=0).stop 2024-04-26T20:51:34.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.268+0000 7f0ac6409640 1 -- 172.21.15.73:0/1851481247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ac00b6c80 msgr2=0x7f0ac00b7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:34.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.268+0000 7f0ac6409640 1 --2- 172.21.15.73:0/1851481247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ac00b6c80 0x7f0ac00b7110 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0161c20 tx=0x7f0ab800ff70 comp rx=0 tx=0).stop 2024-04-26T20:51:34.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.268+0000 7f0ac6409640 1 -- 172.21.15.73:0/1851481247 shutdown_connections 2024-04-26T20:51:34.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.268+0000 7f0ac6409640 1 --2- 172.21.15.73:0/1851481247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ac00b6c80 0x7f0ac00b7110 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:34.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.268+0000 7f0ac6409640 1 --2- 172.21.15.73:0/1851481247 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f0aac06c8f0 0x7f0aac06edb0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:34.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.268+0000 7f0ac6409640 1 --2- 172.21.15.73:0/1851481247 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0ac00b5300 0x7f0ac00ae8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:34.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.268+0000 7f0ac6409640 1 --2- 172.21.15.73:0/1851481247 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0aa8003b70 0x7f0ac00d0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:34.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.268+0000 7f0ac6409640 1 -- 172.21.15.73:0/1851481247 >> 172.21.15.73:0/1851481247 conn(0x7f0ac00d1cf0 msgr2=0x7f0ac015b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:34.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.268+0000 7f0ac6409640 1 -- 172.21.15.73:0/1851481247 shutdown_connections 2024-04-26T20:51:34.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.268+0000 7f0ac6409640 1 -- 172.21.15.73:0/1851481247 wait complete. 2024-04-26T20:51:34.272 INFO:tasks.ceph.mds.c.smithi073.stderr: -5408> 2024-04-26T20:50:36.571+0000 7fcfebb5b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T20:51:34.281 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T20:51:34.302 INFO:tasks.ceph.mds.c.smithi073.stdout:starting mds.c at 2024-04-26T20:51:34.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.380+0000 7ff7ee483640 1 Processor -- start 2024-04-26T20:51:34.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.380+0000 7ff7ee483640 1 -- start start 2024-04-26T20:51:34.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.381+0000 7ff7ee483640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff7e815ab60 0x7ff7e806c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:34.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.381+0000 7ff7ee483640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7e81617f0 0x7ff7e806c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:34.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.381+0000 7ff7ee483640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7ff7e815d5f0 0x7ff7e815daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:34.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.381+0000 7ff7ee483640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7ff7e815af40 con 0x7ff7e815ab60 2024-04-26T20:51:34.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.381+0000 7ff7ee483640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7ff7e815e320 con 0x7ff7e815d5f0 2024-04-26T20:51:34.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.381+0000 7ff7ee483640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7ff7e8160d10 con 0x7ff7e81617f0 2024-04-26T20:51:34.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.381+0000 7ff7e7fff640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff7e815ab60 0x7ff7e806c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:34.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.381+0000 7ff7e7fff640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff7e815ab60 0x7ff7e806c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50250/0 (socket says 172.21.15.73:50250) 2024-04-26T20:51:34.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.381+0000 7ff7e7fff640 1 -- 172.21.15.73:0/2011731838 learned_addr learned my addr 172.21.15.73:0/2011731838 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:34.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.381+0000 7ff7eca20640 1 --2- 172.21.15.73:0/2011731838 >> v2:172.21.15.120:3300/0 conn(0x7ff7e815d5f0 0x7ff7e815daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:34.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.381+0000 7ff7e77fe640 1 --2- 172.21.15.73:0/2011731838 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7e81617f0 0x7ff7e806c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:34.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.381+0000 7ff7e7fff640 1 -- 172.21.15.73:0/2011731838 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7e81617f0 msgr2=0x7ff7e806c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:34.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.381+0000 7ff7e7fff640 1 --2- 172.21.15.73:0/2011731838 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7e81617f0 0x7ff7e806c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:34.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.381+0000 7ff7e7fff640 1 -- 172.21.15.73:0/2011731838 >> v2:172.21.15.120:3300/0 conn(0x7ff7e815d5f0 msgr2=0x7ff7e815daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:34.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.381+0000 7ff7e7fff640 1 --2- 172.21.15.73:0/2011731838 >> v2:172.21.15.120:3300/0 conn(0x7ff7e815d5f0 0x7ff7e815daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:34.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.381+0000 7ff7e7fff640 1 -- 172.21.15.73:0/2011731838 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7e8160ff0 con 0x7ff7e815ab60 2024-04-26T20:51:34.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.382+0000 7ff7e7fff640 1 --2- 172.21.15.73:0/2011731838 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7e815ab60 0x7ff7e806c240 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ff7e80c5620 tx=0x7ff7d802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61df0144681dd64a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:34.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.382+0000 7ff7e6ffd640 1 -- 172.21.15.73:0/2011731838 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d8037070 con 0x7ff7e815ab60 2024-04-26T20:51:34.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.382+0000 7ff7e6ffd640 1 -- 172.21.15.73:0/2011731838 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7d8032440 con 0x7ff7e815ab60 2024-04-26T20:51:34.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.382+0000 7ff7e6ffd640 1 -- 172.21.15.73:0/2011731838 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d8032810 con 0x7ff7e815ab60 2024-04-26T20:51:34.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.383+0000 7ff7ee483640 1 -- 172.21.15.73:0/2011731838 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7e815ab60 msgr2=0x7ff7e806c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:34.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.383+0000 7ff7ee483640 1 --2- 172.21.15.73:0/2011731838 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7e815ab60 0x7ff7e806c240 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ff7e80c5620 tx=0x7ff7d802a1e0 comp rx=0 tx=0).stop 2024-04-26T20:51:34.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.383+0000 7ff7ee483640 1 -- 172.21.15.73:0/2011731838 shutdown_connections 2024-04-26T20:51:34.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.383+0000 7ff7ee483640 1 --2- 172.21.15.73:0/2011731838 >> v2:172.21.15.120:3300/0 conn(0x7ff7e815d5f0 0x7ff7e815daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:34.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.383+0000 7ff7ee483640 1 --2- 172.21.15.73:0/2011731838 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7e81617f0 0x7ff7e806c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:34.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.383+0000 7ff7ee483640 1 --2- 172.21.15.73:0/2011731838 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7e815ab60 0x7ff7e806c240 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:34.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.383+0000 7ff7ee483640 1 -- 172.21.15.73:0/2011731838 >> 172.21.15.73:0/2011731838 conn(0x7ff7e80d1cf0 msgr2=0x7ff7e80d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:34.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.383+0000 7ff7ee483640 1 -- 172.21.15.73:0/2011731838 shutdown_connections 2024-04-26T20:51:34.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.383+0000 7ff7ee483640 1 -- 172.21.15.73:0/2011731838 wait complete. 2024-04-26T20:51:34.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.384+0000 7ff7ee483640 1 Processor -- start 2024-04-26T20:51:34.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.384+0000 7ff7ee483640 1 -- start start 2024-04-26T20:51:34.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.385+0000 7ff7ee483640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7e815ab60 0x7ff7e81f4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:34.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.385+0000 7ff7ee483640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7e815d5f0 0x7ff7e80cd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:34.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.385+0000 7ff7ee483640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7e81617f0 0x7ff7e80cd710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:34.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.385+0000 7ff7ee483640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e8072f70 con 0x7ff7e81617f0 2024-04-26T20:51:34.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.385+0000 7ff7ee483640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e815dfe0 con 0x7ff7e815d5f0 2024-04-26T20:51:34.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.385+0000 7ff7ee483640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7ff7e80730f0 con 0x7ff7e815ab60 2024-04-26T20:51:34.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.385+0000 7ff7eca20640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7e81617f0 0x7ff7e80cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:34.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.385+0000 7ff7eca20640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7e81617f0 0x7ff7e80cd710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50254/0 (socket says 172.21.15.73:50254) 2024-04-26T20:51:34.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.385+0000 7ff7eca20640 1 -- 172.21.15.73:0/2240946179 learned_addr learned my addr 172.21.15.73:0/2240946179 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:34.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.385+0000 7ff7e77fe640 1 --2- 172.21.15.73:0/2240946179 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7e815d5f0 0x7ff7e80cd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:34.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.385+0000 7ff7e7fff640 1 --2- 172.21.15.73:0/2240946179 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7e815ab60 0x7ff7e81f4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:34.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.385+0000 7ff7eca20640 1 -- 172.21.15.73:0/2240946179 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7e815ab60 msgr2=0x7ff7e81f4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:34.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.385+0000 7ff7eca20640 1 --2- 172.21.15.73:0/2240946179 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7e815ab60 0x7ff7e81f4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:34.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.385+0000 7ff7eca20640 1 -- 172.21.15.73:0/2240946179 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7e815d5f0 msgr2=0x7ff7e80cd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:34.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.385+0000 7ff7eca20640 1 --2- 172.21.15.73:0/2240946179 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7e815d5f0 0x7ff7e80cd1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:34.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.385+0000 7ff7eca20640 1 -- 172.21.15.73:0/2240946179 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7e81f5190 con 0x7ff7e81617f0 2024-04-26T20:51:34.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.385+0000 7ff7eca20640 1 --2- 172.21.15.73:0/2240946179 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7e81617f0 0x7ff7e80cd710 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0007e60 tx=0x7ff7e000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:34.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.386+0000 7ff7e4ff9640 1 -- 172.21.15.73:0/2240946179 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e0014620 con 0x7ff7e81617f0 2024-04-26T20:51:34.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.386+0000 7ff7e4ff9640 1 -- 172.21.15.73:0/2240946179 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7e0019070 con 0x7ff7e81617f0 2024-04-26T20:51:34.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.386+0000 7ff7ee483640 1 -- 172.21.15.73:0/2240946179 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7e80b6750 con 0x7ff7e81617f0 2024-04-26T20:51:34.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.386+0000 7ff7e4ff9640 1 -- 172.21.15.73:0/2240946179 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e0014a20 con 0x7ff7e81617f0 2024-04-26T20:51:34.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.386+0000 7ff7ee483640 1 -- 172.21.15.73:0/2240946179 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7e80b6d10 con 0x7ff7e81617f0 2024-04-26T20:51:34.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.387+0000 7ff7ee483640 1 -- 172.21.15.73:0/2240946179 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7e80694f0 con 0x7ff7e81617f0 2024-04-26T20:51:34.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.387+0000 7ff7e4ff9640 1 -- 172.21.15.73:0/2240946179 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff7e0014bc0 con 0x7ff7e81617f0 2024-04-26T20:51:34.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.388+0000 7ff7e4ff9640 1 --2- 172.21.15.73:0/2240946179 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff7d406c8f0 0x7ff7d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:34.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.388+0000 7ff7e7fff640 1 --2- 172.21.15.73:0/2240946179 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff7d406c8f0 0x7ff7d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:34.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.388+0000 7ff7e4ff9640 1 -- 172.21.15.73:0/2240946179 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6821+0+0 (secure 0 0 0) 0x7ff7e008c6b0 con 0x7ff7e81617f0 2024-04-26T20:51:34.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.388+0000 7ff7e7fff640 1 --2- 172.21.15.73:0/2240946179 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff7d406c8f0 0x7ff7d406edb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff7d8034000 tx=0x7ff7d802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:34.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.392+0000 7ff7e4ff9640 1 -- 172.21.15.73:0/2240946179 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff7e005a030 con 0x7ff7e81617f0 2024-04-26T20:51:34.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:34.602+0000 7ff7ee483640 1 -- 172.21.15.73:0/2240946179 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7ff7e815b3a0 con 0x7ff7e81617f0 2024-04-26T20:51:35.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.269+0000 7ff7e4ff9640 1 -- 172.21.15.73:0/2240946179 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7ff7e005e330 con 0x7ff7e81617f0 2024-04-26T20:51:35.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.271+0000 7ff7ee483640 1 -- 172.21.15.73:0/2240946179 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff7d406c8f0 msgr2=0x7ff7d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:35.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.271+0000 7ff7ee483640 1 --2- 172.21.15.73:0/2240946179 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff7d406c8f0 0x7ff7d406edb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff7d8034000 tx=0x7ff7d802e000 comp rx=0 tx=0).stop 2024-04-26T20:51:35.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.272+0000 7ff7ee483640 1 -- 172.21.15.73:0/2240946179 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7e81617f0 msgr2=0x7ff7e80cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:35.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.272+0000 7ff7ee483640 1 --2- 172.21.15.73:0/2240946179 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7e81617f0 0x7ff7e80cd710 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0007e60 tx=0x7ff7e000e4b0 comp rx=0 tx=0).stop 2024-04-26T20:51:35.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.272+0000 7ff7ee483640 1 -- 172.21.15.73:0/2240946179 shutdown_connections 2024-04-26T20:51:35.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.272+0000 7ff7ee483640 1 --2- 172.21.15.73:0/2240946179 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7e81617f0 0x7ff7e80cd710 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:35.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.272+0000 7ff7ee483640 1 --2- 172.21.15.73:0/2240946179 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff7d406c8f0 0x7ff7d406edb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:35.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.272+0000 7ff7ee483640 1 --2- 172.21.15.73:0/2240946179 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7e815d5f0 0x7ff7e80cd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:35.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.272+0000 7ff7ee483640 1 --2- 172.21.15.73:0/2240946179 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7e815ab60 0x7ff7e81f4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:35.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.272+0000 7ff7ee483640 1 -- 172.21.15.73:0/2240946179 >> 172.21.15.73:0/2240946179 conn(0x7ff7e80d1cf0 msgr2=0x7ff7e80b2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:35.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.272+0000 7ff7ee483640 1 -- 172.21.15.73:0/2240946179 shutdown_connections 2024-04-26T20:51:35.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.272+0000 7ff7ee483640 1 -- 172.21.15.73:0/2240946179 wait complete. 2024-04-26T20:51:35.286 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T20:51:35.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.432+0000 7ff6344d9640 1 Processor -- start 2024-04-26T20:51:35.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.432+0000 7ff6344d9640 1 -- start start 2024-04-26T20:51:35.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.433+0000 7ff6344d9640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff62c15d0d0 0x7ff62c0afb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:35.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.433+0000 7ff6344d9640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff62c0b0060 0x7ff62c0d2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:35.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.433+0000 7ff6344d9640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7ff62c0d32c0 0x7ff62c070970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:35.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.433+0000 7ff6344d9640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7ff62c0b1120 con 0x7ff62c15d0d0 2024-04-26T20:51:35.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.433+0000 7ff6344d9640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7ff62c0d36d0 con 0x7ff62c0d32c0 2024-04-26T20:51:35.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.433+0000 7ff6344d9640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7ff62c0b0510 con 0x7ff62c0b0060 2024-04-26T20:51:35.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.433+0000 7ff632275640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff62c15d0d0 0x7ff62c0afb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:35.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.434+0000 7ff632275640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff62c15d0d0 0x7ff62c0afb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50258/0 (socket says 172.21.15.73:50258) 2024-04-26T20:51:35.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.434+0000 7ff632275640 1 -- 172.21.15.73:0/162498946 learned_addr learned my addr 172.21.15.73:0/162498946 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:35.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.434+0000 7ff632a76640 1 --2- 172.21.15.73:0/162498946 >> v2:172.21.15.120:3300/0 conn(0x7ff62c0d32c0 0x7ff62c070970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:35.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.434+0000 7ff631a74640 1 --2- 172.21.15.73:0/162498946 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff62c0b0060 0x7ff62c0d2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:35.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.434+0000 7ff632275640 1 -- 172.21.15.73:0/162498946 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff62c0b0060 msgr2=0x7ff62c0d2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:35.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.434+0000 7ff632275640 1 --2- 172.21.15.73:0/162498946 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff62c0b0060 0x7ff62c0d2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:35.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.434+0000 7ff632275640 1 -- 172.21.15.73:0/162498946 >> v2:172.21.15.120:3300/0 conn(0x7ff62c0d32c0 msgr2=0x7ff62c070970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:35.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.434+0000 7ff632275640 1 --2- 172.21.15.73:0/162498946 >> v2:172.21.15.120:3300/0 conn(0x7ff62c0d32c0 0x7ff62c070970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:35.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.434+0000 7ff632275640 1 -- 172.21.15.73:0/162498946 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff62c0b07f0 con 0x7ff62c15d0d0 2024-04-26T20:51:35.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.434+0000 7ff632275640 1 --2- 172.21.15.73:0/162498946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff62c15d0d0 0x7ff62c0afb20 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff62c0c9e90 tx=0x7ff62802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b16a3e40269381ac server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:35.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.435+0000 7ff631273640 1 -- 172.21.15.73:0/162498946 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff628037070 con 0x7ff62c15d0d0 2024-04-26T20:51:35.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.435+0000 7ff631273640 1 -- 172.21.15.73:0/162498946 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff628032440 con 0x7ff62c15d0d0 2024-04-26T20:51:35.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.435+0000 7ff631273640 1 -- 172.21.15.73:0/162498946 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff628032810 con 0x7ff62c15d0d0 2024-04-26T20:51:35.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.435+0000 7ff6344d9640 1 -- 172.21.15.73:0/162498946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff62c15d0d0 msgr2=0x7ff62c0afb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:35.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.435+0000 7ff6344d9640 1 --2- 172.21.15.73:0/162498946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff62c15d0d0 0x7ff62c0afb20 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff62c0c9e90 tx=0x7ff62802a1e0 comp rx=0 tx=0).stop 2024-04-26T20:51:35.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.435+0000 7ff6344d9640 1 -- 172.21.15.73:0/162498946 shutdown_connections 2024-04-26T20:51:35.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.435+0000 7ff6344d9640 1 --2- 172.21.15.73:0/162498946 >> v2:172.21.15.120:3300/0 conn(0x7ff62c0d32c0 0x7ff62c070970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:35.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.435+0000 7ff6344d9640 1 --2- 172.21.15.73:0/162498946 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff62c0b0060 0x7ff62c0d2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:35.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.435+0000 7ff6344d9640 1 --2- 172.21.15.73:0/162498946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff62c15d0d0 0x7ff62c0afb20 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:35.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.435+0000 7ff6344d9640 1 -- 172.21.15.73:0/162498946 >> 172.21.15.73:0/162498946 conn(0x7ff62c09ba50 msgr2=0x7ff62c09be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:35.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.435+0000 7ff6344d9640 1 -- 172.21.15.73:0/162498946 shutdown_connections 2024-04-26T20:51:35.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.435+0000 7ff6344d9640 1 -- 172.21.15.73:0/162498946 wait complete. 2024-04-26T20:51:35.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.436+0000 7ff6344d9640 1 Processor -- start 2024-04-26T20:51:35.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.436+0000 7ff6344d9640 1 -- start start 2024-04-26T20:51:35.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.437+0000 7ff6344d9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff62c0b0060 0x7ff62c15a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:35.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.437+0000 7ff6344d9640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff62c0d32c0 0x7ff62c0c7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:35.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.437+0000 7ff6344d9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff62c15d0d0 0x7ff62c15f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:35.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.437+0000 7ff6344d9640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff62c15d4b0 con 0x7ff62c15d0d0 2024-04-26T20:51:35.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.437+0000 7ff6344d9640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff62c0b0db0 con 0x7ff62c0b0060 2024-04-26T20:51:35.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.437+0000 7ff6344d9640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7ff62c0cd7f0 con 0x7ff62c0d32c0 2024-04-26T20:51:35.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.437+0000 7ff632a76640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff62c15d0d0 0x7ff62c15f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:35.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.437+0000 7ff632a76640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff62c15d0d0 0x7ff62c15f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50264/0 (socket says 172.21.15.73:50264) 2024-04-26T20:51:35.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.437+0000 7ff632a76640 1 -- 172.21.15.73:0/4162356954 learned_addr learned my addr 172.21.15.73:0/4162356954 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:35.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.437+0000 7ff632275640 1 --2- 172.21.15.73:0/4162356954 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff62c0b0060 0x7ff62c15a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:35.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.437+0000 7ff631a74640 1 --2- 172.21.15.73:0/4162356954 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff62c0d32c0 0x7ff62c0c7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:35.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.437+0000 7ff632a76640 1 -- 172.21.15.73:0/4162356954 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff62c0d32c0 msgr2=0x7ff62c0c7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:35.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.437+0000 7ff632a76640 1 --2- 172.21.15.73:0/4162356954 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff62c0d32c0 0x7ff62c0c7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:35.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.437+0000 7ff632a76640 1 -- 172.21.15.73:0/4162356954 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff62c0b0060 msgr2=0x7ff62c15a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:35.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.437+0000 7ff632a76640 1 --2- 172.21.15.73:0/4162356954 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff62c0b0060 0x7ff62c15a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:35.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.437+0000 7ff632a76640 1 -- 172.21.15.73:0/4162356954 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff62c1f3130 con 0x7ff62c15d0d0 2024-04-26T20:51:35.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.437+0000 7ff631a74640 1 --2- 172.21.15.73:0/4162356954 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff62c0d32c0 0x7ff62c0c7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:35.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.438+0000 7ff632a76640 1 --2- 172.21.15.73:0/4162356954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff62c15d0d0 0x7ff62c15f8b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff624007e40 tx=0x7ff62400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:35.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.438+0000 7ff622ffd640 1 -- 172.21.15.73:0/4162356954 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff624014620 con 0x7ff62c15d0d0 2024-04-26T20:51:35.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.438+0000 7ff622ffd640 1 -- 172.21.15.73:0/4162356954 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff624019070 con 0x7ff62c15d0d0 2024-04-26T20:51:35.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.438+0000 7ff6344d9640 1 -- 172.21.15.73:0/4162356954 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff62c1f3410 con 0x7ff62c15d0d0 2024-04-26T20:51:35.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.438+0000 7ff622ffd640 1 -- 172.21.15.73:0/4162356954 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff624014a20 con 0x7ff62c15d0d0 2024-04-26T20:51:35.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.438+0000 7ff6344d9640 1 -- 172.21.15.73:0/4162356954 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff62c1f39d0 con 0x7ff62c15d0d0 2024-04-26T20:51:35.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.439+0000 7ff6344d9640 1 -- 172.21.15.73:0/4162356954 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff62c069600 con 0x7ff62c15d0d0 2024-04-26T20:51:35.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.439+0000 7ff622ffd640 1 -- 172.21.15.73:0/4162356954 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff624014bc0 con 0x7ff62c15d0d0 2024-04-26T20:51:35.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.440+0000 7ff622ffd640 1 --2- 172.21.15.73:0/4162356954 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff61006c920 0x7ff61006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:35.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.440+0000 7ff622ffd640 1 -- 172.21.15.73:0/4162356954 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6821+0+0 (secure 0 0 0) 0x7ff62408c780 con 0x7ff62c15d0d0 2024-04-26T20:51:35.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.440+0000 7ff632275640 1 --2- 172.21.15.73:0/4162356954 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff61006c920 0x7ff61006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:35.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.440+0000 7ff632275640 1 --2- 172.21.15.73:0/4162356954 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff61006c920 0x7ff61006ede0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff628034000 tx=0x7ff62802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:35.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.444+0000 7ff622ffd640 1 -- 172.21.15.73:0/4162356954 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff62405a750 con 0x7ff62c15d0d0 2024-04-26T20:51:35.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:35.620+0000 7ff6344d9640 1 -- 172.21.15.73:0/4162356954 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff62c071fe0 con 0x7ff62c15d0d0 2024-04-26T20:51:36.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.273+0000 7ff622ffd640 1 -- 172.21.15.73:0/4162356954 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v29) ==== 181+0+0 (secure 0 0 0) 0x7ff62405a050 con 0x7ff62c15d0d0 2024-04-26T20:51:36.274 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_metadata' removed 2024-04-26T20:51:36.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.275+0000 7ff6344d9640 1 -- 172.21.15.73:0/4162356954 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff61006c920 msgr2=0x7ff61006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:36.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.275+0000 7ff6344d9640 1 --2- 172.21.15.73:0/4162356954 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff61006c920 0x7ff61006ede0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff628034000 tx=0x7ff62802e000 comp rx=0 tx=0).stop 2024-04-26T20:51:36.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.275+0000 7ff6344d9640 1 -- 172.21.15.73:0/4162356954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff62c15d0d0 msgr2=0x7ff62c15f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:36.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.275+0000 7ff6344d9640 1 --2- 172.21.15.73:0/4162356954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff62c15d0d0 0x7ff62c15f8b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff624007e40 tx=0x7ff62400e4b0 comp rx=0 tx=0).stop 2024-04-26T20:51:36.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.275+0000 7ff6344d9640 1 -- 172.21.15.73:0/4162356954 shutdown_connections 2024-04-26T20:51:36.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.275+0000 7ff6344d9640 1 --2- 172.21.15.73:0/4162356954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff62c15d0d0 0x7ff62c15f8b0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:36.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.275+0000 7ff6344d9640 1 --2- 172.21.15.73:0/4162356954 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff61006c920 0x7ff61006ede0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:36.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.275+0000 7ff6344d9640 1 --2- 172.21.15.73:0/4162356954 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff62c0d32c0 0x7ff62c0c7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:36.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.275+0000 7ff6344d9640 1 --2- 172.21.15.73:0/4162356954 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff62c0b0060 0x7ff62c15a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:36.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.275+0000 7ff6344d9640 1 -- 172.21.15.73:0/4162356954 >> 172.21.15.73:0/4162356954 conn(0x7ff62c09ba50 msgr2=0x7ff62c15d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:36.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.276+0000 7ff6344d9640 1 -- 172.21.15.73:0/4162356954 shutdown_connections 2024-04-26T20:51:36.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.276+0000 7ff6344d9640 1 -- 172.21.15.73:0/4162356954 wait complete. 2024-04-26T20:51:36.291 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T20:51:36.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.393+0000 7ff72d3f2640 1 Processor -- start 2024-04-26T20:51:36.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.393+0000 7ff72d3f2640 1 -- start start 2024-04-26T20:51:36.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.394+0000 7ff72d3f2640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7ff72815f3e0 0x7ff7280d2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:36.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.394+0000 7ff72d3f2640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7280d31e0 0x7ff7280d3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:36.399 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.394+0000 7ff72d3f2640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff728160930 con 0x7ff72815f7c0 2024-04-26T20:51:36.399 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.394+0000 7ff72d3f2640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7ff72815fd20 con 0x7ff72815f3e0 2024-04-26T20:51:36.399 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.394+0000 7ff72d3f2640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7ff728160000 con 0x7ff7280d31e0 2024-04-26T20:51:36.399 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.394+0000 7ff7277fe640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7ff72815f7c0 0x7ff7281620a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40786/0 (socket says 172.21.15.73:40786) 2024-04-26T20:51:36.399 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.394+0000 7ff7277fe640 1 -- 172.21.15.73:0/2823236620 learned_addr learned my addr 172.21.15.73:0/2823236620 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:36.399 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.394+0000 7ff726ffd640 1 --2- 172.21.15.73:0/2823236620 >> v2:172.21.15.120:3300/0 conn(0x7ff72815f3e0 0x7ff7280d2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:36.399 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.394+0000 7ff7267fc640 1 --2- 172.21.15.73:0/2823236620 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7280d31e0 0x7ff7280d3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:36.399 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.394+0000 7ff725ffb640 1 -- 172.21.15.73:0/2823236620 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 374371764 0 0) 0x7ff728160930 con 0x7ff72815f7c0 2024-04-26T20:51:36.399 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.394+0000 7ff725ffb640 1 -- 172.21.15.73:0/2823236620 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff710003520 con 0x7ff72815f7c0 2024-04-26T20:51:36.399 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.394+0000 7ff725ffb640 1 -- 172.21.15.73:0/2823236620 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3708189358 0 0) 0x7ff710003520 con 0x7ff72815f7c0 2024-04-26T20:51:36.399 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.394+0000 7ff725ffb640 1 -- 172.21.15.73:0/2823236620 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7280d31e0 msgr2=0x7ff7280d3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:36.399 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.394+0000 7ff725ffb640 1 --2- 172.21.15.73:0/2823236620 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7280d31e0 0x7ff7280d3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:36.399 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.394+0000 7ff725ffb640 1 -- 172.21.15.73:0/2823236620 >> v2:172.21.15.120:3300/0 conn(0x7ff72815f3e0 msgr2=0x7ff7280d2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:36.399 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.394+0000 7ff725ffb640 1 --2- 172.21.15.73:0/2823236620 >> v2:172.21.15.120:3300/0 conn(0x7ff72815f3e0 0x7ff7280d2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:36.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.394+0000 7ff725ffb640 1 -- 172.21.15.73:0/2823236620 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7281602e0 con 0x7ff72815f7c0 2024-04-26T20:51:36.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.394+0000 7ff725ffb640 1 -- 172.21.15.73:0/2823236620 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7ff720001670 con 0x7ff72815f7c0 2024-04-26T20:51:36.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.394+0000 7ff726ffd640 1 --2- 172.21.15.73:0/2823236620 >> v2:172.21.15.120:3300/0 conn(0x7ff72815f3e0 0x7ff7280d2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:51:36.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.395+0000 7ff725ffb640 1 -- 172.21.15.73:0/2823236620 >> v1:172.21.15.73:6789/0 conn(0x7ff72815f7c0 legacy=0x7ff7281620a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:36.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.395+0000 7ff725ffb640 1 --2- 172.21.15.73:0/2823236620 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7100039b0 0x7ff710003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:36.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.395+0000 7ff725ffb640 1 --2- 172.21.15.73:0/2823236620 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff710004800 0x7ff710004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:36.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.395+0000 7ff725ffb640 1 --2- 172.21.15.73:0/2823236620 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7280d31e0 0x7ff7100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:36.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.395+0000 7ff725ffb640 1 -- 172.21.15.73:0/2823236620 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff72815fd20 con 0x7ff710004800 2024-04-26T20:51:36.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.395+0000 7ff725ffb640 1 -- 172.21.15.73:0/2823236620 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff728160000 con 0x7ff7100039b0 2024-04-26T20:51:36.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.395+0000 7ff725ffb640 1 -- 172.21.15.73:0/2823236620 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7ff71000b9c0 con 0x7ff7280d31e0 2024-04-26T20:51:36.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.395+0000 7ff7267fc640 1 --2- 172.21.15.73:0/2823236620 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff710004800 0x7ff710004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:36.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.395+0000 7ff726ffd640 1 --2- 172.21.15.73:0/2823236620 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7100039b0 0x7ff710003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:36.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.395+0000 7ff7277fe640 1 --2- 172.21.15.73:0/2823236620 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7280d31e0 0x7ff7100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:36.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.395+0000 7ff7267fc640 1 -- 172.21.15.73:0/2823236620 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7280d31e0 msgr2=0x7ff7100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:36.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.395+0000 7ff7267fc640 1 --2- 172.21.15.73:0/2823236620 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7280d31e0 0x7ff7100051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:36.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.395+0000 7ff7267fc640 1 -- 172.21.15.73:0/2823236620 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7100039b0 msgr2=0x7ff710003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:36.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.395+0000 7ff7267fc640 1 --2- 172.21.15.73:0/2823236620 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7100039b0 0x7ff710003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:36.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.395+0000 7ff7267fc640 1 -- 172.21.15.73:0/2823236620 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff710003520 con 0x7ff710004800 2024-04-26T20:51:36.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.395+0000 7ff7277fe640 1 --2- 172.21.15.73:0/2823236620 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7280d31e0 0x7ff7100051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:36.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.396+0000 7ff7267fc640 1 --2- 172.21.15.73:0/2823236620 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff710004800 0x7ff710004c80 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ff71c009ac0 tx=0x7ff710005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65de19f7997ed8c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:36.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.396+0000 7ff725ffb640 1 -- 172.21.15.73:0/2823236620 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71c036070 con 0x7ff710004800 2024-04-26T20:51:36.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.396+0000 7ff725ffb640 1 -- 172.21.15.73:0/2823236620 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff71c031400 con 0x7ff710004800 2024-04-26T20:51:36.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.396+0000 7ff725ffb640 1 -- 172.21.15.73:0/2823236620 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71c009690 con 0x7ff710004800 2024-04-26T20:51:36.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.396+0000 7ff72d3f2640 1 -- 172.21.15.73:0/2823236620 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff710004800 msgr2=0x7ff710004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:36.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.396+0000 7ff72d3f2640 1 --2- 172.21.15.73:0/2823236620 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff710004800 0x7ff710004c80 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ff71c009ac0 tx=0x7ff710005830 comp rx=0 tx=0).stop 2024-04-26T20:51:36.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.396+0000 7ff7277fe640 1 -- 172.21.15.73:0/2823236620 reap_dead start 2024-04-26T20:51:36.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.396+0000 7ff72d3f2640 1 -- 172.21.15.73:0/2823236620 shutdown_connections 2024-04-26T20:51:36.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.396+0000 7ff72d3f2640 1 -- 172.21.15.73:0/2823236620 >> 172.21.15.73:0/2823236620 conn(0x7ff72809e3c0 msgr2=0x7ff7280d50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:36.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.396+0000 7ff72d3f2640 1 -- 172.21.15.73:0/2823236620 shutdown_connections 2024-04-26T20:51:36.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.396+0000 7ff72d3f2640 1 -- 172.21.15.73:0/2823236620 wait complete. 2024-04-26T20:51:36.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.398+0000 7ff72d3f2640 1 Processor -- start 2024-04-26T20:51:36.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.398+0000 7ff72d3f2640 1 -- start start 2024-04-26T20:51:36.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.398+0000 7ff72d3f2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff710004800 0x7ff7280cf120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:36.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.398+0000 7ff72d3f2640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7280c8760 0x7ff7281f8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:36.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.398+0000 7ff72d3f2640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff728159c90 0x7ff7280c9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:36.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.398+0000 7ff72d3f2640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff7281627b0 con 0x7ff728159c90 2024-04-26T20:51:36.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.398+0000 7ff72d3f2640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff728160d40 con 0x7ff710004800 2024-04-26T20:51:36.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.398+0000 7ff72d3f2640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7ff728162930 con 0x7ff7280c8760 2024-04-26T20:51:36.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.398+0000 7ff7277fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff728159c90 0x7ff7280c9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:36.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.398+0000 7ff7277fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff728159c90 0x7ff7280c9ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50276/0 (socket says 172.21.15.73:50276) 2024-04-26T20:51:36.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.398+0000 7ff7277fe640 1 -- 172.21.15.73:0/3384988387 learned_addr learned my addr 172.21.15.73:0/3384988387 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:36.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.399+0000 7ff726ffd640 1 --2- 172.21.15.73:0/3384988387 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff710004800 0x7ff7280cf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:36.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.399+0000 7ff7267fc640 1 --2- 172.21.15.73:0/3384988387 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7280c8760 0x7ff7281f8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:36.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.399+0000 7ff7277fe640 1 -- 172.21.15.73:0/3384988387 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7280c8760 msgr2=0x7ff7281f8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:36.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.399+0000 7ff7277fe640 1 --2- 172.21.15.73:0/3384988387 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7280c8760 0x7ff7281f8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:36.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.399+0000 7ff7277fe640 1 -- 172.21.15.73:0/3384988387 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff710004800 msgr2=0x7ff7280cf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:36.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.399+0000 7ff7277fe640 1 --2- 172.21.15.73:0/3384988387 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff710004800 0x7ff7280cf120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:36.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.399+0000 7ff7277fe640 1 -- 172.21.15.73:0/3384988387 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7280c8bc0 con 0x7ff728159c90 2024-04-26T20:51:36.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.399+0000 7ff7267fc640 1 --2- 172.21.15.73:0/3384988387 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7280c8760 0x7ff7281f8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:36.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.399+0000 7ff7277fe640 1 --2- 172.21.15.73:0/3384988387 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff728159c90 0x7ff7280c9ff0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ff7280bb180 tx=0x7ff72000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:36.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.399+0000 7ff70f7fe640 1 -- 172.21.15.73:0/3384988387 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff720019070 con 0x7ff728159c90 2024-04-26T20:51:36.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.399+0000 7ff70f7fe640 1 -- 172.21.15.73:0/3384988387 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff72000c590 con 0x7ff728159c90 2024-04-26T20:51:36.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.399+0000 7ff72d3f2640 1 -- 172.21.15.73:0/3384988387 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7281f9660 con 0x7ff728159c90 2024-04-26T20:51:36.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.399+0000 7ff70f7fe640 1 -- 172.21.15.73:0/3384988387 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72000c980 con 0x7ff728159c90 2024-04-26T20:51:36.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.399+0000 7ff72d3f2640 1 -- 172.21.15.73:0/3384988387 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7281f9c20 con 0x7ff728159c90 2024-04-26T20:51:36.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.400+0000 7ff72d3f2640 1 -- 172.21.15.73:0/3384988387 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff72809e7d0 con 0x7ff728159c90 2024-04-26T20:51:36.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.401+0000 7ff70f7fe640 1 -- 172.21.15.73:0/3384988387 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff72001c070 con 0x7ff728159c90 2024-04-26T20:51:36.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.401+0000 7ff70f7fe640 1 --2- 172.21.15.73:0/3384988387 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff71406c920 0x7ff71406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:36.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.401+0000 7ff726ffd640 1 --2- 172.21.15.73:0/3384988387 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff71406c920 0x7ff71406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:36.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.401+0000 7ff70f7fe640 1 -- 172.21.15.73:0/3384988387 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6383+0+0 (secure 0 0 0) 0x7ff72008b090 con 0x7ff728159c90 2024-04-26T20:51:36.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.401+0000 7ff726ffd640 1 --2- 172.21.15.73:0/3384988387 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff71406c920 0x7ff71406ede0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff718009fd0 tx=0x7ff71800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:36.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.405+0000 7ff70f7fe640 1 -- 172.21.15.73:0/3384988387 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff72809e7d0 con 0x7ff728159c90 2024-04-26T20:51:36.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:36.582+0000 7ff72d3f2640 1 -- 172.21.15.73:0/3384988387 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff728160800 con 0x7ff728159c90 2024-04-26T20:51:37.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.274+0000 7ff70f7fe640 1 -- 172.21.15.73:0/3384988387 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v30) ==== 169+0+0 (secure 0 0 0) 0x7ff728160800 con 0x7ff728159c90 2024-04-26T20:51:37.275 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data' removed 2024-04-26T20:51:37.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.276+0000 7ff72d3f2640 1 -- 172.21.15.73:0/3384988387 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff71406c920 msgr2=0x7ff71406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:37.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.276+0000 7ff72d3f2640 1 --2- 172.21.15.73:0/3384988387 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff71406c920 0x7ff71406ede0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff718009fd0 tx=0x7ff71800b040 comp rx=0 tx=0).stop 2024-04-26T20:51:37.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.276+0000 7ff72d3f2640 1 -- 172.21.15.73:0/3384988387 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff728159c90 msgr2=0x7ff7280c9ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:37.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.276+0000 7ff72d3f2640 1 --2- 172.21.15.73:0/3384988387 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff728159c90 0x7ff7280c9ff0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ff7280bb180 tx=0x7ff72000eb90 comp rx=0 tx=0).stop 2024-04-26T20:51:37.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.277+0000 7ff72d3f2640 1 -- 172.21.15.73:0/3384988387 shutdown_connections 2024-04-26T20:51:37.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.277+0000 7ff72d3f2640 1 --2- 172.21.15.73:0/3384988387 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff728159c90 0x7ff7280c9ff0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:37.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.277+0000 7ff72d3f2640 1 --2- 172.21.15.73:0/3384988387 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff71406c920 0x7ff71406ede0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:37.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.277+0000 7ff72d3f2640 1 --2- 172.21.15.73:0/3384988387 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7280c8760 0x7ff7281f8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:37.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.277+0000 7ff72d3f2640 1 --2- 172.21.15.73:0/3384988387 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff710004800 0x7ff7280cf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:37.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.277+0000 7ff72d3f2640 1 -- 172.21.15.73:0/3384988387 >> 172.21.15.73:0/3384988387 conn(0x7ff72809e3c0 msgr2=0x7ff72815d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:37.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.277+0000 7ff72d3f2640 1 -- 172.21.15.73:0/3384988387 shutdown_connections 2024-04-26T20:51:37.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.277+0000 7ff72d3f2640 1 -- 172.21.15.73:0/3384988387 wait complete. 2024-04-26T20:51:37.290 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-26T20:51:37.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.389+0000 7f087f36a640 1 Processor -- start 2024-04-26T20:51:37.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.389+0000 7f087f36a640 1 -- start start 2024-04-26T20:51:37.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.390+0000 7f087f36a640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f08780cfb60 0x7f0878167d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:37.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.390+0000 7f087f36a640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0878071900 con 0x7f08780b7900 2024-04-26T20:51:37.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.390+0000 7f087f36a640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08780d00a0 con 0x7f0878145d80 2024-04-26T20:51:37.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.390+0000 7f087f36a640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f0878160a80 con 0x7f08780cfb60 2024-04-26T20:51:37.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.390+0000 7f087c905640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f08780b7900 0x7f08780cf450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40794/0 (socket says 172.21.15.73:40794) 2024-04-26T20:51:37.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.390+0000 7f087c905640 1 -- 172.21.15.73:0/2149976267 learned_addr learned my addr 172.21.15.73:0/2149976267 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:37.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.391+0000 7f086ffff640 1 -- 172.21.15.73:0/2149976267 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1219741977 0 0) 0x7f0878071900 con 0x7f08780b7900 2024-04-26T20:51:37.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.391+0000 7f087d907640 1 --2- 172.21.15.73:0/2149976267 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f08780cfb60 0x7f0878167d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:37.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.391+0000 7f086ffff640 1 -- 172.21.15.73:0/2149976267 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0864003520 con 0x7f08780b7900 2024-04-26T20:51:37.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.391+0000 7f086ffff640 1 -- 172.21.15.73:0/2149976267 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2833963681 0 0) 0x7f0864003520 con 0x7f08780b7900 2024-04-26T20:51:37.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.391+0000 7f086ffff640 1 -- 172.21.15.73:0/2149976267 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f08780cfb60 msgr2=0x7f0878167d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:37.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.391+0000 7f086ffff640 1 --2- 172.21.15.73:0/2149976267 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f08780cfb60 0x7f0878167d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:37.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.391+0000 7f086ffff640 1 -- 172.21.15.73:0/2149976267 >> v1:172.21.15.120:6789/0 conn(0x7f0878145d80 legacy=0x7f08780b71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:37.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.391+0000 7f086ffff640 1 -- 172.21.15.73:0/2149976267 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0878160d60 con 0x7f08780b7900 2024-04-26T20:51:37.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.391+0000 7f086ffff640 1 -- 172.21.15.73:0/2149976267 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7f0868001670 con 0x7f08780b7900 2024-04-26T20:51:37.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.391+0000 7f086ffff640 1 -- 172.21.15.73:0/2149976267 >> v1:172.21.15.73:6789/0 conn(0x7f08780b7900 legacy=0x7f08780cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:37.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.391+0000 7f086ffff640 1 --2- 172.21.15.73:0/2149976267 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f08640039e0 0x7f0864003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:37.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.391+0000 7f086ffff640 1 --2- 172.21.15.73:0/2149976267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0864004800 0x7f0864004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:37.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.391+0000 7f086ffff640 1 --2- 172.21.15.73:0/2149976267 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f08780cfb60 0x7f08640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:37.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.391+0000 7f086ffff640 1 -- 172.21.15.73:0/2149976267 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0878160a80 con 0x7f0864004800 2024-04-26T20:51:37.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.391+0000 7f086ffff640 1 -- 172.21.15.73:0/2149976267 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f086400ba70 con 0x7f08640039e0 2024-04-26T20:51:37.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.391+0000 7f086ffff640 1 -- 172.21.15.73:0/2149976267 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f086400bc70 con 0x7f08780cfb60 2024-04-26T20:51:37.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.391+0000 7f087c905640 1 --2- 172.21.15.73:0/2149976267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0864004800 0x7f0864004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:37.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.391+0000 7f087d907640 1 --2- 172.21.15.73:0/2149976267 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f08780cfb60 0x7f08640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:37.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.391+0000 7f087d106640 1 --2- 172.21.15.73:0/2149976267 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f08640039e0 0x7f0864003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:37.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.391+0000 7f087c905640 1 -- 172.21.15.73:0/2149976267 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f08780cfb60 msgr2=0x7f08640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:37.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.391+0000 7f087c905640 1 --2- 172.21.15.73:0/2149976267 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f08780cfb60 0x7f08640051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:37.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.391+0000 7f087c905640 1 -- 172.21.15.73:0/2149976267 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f08640039e0 msgr2=0x7f0864003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:37.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.391+0000 7f087c905640 1 --2- 172.21.15.73:0/2149976267 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f08640039e0 0x7f0864003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:37.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.391+0000 7f087c905640 1 -- 172.21.15.73:0/2149976267 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0864003520 con 0x7f0864004800 2024-04-26T20:51:37.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.392+0000 7f087c905640 1 --2- 172.21.15.73:0/2149976267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0864004800 0x7f0864004c80 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f086800c210 tx=0x7f0864005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f83b180bfe66e83a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:37.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.392+0000 7f086ffff640 1 -- 172.21.15.73:0/2149976267 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0868007cf0 con 0x7f0864004800 2024-04-26T20:51:37.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.392+0000 7f086ffff640 1 -- 172.21.15.73:0/2149976267 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0868005af0 con 0x7f0864004800 2024-04-26T20:51:37.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.392+0000 7f086ffff640 1 -- 172.21.15.73:0/2149976267 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0868007070 con 0x7f0864004800 2024-04-26T20:51:37.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.393+0000 7f087f36a640 1 -- 172.21.15.73:0/2149976267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0864004800 msgr2=0x7f0864004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:37.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.393+0000 7f087f36a640 1 --2- 172.21.15.73:0/2149976267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0864004800 0x7f0864004c80 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f086800c210 tx=0x7f0864005830 comp rx=0 tx=0).stop 2024-04-26T20:51:37.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.393+0000 7f087d907640 1 -- 172.21.15.73:0/2149976267 reap_dead start 2024-04-26T20:51:37.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.393+0000 7f087f36a640 1 -- 172.21.15.73:0/2149976267 shutdown_connections 2024-04-26T20:51:37.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.393+0000 7f087f36a640 1 -- 172.21.15.73:0/2149976267 >> 172.21.15.73:0/2149976267 conn(0x7f08780b3be0 msgr2=0x7f08780b3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:37.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.393+0000 7f087f36a640 1 -- 172.21.15.73:0/2149976267 shutdown_connections 2024-04-26T20:51:37.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.393+0000 7f087f36a640 1 -- 172.21.15.73:0/2149976267 wait complete. 2024-04-26T20:51:37.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.394+0000 7f087f36a640 1 Processor -- start 2024-04-26T20:51:37.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.394+0000 7f087f36a640 1 -- start start 2024-04-26T20:51:37.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.395+0000 7f087f36a640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0864004800 0x7f087813d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:37.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.395+0000 7f087f36a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f087813be90 0x7f087815c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:37.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.395+0000 7f087f36a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f08781f2d70 0x7f08780b2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:37.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.395+0000 7f087f36a640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0878161840 con 0x7f087813be90 2024-04-26T20:51:37.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.395+0000 7f087f36a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0878161320 con 0x7f08781f2d70 2024-04-26T20:51:37.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.395+0000 7f087f36a640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f0878165d60 con 0x7f0864004800 2024-04-26T20:51:37.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.395+0000 7f087c905640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f087813be90 0x7f087815c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:37.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.395+0000 7f087c905640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f087813be90 0x7f087815c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50298/0 (socket says 172.21.15.73:50298) 2024-04-26T20:51:37.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.395+0000 7f087c905640 1 -- 172.21.15.73:0/772525626 learned_addr learned my addr 172.21.15.73:0/772525626 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:37.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.395+0000 7f087d907640 1 --2- 172.21.15.73:0/772525626 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f08781f2d70 0x7f08780b2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:37.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.395+0000 7f087d106640 1 --2- 172.21.15.73:0/772525626 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0864004800 0x7f087813d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:37.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.395+0000 7f087c905640 1 -- 172.21.15.73:0/772525626 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0864004800 msgr2=0x7f087813d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:37.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.395+0000 7f087c905640 1 --2- 172.21.15.73:0/772525626 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0864004800 0x7f087813d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:37.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.395+0000 7f087c905640 1 -- 172.21.15.73:0/772525626 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f08781f2d70 msgr2=0x7f08780b2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:37.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.395+0000 7f087c905640 1 --2- 172.21.15.73:0/772525626 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f08781f2d70 0x7f08780b2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:37.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.395+0000 7f087c905640 1 -- 172.21.15.73:0/772525626 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f087813de40 con 0x7f087813be90 2024-04-26T20:51:37.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.395+0000 7f087d907640 1 --2- 172.21.15.73:0/772525626 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f08781f2d70 0x7f08780b2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:37.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.395+0000 7f087c905640 1 --2- 172.21.15.73:0/772525626 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f087813be90 0x7f087815c420 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f086800c330 tx=0x7f08780d0270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:37.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.395+0000 7f086dffb640 1 -- 172.21.15.73:0/772525626 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0868001630 con 0x7f087813be90 2024-04-26T20:51:37.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.395+0000 7f086dffb640 1 -- 172.21.15.73:0/772525626 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0868042070 con 0x7f087813be90 2024-04-26T20:51:37.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.395+0000 7f086dffb640 1 -- 172.21.15.73:0/772525626 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08680072b0 con 0x7f087813be90 2024-04-26T20:51:37.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.395+0000 7f087f36a640 1 -- 172.21.15.73:0/772525626 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f087813e0c0 con 0x7f087813be90 2024-04-26T20:51:37.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.396+0000 7f087f36a640 1 -- 172.21.15.73:0/772525626 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f087813e680 con 0x7f087813be90 2024-04-26T20:51:37.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.396+0000 7f087f36a640 1 -- 172.21.15.73:0/772525626 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0878069600 con 0x7f087813be90 2024-04-26T20:51:37.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.397+0000 7f086dffb640 1 -- 172.21.15.73:0/772525626 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0868007450 con 0x7f087813be90 2024-04-26T20:51:37.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.397+0000 7f086dffb640 1 --2- 172.21.15.73:0/772525626 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f085c06c920 0x7f085c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:37.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.397+0000 7f087d106640 1 --2- 172.21.15.73:0/772525626 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f085c06c920 0x7f085c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:37.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.397+0000 7f086dffb640 1 -- 172.21.15.73:0/772525626 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5853+0+0 (secure 0 0 0) 0x7f08680abb90 con 0x7f087813be90 2024-04-26T20:51:37.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.398+0000 7f087d106640 1 --2- 172.21.15.73:0/772525626 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f085c06c920 0x7f085c06ede0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0874003f10 tx=0x7f0874009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:37.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.402+0000 7f086dffb640 1 -- 172.21.15.73:0/772525626 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f087813de40 con 0x7f087813be90 2024-04-26T20:51:37.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:37.574+0000 7f087f36a640 1 -- 172.21.15.73:0/772525626 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f08780b49e0 con 0x7f087813be90 2024-04-26T20:51:38.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.293+0000 7f086dffb640 1 -- 172.21.15.73:0/772525626 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v31) ==== 178+0+0 (secure 0 0 0) 0x7f0868079fc0 con 0x7f087813be90 2024-04-26T20:51:38.294 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data_ec' removed 2024-04-26T20:51:38.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.296+0000 7f087f36a640 1 -- 172.21.15.73:0/772525626 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f085c06c920 msgr2=0x7f085c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:38.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.296+0000 7f087f36a640 1 --2- 172.21.15.73:0/772525626 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f085c06c920 0x7f085c06ede0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0874003f10 tx=0x7f0874009040 comp rx=0 tx=0).stop 2024-04-26T20:51:38.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.296+0000 7f087f36a640 1 -- 172.21.15.73:0/772525626 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f087813be90 msgr2=0x7f087815c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:38.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.296+0000 7f087f36a640 1 --2- 172.21.15.73:0/772525626 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f087813be90 0x7f087815c420 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f086800c330 tx=0x7f08780d0270 comp rx=0 tx=0).stop 2024-04-26T20:51:38.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.296+0000 7f087f36a640 1 -- 172.21.15.73:0/772525626 shutdown_connections 2024-04-26T20:51:38.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.296+0000 7f087f36a640 1 --2- 172.21.15.73:0/772525626 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f087813be90 0x7f087815c420 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:38.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.296+0000 7f087f36a640 1 --2- 172.21.15.73:0/772525626 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f085c06c920 0x7f085c06ede0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:38.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.296+0000 7f087f36a640 1 --2- 172.21.15.73:0/772525626 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f08781f2d70 0x7f08780b2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:38.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.296+0000 7f087f36a640 1 --2- 172.21.15.73:0/772525626 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0864004800 0x7f087813d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:38.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.296+0000 7f087f36a640 1 -- 172.21.15.73:0/772525626 >> 172.21.15.73:0/772525626 conn(0x7f08780b3be0 msgr2=0x7f08780cf450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:38.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.296+0000 7f087f36a640 1 -- 172.21.15.73:0/772525626 shutdown_connections 2024-04-26T20:51:38.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.296+0000 7f087f36a640 1 -- 172.21.15.73:0/772525626 wait complete. 2024-04-26T20:51:38.310 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T20:51:38.311 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-26T20:51:38.311 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T20:51:38.311 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:51:38.311 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T20:51:38.311 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-26T20:51:38.311 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T20:51:38.311 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:51:38.312 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T20:51:38.312 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-26T20:51:38.312 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T20:51:38.312 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:51:38.312 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T20:51:38.312 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-26T20:51:38.312 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T20:51:38.312 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:51:38.322 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T20:51:38.322 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T20:51:38.323 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T20:51:38.323 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T20:51:38.324 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T20:51:38.324 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T20:51:38.325 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T20:51:38.325 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T20:51:38.364 INFO:tasks.ceph.mds.a:Started 2024-04-26T20:51:38.365 DEBUG:teuthology.parallel:result is None 2024-04-26T20:51:38.365 INFO:tasks.ceph.mds.c:Started 2024-04-26T20:51:38.366 DEBUG:teuthology.parallel:result is None 2024-04-26T20:51:38.367 INFO:tasks.ceph.mds.b:Started 2024-04-26T20:51:38.367 DEBUG:teuthology.parallel:result is None 2024-04-26T20:51:38.368 INFO:tasks.ceph.mds.d:Started 2024-04-26T20:51:38.368 DEBUG:teuthology.parallel:result is None 2024-04-26T20:51:38.368 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-26T20:51:38.443 INFO:tasks.ceph.mds.c.smithi073.stdout:starting mds.c at 2024-04-26T20:51:38.444 INFO:tasks.ceph.mds.a.smithi073.stdout:starting mds.a at 2024-04-26T20:51:38.447 INFO:tasks.ceph.mds.b.smithi120.stdout:starting mds.b at 2024-04-26T20:51:38.448 INFO:tasks.ceph.mds.d.smithi120.stdout:starting mds.d at 2024-04-26T20:51:38.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.473+0000 7f6aeb571640 1 Processor -- start 2024-04-26T20:51:38.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.473+0000 7f6aeb571640 1 -- start start 2024-04-26T20:51:38.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.474+0000 7f6aeb571640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f6ae40d0070 0x7f6ae40d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:38.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.474+0000 7f6aeb571640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6ae406d560 0x7f6ae406db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:38.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.474+0000 7f6aeb571640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ae40d2420 con 0x7f6ae406e0a0 2024-04-26T20:51:38.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.474+0000 7f6aeb571640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f6ae406e510 con 0x7f6ae406d560 2024-04-26T20:51:38.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.474+0000 7f6aeb571640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f6ae4162d00 con 0x7f6ae40d0070 2024-04-26T20:51:38.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.474+0000 7f6ae8b0c640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6ae406d560 0x7f6ae406db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:38.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.474+0000 7f6ae9b0e640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f6ae406e0a0 0x7f6ae4162f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40824/0 (socket says 172.21.15.73:40824) 2024-04-26T20:51:38.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.474+0000 7f6ae9b0e640 1 -- 172.21.15.73:0/220617613 learned_addr learned my addr 172.21.15.73:0/220617613 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:38.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.474+0000 7f6ae930d640 1 --2- 172.21.15.73:0/220617613 >> v2:172.21.15.120:3300/0 conn(0x7f6ae40d0070 0x7f6ae40d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:38.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.474+0000 7f6ae8b0c640 1 -- 172.21.15.73:0/220617613 >> v2:172.21.15.120:3300/0 conn(0x7f6ae40d0070 msgr2=0x7f6ae40d1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:38.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.474+0000 7f6ae8b0c640 1 --2- 172.21.15.73:0/220617613 >> v2:172.21.15.120:3300/0 conn(0x7f6ae40d0070 0x7f6ae40d1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:38.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.474+0000 7f6ae8b0c640 1 -- 172.21.15.73:0/220617613 >> v1:172.21.15.73:6789/0 conn(0x7f6ae406e0a0 legacy=0x7f6ae4162f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:38.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.474+0000 7f6adbfff640 1 -- 172.21.15.73:0/220617613 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1238630126 0 0) 0x7f6ae40d2420 con 0x7f6ae406e0a0 2024-04-26T20:51:38.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.474+0000 7f6ae8b0c640 1 -- 172.21.15.73:0/220617613 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ae40cd6b0 con 0x7f6ae406d560 2024-04-26T20:51:38.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.475+0000 7f6ae8b0c640 1 --2- 172.21.15.73:0/220617613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ae406d560 0x7f6ae406db60 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f6ae40731d0 tx=0x7f6ad402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dcdf197e02afaf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:38.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.475+0000 7f6adbfff640 1 -- 172.21.15.73:0/220617613 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad4037070 con 0x7f6ae406d560 2024-04-26T20:51:38.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.475+0000 7f6adbfff640 1 -- 172.21.15.73:0/220617613 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ad4032440 con 0x7f6ae406d560 2024-04-26T20:51:38.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.475+0000 7f6adbfff640 1 -- 172.21.15.73:0/220617613 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad4032810 con 0x7f6ae406d560 2024-04-26T20:51:38.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.476+0000 7f6aeb571640 1 -- 172.21.15.73:0/220617613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ae406d560 msgr2=0x7f6ae406db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:38.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.476+0000 7f6aeb571640 1 --2- 172.21.15.73:0/220617613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ae406d560 0x7f6ae406db60 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f6ae40731d0 tx=0x7f6ad402a1a0 comp rx=0 tx=0).stop 2024-04-26T20:51:38.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.476+0000 7f6aeb571640 1 -- 172.21.15.73:0/220617613 shutdown_connections 2024-04-26T20:51:38.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.476+0000 7f6aeb571640 1 --2- 172.21.15.73:0/220617613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ae406d560 0x7f6ae406db60 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:38.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.476+0000 7f6aeb571640 1 --2- 172.21.15.73:0/220617613 >> v2:172.21.15.120:3300/0 conn(0x7f6ae40d0070 0x7f6ae40d1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:38.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.476+0000 7f6aeb571640 1 -- 172.21.15.73:0/220617613 >> 172.21.15.73:0/220617613 conn(0x7f6ae40d2ec0 msgr2=0x7f6ae40d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:38.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.476+0000 7f6aeb571640 1 -- 172.21.15.73:0/220617613 shutdown_connections 2024-04-26T20:51:38.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.476+0000 7f6aeb571640 1 -- 172.21.15.73:0/220617613 wait complete. 2024-04-26T20:51:38.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.477+0000 7f6aeb571640 1 Processor -- start 2024-04-26T20:51:38.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.477+0000 7f6aeb571640 1 -- start start 2024-04-26T20:51:38.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.478+0000 7f6aeb571640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ae406d560 0x7f6ae415d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:38.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.478+0000 7f6aeb571640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ae406e0a0 0x7f6ae40c7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:38.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.478+0000 7f6aeb571640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6ae40d0070 0x7f6ae40ce090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:38.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.478+0000 7f6aeb571640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae40cdc70 con 0x7f6ae406d560 2024-04-26T20:51:38.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.478+0000 7f6aeb571640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae40d2d10 con 0x7f6ae406e0a0 2024-04-26T20:51:38.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.478+0000 7f6aeb571640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f6ae40cf1a0 con 0x7f6ae40d0070 2024-04-26T20:51:38.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.478+0000 7f6ae930d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ae406d560 0x7f6ae415d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:38.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.478+0000 7f6ae930d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ae406d560 0x7f6ae415d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50346/0 (socket says 172.21.15.73:50346) 2024-04-26T20:51:38.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.478+0000 7f6ae930d640 1 -- 172.21.15.73:0/2092271072 learned_addr learned my addr 172.21.15.73:0/2092271072 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:38.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.478+0000 7f6ae9b0e640 1 --2- 172.21.15.73:0/2092271072 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6ae40d0070 0x7f6ae40ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:38.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.478+0000 7f6ae930d640 1 -- 172.21.15.73:0/2092271072 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6ae40d0070 msgr2=0x7f6ae40ce090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:38.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.478+0000 7f6ae8b0c640 1 --2- 172.21.15.73:0/2092271072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ae406e0a0 0x7f6ae40c7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:38.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.478+0000 7f6ae930d640 1 --2- 172.21.15.73:0/2092271072 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6ae40d0070 0x7f6ae40ce090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:38.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.478+0000 7f6ae930d640 1 -- 172.21.15.73:0/2092271072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ae406e0a0 msgr2=0x7f6ae40c7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:38.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.478+0000 7f6ae930d640 1 --2- 172.21.15.73:0/2092271072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ae406e0a0 0x7f6ae40c7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:38.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.478+0000 7f6ae930d640 1 -- 172.21.15.73:0/2092271072 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ae409f710 con 0x7f6ae406d560 2024-04-26T20:51:38.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.478+0000 7f6ae9b0e640 1 --2- 172.21.15.73:0/2092271072 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6ae40d0070 0x7f6ae40ce090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:38.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.478+0000 7f6ae8b0c640 1 --2- 172.21.15.73:0/2092271072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ae406e0a0 0x7f6ae40c7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:38.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.478+0000 7f6ae930d640 1 --2- 172.21.15.73:0/2092271072 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ae406d560 0x7f6ae415d7b0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f6ae000e8a0 tx=0x7f6ae000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:38.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.478+0000 7f6ad9ffb640 1 -- 172.21.15.73:0/2092271072 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0017070 con 0x7f6ae406d560 2024-04-26T20:51:38.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.478+0000 7f6ad9ffb640 1 -- 172.21.15.73:0/2092271072 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ae000c940 con 0x7f6ae406d560 2024-04-26T20:51:38.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.478+0000 7f6ad9ffb640 1 -- 172.21.15.73:0/2092271072 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae000cd30 con 0x7f6ae406d560 2024-04-26T20:51:38.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.478+0000 7f6aeb571640 1 -- 172.21.15.73:0/2092271072 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ae409f9f0 con 0x7f6ae406d560 2024-04-26T20:51:38.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.478+0000 7f6aeb571640 1 -- 172.21.15.73:0/2092271072 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ae409ffb0 con 0x7f6ae406d560 2024-04-26T20:51:38.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.480+0000 7f6aeb571640 1 -- 172.21.15.73:0/2092271072 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ae40694f0 con 0x7f6ae406d560 2024-04-26T20:51:38.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.480+0000 7f6ad9ffb640 1 -- 172.21.15.73:0/2092271072 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6ae001a070 con 0x7f6ae406d560 2024-04-26T20:51:38.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.480+0000 7f6ad9ffb640 1 --2- 172.21.15.73:0/2092271072 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f6ac806c8f0 0x7f6ac806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:38.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.480+0000 7f6ae8b0c640 1 --2- 172.21.15.73:0/2092271072 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f6ac806c8f0 0x7f6ac806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:38.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.481+0000 7f6ad9ffb640 1 -- 172.21.15.73:0/2092271072 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5335+0+0 (secure 0 0 0) 0x7f6ae008bb90 con 0x7f6ae406d560 2024-04-26T20:51:38.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.481+0000 7f6ae8b0c640 1 --2- 172.21.15.73:0/2092271072 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f6ac806c8f0 0x7f6ac806edb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4034000 tx=0x7f6ad402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:38.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.485+0000 7f6ad9ffb640 1 -- 172.21.15.73:0/2092271072 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6ae005a1c0 con 0x7f6ae406d560 2024-04-26T20:51:38.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:38.674+0000 7f6aeb571640 1 -- 172.21.15.73:0/2092271072 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f6ae40d0470 con 0x7f6ae406d560 2024-04-26T20:51:39.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.310+0000 7f6ad9ffb640 1 -- 172.21.15.73:0/2092271072 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v33) ==== 97+0+0 (secure 0 0 0) 0x7f6ae0059ac0 con 0x7f6ae406d560 2024-04-26T20:51:39.311 INFO:teuthology.orchestra.run.smithi073.stderr: removed all blocklist entries 2024-04-26T20:51:39.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.312+0000 7f6aeb571640 1 -- 172.21.15.73:0/2092271072 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f6ac806c8f0 msgr2=0x7f6ac806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:39.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.312+0000 7f6aeb571640 1 --2- 172.21.15.73:0/2092271072 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f6ac806c8f0 0x7f6ac806edb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4034000 tx=0x7f6ad402e000 comp rx=0 tx=0).stop 2024-04-26T20:51:39.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.312+0000 7f6aeb571640 1 -- 172.21.15.73:0/2092271072 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ae406d560 msgr2=0x7f6ae415d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:39.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.312+0000 7f6aeb571640 1 --2- 172.21.15.73:0/2092271072 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ae406d560 0x7f6ae415d7b0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f6ae000e8a0 tx=0x7f6ae000ed60 comp rx=0 tx=0).stop 2024-04-26T20:51:39.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.312+0000 7f6aeb571640 1 -- 172.21.15.73:0/2092271072 shutdown_connections 2024-04-26T20:51:39.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.312+0000 7f6aeb571640 1 --2- 172.21.15.73:0/2092271072 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f6ac806c8f0 0x7f6ac806edb0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:39.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.312+0000 7f6aeb571640 1 --2- 172.21.15.73:0/2092271072 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6ae40d0070 0x7f6ae40ce090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:39.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.312+0000 7f6aeb571640 1 --2- 172.21.15.73:0/2092271072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6ae406e0a0 0x7f6ae40c7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:39.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.312+0000 7f6aeb571640 1 --2- 172.21.15.73:0/2092271072 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ae406d560 0x7f6ae415d7b0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:39.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.312+0000 7f6aeb571640 1 -- 172.21.15.73:0/2092271072 >> 172.21.15.73:0/2092271072 conn(0x7f6ae40d2ec0 msgr2=0x7f6ae40701d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:39.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.312+0000 7f6aeb571640 1 -- 172.21.15.73:0/2092271072 shutdown_connections 2024-04-26T20:51:39.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.312+0000 7f6aeb571640 1 -- 172.21.15.73:0/2092271072 wait complete. 2024-04-26T20:51:39.328 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-26T20:51:39.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.429+0000 7f34b67b4640 1 Processor -- start 2024-04-26T20:51:39.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.429+0000 7f34b67b4640 1 -- start start 2024-04-26T20:51:39.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.430+0000 7f34b67b4640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f34b00b0380 0x7f34b00ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:39.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.430+0000 7f34b67b4640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f34b00b10b0 0x7f34b006c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:39.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.430+0000 7f34b67b4640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34b00cfd80 con 0x7f34b006c780 2024-04-26T20:51:39.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.430+0000 7f34b67b4640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f34b006cbd0 con 0x7f34b00b0380 2024-04-26T20:51:39.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.430+0000 7f34b67b4640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f34b00af8e0 con 0x7f34b00b10b0 2024-04-26T20:51:39.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.430+0000 7f34affff640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f34b00b0380 0x7f34b00ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:39.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.430+0000 7f34b4d51640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f34b006c780 0x7f34b0072f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40832/0 (socket says 172.21.15.73:40832) 2024-04-26T20:51:39.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.430+0000 7f34b4d51640 1 -- 172.21.15.73:0/3376156091 learned_addr learned my addr 172.21.15.73:0/3376156091 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:39.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.430+0000 7f34af7fe640 1 --2- 172.21.15.73:0/3376156091 >> v2:172.21.15.120:3300/0 conn(0x7f34b00b10b0 0x7f34b006c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:39.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.431+0000 7f34affff640 1 -- 172.21.15.73:0/3376156091 >> v2:172.21.15.120:3300/0 conn(0x7f34b00b10b0 msgr2=0x7f34b006c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:39.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.431+0000 7f34affff640 1 --2- 172.21.15.73:0/3376156091 >> v2:172.21.15.120:3300/0 conn(0x7f34b00b10b0 0x7f34b006c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:39.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.431+0000 7f34affff640 1 -- 172.21.15.73:0/3376156091 >> v1:172.21.15.73:6789/0 conn(0x7f34b006c780 legacy=0x7f34b0072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:39.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.431+0000 7f34affff640 1 -- 172.21.15.73:0/3376156091 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34b00d0020 con 0x7f34b00b0380 2024-04-26T20:51:39.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.431+0000 7f34aeffd640 1 -- 172.21.15.73:0/3376156091 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4031394115 0 0) 0x7f34b00cfd80 con 0x7f34b006c780 2024-04-26T20:51:39.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.431+0000 7f34affff640 1 --2- 172.21.15.73:0/3376156091 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34b00b0380 0x7f34b00ccd50 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f34b00b48c0 tx=0x7f34a002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9528ac3c6de72388 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:39.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.431+0000 7f34aeffd640 1 -- 172.21.15.73:0/3376156091 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a0037070 con 0x7f34b00b0380 2024-04-26T20:51:39.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.431+0000 7f34aeffd640 1 -- 172.21.15.73:0/3376156091 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34a0032440 con 0x7f34b00b0380 2024-04-26T20:51:39.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.431+0000 7f34aeffd640 1 -- 172.21.15.73:0/3376156091 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a002ae60 con 0x7f34b00b0380 2024-04-26T20:51:39.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.432+0000 7f34b67b4640 1 -- 172.21.15.73:0/3376156091 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34b00b0380 msgr2=0x7f34b00ccd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:39.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.432+0000 7f34b67b4640 1 --2- 172.21.15.73:0/3376156091 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34b00b0380 0x7f34b00ccd50 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f34b00b48c0 tx=0x7f34a002a110 comp rx=0 tx=0).stop 2024-04-26T20:51:39.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.432+0000 7f34b67b4640 1 -- 172.21.15.73:0/3376156091 shutdown_connections 2024-04-26T20:51:39.434 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.432+0000 7f34b67b4640 1 --2- 172.21.15.73:0/3376156091 >> v2:172.21.15.120:3300/0 conn(0x7f34b00b10b0 0x7f34b006c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:39.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.432+0000 7f34b67b4640 1 --2- 172.21.15.73:0/3376156091 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34b00b0380 0x7f34b00ccd50 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:39.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.432+0000 7f34b67b4640 1 -- 172.21.15.73:0/3376156091 >> 172.21.15.73:0/3376156091 conn(0x7f34b00d1cf0 msgr2=0x7f34b00d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:39.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.432+0000 7f34b67b4640 1 -- 172.21.15.73:0/3376156091 shutdown_connections 2024-04-26T20:51:39.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.432+0000 7f34b67b4640 1 -- 172.21.15.73:0/3376156091 wait complete. 2024-04-26T20:51:39.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.433+0000 7f34b67b4640 1 Processor -- start 2024-04-26T20:51:39.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.433+0000 7f34b67b4640 1 -- start start 2024-04-26T20:51:39.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.434+0000 7f34b67b4640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f34b006c780 0x7f34b01526e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:39.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.434+0000 7f34b67b4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34b00b0380 0x7f34b015f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:39.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.434+0000 7f34b67b4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f34b00b10b0 0x7f34b006fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:39.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.434+0000 7f34b67b4640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f34b00d1980 con 0x7f34b00b0380 2024-04-26T20:51:39.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.434+0000 7f34b67b4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f34b00d05e0 con 0x7f34b00b10b0 2024-04-26T20:51:39.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.434+0000 7f34b67b4640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f34b00b0760 con 0x7f34b006c780 2024-04-26T20:51:39.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.434+0000 7f34af7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34b00b0380 0x7f34b015f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:39.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.434+0000 7f34af7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34b00b0380 0x7f34b015f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50368/0 (socket says 172.21.15.73:50368) 2024-04-26T20:51:39.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.434+0000 7f34af7fe640 1 -- 172.21.15.73:0/3032677048 learned_addr learned my addr 172.21.15.73:0/3032677048 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:39.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.434+0000 7f34b4d51640 1 --2- 172.21.15.73:0/3032677048 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f34b00b10b0 0x7f34b006fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:39.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.434+0000 7f34affff640 1 --2- 172.21.15.73:0/3032677048 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f34b006c780 0x7f34b01526e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:39.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.434+0000 7f34af7fe640 1 -- 172.21.15.73:0/3032677048 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f34b006c780 msgr2=0x7f34b01526e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:39.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.434+0000 7f34af7fe640 1 --2- 172.21.15.73:0/3032677048 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f34b006c780 0x7f34b01526e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:39.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.434+0000 7f34af7fe640 1 -- 172.21.15.73:0/3032677048 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f34b00b10b0 msgr2=0x7f34b006fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:39.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.434+0000 7f34af7fe640 1 --2- 172.21.15.73:0/3032677048 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f34b00b10b0 0x7f34b006fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:39.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.434+0000 7f34af7fe640 1 -- 172.21.15.73:0/3032677048 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34b0153020 con 0x7f34b00b0380 2024-04-26T20:51:39.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.434+0000 7f34b4d51640 1 --2- 172.21.15.73:0/3032677048 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f34b00b10b0 0x7f34b006fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:39.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.435+0000 7f34af7fe640 1 --2- 172.21.15.73:0/3032677048 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34b00b0380 0x7f34b015f790 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f34a400e9b0 tx=0x7f34a400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:39.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.435+0000 7f34acff9640 1 -- 172.21.15.73:0/3032677048 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a4017070 con 0x7f34b00b0380 2024-04-26T20:51:39.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.435+0000 7f34acff9640 1 -- 172.21.15.73:0/3032677048 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34a400c9a0 con 0x7f34b00b0380 2024-04-26T20:51:39.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.435+0000 7f34acff9640 1 -- 172.21.15.73:0/3032677048 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a400cd90 con 0x7f34b00b0380 2024-04-26T20:51:39.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.435+0000 7f34b67b4640 1 -- 172.21.15.73:0/3032677048 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34b015d860 con 0x7f34b00b0380 2024-04-26T20:51:39.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.435+0000 7f34b67b4640 1 -- 172.21.15.73:0/3032677048 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34b015de20 con 0x7f34b00b0380 2024-04-26T20:51:39.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.436+0000 7f34b67b4640 1 -- 172.21.15.73:0/3032677048 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34b00694f0 con 0x7f34b00b0380 2024-04-26T20:51:39.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.436+0000 7f34acff9640 1 -- 172.21.15.73:0/3032677048 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f34a401a070 con 0x7f34b00b0380 2024-04-26T20:51:39.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.437+0000 7f34acff9640 1 --2- 172.21.15.73:0/3032677048 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f349c06c920 0x7f349c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:39.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.437+0000 7f34affff640 1 --2- 172.21.15.73:0/3032677048 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f349c06c920 0x7f349c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:39.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.437+0000 7f34acff9640 1 -- 172.21.15.73:0/3032677048 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f34a408b900 con 0x7f34b00b0380 2024-04-26T20:51:39.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.437+0000 7f34affff640 1 --2- 172.21.15.73:0/3032677048 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f349c06c920 0x7f349c06ede0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f34a0034000 tx=0x7f34a002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:39.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.441+0000 7f34acff9640 1 -- 172.21.15.73:0/3032677048 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f34b0153020 con 0x7f34b00b0380 2024-04-26T20:51:39.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.687+0000 7f34b67b4640 1 -- 172.21.15.73:0/3032677048 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f34b006f5f0 con 0x7f34b00b0380 2024-04-26T20:51:39.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.688+0000 7f34acff9640 1 -- 172.21.15.73:0/3032677048 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) ==== 80+0+6645 (secure 0 0 0) 0x7f34a4059f80 con 0x7f34b00b0380 2024-04-26T20:51:39.689 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:51:39.689 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-26T20:51:39.689 INFO:teuthology.orchestra.run.smithi073.stdout: "auth_dump": [ 2024-04-26T20:51:39.689 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-26T20:51:39.689 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.a", 2024-04-26T20:51:39.689 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQBUEyxmAE6mGhAAe52HRhOCVM94+tYM6+D8CQ==", 2024-04-26T20:51:39.689 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-26T20:51:39.690 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-26T20:51:39.690 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-26T20:51:39.690 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-26T20:51:39.690 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-26T20:51:39.690 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-26T20:51:39.690 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-26T20:51:39.690 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-26T20:51:39.690 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.b", 2024-04-26T20:51:39.690 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQBUEyxmS7XbJRAAqlMo8v1BqQcHUrDIScQh/Q==", 2024-04-26T20:51:39.690 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-26T20:51:39.690 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-26T20:51:39.690 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-26T20:51:39.690 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-26T20:51:39.690 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-26T20:51:39.691 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-26T20:51:39.691 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-26T20:51:39.691 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-26T20:51:39.691 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.c", 2024-04-26T20:51:39.691 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQBUEyxmpyRtIRAAfZQyLswSVyCCJ/G/i2R+XQ==", 2024-04-26T20:51:39.691 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-26T20:51:39.691 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-26T20:51:39.691 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-26T20:51:39.691 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-26T20:51:39.691 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-26T20:51:39.691 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-26T20:51:39.691 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-26T20:51:39.691 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-26T20:51:39.691 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.d", 2024-04-26T20:51:39.692 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQBUEyxmzMqgLBAAF33EqesRSqAWg392qc9ZIQ==", 2024-04-26T20:51:39.692 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-26T20:51:39.692 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-26T20:51:39.692 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-26T20:51:39.692 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-26T20:51:39.692 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-26T20:51:39.692 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-26T20:51:39.692 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-26T20:51:39.692 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-26T20:51:39.692 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.0", 2024-04-26T20:51:39.692 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQBYEyxmGSyTCRAAHyan+CyQu5SN2n3m7nvHKg==", 2024-04-26T20:51:39.692 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-26T20:51:39.692 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-26T20:51:39.692 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-26T20:51:39.692 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-26T20:51:39.693 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-26T20:51:39.693 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-26T20:51:39.693 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-26T20:51:39.693 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.1", 2024-04-26T20:51:39.693 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQBbEyxmD2XdBxAA80mAzCXbmZGi80w3Dx6CHg==", 2024-04-26T20:51:39.693 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-26T20:51:39.693 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-26T20:51:39.693 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-26T20:51:39.693 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-26T20:51:39.693 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-26T20:51:39.693 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-26T20:51:39.693 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-26T20:51:39.694 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.2", 2024-04-26T20:51:39.694 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQBeEyxmxfWzBRAAufFGKiy5pPW++6QRW3PHgQ==", 2024-04-26T20:51:39.694 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-26T20:51:39.694 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-26T20:51:39.694 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-26T20:51:39.694 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-26T20:51:39.694 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-26T20:51:39.694 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-26T20:51:39.694 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-26T20:51:39.694 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.3", 2024-04-26T20:51:39.694 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQBhEyxmT1dmAxAANZmJAcTSTsZUotOhTQiTKA==", 2024-04-26T20:51:39.694 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-26T20:51:39.694 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-26T20:51:39.695 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-26T20:51:39.695 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-26T20:51:39.695 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-26T20:51:39.695 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-26T20:51:39.695 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-26T20:51:39.695 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.4", 2024-04-26T20:51:39.695 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQBnEyxmjwN8ChAA4uHxtDWrxi+NX5t0VCNGag==", 2024-04-26T20:51:39.695 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-26T20:51:39.695 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-26T20:51:39.695 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-26T20:51:39.695 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-26T20:51:39.695 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-26T20:51:39.695 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-26T20:51:39.695 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-26T20:51:39.695 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.5", 2024-04-26T20:51:39.695 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQBqEyxmnCEwCBAAGsOlPpl0/bxbbcDlxufpBw==", 2024-04-26T20:51:39.696 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-26T20:51:39.696 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-26T20:51:39.696 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-26T20:51:39.696 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-26T20:51:39.696 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-26T20:51:39.696 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-26T20:51:39.696 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-26T20:51:39.696 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.6", 2024-04-26T20:51:39.696 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQBtEyxmRmZ5BhAAG/JdOd2r48xb8wHeXJNK0A==", 2024-04-26T20:51:39.696 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-26T20:51:39.696 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-26T20:51:39.696 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-26T20:51:39.696 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-26T20:51:39.696 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-26T20:51:39.697 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-26T20:51:39.697 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-26T20:51:39.697 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.7", 2024-04-26T20:51:39.697 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQBwEyxmtK3mBhAAEe5+8WbXSYIWx2YstWwE6g==", 2024-04-26T20:51:39.697 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-26T20:51:39.697 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-26T20:51:39.697 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-26T20:51:39.697 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-26T20:51:39.697 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-26T20:51:39.697 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-26T20:51:39.697 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-26T20:51:39.697 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.0", 2024-04-26T20:51:39.697 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQBUEyxmch+BMBAAap3kvW3+/FlCuAkhRtbWxQ==", 2024-04-26T20:51:39.697 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-26T20:51:39.697 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-26T20:51:39.698 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2024-04-26T20:51:39.698 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2024-04-26T20:51:39.698 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2024-04-26T20:51:39.698 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-26T20:51:39.698 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-26T20:51:39.698 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-26T20:51:39.698 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.1", 2024-04-26T20:51:39.698 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQBUEyxmg2vgOBAAPAq4hzJxsRywJoOawl7r7g==", 2024-04-26T20:51:39.698 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-26T20:51:39.699 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-26T20:51:39.699 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2024-04-26T20:51:39.699 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2024-04-26T20:51:39.699 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2024-04-26T20:51:39.699 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-26T20:51:39.699 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-26T20:51:39.699 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-26T20:51:39.699 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.2", 2024-04-26T20:51:39.699 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQBUEyxmHJ7fNRAAbjIeTP+R5KsjQ5IYlmpqFg==", 2024-04-26T20:51:39.699 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-26T20:51:39.699 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-26T20:51:39.699 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2024-04-26T20:51:39.699 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2024-04-26T20:51:39.699 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2024-04-26T20:51:39.699 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-26T20:51:39.700 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-26T20:51:39.700 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-26T20:51:39.700 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.3", 2024-04-26T20:51:39.700 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQBVEyxmt+GfAhAA5qvKp2+NlXcGj975bUQS9Q==", 2024-04-26T20:51:39.700 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-26T20:51:39.700 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-26T20:51:39.700 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2024-04-26T20:51:39.700 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2024-04-26T20:51:39.700 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2024-04-26T20:51:39.700 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-26T20:51:39.700 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-26T20:51:39.700 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-26T20:51:39.700 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.admin", 2024-04-26T20:51:39.701 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQBUEyxmPhJRABAAwL7GwQXnfU+Erheps382PQ==", 2024-04-26T20:51:39.701 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-26T20:51:39.701 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2024-04-26T20:51:39.701 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-26T20:51:39.701 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-26T20:51:39.701 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-26T20:51:39.701 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-26T20:51:39.701 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-26T20:51:39.701 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-26T20:51:39.701 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.bootstrap-mds", 2024-04-26T20:51:39.701 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQB/EyxmmoNLNBAAXdfnTiavPM1/UXjoAlEgBQ==", 2024-04-26T20:51:39.701 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-26T20:51:39.701 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile bootstrap-mds" 2024-04-26T20:51:39.701 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-26T20:51:39.701 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-26T20:51:39.702 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-26T20:51:39.702 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.bootstrap-mgr", 2024-04-26T20:51:39.702 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQB/EyxmiItLNBAAVB5DKeLPB70PXP+6MAQinw==", 2024-04-26T20:51:39.702 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-26T20:51:39.702 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-26T20:51:39.702 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-26T20:51:39.702 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-26T20:51:39.702 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-26T20:51:39.702 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.bootstrap-osd", 2024-04-26T20:51:39.702 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQB/EyxmepRLNBAAVYmgncMa3h1x7VV2RQGwcg==", 2024-04-26T20:51:39.702 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-26T20:51:39.702 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile bootstrap-osd" 2024-04-26T20:51:39.702 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-26T20:51:39.703 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-26T20:51:39.703 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-26T20:51:39.703 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.bootstrap-rbd", 2024-04-26T20:51:39.703 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQB/EyxmSZtLNBAAjrGbWExrNlU5sRukMmYpkg==", 2024-04-26T20:51:39.703 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-26T20:51:39.703 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-26T20:51:39.703 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-26T20:51:39.703 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-26T20:51:39.703 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-26T20:51:39.703 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-26T20:51:39.703 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQB/EyxmjaJLNBAAaQw6PB9Sv5KI9YmfToFn/Q==", 2024-04-26T20:51:39.703 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-26T20:51:39.703 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-26T20:51:39.703 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-26T20:51:39.703 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-26T20:51:39.704 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-26T20:51:39.704 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.bootstrap-rgw", 2024-04-26T20:51:39.704 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQB/EyxmmalLNBAAGpd5vh4QWeQ6BZfyDoULwA==", 2024-04-26T20:51:39.704 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-26T20:51:39.704 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-26T20:51:39.704 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-26T20:51:39.704 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-26T20:51:39.704 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-26T20:51:39.704 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mgr.x", 2024-04-26T20:51:39.704 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQBUEyxmLbWjFxAAo2evZCpMUZUHkU6N+eDa4A==", 2024-04-26T20:51:39.704 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-26T20:51:39.704 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2024-04-26T20:51:39.704 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile mgr", 2024-04-26T20:51:39.704 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-26T20:51:39.704 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-26T20:51:39.705 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-26T20:51:39.705 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-26T20:51:39.705 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mgr.y", 2024-04-26T20:51:39.705 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQBUEyxmZnhRFBAAB2V6HQEbpoUNUbAflZok8A==", 2024-04-26T20:51:39.705 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-26T20:51:39.705 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2024-04-26T20:51:39.705 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile mgr", 2024-04-26T20:51:39.705 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-26T20:51:39.705 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-26T20:51:39.705 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-26T20:51:39.705 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-26T20:51:39.705 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-26T20:51:39.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.690+0000 7f34b67b4640 1 -- 172.21.15.73:0/3032677048 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f349c06c920 msgr2=0x7f349c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:39.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.690+0000 7f34b67b4640 1 --2- 172.21.15.73:0/3032677048 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f349c06c920 0x7f349c06ede0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f34a0034000 tx=0x7f34a002e000 comp rx=0 tx=0).stop 2024-04-26T20:51:39.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.690+0000 7f34b67b4640 1 -- 172.21.15.73:0/3032677048 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34b00b0380 msgr2=0x7f34b015f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:39.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.690+0000 7f34b67b4640 1 --2- 172.21.15.73:0/3032677048 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34b00b0380 0x7f34b015f790 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f34a400e9b0 tx=0x7f34a400ee70 comp rx=0 tx=0).stop 2024-04-26T20:51:39.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.690+0000 7f34b67b4640 1 -- 172.21.15.73:0/3032677048 shutdown_connections 2024-04-26T20:51:39.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.690+0000 7f34b67b4640 1 --2- 172.21.15.73:0/3032677048 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34b00b0380 0x7f34b015f790 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:39.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.690+0000 7f34b67b4640 1 --2- 172.21.15.73:0/3032677048 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f349c06c920 0x7f349c06ede0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:39.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.690+0000 7f34b67b4640 1 --2- 172.21.15.73:0/3032677048 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f34b00b10b0 0x7f34b006fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:39.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.690+0000 7f34b67b4640 1 --2- 172.21.15.73:0/3032677048 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f34b006c780 0x7f34b01526e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:39.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.690+0000 7f34b67b4640 1 -- 172.21.15.73:0/3032677048 >> 172.21.15.73:0/3032677048 conn(0x7f34b00d1cf0 msgr2=0x7f34b0072f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:39.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.690+0000 7f34b67b4640 1 -- 172.21.15.73:0/3032677048 shutdown_connections 2024-04-26T20:51:39.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.690+0000 7f34b67b4640 1 -- 172.21.15.73:0/3032677048 wait complete. 2024-04-26T20:51:39.709 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-26T20:51:39.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.805+0000 7efd8059d640 1 Processor -- start 2024-04-26T20:51:39.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.805+0000 7efd8059d640 1 -- start start 2024-04-26T20:51:39.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.806+0000 7efd8059d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7efd7815f1d0 0x7efd7806f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:39.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.806+0000 7efd8059d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efd780afd60 0x7efd7806c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:39.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.806+0000 7efd8059d640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd780b4480 con 0x7efd7806c240 2024-04-26T20:51:39.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.806+0000 7efd8059d640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7efd7806cbe0 con 0x7efd7815f1d0 2024-04-26T20:51:39.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.806+0000 7efd8059d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7efd780ae8c0 con 0x7efd780afd60 2024-04-26T20:51:39.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.806+0000 7efd7e339640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7efd7815f1d0 0x7efd7806f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:39.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.806+0000 7efd7eb3a640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efd780afd60 0x7efd7806c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:39.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.806+0000 7efd7db38640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7efd7806c240 0x7efd780726e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.73:57954/0 (socket says 172.21.15.73:57954) 2024-04-26T20:51:39.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.806+0000 7efd7db38640 1 -- 172.21.15.73:0/2429039383 learned_addr learned my addr 172.21.15.73:0/2429039383 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:39.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.806+0000 7efd7e339640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7efd7815f1d0 0x7efd7806f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.73:38542/0 (socket says 172.21.15.73:38542) 2024-04-26T20:51:39.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.807+0000 7efd7e339640 1 -- 172.21.15.73:0/2429039383 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efd780afd60 msgr2=0x7efd7806c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:39.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.807+0000 7efd7e339640 1 --2- 172.21.15.73:0/2429039383 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efd780afd60 0x7efd7806c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:39.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.807+0000 7efd7e339640 1 -- 172.21.15.73:0/2429039383 >> v1:172.21.15.120:6789/0 conn(0x7efd7806c240 legacy=0x7efd780726e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:39.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.807+0000 7efd7e339640 1 -- 172.21.15.73:0/2429039383 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd780b4720 con 0x7efd7815f1d0 2024-04-26T20:51:39.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.807+0000 7efd7e339640 1 --2- 172.21.15.73:0/2429039383 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd7815f1d0 0x7efd7806f170 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efd74009d90 tx=0x7efd7402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1a68c8dd414b296 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:39.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.808+0000 7efd7d337640 1 -- 172.21.15.73:0/2429039383 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd74007740 con 0x7efd7815f1d0 2024-04-26T20:51:39.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.808+0000 7efd7d337640 1 -- 172.21.15.73:0/2429039383 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd740078e0 con 0x7efd7815f1d0 2024-04-26T20:51:39.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.808+0000 7efd7d337640 1 -- 172.21.15.73:0/2429039383 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd74007cb0 con 0x7efd7815f1d0 2024-04-26T20:51:39.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.808+0000 7efd8059d640 1 -- 172.21.15.73:0/2429039383 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd7815f1d0 msgr2=0x7efd7806f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:39.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.808+0000 7efd8059d640 1 --2- 172.21.15.73:0/2429039383 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd7815f1d0 0x7efd7806f170 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efd74009d90 tx=0x7efd7402a660 comp rx=0 tx=0).stop 2024-04-26T20:51:39.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.808+0000 7efd8059d640 1 -- 172.21.15.73:0/2429039383 shutdown_connections 2024-04-26T20:51:39.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.808+0000 7efd8059d640 1 --2- 172.21.15.73:0/2429039383 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efd780afd60 0x7efd7806c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:39.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.808+0000 7efd8059d640 1 --2- 172.21.15.73:0/2429039383 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd7815f1d0 0x7efd7806f170 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:39.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.808+0000 7efd8059d640 1 -- 172.21.15.73:0/2429039383 >> 172.21.15.73:0/2429039383 conn(0x7efd780b3180 msgr2=0x7efd780b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:39.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.808+0000 7efd8059d640 1 -- 172.21.15.73:0/2429039383 shutdown_connections 2024-04-26T20:51:39.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.808+0000 7efd8059d640 1 -- 172.21.15.73:0/2429039383 wait complete. 2024-04-26T20:51:39.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.809+0000 7efd8059d640 1 Processor -- start 2024-04-26T20:51:39.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.809+0000 7efd8059d640 1 -- start start 2024-04-26T20:51:39.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.810+0000 7efd8059d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd7806c240 0x7efd781eaff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:39.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.810+0000 7efd8059d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efd780afd60 0x7efd7815b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:39.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.810+0000 7efd8059d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd7815f1d0 0x7efd781574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:39.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.810+0000 7efd8059d640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efd780b4fc0 con 0x7efd7806c240 2024-04-26T20:51:39.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.810+0000 7efd8059d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efd780b4ce0 con 0x7efd7815f1d0 2024-04-26T20:51:39.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.810+0000 7efd8059d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7efd7806f6b0 con 0x7efd780afd60 2024-04-26T20:51:39.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.810+0000 7efd7e339640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd7806c240 0x7efd781eaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:39.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.810+0000 7efd7e339640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd7806c240 0x7efd781eaff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54346/0 (socket says 172.21.15.73:54346) 2024-04-26T20:51:39.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.810+0000 7efd7e339640 1 -- 172.21.15.73:0/2849501342 learned_addr learned my addr 172.21.15.73:0/2849501342 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:39.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.810+0000 7efd7eb3a640 1 --2- 172.21.15.73:0/2849501342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd7815f1d0 0x7efd781574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:39.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.810+0000 7efd7db38640 1 --2- 172.21.15.73:0/2849501342 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efd780afd60 0x7efd7815b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:39.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.811+0000 7efd7e339640 1 -- 172.21.15.73:0/2849501342 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efd780afd60 msgr2=0x7efd7815b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:39.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.811+0000 7efd7e339640 1 --2- 172.21.15.73:0/2849501342 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efd780afd60 0x7efd7815b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:39.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.811+0000 7efd7e339640 1 -- 172.21.15.73:0/2849501342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd7815f1d0 msgr2=0x7efd781574f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:39.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.811+0000 7efd7e339640 1 --2- 172.21.15.73:0/2849501342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd7815f1d0 0x7efd781574f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:39.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.811+0000 7efd7e339640 1 -- 172.21.15.73:0/2849501342 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd7815b650 con 0x7efd7806c240 2024-04-26T20:51:39.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.811+0000 7efd7db38640 1 --2- 172.21.15.73:0/2849501342 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efd780afd60 0x7efd7815b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:39.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.811+0000 7efd7e339640 1 --2- 172.21.15.73:0/2849501342 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd7806c240 0x7efd781eaff0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7efd74009eb0 tx=0x7efd74004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:39.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.811+0000 7efd6effd640 1 -- 172.21.15.73:0/2849501342 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd74040070 con 0x7efd7806c240 2024-04-26T20:51:39.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.811+0000 7efd6effd640 1 -- 172.21.15.73:0/2849501342 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd74031bc0 con 0x7efd7806c240 2024-04-26T20:51:39.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.811+0000 7efd8059d640 1 -- 172.21.15.73:0/2849501342 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd781eb690 con 0x7efd7806c240 2024-04-26T20:51:39.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.811+0000 7efd6effd640 1 -- 172.21.15.73:0/2849501342 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7403c420 con 0x7efd7806c240 2024-04-26T20:51:39.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.811+0000 7efd8059d640 1 -- 172.21.15.73:0/2849501342 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd781e7e10 con 0x7efd7806c240 2024-04-26T20:51:39.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.812+0000 7efd8059d640 1 -- 172.21.15.73:0/2849501342 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd78068fc0 con 0x7efd7806c240 2024-04-26T20:51:39.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.812+0000 7efd6effd640 1 -- 172.21.15.73:0/2849501342 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efd74039040 con 0x7efd7806c240 2024-04-26T20:51:39.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.813+0000 7efd6effd640 1 --2- 172.21.15.73:0/2849501342 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7efd5c06c8f0 0x7efd5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:39.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.813+0000 7efd6effd640 1 -- 172.21.15.73:0/2849501342 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7efd740ace70 con 0x7efd7806c240 2024-04-26T20:51:39.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.813+0000 7efd7db38640 1 --2- 172.21.15.73:0/2849501342 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7efd5c06c8f0 0x7efd5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:39.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.813+0000 7efd7db38640 1 --2- 172.21.15.73:0/2849501342 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7efd5c06c8f0 0x7efd5c06edb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efd780d0700 tx=0x7efd68009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:39.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:39.817+0000 7efd6effd640 1 -- 172.21.15.73:0/2849501342 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efd7404c350 con 0x7efd7806c240 2024-04-26T20:51:40.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.062+0000 7efd8059d640 1 -- 172.21.15.73:0/2849501342 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7efd781629f0 con 0x7efd7806c240 2024-04-26T20:51:40.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.064+0000 7efd6effd640 1 -- 172.21.15.73:0/2849501342 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7efd7407b3e0 con 0x7efd7806c240 2024-04-26T20:51:40.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.066+0000 7efd8059d640 1 -- 172.21.15.73:0/2849501342 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7efd5c06c8f0 msgr2=0x7efd5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:40.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.066+0000 7efd8059d640 1 --2- 172.21.15.73:0/2849501342 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7efd5c06c8f0 0x7efd5c06edb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efd780d0700 tx=0x7efd68009040 comp rx=0 tx=0).stop 2024-04-26T20:51:40.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.066+0000 7efd8059d640 1 -- 172.21.15.73:0/2849501342 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd7806c240 msgr2=0x7efd781eaff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:40.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.066+0000 7efd8059d640 1 --2- 172.21.15.73:0/2849501342 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd7806c240 0x7efd781eaff0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7efd74009eb0 tx=0x7efd74004a40 comp rx=0 tx=0).stop 2024-04-26T20:51:40.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.066+0000 7efd8059d640 1 -- 172.21.15.73:0/2849501342 shutdown_connections 2024-04-26T20:51:40.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.066+0000 7efd8059d640 1 --2- 172.21.15.73:0/2849501342 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7efd5c06c8f0 0x7efd5c06edb0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:40.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.066+0000 7efd8059d640 1 --2- 172.21.15.73:0/2849501342 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd7815f1d0 0x7efd781574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:40.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.066+0000 7efd8059d640 1 --2- 172.21.15.73:0/2849501342 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efd780afd60 0x7efd7815b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:40.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.066+0000 7efd8059d640 1 --2- 172.21.15.73:0/2849501342 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd7806c240 0x7efd781eaff0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:40.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.066+0000 7efd8059d640 1 -- 172.21.15.73:0/2849501342 >> 172.21.15.73:0/2849501342 conn(0x7efd780b3180 msgr2=0x7efd780d14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:40.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.066+0000 7efd8059d640 1 -- 172.21.15.73:0/2849501342 shutdown_connections 2024-04-26T20:51:40.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.066+0000 7efd8059d640 1 -- 172.21.15.73:0/2849501342 wait complete. 2024-04-26T20:51:40.078 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-26T20:51:40.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.175+0000 7f88623b6640 1 Processor -- start 2024-04-26T20:51:40.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.175+0000 7f88623b6640 1 -- start start 2024-04-26T20:51:40.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.176+0000 7f88623b6640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f885c1606f0 0x7f885c0ce890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:40.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.176+0000 7f88623b6640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f885c0d38f0 0x7f885c0d09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:40.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.176+0000 7f88623b6640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f885c0d10f0 0x7f885c0d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:40.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.176+0000 7f88623b6640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f885c0cdb10 con 0x7f885c0d10f0 2024-04-26T20:51:40.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.176+0000 7f88623b6640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f885c0cddb0 con 0x7f885c0d38f0 2024-04-26T20:51:40.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.176+0000 7f88623b6640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f885c0ce090 con 0x7f885c1606f0 2024-04-26T20:51:40.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.176+0000 7f8860953640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f885c0d10f0 0x7f885c0d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:40.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.176+0000 7f8860953640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f885c0d10f0 0x7f885c0d1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54352/0 (socket says 172.21.15.73:54352) 2024-04-26T20:51:40.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.176+0000 7f8860953640 1 -- 172.21.15.73:0/1369721831 learned_addr learned my addr 172.21.15.73:0/1369721831 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:40.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.177+0000 7f8860953640 1 -- 172.21.15.73:0/1369721831 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f885c1606f0 msgr2=0x7f885c0ce890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:40.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.177+0000 7f8860953640 1 --2- 172.21.15.73:0/1369721831 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f885c1606f0 0x7f885c0ce890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:40.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.177+0000 7f8860953640 1 -- 172.21.15.73:0/1369721831 >> v2:172.21.15.120:3300/0 conn(0x7f885c0d38f0 msgr2=0x7f885c0d09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:40.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.177+0000 7f8860953640 1 --2- 172.21.15.73:0/1369721831 >> v2:172.21.15.120:3300/0 conn(0x7f885c0d38f0 0x7f885c0d09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:40.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.177+0000 7f8860953640 1 -- 172.21.15.73:0/1369721831 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f885c0ce370 con 0x7f885c0d10f0 2024-04-26T20:51:40.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.177+0000 7f8860953640 1 --2- 172.21.15.73:0/1369721831 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f885c0d10f0 0x7f885c0d1580 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f8854009e50 tx=0x7f885402a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fde2efd08454da54 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:40.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.177+0000 7f885affd640 1 -- 172.21.15.73:0/1369721831 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8854036070 con 0x7f885c0d10f0 2024-04-26T20:51:40.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.177+0000 7f885affd640 1 -- 172.21.15.73:0/1369721831 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8854031600 con 0x7f885c0d10f0 2024-04-26T20:51:40.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.177+0000 7f885affd640 1 -- 172.21.15.73:0/1369721831 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88540319d0 con 0x7f885c0d10f0 2024-04-26T20:51:40.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.178+0000 7f88623b6640 1 -- 172.21.15.73:0/1369721831 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f885c0d10f0 msgr2=0x7f885c0d1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:40.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.178+0000 7f88623b6640 1 --2- 172.21.15.73:0/1369721831 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f885c0d10f0 0x7f885c0d1580 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f8854009e50 tx=0x7f885402a630 comp rx=0 tx=0).stop 2024-04-26T20:51:40.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.178+0000 7f88623b6640 1 -- 172.21.15.73:0/1369721831 shutdown_connections 2024-04-26T20:51:40.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.178+0000 7f88623b6640 1 --2- 172.21.15.73:0/1369721831 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f885c0d10f0 0x7f885c0d1580 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:40.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.178+0000 7f88623b6640 1 --2- 172.21.15.73:0/1369721831 >> v2:172.21.15.120:3300/0 conn(0x7f885c0d38f0 0x7f885c0d09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:40.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.178+0000 7f88623b6640 1 --2- 172.21.15.73:0/1369721831 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f885c1606f0 0x7f885c0ce890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:40.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.178+0000 7f88623b6640 1 -- 172.21.15.73:0/1369721831 >> 172.21.15.73:0/1369721831 conn(0x7f885c0d2dd0 msgr2=0x7f885c0d31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:40.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.178+0000 7f88623b6640 1 -- 172.21.15.73:0/1369721831 shutdown_connections 2024-04-26T20:51:40.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.178+0000 7f88623b6640 1 -- 172.21.15.73:0/1369721831 wait complete. 2024-04-26T20:51:40.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.179+0000 7f88623b6640 1 Processor -- start 2024-04-26T20:51:40.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.179+0000 7f88623b6640 1 -- start start 2024-04-26T20:51:40.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.180+0000 7f88623b6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f885c0d10f0 0x7f885c14f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:40.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.180+0000 7f88623b6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f885c0d38f0 0x7f885c14b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:40.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.180+0000 7f88623b6640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f885c1606f0 0x7f885c14b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:40.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.180+0000 7f88623b6640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f885c0b5c20 con 0x7f885c0d38f0 2024-04-26T20:51:40.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.180+0000 7f88623b6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f885c0d1cd0 con 0x7f885c0d10f0 2024-04-26T20:51:40.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.180+0000 7f88623b6640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f885c0b5da0 con 0x7f885c1606f0 2024-04-26T20:51:40.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.180+0000 7f885b7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f885c0d38f0 0x7f885c14b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:40.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.180+0000 7f885b7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f885c0d38f0 0x7f885c14b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54360/0 (socket says 172.21.15.73:54360) 2024-04-26T20:51:40.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.180+0000 7f885b7fe640 1 -- 172.21.15.73:0/1448951308 learned_addr learned my addr 172.21.15.73:0/1448951308 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:40.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.180+0000 7f8860953640 1 --2- 172.21.15.73:0/1448951308 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f885c1606f0 0x7f885c14b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:40.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.180+0000 7f885bfff640 1 --2- 172.21.15.73:0/1448951308 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f885c0d10f0 0x7f885c14f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:40.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.181+0000 7f885b7fe640 1 -- 172.21.15.73:0/1448951308 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f885c1606f0 msgr2=0x7f885c14b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:40.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.181+0000 7f885b7fe640 1 --2- 172.21.15.73:0/1448951308 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f885c1606f0 0x7f885c14b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:40.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.181+0000 7f885b7fe640 1 -- 172.21.15.73:0/1448951308 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f885c0d10f0 msgr2=0x7f885c14f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:40.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.181+0000 7f885b7fe640 1 --2- 172.21.15.73:0/1448951308 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f885c0d10f0 0x7f885c14f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:40.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.181+0000 7f885b7fe640 1 -- 172.21.15.73:0/1448951308 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f885c149320 con 0x7f885c0d38f0 2024-04-26T20:51:40.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.181+0000 7f885b7fe640 1 --2- 172.21.15.73:0/1448951308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f885c0d38f0 0x7f885c14b2d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f885000b4a0 tx=0x7f885000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:40.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.181+0000 7f8858ff9640 1 -- 172.21.15.73:0/1448951308 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8850014070 con 0x7f885c0d38f0 2024-04-26T20:51:40.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.181+0000 7f8858ff9640 1 -- 172.21.15.73:0/1448951308 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8850010610 con 0x7f885c0d38f0 2024-04-26T20:51:40.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.181+0000 7f88623b6640 1 -- 172.21.15.73:0/1448951308 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f885c1e6d90 con 0x7f885c0d38f0 2024-04-26T20:51:40.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.181+0000 7f8858ff9640 1 -- 172.21.15.73:0/1448951308 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8850010a00 con 0x7f885c0d38f0 2024-04-26T20:51:40.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.181+0000 7f88623b6640 1 -- 172.21.15.73:0/1448951308 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f885c149870 con 0x7f885c0d38f0 2024-04-26T20:51:40.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.182+0000 7f88623b6640 1 -- 172.21.15.73:0/1448951308 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f885c0694f0 con 0x7f885c0d38f0 2024-04-26T20:51:40.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.182+0000 7f8858ff9640 1 -- 172.21.15.73:0/1448951308 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8850017070 con 0x7f885c0d38f0 2024-04-26T20:51:40.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.182+0000 7f8858ff9640 1 --2- 172.21.15.73:0/1448951308 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f884806c8f0 0x7f884806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:40.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.183+0000 7f8858ff9640 1 -- 172.21.15.73:0/1448951308 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f885008b330 con 0x7f885c0d38f0 2024-04-26T20:51:40.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.183+0000 7f885bfff640 1 --2- 172.21.15.73:0/1448951308 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f884806c8f0 0x7f884806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:40.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.183+0000 7f885bfff640 1 --2- 172.21.15.73:0/1448951308 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f884806c8f0 0x7f884806edb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f884c0098e0 tx=0x7f884c009060 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:40.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.187+0000 7f8858ff9640 1 -- 172.21.15.73:0/1448951308 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f88500592d0 con 0x7f885c0d38f0 2024-04-26T20:51:40.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.432+0000 7f88623b6640 1 -- 172.21.15.73:0/1448951308 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f885c0b8d50 con 0x7f885c0d38f0 2024-04-26T20:51:40.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.436+0000 7f8858ff9640 1 -- 172.21.15.73:0/1448951308 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7f885005d5d0 con 0x7f885c0d38f0 2024-04-26T20:51:40.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.438+0000 7f88623b6640 1 -- 172.21.15.73:0/1448951308 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f884806c8f0 msgr2=0x7f884806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:40.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.438+0000 7f88623b6640 1 --2- 172.21.15.73:0/1448951308 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f884806c8f0 0x7f884806edb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f884c0098e0 tx=0x7f884c009060 comp rx=0 tx=0).stop 2024-04-26T20:51:40.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.438+0000 7f88623b6640 1 -- 172.21.15.73:0/1448951308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f885c0d38f0 msgr2=0x7f885c14b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:40.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.438+0000 7f88623b6640 1 --2- 172.21.15.73:0/1448951308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f885c0d38f0 0x7f885c14b2d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f885000b4a0 tx=0x7f885000b960 comp rx=0 tx=0).stop 2024-04-26T20:51:40.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.438+0000 7f88623b6640 1 -- 172.21.15.73:0/1448951308 shutdown_connections 2024-04-26T20:51:40.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.438+0000 7f88623b6640 1 --2- 172.21.15.73:0/1448951308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f885c0d38f0 0x7f885c14b2d0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:40.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.438+0000 7f88623b6640 1 --2- 172.21.15.73:0/1448951308 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f884806c8f0 0x7f884806edb0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:40.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.438+0000 7f88623b6640 1 --2- 172.21.15.73:0/1448951308 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f885c1606f0 0x7f885c14b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:40.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.438+0000 7f88623b6640 1 --2- 172.21.15.73:0/1448951308 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f885c0d10f0 0x7f885c14f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:40.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.438+0000 7f88623b6640 1 -- 172.21.15.73:0/1448951308 >> 172.21.15.73:0/1448951308 conn(0x7f885c0d2dd0 msgr2=0x7f885c0cf400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:40.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.438+0000 7f88623b6640 1 -- 172.21.15.73:0/1448951308 shutdown_connections 2024-04-26T20:51:40.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.438+0000 7f88623b6640 1 -- 172.21.15.73:0/1448951308 wait complete. 2024-04-26T20:51:40.450 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-26T20:51:40.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.546+0000 7f3234c20640 1 Processor -- start 2024-04-26T20:51:40.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.546+0000 7f3234c20640 1 -- start start 2024-04-26T20:51:40.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.547+0000 7f3234c20640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f32300b0380 0x7f32300ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:40.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.547+0000 7f3234c20640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f323006c950 0x7f323015e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:40.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.547+0000 7f3234c20640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32300cfd80 con 0x7f32300b10b0 2024-04-26T20:51:40.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.547+0000 7f3234c20640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f32300af8e0 con 0x7f323006c950 2024-04-26T20:51:40.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.547+0000 7f3234c20640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f32300d0020 con 0x7f32300b0380 2024-04-26T20:51:40.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.547+0000 7f322dd9b640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f32300b10b0 0x7f323006c240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:41576/0 (socket says 172.21.15.73:41576) 2024-04-26T20:51:40.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.547+0000 7f322dd9b640 1 -- 172.21.15.73:0/1684744218 learned_addr learned my addr 172.21.15.73:0/1684744218 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:40.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.547+0000 7f322ed9d640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f323006c950 0x7f323015e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:40.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.547+0000 7f322e59c640 1 --2- 172.21.15.73:0/1684744218 >> v2:172.21.15.120:3300/0 conn(0x7f32300b0380 0x7f32300ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:40.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.547+0000 7f322d59a640 1 -- 172.21.15.73:0/1684744218 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1465219820 0 0) 0x7f32300cfd80 con 0x7f32300b10b0 2024-04-26T20:51:40.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.547+0000 7f322d59a640 1 -- 172.21.15.73:0/1684744218 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3218003520 con 0x7f32300b10b0 2024-04-26T20:51:40.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.547+0000 7f322ed9d640 1 -- 172.21.15.73:0/1684744218 >> v2:172.21.15.120:3300/0 conn(0x7f32300b0380 msgr2=0x7f32300ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:40.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.547+0000 7f322ed9d640 1 --2- 172.21.15.73:0/1684744218 >> v2:172.21.15.120:3300/0 conn(0x7f32300b0380 0x7f32300ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:40.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.547+0000 7f322ed9d640 1 -- 172.21.15.73:0/1684744218 >> v1:172.21.15.73:6789/0 conn(0x7f32300b10b0 legacy=0x7f323006c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:40.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.547+0000 7f322ed9d640 1 -- 172.21.15.73:0/1684744218 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32300d0300 con 0x7f323006c950 2024-04-26T20:51:40.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.548+0000 7f322ed9d640 1 --2- 172.21.15.73:0/1684744218 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f323006c950 0x7f323015e970 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f323006d290 tx=0x7f322802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86998f0afc984fca server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:40.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.548+0000 7f322d59a640 1 -- 172.21.15.73:0/1684744218 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3228007740 con 0x7f323006c950 2024-04-26T20:51:40.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.548+0000 7f322d59a640 1 -- 172.21.15.73:0/1684744218 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32280078e0 con 0x7f323006c950 2024-04-26T20:51:40.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.548+0000 7f322d59a640 1 -- 172.21.15.73:0/1684744218 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3228007cb0 con 0x7f323006c950 2024-04-26T20:51:40.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.549+0000 7f3234c20640 1 -- 172.21.15.73:0/1684744218 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f323006c950 msgr2=0x7f323015e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:40.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.549+0000 7f3234c20640 1 --2- 172.21.15.73:0/1684744218 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f323006c950 0x7f323015e970 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f323006d290 tx=0x7f322802a5f0 comp rx=0 tx=0).stop 2024-04-26T20:51:40.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.549+0000 7f3234c20640 1 -- 172.21.15.73:0/1684744218 shutdown_connections 2024-04-26T20:51:40.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.549+0000 7f3234c20640 1 --2- 172.21.15.73:0/1684744218 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f323006c950 0x7f323015e970 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:40.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.549+0000 7f3234c20640 1 --2- 172.21.15.73:0/1684744218 >> v2:172.21.15.120:3300/0 conn(0x7f32300b0380 0x7f32300ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:40.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.549+0000 7f3234c20640 1 -- 172.21.15.73:0/1684744218 >> 172.21.15.73:0/1684744218 conn(0x7f32300d1cf0 msgr2=0x7f32300d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:40.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.549+0000 7f3234c20640 1 -- 172.21.15.73:0/1684744218 shutdown_connections 2024-04-26T20:51:40.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.549+0000 7f3234c20640 1 -- 172.21.15.73:0/1684744218 wait complete. 2024-04-26T20:51:40.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.550+0000 7f3234c20640 1 Processor -- start 2024-04-26T20:51:40.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.550+0000 7f3234c20640 1 -- start start 2024-04-26T20:51:40.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.551+0000 7f3234c20640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f323006c950 0x7f32301570b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:40.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.551+0000 7f3234c20640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32300b0380 0x7f32300b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:40.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.551+0000 7f3234c20640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32300b10b0 0x7f32301620a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:40.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.551+0000 7f3234c20640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f32300b0760 con 0x7f32300b0380 2024-04-26T20:51:40.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.551+0000 7f3234c20640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f32300d1980 con 0x7f32300b10b0 2024-04-26T20:51:40.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.551+0000 7f3234c20640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f32300d15c0 con 0x7f323006c950 2024-04-26T20:51:40.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.551+0000 7f322dd9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32300b0380 0x7f32300b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:40.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.551+0000 7f322dd9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32300b0380 0x7f32300b72b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54372/0 (socket says 172.21.15.73:54372) 2024-04-26T20:51:40.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.551+0000 7f322dd9b640 1 -- 172.21.15.73:0/840296035 learned_addr learned my addr 172.21.15.73:0/840296035 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:40.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.551+0000 7f322ed9d640 1 --2- 172.21.15.73:0/840296035 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32300b10b0 0x7f32301620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:40.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.551+0000 7f322e59c640 1 --2- 172.21.15.73:0/840296035 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f323006c950 0x7f32301570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:40.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.551+0000 7f322dd9b640 1 -- 172.21.15.73:0/840296035 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f323006c950 msgr2=0x7f32301570b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:40.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.551+0000 7f322dd9b640 1 --2- 172.21.15.73:0/840296035 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f323006c950 0x7f32301570b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:40.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.551+0000 7f322dd9b640 1 -- 172.21.15.73:0/840296035 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32300b10b0 msgr2=0x7f32301620a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:40.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.551+0000 7f322dd9b640 1 --2- 172.21.15.73:0/840296035 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32300b10b0 0x7f32301620a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:40.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.551+0000 7f322dd9b640 1 -- 172.21.15.73:0/840296035 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32300c7350 con 0x7f32300b0380 2024-04-26T20:51:40.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.551+0000 7f322ed9d640 1 --2- 172.21.15.73:0/840296035 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32300b10b0 0x7f32301620a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:40.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.551+0000 7f322dd9b640 1 --2- 172.21.15.73:0/840296035 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32300b0380 0x7f32300b72b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f322400c400 tx=0x7f322400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:40.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.551+0000 7f321effd640 1 -- 172.21.15.73:0/840296035 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3224017070 con 0x7f32300b0380 2024-04-26T20:51:40.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.551+0000 7f321effd640 1 -- 172.21.15.73:0/840296035 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3224005d00 con 0x7f32300b0380 2024-04-26T20:51:40.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.551+0000 7f3234c20640 1 -- 172.21.15.73:0/840296035 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32301f8390 con 0x7f32300b0380 2024-04-26T20:51:40.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.551+0000 7f321effd640 1 -- 172.21.15.73:0/840296035 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3224007310 con 0x7f32300b0380 2024-04-26T20:51:40.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.551+0000 7f3234c20640 1 -- 172.21.15.73:0/840296035 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32301f8950 con 0x7f32300b0380 2024-04-26T20:51:40.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.552+0000 7f3234c20640 1 -- 172.21.15.73:0/840296035 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32300694f0 con 0x7f32300b0380 2024-04-26T20:51:40.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.553+0000 7f321effd640 1 -- 172.21.15.73:0/840296035 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f322401a070 con 0x7f32300b0380 2024-04-26T20:51:40.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.553+0000 7f321effd640 1 --2- 172.21.15.73:0/840296035 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f321006c8f0 0x7f321006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:40.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.553+0000 7f321effd640 1 -- 172.21.15.73:0/840296035 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f322408ad50 con 0x7f32300b0380 2024-04-26T20:51:40.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.553+0000 7f322e59c640 1 --2- 172.21.15.73:0/840296035 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f321006c8f0 0x7f321006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:40.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.554+0000 7f322e59c640 1 --2- 172.21.15.73:0/840296035 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f321006c8f0 0x7f321006edb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f32300b2430 tx=0x7f3220008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:40.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.558+0000 7f321effd640 1 -- 172.21.15.73:0/840296035 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f322408e030 con 0x7f32300b0380 2024-04-26T20:51:40.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.801+0000 7f3234c20640 1 -- 172.21.15.73:0/840296035 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f32300cc210 con 0x7f32300b0380 2024-04-26T20:51:40.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.803+0000 7f321effd640 1 -- 172.21.15.73:0/840296035 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7f3224059340 con 0x7f32300b0380 2024-04-26T20:51:40.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.805+0000 7f3234c20640 1 -- 172.21.15.73:0/840296035 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f321006c8f0 msgr2=0x7f321006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:40.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.805+0000 7f3234c20640 1 --2- 172.21.15.73:0/840296035 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f321006c8f0 0x7f321006edb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f32300b2430 tx=0x7f3220008040 comp rx=0 tx=0).stop 2024-04-26T20:51:40.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.805+0000 7f3234c20640 1 -- 172.21.15.73:0/840296035 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32300b0380 msgr2=0x7f32300b72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:40.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.805+0000 7f3234c20640 1 --2- 172.21.15.73:0/840296035 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32300b0380 0x7f32300b72b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f322400c400 tx=0x7f322400c8c0 comp rx=0 tx=0).stop 2024-04-26T20:51:40.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.805+0000 7f3234c20640 1 -- 172.21.15.73:0/840296035 shutdown_connections 2024-04-26T20:51:40.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.805+0000 7f3234c20640 1 --2- 172.21.15.73:0/840296035 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32300b0380 0x7f32300b72b0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:40.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.805+0000 7f3234c20640 1 --2- 172.21.15.73:0/840296035 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f321006c8f0 0x7f321006edb0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:40.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.805+0000 7f3234c20640 1 --2- 172.21.15.73:0/840296035 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32300b10b0 0x7f32301620a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:40.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.805+0000 7f3234c20640 1 --2- 172.21.15.73:0/840296035 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f323006c950 0x7f32301570b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:40.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.805+0000 7f3234c20640 1 -- 172.21.15.73:0/840296035 >> 172.21.15.73:0/840296035 conn(0x7f32300d1cf0 msgr2=0x7f323006c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:40.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.805+0000 7f3234c20640 1 -- 172.21.15.73:0/840296035 shutdown_connections 2024-04-26T20:51:40.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.805+0000 7f3234c20640 1 -- 172.21.15.73:0/840296035 wait complete. 2024-04-26T20:51:40.818 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-26T20:51:40.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.914+0000 7f22dcd1c640 1 Processor -- start 2024-04-26T20:51:40.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.914+0000 7f22dcd1c640 1 -- start start 2024-04-26T20:51:40.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.914+0000 7f22dcd1c640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f22d80cf5e0 0x7f22d80ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:40.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.914+0000 7f22dcd1c640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f22d80cbbd0 0x7f22d80cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:40.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.914+0000 7f22dcd1c640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22d815b9e0 con 0x7f22d815b600 2024-04-26T20:51:40.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.914+0000 7f22dcd1c640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f22d80cb9b0 con 0x7f22d80cbbd0 2024-04-26T20:51:40.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.914+0000 7f22dcd1c640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f22d80d0740 con 0x7f22d80cf5e0 2024-04-26T20:51:40.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.914+0000 7f22d6d9d640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f22d80cbbd0 0x7f22d80cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:40.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.915+0000 7f22d6d9d640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f22d80cbbd0 0x7f22d80cef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54382/0 (socket says 172.21.15.73:54382) 2024-04-26T20:51:40.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.915+0000 7f22d6d9d640 1 -- 172.21.15.73:0/1807478748 learned_addr learned my addr 172.21.15.73:0/1807478748 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:40.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.915+0000 7f22cdd9b640 1 --2- 172.21.15.73:0/1807478748 >> v2:172.21.15.120:3300/0 conn(0x7f22d80cf5e0 0x7f22d80ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:40.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.915+0000 7f22d6d9d640 1 -- 172.21.15.73:0/1807478748 >> v2:172.21.15.120:3300/0 conn(0x7f22d80cf5e0 msgr2=0x7f22d80ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:40.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.915+0000 7f22d6d9d640 1 --2- 172.21.15.73:0/1807478748 >> v2:172.21.15.120:3300/0 conn(0x7f22d80cf5e0 0x7f22d80ce9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:40.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.915+0000 7f22d6d9d640 1 -- 172.21.15.73:0/1807478748 >> v1:172.21.15.120:6789/0 conn(0x7f22d815b600 legacy=0x7f22d8071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:40.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.915+0000 7f22d6d9d640 1 -- 172.21.15.73:0/1807478748 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22d80cb360 con 0x7f22d80cbbd0 2024-04-26T20:51:40.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.915+0000 7f22d6d9d640 1 --2- 172.21.15.73:0/1807478748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22d80cbbd0 0x7f22d80cef10 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f22d815bbb0 tx=0x7f22d002c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8664ea29fd01838a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:40.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.916+0000 7f22d5d9b640 1 -- 172.21.15.73:0/1807478748 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d0035400 con 0x7f22d80cbbd0 2024-04-26T20:51:40.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.916+0000 7f22d5d9b640 1 -- 172.21.15.73:0/1807478748 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22d00355a0 con 0x7f22d80cbbd0 2024-04-26T20:51:40.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.916+0000 7f22d5d9b640 1 -- 172.21.15.73:0/1807478748 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d0035970 con 0x7f22d80cbbd0 2024-04-26T20:51:40.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.916+0000 7f22dcd1c640 1 -- 172.21.15.73:0/1807478748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22d80cbbd0 msgr2=0x7f22d80cef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:40.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.916+0000 7f22dcd1c640 1 --2- 172.21.15.73:0/1807478748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22d80cbbd0 0x7f22d80cef10 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f22d815bbb0 tx=0x7f22d002c8a0 comp rx=0 tx=0).stop 2024-04-26T20:51:40.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.916+0000 7f22dcd1c640 1 -- 172.21.15.73:0/1807478748 shutdown_connections 2024-04-26T20:51:40.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.916+0000 7f22dcd1c640 1 --2- 172.21.15.73:0/1807478748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22d80cbbd0 0x7f22d80cef10 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:40.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.916+0000 7f22dcd1c640 1 --2- 172.21.15.73:0/1807478748 >> v2:172.21.15.120:3300/0 conn(0x7f22d80cf5e0 0x7f22d80ce9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:40.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.916+0000 7f22dcd1c640 1 -- 172.21.15.73:0/1807478748 >> 172.21.15.73:0/1807478748 conn(0x7f22d80d0be0 msgr2=0x7f22d80d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:40.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.916+0000 7f22dcd1c640 1 -- 172.21.15.73:0/1807478748 shutdown_connections 2024-04-26T20:51:40.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.916+0000 7f22dcd1c640 1 -- 172.21.15.73:0/1807478748 wait complete. 2024-04-26T20:51:40.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.918+0000 7f22dcd1c640 1 Processor -- start 2024-04-26T20:51:40.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.918+0000 7f22dcd1c640 1 -- start start 2024-04-26T20:51:40.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.918+0000 7f22dcd1c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22d80cbbd0 0x7f22d81541b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:40.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.918+0000 7f22dcd1c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f22d80cf5e0 0x7f22d80c9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:40.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.918+0000 7f22dcd1c640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f22d815b600 0x7f22d80ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:40.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.918+0000 7f22dcd1c640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f22d80d0470 con 0x7f22d80cbbd0 2024-04-26T20:51:40.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.918+0000 7f22dcd1c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f22d80d02f0 con 0x7f22d80cf5e0 2024-04-26T20:51:40.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.918+0000 7f22dcd1c640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f22d806ddf0 con 0x7f22d815b600 2024-04-26T20:51:40.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.918+0000 7f22d659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22d80cbbd0 0x7f22d81541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:40.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.918+0000 7f22d659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22d80cbbd0 0x7f22d81541b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54390/0 (socket says 172.21.15.73:54390) 2024-04-26T20:51:40.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.918+0000 7f22d659c640 1 -- 172.21.15.73:0/561905907 learned_addr learned my addr 172.21.15.73:0/561905907 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:40.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.919+0000 7f22d659c640 1 -- 172.21.15.73:0/561905907 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f22d815b600 msgr2=0x7f22d80ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:40.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.919+0000 7f22d659c640 1 --2- 172.21.15.73:0/561905907 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f22d815b600 0x7f22d80ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:40.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.919+0000 7f22d659c640 1 -- 172.21.15.73:0/561905907 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f22d80cf5e0 msgr2=0x7f22d80c9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:40.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.919+0000 7f22d659c640 1 --2- 172.21.15.73:0/561905907 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f22d80cf5e0 0x7f22d80c9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:40.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.919+0000 7f22d659c640 1 -- 172.21.15.73:0/561905907 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22d80b60c0 con 0x7f22d80cbbd0 2024-04-26T20:51:40.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.919+0000 7f22d659c640 1 --2- 172.21.15.73:0/561905907 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22d80cbbd0 0x7f22d81541b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f22c800c3c0 tx=0x7f22c800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:40.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.919+0000 7f22cf7fe640 1 -- 172.21.15.73:0/561905907 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c80052d0 con 0x7f22d80cbbd0 2024-04-26T20:51:40.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.919+0000 7f22cf7fe640 1 -- 172.21.15.73:0/561905907 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22c8005490 con 0x7f22d80cbbd0 2024-04-26T20:51:40.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.919+0000 7f22dcd1c640 1 -- 172.21.15.73:0/561905907 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22d80b63a0 con 0x7f22d80cbbd0 2024-04-26T20:51:40.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.919+0000 7f22cf7fe640 1 -- 172.21.15.73:0/561905907 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c8007260 con 0x7f22d80cbbd0 2024-04-26T20:51:40.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.919+0000 7f22dcd1c640 1 -- 172.21.15.73:0/561905907 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22d80b6960 con 0x7f22d80cbbd0 2024-04-26T20:51:40.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.920+0000 7f22dcd1c640 1 -- 172.21.15.73:0/561905907 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22d80694f0 con 0x7f22d80cbbd0 2024-04-26T20:51:40.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.921+0000 7f22cf7fe640 1 -- 172.21.15.73:0/561905907 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f22c8007400 con 0x7f22d80cbbd0 2024-04-26T20:51:40.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.921+0000 7f22cf7fe640 1 --2- 172.21.15.73:0/561905907 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f22b806c8f0 0x7f22b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:40.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.921+0000 7f22cf7fe640 1 -- 172.21.15.73:0/561905907 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f22c808aed0 con 0x7f22d80cbbd0 2024-04-26T20:51:40.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.921+0000 7f22cdd9b640 1 --2- 172.21.15.73:0/561905907 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f22b806c8f0 0x7f22b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:40.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.921+0000 7f22cdd9b640 1 --2- 172.21.15.73:0/561905907 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f22b806c8f0 0x7f22b806edb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f22d8072390 tx=0x7f22c0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:40.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:40.925+0000 7f22cf7fe640 1 -- 172.21.15.73:0/561905907 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f22c8059440 con 0x7f22d80cbbd0 2024-04-26T20:51:41.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.168+0000 7f22dcd1c640 1 -- 172.21.15.73:0/561905907 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f22d80725f0 con 0x7f22d80cbbd0 2024-04-26T20:51:41.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.171+0000 7f22cf7fe640 1 -- 172.21.15.73:0/561905907 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f22c8058d40 con 0x7f22d80cbbd0 2024-04-26T20:51:41.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.172+0000 7f22dcd1c640 1 -- 172.21.15.73:0/561905907 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f22b806c8f0 msgr2=0x7f22b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:41.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.172+0000 7f22dcd1c640 1 --2- 172.21.15.73:0/561905907 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f22b806c8f0 0x7f22b806edb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f22d8072390 tx=0x7f22c0008040 comp rx=0 tx=0).stop 2024-04-26T20:51:41.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.172+0000 7f22dcd1c640 1 -- 172.21.15.73:0/561905907 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22d80cbbd0 msgr2=0x7f22d81541b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:41.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.172+0000 7f22dcd1c640 1 --2- 172.21.15.73:0/561905907 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22d80cbbd0 0x7f22d81541b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f22c800c3c0 tx=0x7f22c800c880 comp rx=0 tx=0).stop 2024-04-26T20:51:41.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.173+0000 7f22dcd1c640 1 -- 172.21.15.73:0/561905907 shutdown_connections 2024-04-26T20:51:41.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.173+0000 7f22dcd1c640 1 --2- 172.21.15.73:0/561905907 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f22b806c8f0 0x7f22b806edb0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:41.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.173+0000 7f22dcd1c640 1 --2- 172.21.15.73:0/561905907 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f22d815b600 0x7f22d80ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:41.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.173+0000 7f22dcd1c640 1 --2- 172.21.15.73:0/561905907 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f22d80cf5e0 0x7f22d80c9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:41.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.173+0000 7f22dcd1c640 1 --2- 172.21.15.73:0/561905907 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22d80cbbd0 0x7f22d81541b0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:41.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.173+0000 7f22dcd1c640 1 -- 172.21.15.73:0/561905907 >> 172.21.15.73:0/561905907 conn(0x7f22d80d0be0 msgr2=0x7f22d806f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:41.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.173+0000 7f22dcd1c640 1 -- 172.21.15.73:0/561905907 shutdown_connections 2024-04-26T20:51:41.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.173+0000 7f22dcd1c640 1 -- 172.21.15.73:0/561905907 wait complete. 2024-04-26T20:51:41.185 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-26T20:51:41.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.280+0000 7f579827b640 1 Processor -- start 2024-04-26T20:51:41.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.280+0000 7f579827b640 1 -- start start 2024-04-26T20:51:41.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.281+0000 7f579827b640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f579015ab60 0x7f579006c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:41.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.281+0000 7f579827b640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f57901617f0 0x7f579006c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:41.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.281+0000 7f579827b640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f579015af40 con 0x7f579015d5f0 2024-04-26T20:51:41.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.281+0000 7f579827b640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f579015e320 con 0x7f57901617f0 2024-04-26T20:51:41.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.281+0000 7f579827b640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f5790160d10 con 0x7f579015ab60 2024-04-26T20:51:41.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.281+0000 7f5795816640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f57901617f0 0x7f579006c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:41.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.282+0000 7f5795816640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f57901617f0 0x7f579006c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54400/0 (socket says 172.21.15.73:54400) 2024-04-26T20:51:41.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.282+0000 7f5795816640 1 -- 172.21.15.73:0/1888278325 learned_addr learned my addr 172.21.15.73:0/1888278325 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:41.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.282+0000 7f5796017640 1 --2- 172.21.15.73:0/1888278325 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f579015ab60 0x7f579006c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:41.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.282+0000 7f5795816640 1 -- 172.21.15.73:0/1888278325 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f579015ab60 msgr2=0x7f579006c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:41.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.282+0000 7f5795816640 1 --2- 172.21.15.73:0/1888278325 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f579015ab60 0x7f579006c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:41.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.282+0000 7f5795816640 1 -- 172.21.15.73:0/1888278325 >> v1:172.21.15.120:6789/0 conn(0x7f579015d5f0 legacy=0x7f579015daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:41.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.282+0000 7f5795816640 1 -- 172.21.15.73:0/1888278325 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5790160ff0 con 0x7f57901617f0 2024-04-26T20:51:41.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.282+0000 7f5796017640 1 --2- 172.21.15.73:0/1888278325 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f579015ab60 0x7f579006c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:41.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.282+0000 7f5795816640 1 --2- 172.21.15.73:0/1888278325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57901617f0 0x7f579006c780 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f578c009870 tx=0x7f578c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a8dfa50d425c27 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:41.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.283+0000 7f5795015640 1 -- 172.21.15.73:0/1888278325 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f578c037070 con 0x7f57901617f0 2024-04-26T20:51:41.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.283+0000 7f5795015640 1 -- 172.21.15.73:0/1888278325 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f578c032440 con 0x7f57901617f0 2024-04-26T20:51:41.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.283+0000 7f5795015640 1 -- 172.21.15.73:0/1888278325 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f578c032810 con 0x7f57901617f0 2024-04-26T20:51:41.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.283+0000 7f579827b640 1 -- 172.21.15.73:0/1888278325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57901617f0 msgr2=0x7f579006c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:41.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.283+0000 7f579827b640 1 --2- 172.21.15.73:0/1888278325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57901617f0 0x7f579006c780 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f578c009870 tx=0x7f578c02a270 comp rx=0 tx=0).stop 2024-04-26T20:51:41.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.283+0000 7f579827b640 1 -- 172.21.15.73:0/1888278325 shutdown_connections 2024-04-26T20:51:41.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.283+0000 7f579827b640 1 --2- 172.21.15.73:0/1888278325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57901617f0 0x7f579006c780 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:41.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.283+0000 7f579827b640 1 --2- 172.21.15.73:0/1888278325 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f579015ab60 0x7f579006c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:41.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.283+0000 7f579827b640 1 -- 172.21.15.73:0/1888278325 >> 172.21.15.73:0/1888278325 conn(0x7f57900d1cf0 msgr2=0x7f57900d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:41.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.283+0000 7f579827b640 1 -- 172.21.15.73:0/1888278325 shutdown_connections 2024-04-26T20:51:41.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.283+0000 7f579827b640 1 -- 172.21.15.73:0/1888278325 wait complete. 2024-04-26T20:51:41.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.284+0000 7f579827b640 1 Processor -- start 2024-04-26T20:51:41.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.284+0000 7f579827b640 1 -- start start 2024-04-26T20:51:41.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.285+0000 7f579827b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f579015ab60 0x7f579015c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:41.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.285+0000 7f579827b640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f579015d5f0 0x7f57901592d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:41.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.285+0000 7f579827b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57901617f0 0x7f57900caf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:41.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.285+0000 7f579827b640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f57900cc4c0 con 0x7f57901617f0 2024-04-26T20:51:41.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.285+0000 7f579827b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f57900cc340 con 0x7f579015ab60 2024-04-26T20:51:41.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.285+0000 7f579827b640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f57900cc640 con 0x7f579015d5f0 2024-04-26T20:51:41.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.285+0000 7f5796818640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57901617f0 0x7f57900caf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:41.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.285+0000 7f5796818640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57901617f0 0x7f57900caf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54408/0 (socket says 172.21.15.73:54408) 2024-04-26T20:51:41.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.285+0000 7f5796818640 1 -- 172.21.15.73:0/1812184415 learned_addr learned my addr 172.21.15.73:0/1812184415 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:41.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.285+0000 7f5795816640 1 --2- 172.21.15.73:0/1812184415 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f579015d5f0 0x7f57901592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:41.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.285+0000 7f5796017640 1 --2- 172.21.15.73:0/1812184415 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f579015ab60 0x7f579015c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:41.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.286+0000 7f5796818640 1 -- 172.21.15.73:0/1812184415 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f579015d5f0 msgr2=0x7f57901592d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:41.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.286+0000 7f5796818640 1 --2- 172.21.15.73:0/1812184415 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f579015d5f0 0x7f57901592d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:41.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.286+0000 7f5796818640 1 -- 172.21.15.73:0/1812184415 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f579015ab60 msgr2=0x7f579015c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:41.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.286+0000 7f5796818640 1 --2- 172.21.15.73:0/1812184415 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f579015ab60 0x7f579015c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:41.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.286+0000 7f5796818640 1 -- 172.21.15.73:0/1812184415 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57901f3030 con 0x7f57901617f0 2024-04-26T20:51:41.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.286+0000 7f5796818640 1 --2- 172.21.15.73:0/1812184415 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57901617f0 0x7f57900caf90 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f578800c7d0 tx=0x7f578800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:41.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.286+0000 7f5786ffd640 1 -- 172.21.15.73:0/1812184415 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5788003b70 con 0x7f57901617f0 2024-04-26T20:51:41.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.286+0000 7f5786ffd640 1 -- 172.21.15.73:0/1812184415 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5788003d30 con 0x7f57901617f0 2024-04-26T20:51:41.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.286+0000 7f579827b640 1 -- 172.21.15.73:0/1812184415 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57900b4df0 con 0x7f57901617f0 2024-04-26T20:51:41.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.286+0000 7f5786ffd640 1 -- 172.21.15.73:0/1812184415 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f578800aa90 con 0x7f57901617f0 2024-04-26T20:51:41.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.286+0000 7f579827b640 1 -- 172.21.15.73:0/1812184415 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57900b53b0 con 0x7f57901617f0 2024-04-26T20:51:41.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.287+0000 7f579827b640 1 -- 172.21.15.73:0/1812184415 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57900694f0 con 0x7f57901617f0 2024-04-26T20:51:41.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.287+0000 7f5786ffd640 1 -- 172.21.15.73:0/1812184415 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f578800ac30 con 0x7f57901617f0 2024-04-26T20:51:41.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.288+0000 7f5786ffd640 1 --2- 172.21.15.73:0/1812184415 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f577406c8f0 0x7f577406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:41.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.288+0000 7f5786ffd640 1 -- 172.21.15.73:0/1812184415 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f578808b790 con 0x7f57901617f0 2024-04-26T20:51:41.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.288+0000 7f5796017640 1 --2- 172.21.15.73:0/1812184415 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f577406c8f0 0x7f577406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:41.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.288+0000 7f5796017640 1 --2- 172.21.15.73:0/1812184415 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f577406c8f0 0x7f577406edb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f57900cd020 tx=0x7f5780008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:41.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.292+0000 7f5786ffd640 1 -- 172.21.15.73:0/1812184415 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5788059730 con 0x7f57901617f0 2024-04-26T20:51:41.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.536+0000 7f579827b640 1 -- 172.21.15.73:0/1812184415 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f57900d29e0 con 0x7f57901617f0 2024-04-26T20:51:41.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.540+0000 7f5786ffd640 1 -- 172.21.15.73:0/1812184415 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) ==== 97+0+0 (secure 0 0 0) 0x7f578805da30 con 0x7f57901617f0 2024-04-26T20:51:41.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.542+0000 7f579827b640 1 -- 172.21.15.73:0/1812184415 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f577406c8f0 msgr2=0x7f577406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:41.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.542+0000 7f579827b640 1 --2- 172.21.15.73:0/1812184415 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f577406c8f0 0x7f577406edb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f57900cd020 tx=0x7f5780008040 comp rx=0 tx=0).stop 2024-04-26T20:51:41.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.542+0000 7f579827b640 1 -- 172.21.15.73:0/1812184415 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57901617f0 msgr2=0x7f57900caf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:41.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.542+0000 7f579827b640 1 --2- 172.21.15.73:0/1812184415 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57901617f0 0x7f57900caf90 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f578800c7d0 tx=0x7f578800cc90 comp rx=0 tx=0).stop 2024-04-26T20:51:41.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.542+0000 7f579827b640 1 -- 172.21.15.73:0/1812184415 shutdown_connections 2024-04-26T20:51:41.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.542+0000 7f579827b640 1 --2- 172.21.15.73:0/1812184415 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57901617f0 0x7f57900caf90 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:41.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.542+0000 7f579827b640 1 --2- 172.21.15.73:0/1812184415 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f577406c8f0 0x7f577406edb0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:41.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.542+0000 7f579827b640 1 --2- 172.21.15.73:0/1812184415 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f579015d5f0 0x7f57901592d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:41.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.542+0000 7f579827b640 1 --2- 172.21.15.73:0/1812184415 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f579015ab60 0x7f579015c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:41.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.542+0000 7f579827b640 1 -- 172.21.15.73:0/1812184415 >> 172.21.15.73:0/1812184415 conn(0x7f57900d1cf0 msgr2=0x7f579006ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:41.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.542+0000 7f579827b640 1 -- 172.21.15.73:0/1812184415 shutdown_connections 2024-04-26T20:51:41.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.542+0000 7f579827b640 1 -- 172.21.15.73:0/1812184415 wait complete. 2024-04-26T20:51:41.554 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-26T20:51:41.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.650+0000 7f2a459c6640 1 Processor -- start 2024-04-26T20:51:41.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.650+0000 7f2a459c6640 1 -- start start 2024-04-26T20:51:41.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.651+0000 7f2a459c6640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2a400b0380 0x7f2a400ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:41.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.651+0000 7f2a459c6640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a400cfd80 con 0x7f2a400b10b0 2024-04-26T20:51:41.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.651+0000 7f2a459c6640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a400d0020 con 0x7f2a4006c950 2024-04-26T20:51:41.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.651+0000 7f2a459c6640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f2a400af8e0 con 0x7f2a400b0380 2024-04-26T20:51:41.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.651+0000 7f2a3effd640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2a400b0380 0x7f2a400ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:41.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.651+0000 7f2a3e7fc640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f2a400b10b0 0x7f2a4006c240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:41592/0 (socket says 172.21.15.73:41592) 2024-04-26T20:51:41.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.651+0000 7f2a3e7fc640 1 -- 172.21.15.73:0/229314872 learned_addr learned my addr 172.21.15.73:0/229314872 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:41.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.651+0000 7f2a3effd640 1 -- 172.21.15.73:0/229314872 >> v1:172.21.15.120:6789/0 conn(0x7f2a4006c950 legacy=0x7f2a4015e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:41.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.651+0000 7f2a3effd640 1 -- 172.21.15.73:0/229314872 >> v1:172.21.15.73:6789/0 conn(0x7f2a400b10b0 legacy=0x7f2a4006c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:41.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.651+0000 7f2a3effd640 1 -- 172.21.15.73:0/229314872 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a400d0300 con 0x7f2a400b0380 2024-04-26T20:51:41.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.652+0000 7f2a3effd640 1 --2- 172.21.15.73:0/229314872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a400b0380 0x7f2a400ccd50 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f2a30009d90 tx=0x7f2a3002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a032106dece379a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:41.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.652+0000 7f2a3dffb640 1 -- 172.21.15.73:0/229314872 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a30007740 con 0x7f2a400b0380 2024-04-26T20:51:41.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.652+0000 7f2a3dffb640 1 -- 172.21.15.73:0/229314872 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a300078e0 con 0x7f2a400b0380 2024-04-26T20:51:41.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.652+0000 7f2a3dffb640 1 -- 172.21.15.73:0/229314872 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a30007cb0 con 0x7f2a400b0380 2024-04-26T20:51:41.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.653+0000 7f2a459c6640 1 -- 172.21.15.73:0/229314872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a400b0380 msgr2=0x7f2a400ccd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:41.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.653+0000 7f2a459c6640 1 --2- 172.21.15.73:0/229314872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a400b0380 0x7f2a400ccd50 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f2a30009d90 tx=0x7f2a3002a660 comp rx=0 tx=0).stop 2024-04-26T20:51:41.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.653+0000 7f2a459c6640 1 -- 172.21.15.73:0/229314872 shutdown_connections 2024-04-26T20:51:41.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.653+0000 7f2a459c6640 1 --2- 172.21.15.73:0/229314872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a400b0380 0x7f2a400ccd50 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:41.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.653+0000 7f2a459c6640 1 -- 172.21.15.73:0/229314872 >> 172.21.15.73:0/229314872 conn(0x7f2a400d1cf0 msgr2=0x7f2a400d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:41.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.653+0000 7f2a459c6640 1 -- 172.21.15.73:0/229314872 shutdown_connections 2024-04-26T20:51:41.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.653+0000 7f2a459c6640 1 -- 172.21.15.73:0/229314872 wait complete. 2024-04-26T20:51:41.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.654+0000 7f2a459c6640 1 Processor -- start 2024-04-26T20:51:41.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.654+0000 7f2a459c6640 1 -- start start 2024-04-26T20:51:41.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.655+0000 7f2a459c6640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2a4006c950 0x7f2a40151160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:41.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.655+0000 7f2a459c6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a400b0380 0x7f2a400cb900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:41.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.655+0000 7f2a459c6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a400b10b0 0x7f2a401608e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:41.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.655+0000 7f2a459c6640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2a400b0760 con 0x7f2a400b10b0 2024-04-26T20:51:41.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.655+0000 7f2a459c6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2a400d1980 con 0x7f2a400b0380 2024-04-26T20:51:41.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.655+0000 7f2a459c6640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f2a400d15c0 con 0x7f2a4006c950 2024-04-26T20:51:41.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.655+0000 7f2a3f7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a400b10b0 0x7f2a401608e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:41.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.655+0000 7f2a3f7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a400b10b0 0x7f2a401608e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54436/0 (socket says 172.21.15.73:54436) 2024-04-26T20:51:41.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.655+0000 7f2a3f7fe640 1 -- 172.21.15.73:0/2117563794 learned_addr learned my addr 172.21.15.73:0/2117563794 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:41.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.655+0000 7f2a3e7fc640 1 --2- 172.21.15.73:0/2117563794 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a400b0380 0x7f2a400cb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:41.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.655+0000 7f2a3effd640 1 --2- 172.21.15.73:0/2117563794 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2a4006c950 0x7f2a40151160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:41.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.655+0000 7f2a3f7fe640 1 -- 172.21.15.73:0/2117563794 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2a4006c950 msgr2=0x7f2a40151160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:41.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.655+0000 7f2a3f7fe640 1 --2- 172.21.15.73:0/2117563794 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2a4006c950 0x7f2a40151160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:41.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.655+0000 7f2a3f7fe640 1 -- 172.21.15.73:0/2117563794 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a400b0380 msgr2=0x7f2a400cb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:41.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.655+0000 7f2a3f7fe640 1 --2- 172.21.15.73:0/2117563794 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a400b0380 0x7f2a400cb900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:41.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.655+0000 7f2a3f7fe640 1 -- 172.21.15.73:0/2117563794 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a401516a0 con 0x7f2a400b10b0 2024-04-26T20:51:41.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.655+0000 7f2a3e7fc640 1 --2- 172.21.15.73:0/2117563794 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a400b0380 0x7f2a400cb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:41.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.655+0000 7f2a3f7fe640 1 --2- 172.21.15.73:0/2117563794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a400b10b0 0x7f2a401608e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f2a3800c3c0 tx=0x7f2a3800c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:41.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.656+0000 7f2a27fff640 1 -- 172.21.15.73:0/2117563794 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a38017070 con 0x7f2a400b10b0 2024-04-26T20:51:41.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.656+0000 7f2a27fff640 1 -- 172.21.15.73:0/2117563794 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a380052d0 con 0x7f2a400b10b0 2024-04-26T20:51:41.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.656+0000 7f2a459c6640 1 -- 172.21.15.73:0/2117563794 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a400ca860 con 0x7f2a400b10b0 2024-04-26T20:51:41.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.656+0000 7f2a27fff640 1 -- 172.21.15.73:0/2117563794 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a38003830 con 0x7f2a400b10b0 2024-04-26T20:51:41.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.656+0000 7f2a459c6640 1 -- 172.21.15.73:0/2117563794 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a400cb100 con 0x7f2a400b10b0 2024-04-26T20:51:41.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.657+0000 7f2a459c6640 1 -- 172.21.15.73:0/2117563794 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a400694f0 con 0x7f2a400b10b0 2024-04-26T20:51:41.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.657+0000 7f2a27fff640 1 -- 172.21.15.73:0/2117563794 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2a3801a070 con 0x7f2a400b10b0 2024-04-26T20:51:41.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.657+0000 7f2a27fff640 1 --2- 172.21.15.73:0/2117563794 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f2a2c06c8f0 0x7f2a2c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:41.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.658+0000 7f2a27fff640 1 -- 172.21.15.73:0/2117563794 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f2a3808ad10 con 0x7f2a400b10b0 2024-04-26T20:51:41.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.658+0000 7f2a3effd640 1 --2- 172.21.15.73:0/2117563794 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f2a2c06c8f0 0x7f2a2c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:41.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.658+0000 7f2a3effd640 1 --2- 172.21.15.73:0/2117563794 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f2a2c06c8f0 0x7f2a2c06edb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2a30034000 tx=0x7f2a3002f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:41.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.662+0000 7f2a27fff640 1 -- 172.21.15.73:0/2117563794 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2a38059300 con 0x7f2a400b10b0 2024-04-26T20:51:41.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.905+0000 7f2a459c6640 1 -- 172.21.15.73:0/2117563794 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f2a400cd140 con 0x7f2a400b10b0 2024-04-26T20:51:41.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.910+0000 7f2a27fff640 1 -- 172.21.15.73:0/2117563794 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7f2a38058c00 con 0x7f2a400b10b0 2024-04-26T20:51:41.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.911+0000 7f2a459c6640 1 -- 172.21.15.73:0/2117563794 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f2a2c06c8f0 msgr2=0x7f2a2c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:41.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.911+0000 7f2a459c6640 1 --2- 172.21.15.73:0/2117563794 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f2a2c06c8f0 0x7f2a2c06edb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2a30034000 tx=0x7f2a3002f000 comp rx=0 tx=0).stop 2024-04-26T20:51:41.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.911+0000 7f2a459c6640 1 -- 172.21.15.73:0/2117563794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a400b10b0 msgr2=0x7f2a401608e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:41.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.911+0000 7f2a459c6640 1 --2- 172.21.15.73:0/2117563794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a400b10b0 0x7f2a401608e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f2a3800c3c0 tx=0x7f2a3800c7c0 comp rx=0 tx=0).stop 2024-04-26T20:51:41.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.911+0000 7f2a459c6640 1 -- 172.21.15.73:0/2117563794 shutdown_connections 2024-04-26T20:51:41.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.911+0000 7f2a459c6640 1 --2- 172.21.15.73:0/2117563794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a400b10b0 0x7f2a401608e0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:41.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.911+0000 7f2a459c6640 1 --2- 172.21.15.73:0/2117563794 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f2a2c06c8f0 0x7f2a2c06edb0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:41.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.911+0000 7f2a459c6640 1 --2- 172.21.15.73:0/2117563794 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2a400b0380 0x7f2a400cb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:41.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.911+0000 7f2a459c6640 1 --2- 172.21.15.73:0/2117563794 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2a4006c950 0x7f2a40151160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:41.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.911+0000 7f2a459c6640 1 -- 172.21.15.73:0/2117563794 >> 172.21.15.73:0/2117563794 conn(0x7f2a400d1cf0 msgr2=0x7f2a400b2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:41.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.911+0000 7f2a459c6640 1 -- 172.21.15.73:0/2117563794 shutdown_connections 2024-04-26T20:51:41.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:41.911+0000 7f2a459c6640 1 -- 172.21.15.73:0/2117563794 wait complete. 2024-04-26T20:51:41.924 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:51:42.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.021+0000 7f8c46378640 1 Processor -- start 2024-04-26T20:51:42.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.021+0000 7f8c46378640 1 -- start start 2024-04-26T20:51:42.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.022+0000 7f8c46378640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f8c40071e60 0x7f8c4006f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:42.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.022+0000 7f8c46378640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8c400cc480 0x7f8c40166ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:42.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.022+0000 7f8c46378640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c4015d7c0 con 0x7f8c4006fe20 2024-04-26T20:51:42.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.022+0000 7f8c46378640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f8c4015daa0 con 0x7f8c40071e60 2024-04-26T20:51:42.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.022+0000 7f8c46378640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f8c4015dd80 con 0x7f8c400cc480 2024-04-26T20:51:42.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.022+0000 7f8c44915640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8c400cc480 0x7f8c40166ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:42.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.022+0000 7f8c3ffff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f8c40071e60 0x7f8c4006f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:42.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.022+0000 7f8c3f7fe640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f8c4006fe20 0x7f8c40072240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.73:57994/0 (socket says 172.21.15.73:57994) 2024-04-26T20:51:42.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.022+0000 7f8c3f7fe640 1 -- 172.21.15.73:0/4238313183 learned_addr learned my addr 172.21.15.73:0/4238313183 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:42.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.023+0000 7f8c3ffff640 1 -- 172.21.15.73:0/4238313183 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8c400cc480 msgr2=0x7f8c40166ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:42.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.023+0000 7f8c3ffff640 1 --2- 172.21.15.73:0/4238313183 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8c400cc480 0x7f8c40166ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:42.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.023+0000 7f8c3ffff640 1 -- 172.21.15.73:0/4238313183 >> v1:172.21.15.120:6789/0 conn(0x7f8c4006fe20 legacy=0x7f8c40072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:42.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.023+0000 7f8c3effd640 1 -- 172.21.15.73:0/4238313183 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3279285583 0 0) 0x7f8c4015d7c0 con 0x7f8c4006fe20 2024-04-26T20:51:42.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.023+0000 7f8c3ffff640 1 -- 172.21.15.73:0/4238313183 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c4015e060 con 0x7f8c40071e60 2024-04-26T20:51:42.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.023+0000 7f8c44915640 1 --2- 172.21.15.73:0/4238313183 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8c400cc480 0x7f8c40166ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:51:42.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.023+0000 7f8c3ffff640 1 --2- 172.21.15.73:0/4238313183 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c40071e60 0x7f8c4006f8e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8c300097a0 tx=0x7f8c3002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f77541f988ab8e96 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:42.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.023+0000 7f8c3effd640 1 -- 172.21.15.73:0/4238313183 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c30037070 con 0x7f8c40071e60 2024-04-26T20:51:42.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.023+0000 7f8c3effd640 1 -- 172.21.15.73:0/4238313183 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c30032440 con 0x7f8c40071e60 2024-04-26T20:51:42.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.023+0000 7f8c3effd640 1 -- 172.21.15.73:0/4238313183 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c30032810 con 0x7f8c40071e60 2024-04-26T20:51:42.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.024+0000 7f8c46378640 1 -- 172.21.15.73:0/4238313183 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c40071e60 msgr2=0x7f8c4006f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:42.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.024+0000 7f8c46378640 1 --2- 172.21.15.73:0/4238313183 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c40071e60 0x7f8c4006f8e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8c300097a0 tx=0x7f8c3002a180 comp rx=0 tx=0).stop 2024-04-26T20:51:42.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.024+0000 7f8c46378640 1 -- 172.21.15.73:0/4238313183 shutdown_connections 2024-04-26T20:51:42.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.024+0000 7f8c46378640 1 --2- 172.21.15.73:0/4238313183 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8c400cc480 0x7f8c40166ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:42.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.024+0000 7f8c46378640 1 --2- 172.21.15.73:0/4238313183 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c40071e60 0x7f8c4006f8e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:42.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.024+0000 7f8c46378640 1 -- 172.21.15.73:0/4238313183 >> 172.21.15.73:0/4238313183 conn(0x7f8c400d0be0 msgr2=0x7f8c400d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:42.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.024+0000 7f8c46378640 1 -- 172.21.15.73:0/4238313183 shutdown_connections 2024-04-26T20:51:42.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.024+0000 7f8c46378640 1 -- 172.21.15.73:0/4238313183 wait complete. 2024-04-26T20:51:42.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.025+0000 7f8c46378640 1 Processor -- start 2024-04-26T20:51:42.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.025+0000 7f8c46378640 1 -- start start 2024-04-26T20:51:42.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.026+0000 7f8c46378640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8c4006fe20 0x7f8c401561b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:42.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.026+0000 7f8c46378640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c40071e60 0x7f8c400b5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:42.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.026+0000 7f8c46378640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c400cc480 0x7f8c400b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:42.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.026+0000 7f8c46378640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4015d570 con 0x7f8c400cc480 2024-04-26T20:51:42.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.026+0000 7f8c46378640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4015d3f0 con 0x7f8c40071e60 2024-04-26T20:51:42.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.026+0000 7f8c46378640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f8c40072950 con 0x7f8c4006fe20 2024-04-26T20:51:42.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.026+0000 7f8c44915640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c400cc480 0x7f8c400b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:42.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.026+0000 7f8c44915640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c400cc480 0x7f8c400b5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54440/0 (socket says 172.21.15.73:54440) 2024-04-26T20:51:42.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.026+0000 7f8c44915640 1 -- 172.21.15.73:0/4112290678 learned_addr learned my addr 172.21.15.73:0/4112290678 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:42.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.026+0000 7f8c3f7fe640 1 --2- 172.21.15.73:0/4112290678 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c40071e60 0x7f8c400b5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:42.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.027+0000 7f8c3ffff640 1 --2- 172.21.15.73:0/4112290678 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8c4006fe20 0x7f8c401561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:42.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.027+0000 7f8c44915640 1 -- 172.21.15.73:0/4112290678 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8c4006fe20 msgr2=0x7f8c401561b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:42.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.027+0000 7f8c44915640 1 --2- 172.21.15.73:0/4112290678 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8c4006fe20 0x7f8c401561b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:42.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.027+0000 7f8c44915640 1 -- 172.21.15.73:0/4112290678 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c40071e60 msgr2=0x7f8c400b5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:42.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.027+0000 7f8c44915640 1 --2- 172.21.15.73:0/4112290678 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c40071e60 0x7f8c400b5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:42.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.027+0000 7f8c44915640 1 -- 172.21.15.73:0/4112290678 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c400aa6f0 con 0x7f8c400cc480 2024-04-26T20:51:42.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.027+0000 7f8c3ffff640 1 --2- 172.21.15.73:0/4112290678 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8c4006fe20 0x7f8c401561b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:42.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.027+0000 7f8c44915640 1 --2- 172.21.15.73:0/4112290678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c400cc480 0x7f8c400b5f90 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8c3800efc0 tx=0x7f8c3800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:42.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.027+0000 7f8c3cff9640 1 -- 172.21.15.73:0/4112290678 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c38018020 con 0x7f8c400cc480 2024-04-26T20:51:42.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.027+0000 7f8c3cff9640 1 -- 172.21.15.73:0/4112290678 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c38014480 con 0x7f8c400cc480 2024-04-26T20:51:42.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.027+0000 7f8c46378640 1 -- 172.21.15.73:0/4112290678 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c401f7010 con 0x7f8c400cc480 2024-04-26T20:51:42.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.027+0000 7f8c3cff9640 1 -- 172.21.15.73:0/4112290678 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c38014870 con 0x7f8c400cc480 2024-04-26T20:51:42.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.027+0000 7f8c46378640 1 -- 172.21.15.73:0/4112290678 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c401f6d30 con 0x7f8c400cc480 2024-04-26T20:51:42.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.028+0000 7f8c46378640 1 -- 172.21.15.73:0/4112290678 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c400694f0 con 0x7f8c400cc480 2024-04-26T20:51:42.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.028+0000 7f8c3cff9640 1 -- 172.21.15.73:0/4112290678 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8c38005d10 con 0x7f8c400cc480 2024-04-26T20:51:42.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.029+0000 7f8c3cff9640 1 --2- 172.21.15.73:0/4112290678 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f8c2c06c8f0 0x7f8c2c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:42.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.029+0000 7f8c3cff9640 1 -- 172.21.15.73:0/4112290678 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f8c3808c5b0 con 0x7f8c400cc480 2024-04-26T20:51:42.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.029+0000 7f8c3ffff640 1 --2- 172.21.15.73:0/4112290678 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f8c2c06c8f0 0x7f8c2c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:42.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.029+0000 7f8c3ffff640 1 --2- 172.21.15.73:0/4112290678 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f8c2c06c8f0 0x7f8c2c06edb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8c30034000 tx=0x7f8c3002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:42.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.033+0000 7f8c3cff9640 1 -- 172.21.15.73:0/4112290678 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8c38022030 con 0x7f8c400cc480 2024-04-26T20:51:42.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.208+0000 7f8c46378640 1 -- 172.21.15.73:0/4112290678 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8c400b16d0 con 0x7f8c400cc480 2024-04-26T20:51:42.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.208+0000 7f8c3cff9640 1 -- 172.21.15.73:0/4112290678 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+13164 (secure 0 0 0) 0x7f8c38010040 con 0x7f8c400cc480 2024-04-26T20:51:42.210 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:51:42.210 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":35,"fsid":"fbd3f548-b547-4713-856b-09f98f5d1764","created":"2024-04-26T20:50:07.876786+0000","modified":"2024-04-26T20:51:39.320707+0000","last_up_change":"2024-04-26T20:50:19.082741+0000","last_in_change":"2024-04-26T20:50:10.018400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:50:19.993731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d6f4f3-214a-4392-a02f-c33cbcab3c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6809","nonce":3461705124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6811","nonce":3461705124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6815","nonce":3461705124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6813","nonce":3461705124}]},"public_addr":"172.21.15.73:6809/3461705124","cluster_addr":"172.21.15.73:6811/3461705124","heartbeat_back_addr":"172.21.15.73:6815/3461705124","heartbeat_front_addr":"172.21.15.73:6813/3461705124","state":["exists","up"]},{"osd":1,"uuid":"786bf8ca-6ddc-4b3c-be6c-9152398bd5fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6817","nonce":2827619403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6819","nonce":2827619403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6825","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6827","nonce":2827619403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6821","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6823","nonce":2827619403}]},"public_addr":"172.21.15.73:6817/2827619403","cluster_addr":"172.21.15.73:6819/2827619403","heartbeat_back_addr":"172.21.15.73:6827/2827619403","heartbeat_front_addr":"172.21.15.73:6823/2827619403","state":["exists","up"]},{"osd":2,"uuid":"90963b1a-5437-4ef8-ad8b-6b74c534eeb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6801","nonce":2542681064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6803","nonce":2542681064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6808","nonce":2542681064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6805","nonce":2542681064}]},"public_addr":"172.21.15.73:6801/2542681064","cluster_addr":"172.21.15.73:6803/2542681064","heartbeat_back_addr":"172.21.15.73:6808/2542681064","heartbeat_front_addr":"172.21.15.73:6805/2542681064","state":["exists","up"]},{"osd":3,"uuid":"bd3d2148-6e38-41b4-a453-fe6e3b8b4ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6822","nonce":1604042718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6826","nonce":1604042718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6831","nonce":1604042718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6829","nonce":1604042718}]},"public_addr":"172.21.15.73:6822/1604042718","cluster_addr":"172.21.15.73:6826/1604042718","heartbeat_back_addr":"172.21.15.73:6831/1604042718","heartbeat_front_addr":"172.21.15.73:6829/1604042718","state":["exists","up"]},{"osd":4,"uuid":"e8668dc8-a23f-46d2-8311-cb882da08e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6817","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6819","nonce":3591055561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6824","nonce":3591055561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6831","nonce":3591055561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6828","nonce":3591055561}]},"public_addr":"172.21.15.120:6819/3591055561","cluster_addr":"172.21.15.120:6824/3591055561","heartbeat_back_addr":"172.21.15.120:6831/3591055561","heartbeat_front_addr":"172.21.15.120:6828/3591055561","state":["exists","up"]},{"osd":5,"uuid":"cbac0bdb-96bd-4852-bcce-e564495c3fcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6809","nonce":819450433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6811","nonce":819450433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6815","nonce":819450433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6813","nonce":819450433}]},"public_addr":"172.21.15.120:6809/819450433","cluster_addr":"172.21.15.120:6811/819450433","heartbeat_back_addr":"172.21.15.120:6815/819450433","heartbeat_front_addr":"172.21.15.120:6813/819450433","state":["exists","up"]},{"osd":6,"uuid":"1884b2a8-569f-4272-96a9-f25a09364dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6801","nonce":3946210727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6803","nonce":3946210727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6807","nonce":3946210727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6805","nonce":3946210727}]},"public_addr":"172.21.15.120:6801/3946210727","cluster_addr":"172.21.15.120:6803/3946210727","heartbeat_back_addr":"172.21.15.120:6807/3946210727","heartbeat_front_addr":"172.21.15.120:6805/3946210727","state":["exists","up"]},{"osd":7,"uuid":"c0c3bf6d-a4c5-4f1e-aa2d-330951830666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6818","nonce":2915574674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6821","nonce":2915574674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6827","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6829","nonce":2915574674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6823","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6825","nonce":2915574674}]},"public_addr":"172.21.15.120:6818/2915574674","cluster_addr":"172.21.15.120:6821/2915574674","heartbeat_back_addr":"172.21.15.120:6829/2915574674","heartbeat_front_addr":"172.21.15.120:6825/2915574674","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.218216+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.208710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.112705+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.182789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.164324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.174212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.225396+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.261559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:51:42.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.210+0000 7f8c46378640 1 -- 172.21.15.73:0/4112290678 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f8c2c06c8f0 msgr2=0x7f8c2c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:42.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.210+0000 7f8c46378640 1 --2- 172.21.15.73:0/4112290678 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f8c2c06c8f0 0x7f8c2c06edb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8c30034000 tx=0x7f8c3002e000 comp rx=0 tx=0).stop 2024-04-26T20:51:42.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.210+0000 7f8c46378640 1 -- 172.21.15.73:0/4112290678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c400cc480 msgr2=0x7f8c400b5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:42.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.210+0000 7f8c46378640 1 --2- 172.21.15.73:0/4112290678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c400cc480 0x7f8c400b5f90 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8c3800efc0 tx=0x7f8c3800c490 comp rx=0 tx=0).stop 2024-04-26T20:51:42.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.210+0000 7f8c46378640 1 -- 172.21.15.73:0/4112290678 shutdown_connections 2024-04-26T20:51:42.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.210+0000 7f8c46378640 1 --2- 172.21.15.73:0/4112290678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c400cc480 0x7f8c400b5f90 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:42.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.210+0000 7f8c46378640 1 --2- 172.21.15.73:0/4112290678 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f8c2c06c8f0 0x7f8c2c06edb0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:42.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.210+0000 7f8c46378640 1 --2- 172.21.15.73:0/4112290678 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8c40071e60 0x7f8c400b5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:42.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.210+0000 7f8c46378640 1 --2- 172.21.15.73:0/4112290678 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8c4006fe20 0x7f8c401561b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:42.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.210+0000 7f8c46378640 1 -- 172.21.15.73:0/4112290678 >> 172.21.15.73:0/4112290678 conn(0x7f8c400d0be0 msgr2=0x7f8c4015e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:42.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.210+0000 7f8c46378640 1 -- 172.21.15.73:0/4112290678 shutdown_connections 2024-04-26T20:51:42.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.211+0000 7f8c46378640 1 -- 172.21.15.73:0/4112290678 wait complete. 2024-04-26T20:51:42.223 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:50:19.993731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:51:42.223 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:51:42.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.321+0000 7efd57b19640 1 Processor -- start 2024-04-26T20:51:42.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.321+0000 7efd57b19640 1 -- start start 2024-04-26T20:51:42.326 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.322+0000 7efd57b19640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7efd5015f170 0x7efd500d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:42.326 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.322+0000 7efd57b19640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd500d1040 con 0x7efd500d3bc0 2024-04-26T20:51:42.326 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.322+0000 7efd57b19640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd500c9680 con 0x7efd5015f550 2024-04-26T20:51:42.326 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.322+0000 7efd57b19640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7efd500c9430 con 0x7efd5015f170 2024-04-26T20:51:42.326 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.322+0000 7efd560b6640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7efd500d3bc0 0x7efd500ca440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:41594/0 (socket says 172.21.15.73:41594) 2024-04-26T20:51:42.326 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.322+0000 7efd560b6640 1 -- 172.21.15.73:0/3153756458 learned_addr learned my addr 172.21.15.73:0/3153756458 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:42.326 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.322+0000 7efd558b5640 1 --2- 172.21.15.73:0/3153756458 >> v2:172.21.15.120:3300/0 conn(0x7efd5015f170 0x7efd500d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:42.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.322+0000 7efd548b3640 1 -- 172.21.15.73:0/3153756458 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2542473553 0 0) 0x7efd48001490 con 0x7efd500d3bc0 2024-04-26T20:51:42.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.322+0000 7efd548b3640 1 -- 172.21.15.73:0/3153756458 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd3c003520 con 0x7efd500d3bc0 2024-04-26T20:51:42.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.323+0000 7efd548b3640 1 -- 172.21.15.73:0/3153756458 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 437889186 0 0) 0x7efd3c003520 con 0x7efd500d3bc0 2024-04-26T20:51:42.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.323+0000 7efd548b3640 1 -- 172.21.15.73:0/3153756458 >> v2:172.21.15.120:3300/0 conn(0x7efd5015f170 msgr2=0x7efd500d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:42.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.323+0000 7efd548b3640 1 --2- 172.21.15.73:0/3153756458 >> v2:172.21.15.120:3300/0 conn(0x7efd5015f170 0x7efd500d5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:42.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.323+0000 7efd548b3640 1 -- 172.21.15.73:0/3153756458 >> v1:172.21.15.120:6789/0 conn(0x7efd5015f550 legacy=0x7efd500d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:42.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.323+0000 7efd548b3640 1 -- 172.21.15.73:0/3153756458 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd500c9960 con 0x7efd500d3bc0 2024-04-26T20:51:42.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.323+0000 7efd548b3640 1 -- 172.21.15.73:0/3153756458 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7efd48001bc0 con 0x7efd500d3bc0 2024-04-26T20:51:42.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.323+0000 7efd548b3640 1 -- 172.21.15.73:0/3153756458 >> v1:172.21.15.73:6789/0 conn(0x7efd500d3bc0 legacy=0x7efd500ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:42.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.323+0000 7efd548b3640 1 --2- 172.21.15.73:0/3153756458 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efd3c0039b0 0x7efd3c003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:42.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.323+0000 7efd548b3640 1 --2- 172.21.15.73:0/3153756458 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd3c0043d0 0x7efd3c0048a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:42.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.323+0000 7efd548b3640 1 --2- 172.21.15.73:0/3153756458 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd3c0050a0 0x7efd3c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:42.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.323+0000 7efd548b3640 1 -- 172.21.15.73:0/3153756458 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efd500c9430 con 0x7efd3c0050a0 2024-04-26T20:51:42.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.323+0000 7efd548b3640 1 -- 172.21.15.73:0/3153756458 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efd3c00bfe0 con 0x7efd3c0043d0 2024-04-26T20:51:42.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.323+0000 7efd548b3640 1 -- 172.21.15.73:0/3153756458 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7efd3c00c1e0 con 0x7efd3c0039b0 2024-04-26T20:51:42.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.323+0000 7efd560b6640 1 --2- 172.21.15.73:0/3153756458 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd3c0050a0 0x7efd3c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:42.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.323+0000 7efd550b4640 1 --2- 172.21.15.73:0/3153756458 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd3c0043d0 0x7efd3c0048a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:42.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.323+0000 7efd558b5640 1 --2- 172.21.15.73:0/3153756458 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efd3c0039b0 0x7efd3c003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:42.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.323+0000 7efd560b6640 1 -- 172.21.15.73:0/3153756458 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efd3c0039b0 msgr2=0x7efd3c003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:42.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.323+0000 7efd560b6640 1 --2- 172.21.15.73:0/3153756458 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efd3c0039b0 0x7efd3c003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:42.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.323+0000 7efd560b6640 1 -- 172.21.15.73:0/3153756458 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd3c0043d0 msgr2=0x7efd3c0048a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:42.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.323+0000 7efd560b6640 1 --2- 172.21.15.73:0/3153756458 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd3c0043d0 0x7efd3c0048a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:42.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.323+0000 7efd560b6640 1 -- 172.21.15.73:0/3153756458 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd3c003520 con 0x7efd3c0050a0 2024-04-26T20:51:42.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.323+0000 7efd550b4640 1 --2- 172.21.15.73:0/3153756458 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd3c0043d0 0x7efd3c0048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:42.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.324+0000 7efd560b6640 1 -- 172.21.15.73:0/3153756458 reap_dead start 2024-04-26T20:51:42.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.324+0000 7efd560b6640 1 --2- 172.21.15.73:0/3153756458 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd3c0050a0 0x7efd3c00b9c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7efd4800aa50 tx=0x7efd3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e77b001f5aa1a589 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:42.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.324+0000 7efd548b3640 1 -- 172.21.15.73:0/3153756458 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4800a670 con 0x7efd3c0050a0 2024-04-26T20:51:42.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.324+0000 7efd548b3640 1 -- 172.21.15.73:0/3153756458 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd4802f070 con 0x7efd3c0050a0 2024-04-26T20:51:42.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.324+0000 7efd548b3640 1 -- 172.21.15.73:0/3153756458 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd48034470 con 0x7efd3c0050a0 2024-04-26T20:51:42.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.324+0000 7efd57b19640 1 -- 172.21.15.73:0/3153756458 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd3c0050a0 msgr2=0x7efd3c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:42.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.324+0000 7efd57b19640 1 --2- 172.21.15.73:0/3153756458 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd3c0050a0 0x7efd3c00b9c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7efd4800aa50 tx=0x7efd3c005830 comp rx=0 tx=0).stop 2024-04-26T20:51:42.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.325+0000 7efd57b19640 1 -- 172.21.15.73:0/3153756458 shutdown_connections 2024-04-26T20:51:42.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.325+0000 7efd57b19640 1 --2- 172.21.15.73:0/3153756458 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd3c0050a0 0x7efd3c00b9c0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:42.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.325+0000 7efd57b19640 1 -- 172.21.15.73:0/3153756458 >> 172.21.15.73:0/3153756458 conn(0x7efd500d4f60 msgr2=0x7efd500d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:42.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.325+0000 7efd57b19640 1 -- 172.21.15.73:0/3153756458 shutdown_connections 2024-04-26T20:51:42.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.325+0000 7efd57b19640 1 -- 172.21.15.73:0/3153756458 wait complete. 2024-04-26T20:51:42.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.326+0000 7efd57b19640 1 Processor -- start 2024-04-26T20:51:42.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.326+0000 7efd57b19640 1 -- start start 2024-04-26T20:51:42.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.326+0000 7efd57b19640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd3c0050a0 0x7efd501f8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:42.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.327+0000 7efd57b19640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd5015a640 0x7efd501f8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:42.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.327+0000 7efd57b19640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efd501f49e0 0x7efd501f4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:42.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.327+0000 7efd57b19640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efd500d5c00 con 0x7efd3c0050a0 2024-04-26T20:51:42.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.327+0000 7efd57b19640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efd500d5a80 con 0x7efd5015a640 2024-04-26T20:51:42.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.327+0000 7efd57b19640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7efd500d5d80 con 0x7efd501f49e0 2024-04-26T20:51:42.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.327+0000 7efd558b5640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd3c0050a0 0x7efd501f8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:42.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.327+0000 7efd558b5640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd3c0050a0 0x7efd501f8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54456/0 (socket says 172.21.15.73:54456) 2024-04-26T20:51:42.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.327+0000 7efd558b5640 1 -- 172.21.15.73:0/3182724576 learned_addr learned my addr 172.21.15.73:0/3182724576 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:42.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.327+0000 7efd550b4640 1 --2- 172.21.15.73:0/3182724576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd5015a640 0x7efd501f8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:42.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.327+0000 7efd560b6640 1 --2- 172.21.15.73:0/3182724576 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efd501f49e0 0x7efd501f4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:42.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.327+0000 7efd558b5640 1 -- 172.21.15.73:0/3182724576 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efd501f49e0 msgr2=0x7efd501f4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:42.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.327+0000 7efd558b5640 1 --2- 172.21.15.73:0/3182724576 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efd501f49e0 0x7efd501f4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:42.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.327+0000 7efd558b5640 1 -- 172.21.15.73:0/3182724576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd5015a640 msgr2=0x7efd501f8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:42.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.327+0000 7efd558b5640 1 --2- 172.21.15.73:0/3182724576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd5015a640 0x7efd501f8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:42.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.327+0000 7efd558b5640 1 -- 172.21.15.73:0/3182724576 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd5015ac00 con 0x7efd3c0050a0 2024-04-26T20:51:42.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.327+0000 7efd550b4640 1 --2- 172.21.15.73:0/3182724576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd5015a640 0x7efd501f8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:42.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.327+0000 7efd558b5640 1 --2- 172.21.15.73:0/3182724576 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd3c0050a0 0x7efd501f8020 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7efd4c011540 tx=0x7efd4c011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:42.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.327+0000 7efd467fc640 1 -- 172.21.15.73:0/3182724576 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4c015930 con 0x7efd3c0050a0 2024-04-26T20:51:42.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.327+0000 7efd467fc640 1 -- 172.21.15.73:0/3182724576 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd4c015ad0 con 0x7efd3c0050a0 2024-04-26T20:51:42.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.327+0000 7efd57b19640 1 -- 172.21.15.73:0/3182724576 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd5015aee0 con 0x7efd3c0050a0 2024-04-26T20:51:42.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.327+0000 7efd467fc640 1 -- 172.21.15.73:0/3182724576 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4c00f460 con 0x7efd3c0050a0 2024-04-26T20:51:42.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.327+0000 7efd57b19640 1 -- 172.21.15.73:0/3182724576 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd5015b4a0 con 0x7efd3c0050a0 2024-04-26T20:51:42.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.328+0000 7efd57b19640 1 -- 172.21.15.73:0/3182724576 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd500694f0 con 0x7efd3c0050a0 2024-04-26T20:51:42.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.329+0000 7efd467fc640 1 -- 172.21.15.73:0/3182724576 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efd4c00f600 con 0x7efd3c0050a0 2024-04-26T20:51:42.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.329+0000 7efd467fc640 1 --2- 172.21.15.73:0/3182724576 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7efd3406c920 0x7efd3406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:42.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.329+0000 7efd467fc640 1 -- 172.21.15.73:0/3182724576 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7efd4c08c170 con 0x7efd3c0050a0 2024-04-26T20:51:42.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.329+0000 7efd550b4640 1 --2- 172.21.15.73:0/3182724576 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7efd3406c920 0x7efd3406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:42.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.330+0000 7efd550b4640 1 --2- 172.21.15.73:0/3182724576 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7efd3406c920 0x7efd3406ede0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7efd500baff0 tx=0x7efd40009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:42.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.334+0000 7efd467fc640 1 -- 172.21.15.73:0/3182724576 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efd4c019030 con 0x7efd3c0050a0 2024-04-26T20:51:42.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.506+0000 7efd57b19640 1 -- 172.21.15.73:0/3182724576 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efd500d53e0 con 0x7efd3c0050a0 2024-04-26T20:51:42.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.507+0000 7efd467fc640 1 -- 172.21.15.73:0/3182724576 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) ==== 93+0+10 (secure 0 0 0) 0x7efd4c05a7f0 con 0x7efd3c0050a0 2024-04-26T20:51:42.508 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-26T20:51:42.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.509+0000 7efd57b19640 1 -- 172.21.15.73:0/3182724576 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7efd3406c920 msgr2=0x7efd3406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:42.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.509+0000 7efd57b19640 1 --2- 172.21.15.73:0/3182724576 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7efd3406c920 0x7efd3406ede0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7efd500baff0 tx=0x7efd40009040 comp rx=0 tx=0).stop 2024-04-26T20:51:42.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.509+0000 7efd57b19640 1 -- 172.21.15.73:0/3182724576 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd3c0050a0 msgr2=0x7efd501f8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:42.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.509+0000 7efd57b19640 1 --2- 172.21.15.73:0/3182724576 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd3c0050a0 0x7efd501f8020 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7efd4c011540 tx=0x7efd4c011a00 comp rx=0 tx=0).stop 2024-04-26T20:51:42.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.509+0000 7efd57b19640 1 -- 172.21.15.73:0/3182724576 shutdown_connections 2024-04-26T20:51:42.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.509+0000 7efd57b19640 1 --2- 172.21.15.73:0/3182724576 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7efd3406c920 0x7efd3406ede0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:42.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.509+0000 7efd57b19640 1 --2- 172.21.15.73:0/3182724576 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efd501f49e0 0x7efd501f4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:42.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.509+0000 7efd57b19640 1 --2- 172.21.15.73:0/3182724576 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efd5015a640 0x7efd501f8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:42.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.509+0000 7efd57b19640 1 --2- 172.21.15.73:0/3182724576 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd3c0050a0 0x7efd501f8020 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:42.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.509+0000 7efd57b19640 1 -- 172.21.15.73:0/3182724576 >> 172.21.15.73:0/3182724576 conn(0x7efd500d4f60 msgr2=0x7efd5006e030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:42.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.509+0000 7efd57b19640 1 -- 172.21.15.73:0/3182724576 shutdown_connections 2024-04-26T20:51:42.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.509+0000 7efd57b19640 1 -- 172.21.15.73:0/3182724576 wait complete. 2024-04-26T20:51:42.521 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T20:51:42.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.617+0000 7f236fd66640 1 Processor -- start 2024-04-26T20:51:42.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.617+0000 7f236fd66640 1 -- start start 2024-04-26T20:51:42.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.618+0000 7f236fd66640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f23680b2340 0x7f23680cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:42.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.618+0000 7f236fd66640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f23680cd930 0x7f23680d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:42.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.618+0000 7f236fd66640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23680d10d0 con 0x7f2368162810 2024-04-26T20:51:42.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.618+0000 7f236fd66640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f23680d0750 con 0x7f23680b2340 2024-04-26T20:51:42.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.618+0000 7f236fd66640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f23680d16a0 con 0x7f23680cd930 2024-04-26T20:51:42.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.618+0000 7f236d301640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f23680b2340 0x7f23680cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:42.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.618+0000 7f236d301640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f23680b2340 0x7f23680cd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54460/0 (socket says 172.21.15.73:54460) 2024-04-26T20:51:42.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.618+0000 7f236d301640 1 -- 172.21.15.73:0/4164357406 learned_addr learned my addr 172.21.15.73:0/4164357406 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:42.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.618+0000 7f236d301640 1 -- 172.21.15.73:0/4164357406 >> v2:172.21.15.120:3300/0 conn(0x7f23680cd930 msgr2=0x7f23680d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:42.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.618+0000 7f236d301640 1 --2- 172.21.15.73:0/4164357406 >> v2:172.21.15.120:3300/0 conn(0x7f23680cd930 0x7f23680d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:42.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.618+0000 7f236d301640 1 -- 172.21.15.73:0/4164357406 >> v1:172.21.15.120:6789/0 conn(0x7f2368162810 legacy=0x7f2368070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:42.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.618+0000 7f236d301640 1 -- 172.21.15.73:0/4164357406 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23680d1370 con 0x7f23680b2340 2024-04-26T20:51:42.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.619+0000 7f236d301640 1 --2- 172.21.15.73:0/4164357406 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23680b2340 0x7f23680cd3f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f23680b2280 tx=0x7f235802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=566740efd014f49a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:42.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.619+0000 7f236cb00640 1 -- 172.21.15.73:0/4164357406 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2358037070 con 0x7f23680b2340 2024-04-26T20:51:42.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.619+0000 7f236cb00640 1 -- 172.21.15.73:0/4164357406 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2358032440 con 0x7f23680b2340 2024-04-26T20:51:42.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.619+0000 7f236cb00640 1 -- 172.21.15.73:0/4164357406 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2358032810 con 0x7f23680b2340 2024-04-26T20:51:42.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.619+0000 7f236fd66640 1 -- 172.21.15.73:0/4164357406 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23680b2340 msgr2=0x7f23680cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:42.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.619+0000 7f236fd66640 1 --2- 172.21.15.73:0/4164357406 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23680b2340 0x7f23680cd3f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f23680b2280 tx=0x7f235802a250 comp rx=0 tx=0).stop 2024-04-26T20:51:42.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.619+0000 7f236fd66640 1 -- 172.21.15.73:0/4164357406 shutdown_connections 2024-04-26T20:51:42.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.619+0000 7f236fd66640 1 --2- 172.21.15.73:0/4164357406 >> v2:172.21.15.120:3300/0 conn(0x7f23680cd930 0x7f23680d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:42.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.619+0000 7f236fd66640 1 --2- 172.21.15.73:0/4164357406 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23680b2340 0x7f23680cd3f0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:42.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.619+0000 7f236fd66640 1 -- 172.21.15.73:0/4164357406 >> 172.21.15.73:0/4164357406 conn(0x7f23680d2da0 msgr2=0x7f23680d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:42.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.620+0000 7f236fd66640 1 -- 172.21.15.73:0/4164357406 shutdown_connections 2024-04-26T20:51:42.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.620+0000 7f236fd66640 1 -- 172.21.15.73:0/4164357406 wait complete. 2024-04-26T20:51:42.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.621+0000 7f236fd66640 1 Processor -- start 2024-04-26T20:51:42.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.621+0000 7f236fd66640 1 -- start start 2024-04-26T20:51:42.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.621+0000 7f236fd66640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f23680b2340 0x7f236814c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:42.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.621+0000 7f236fd66640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23680cd930 0x7f2368160760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:42.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.621+0000 7f236fd66640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2368162810 0x7f2368160ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:42.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.621+0000 7f236fd66640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f23680d2070 con 0x7f23680cd930 2024-04-26T20:51:42.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.621+0000 7f236fd66640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f23680d0470 con 0x7f2368162810 2024-04-26T20:51:42.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.621+0000 7f236fd66640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f23680d21f0 con 0x7f23680b2340 2024-04-26T20:51:42.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.621+0000 7f236d301640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23680cd930 0x7f2368160760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:42.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.621+0000 7f236d301640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23680cd930 0x7f2368160760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54464/0 (socket says 172.21.15.73:54464) 2024-04-26T20:51:42.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.622+0000 7f236d301640 1 -- 172.21.15.73:0/2639443881 learned_addr learned my addr 172.21.15.73:0/2639443881 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:42.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.622+0000 7f236e303640 1 --2- 172.21.15.73:0/2639443881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2368162810 0x7f2368160ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:42.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.622+0000 7f236db02640 1 --2- 172.21.15.73:0/2639443881 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f23680b2340 0x7f236814c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:42.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.622+0000 7f236d301640 1 -- 172.21.15.73:0/2639443881 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f23680b2340 msgr2=0x7f236814c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:42.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.622+0000 7f236d301640 1 --2- 172.21.15.73:0/2639443881 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f23680b2340 0x7f236814c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:42.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.622+0000 7f236d301640 1 -- 172.21.15.73:0/2639443881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2368162810 msgr2=0x7f2368160ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:42.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.622+0000 7f236d301640 1 --2- 172.21.15.73:0/2639443881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2368162810 0x7f2368160ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:42.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.622+0000 7f236d301640 1 -- 172.21.15.73:0/2639443881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23680cab10 con 0x7f23680cd930 2024-04-26T20:51:42.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.622+0000 7f236e303640 1 --2- 172.21.15.73:0/2639443881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2368162810 0x7f2368160ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:42.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.622+0000 7f236d301640 1 --2- 172.21.15.73:0/2639443881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23680cd930 0x7f2368160760 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2358000c00 tx=0x7f23580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:42.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.622+0000 7f235e7fc640 1 -- 172.21.15.73:0/2639443881 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2358041070 con 0x7f23680cd930 2024-04-26T20:51:42.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.622+0000 7f235e7fc640 1 -- 172.21.15.73:0/2639443881 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f235803c870 con 0x7f23680cd930 2024-04-26T20:51:42.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.622+0000 7f235e7fc640 1 -- 172.21.15.73:0/2639443881 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2358037040 con 0x7f23680cd930 2024-04-26T20:51:42.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.622+0000 7f236fd66640 1 -- 172.21.15.73:0/2639443881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23680cadf0 con 0x7f23680cd930 2024-04-26T20:51:42.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.622+0000 7f236fd66640 1 -- 172.21.15.73:0/2639443881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23681f6b10 con 0x7f23680cd930 2024-04-26T20:51:42.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.623+0000 7f236fd66640 1 -- 172.21.15.73:0/2639443881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23680694f0 con 0x7f23680cd930 2024-04-26T20:51:42.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.624+0000 7f235e7fc640 1 -- 172.21.15.73:0/2639443881 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2358044070 con 0x7f23680cd930 2024-04-26T20:51:42.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.624+0000 7f235e7fc640 1 --2- 172.21.15.73:0/2639443881 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f234c06c8f0 0x7f234c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:42.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.624+0000 7f235e7fc640 1 -- 172.21.15.73:0/2639443881 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f23580ac580 con 0x7f23680cd930 2024-04-26T20:51:42.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.624+0000 7f236db02640 1 --2- 172.21.15.73:0/2639443881 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f234c06c8f0 0x7f234c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:42.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.624+0000 7f236db02640 1 --2- 172.21.15.73:0/2639443881 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f234c06c8f0 0x7f234c06edb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2364006fd0 tx=0x7f2364008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:42.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.628+0000 7f235e7fc640 1 -- 172.21.15.73:0/2639443881 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f235807ac90 con 0x7f23680cd930 2024-04-26T20:51:42.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.806+0000 7f236fd66640 1 -- 172.21.15.73:0/2639443881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f23680b45e0 con 0x7f23680cd930 2024-04-26T20:51:42.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.806+0000 7f235e7fc640 1 -- 172.21.15.73:0/2639443881 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v35) ==== 84+0+65 (secure 0 0 0) 0x7f235807a590 con 0x7f23680cd930 2024-04-26T20:51:42.807 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:51:42.807 INFO:teuthology.orchestra.run.smithi073.stdout:[ 2024-04-26T20:51:42.807 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-26T20:51:42.807 INFO:teuthology.orchestra.run.smithi073.stdout: "poolnum": 1, 2024-04-26T20:51:42.808 INFO:teuthology.orchestra.run.smithi073.stdout: "poolname": ".mgr" 2024-04-26T20:51:42.808 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-26T20:51:42.808 INFO:teuthology.orchestra.run.smithi073.stdout:] 2024-04-26T20:51:42.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.808+0000 7f236fd66640 1 -- 172.21.15.73:0/2639443881 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f234c06c8f0 msgr2=0x7f234c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:42.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.808+0000 7f236fd66640 1 --2- 172.21.15.73:0/2639443881 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f234c06c8f0 0x7f234c06edb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2364006fd0 tx=0x7f2364008040 comp rx=0 tx=0).stop 2024-04-26T20:51:42.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.808+0000 7f236fd66640 1 -- 172.21.15.73:0/2639443881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23680cd930 msgr2=0x7f2368160760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:42.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.808+0000 7f236fd66640 1 --2- 172.21.15.73:0/2639443881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23680cd930 0x7f2368160760 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2358000c00 tx=0x7f23580046c0 comp rx=0 tx=0).stop 2024-04-26T20:51:42.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.808+0000 7f236fd66640 1 -- 172.21.15.73:0/2639443881 shutdown_connections 2024-04-26T20:51:42.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.808+0000 7f236fd66640 1 --2- 172.21.15.73:0/2639443881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23680cd930 0x7f2368160760 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:42.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.808+0000 7f236fd66640 1 --2- 172.21.15.73:0/2639443881 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f234c06c8f0 0x7f234c06edb0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:42.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.808+0000 7f236fd66640 1 --2- 172.21.15.73:0/2639443881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2368162810 0x7f2368160ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:42.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.808+0000 7f236fd66640 1 --2- 172.21.15.73:0/2639443881 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f23680b2340 0x7f236814c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:42.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.808+0000 7f236fd66640 1 -- 172.21.15.73:0/2639443881 >> 172.21.15.73:0/2639443881 conn(0x7f23680d2da0 msgr2=0x7f2368070b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:42.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.808+0000 7f236fd66640 1 -- 172.21.15.73:0/2639443881 shutdown_connections 2024-04-26T20:51:42.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.808+0000 7f236fd66640 1 -- 172.21.15.73:0/2639443881 wait complete. 2024-04-26T20:51:42.820 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T20:51:42.820 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T20:51:42.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.921+0000 7fc0f6771640 1 Processor -- start 2024-04-26T20:51:42.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.921+0000 7fc0f6771640 1 -- start start 2024-04-26T20:51:42.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.922+0000 7fc0f6771640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc0f015d100 0x7fc0f006c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:42.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.922+0000 7fc0f6771640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc0f0071720 0x7fc0f00b0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:42.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.922+0000 7fc0f6771640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0f015d4e0 con 0x7fc0f006c780 2024-04-26T20:51:42.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.922+0000 7fc0f6771640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7fc0f0161ae0 con 0x7fc0f015d100 2024-04-26T20:51:42.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.922+0000 7fc0f6771640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fc0f006cc70 con 0x7fc0f0071720 2024-04-26T20:51:42.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.922+0000 7fc0effff640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc0f015d100 0x7fc0f006c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:42.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.922+0000 7fc0effff640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc0f015d100 0x7fc0f006c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54470/0 (socket says 172.21.15.73:54470) 2024-04-26T20:51:42.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.922+0000 7fc0effff640 1 -- 172.21.15.73:0/2013922502 learned_addr learned my addr 172.21.15.73:0/2013922502 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:42.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.922+0000 7fc0f4d0e640 1 --2- 172.21.15.73:0/2013922502 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc0f0071720 0x7fc0f00b0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:42.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.922+0000 7fc0effff640 1 -- 172.21.15.73:0/2013922502 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc0f0071720 msgr2=0x7fc0f00b0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:42.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.922+0000 7fc0effff640 1 --2- 172.21.15.73:0/2013922502 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc0f0071720 0x7fc0f00b0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:42.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.922+0000 7fc0effff640 1 -- 172.21.15.73:0/2013922502 >> v1:172.21.15.120:6789/0 conn(0x7fc0f006c780 legacy=0x7fc0f0071010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:42.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.922+0000 7fc0effff640 1 -- 172.21.15.73:0/2013922502 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0f0160d20 con 0x7fc0f015d100 2024-04-26T20:51:42.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.923+0000 7fc0effff640 1 --2- 172.21.15.73:0/2013922502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0f015d100 0x7fc0f006c240 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fc0e00098a0 tx=0x7fc0e002a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2246841a7dfc659e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:42.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.923+0000 7fc0eeffd640 1 -- 172.21.15.73:0/2013922502 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e0037070 con 0x7fc0f015d100 2024-04-26T20:51:42.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.923+0000 7fc0eeffd640 1 -- 172.21.15.73:0/2013922502 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0e0032440 con 0x7fc0f015d100 2024-04-26T20:51:42.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.923+0000 7fc0eeffd640 1 -- 172.21.15.73:0/2013922502 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e0032810 con 0x7fc0f015d100 2024-04-26T20:51:42.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.923+0000 7fc0f6771640 1 -- 172.21.15.73:0/2013922502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0f015d100 msgr2=0x7fc0f006c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:42.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.923+0000 7fc0f6771640 1 --2- 172.21.15.73:0/2013922502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0f015d100 0x7fc0f006c240 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fc0e00098a0 tx=0x7fc0e002a200 comp rx=0 tx=0).stop 2024-04-26T20:51:42.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.923+0000 7fc0f6771640 1 -- 172.21.15.73:0/2013922502 shutdown_connections 2024-04-26T20:51:42.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.923+0000 7fc0f6771640 1 --2- 172.21.15.73:0/2013922502 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc0f0071720 0x7fc0f00b0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:42.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.923+0000 7fc0f6771640 1 --2- 172.21.15.73:0/2013922502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0f015d100 0x7fc0f006c240 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:42.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.923+0000 7fc0f6771640 1 -- 172.21.15.73:0/2013922502 >> 172.21.15.73:0/2013922502 conn(0x7fc0f00d1d60 msgr2=0x7fc0f00d2140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:42.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.923+0000 7fc0f6771640 1 -- 172.21.15.73:0/2013922502 shutdown_connections 2024-04-26T20:51:42.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.923+0000 7fc0f6771640 1 -- 172.21.15.73:0/2013922502 wait complete. 2024-04-26T20:51:42.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.924+0000 7fc0f6771640 1 Processor -- start 2024-04-26T20:51:42.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.925+0000 7fc0f6771640 1 -- start start 2024-04-26T20:51:42.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.925+0000 7fc0f6771640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0f006c780 0x7fc0f015b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:42.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.925+0000 7fc0f6771640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0f0071720 0x7fc0f00b0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:42.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.925+0000 7fc0f6771640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc0f015d100 0x7fc0f00b0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:42.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.925+0000 7fc0f6771640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f01615c0 con 0x7fc0f0071720 2024-04-26T20:51:42.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.925+0000 7fc0f6771640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc0f01612e0 con 0x7fc0f006c780 2024-04-26T20:51:42.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.925+0000 7fc0f6771640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fc0f0073830 con 0x7fc0f015d100 2024-04-26T20:51:42.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.925+0000 7fc0ef7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0f0071720 0x7fc0f00b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:42.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.925+0000 7fc0ef7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0f0071720 0x7fc0f00b0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54478/0 (socket says 172.21.15.73:54478) 2024-04-26T20:51:42.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.925+0000 7fc0ef7fe640 1 -- 172.21.15.73:0/952375127 learned_addr learned my addr 172.21.15.73:0/952375127 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:42.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.926+0000 7fc0effff640 1 --2- 172.21.15.73:0/952375127 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0f006c780 0x7fc0f015b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:42.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.926+0000 7fc0f4d0e640 1 --2- 172.21.15.73:0/952375127 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc0f015d100 0x7fc0f00b0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:42.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.926+0000 7fc0ef7fe640 1 -- 172.21.15.73:0/952375127 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc0f015d100 msgr2=0x7fc0f00b0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:42.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.926+0000 7fc0ef7fe640 1 --2- 172.21.15.73:0/952375127 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc0f015d100 0x7fc0f00b0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:42.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.926+0000 7fc0ef7fe640 1 -- 172.21.15.73:0/952375127 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0f006c780 msgr2=0x7fc0f015b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:42.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.926+0000 7fc0ef7fe640 1 --2- 172.21.15.73:0/952375127 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0f006c780 0x7fc0f015b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:42.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.926+0000 7fc0ef7fe640 1 -- 172.21.15.73:0/952375127 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0f015e1d0 con 0x7fc0f0071720 2024-04-26T20:51:42.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.926+0000 7fc0ef7fe640 1 --2- 172.21.15.73:0/952375127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0f0071720 0x7fc0f00b0220 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc0e400c400 tx=0x7fc0e400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:42.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.926+0000 7fc0ecff9640 1 -- 172.21.15.73:0/952375127 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e40052d0 con 0x7fc0f0071720 2024-04-26T20:51:42.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.926+0000 7fc0ecff9640 1 -- 172.21.15.73:0/952375127 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0e4005490 con 0x7fc0f0071720 2024-04-26T20:51:42.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.926+0000 7fc0f6771640 1 -- 172.21.15.73:0/952375127 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0f00b2490 con 0x7fc0f0071720 2024-04-26T20:51:42.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.926+0000 7fc0ecff9640 1 -- 172.21.15.73:0/952375127 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4007260 con 0x7fc0f0071720 2024-04-26T20:51:42.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.926+0000 7fc0f6771640 1 -- 172.21.15.73:0/952375127 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0f00b2a50 con 0x7fc0f0071720 2024-04-26T20:51:42.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.927+0000 7fc0f6771640 1 -- 172.21.15.73:0/952375127 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0f0069600 con 0x7fc0f0071720 2024-04-26T20:51:42.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.927+0000 7fc0ecff9640 1 -- 172.21.15.73:0/952375127 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc0e4007400 con 0x7fc0f0071720 2024-04-26T20:51:42.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.928+0000 7fc0ecff9640 1 --2- 172.21.15.73:0/952375127 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc0dc06c8f0 0x7fc0dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:42.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.928+0000 7fc0ecff9640 1 -- 172.21.15.73:0/952375127 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7fc0e408af20 con 0x7fc0f0071720 2024-04-26T20:51:42.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.928+0000 7fc0effff640 1 --2- 172.21.15.73:0/952375127 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc0dc06c8f0 0x7fc0dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:42.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.928+0000 7fc0effff640 1 --2- 172.21.15.73:0/952375127 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc0dc06c8f0 0x7fc0dc06edb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0034000 tx=0x7fc0e002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:42.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:42.932+0000 7fc0ecff9640 1 -- 172.21.15.73:0/952375127 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc0e4059510 con 0x7fc0f0071720 2024-04-26T20:51:43.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:43.106+0000 7fc0f6771640 1 -- 172.21.15.73:0/952375127 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fc0f0073130 con 0x7fc0f0071720 2024-04-26T20:51:43.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:43.914+0000 7fc0ecff9640 1 -- 172.21.15.73:0/952375127 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v36) ==== 138+0+0 (secure 0 0 0) 0x7fc0e4058e10 con 0x7fc0f0071720 2024-04-26T20:51:43.915 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_metadata' created 2024-04-26T20:51:43.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:43.917+0000 7fc0f6771640 1 -- 172.21.15.73:0/952375127 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc0dc06c8f0 msgr2=0x7fc0dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:43.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:43.917+0000 7fc0f6771640 1 --2- 172.21.15.73:0/952375127 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc0dc06c8f0 0x7fc0dc06edb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0034000 tx=0x7fc0e002e000 comp rx=0 tx=0).stop 2024-04-26T20:51:43.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:43.917+0000 7fc0f6771640 1 -- 172.21.15.73:0/952375127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0f0071720 msgr2=0x7fc0f00b0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:43.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:43.917+0000 7fc0f6771640 1 --2- 172.21.15.73:0/952375127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0f0071720 0x7fc0f00b0220 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc0e400c400 tx=0x7fc0e400c8c0 comp rx=0 tx=0).stop 2024-04-26T20:51:43.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:43.917+0000 7fc0f6771640 1 -- 172.21.15.73:0/952375127 shutdown_connections 2024-04-26T20:51:43.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:43.917+0000 7fc0f6771640 1 --2- 172.21.15.73:0/952375127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0f0071720 0x7fc0f00b0220 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:43.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:43.917+0000 7fc0f6771640 1 --2- 172.21.15.73:0/952375127 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc0dc06c8f0 0x7fc0dc06edb0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:43.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:43.917+0000 7fc0f6771640 1 --2- 172.21.15.73:0/952375127 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc0f015d100 0x7fc0f00b0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:43.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:43.917+0000 7fc0f6771640 1 --2- 172.21.15.73:0/952375127 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc0f006c780 0x7fc0f015b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:43.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:43.917+0000 7fc0f6771640 1 -- 172.21.15.73:0/952375127 >> 172.21.15.73:0/952375127 conn(0x7fc0f00d1d60 msgr2=0x7fc0f006daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:43.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:43.917+0000 7fc0f6771640 1 -- 172.21.15.73:0/952375127 shutdown_connections 2024-04-26T20:51:43.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:43.917+0000 7fc0f6771640 1 -- 172.21.15.73:0/952375127 wait complete. 2024-04-26T20:51:43.935 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T20:51:44.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.043+0000 7f218c5c1640 1 Processor -- start 2024-04-26T20:51:44.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.043+0000 7f218c5c1640 1 -- start start 2024-04-26T20:51:44.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.044+0000 7f218c5c1640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f218406f790 0x7f218406fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:44.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.044+0000 7f218c5c1640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2184072c50 0x7f2184073060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:44.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.044+0000 7f218c5c1640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2184156840 con 0x7f2184156460 2024-04-26T20:51:44.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.044+0000 7f218c5c1640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f21840cbf20 con 0x7f2184072c50 2024-04-26T20:51:44.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.044+0000 7f218c5c1640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f21840cc200 con 0x7f218406f790 2024-04-26T20:51:44.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.044+0000 7f218ab5e640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2184072c50 0x7f2184073060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:44.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.044+0000 7f218ab5e640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2184072c50 0x7f2184073060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54480/0 (socket says 172.21.15.73:54480) 2024-04-26T20:51:44.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.044+0000 7f218ab5e640 1 -- 172.21.15.73:0/1936556663 learned_addr learned my addr 172.21.15.73:0/1936556663 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:44.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.044+0000 7f2189b5c640 1 --2- 172.21.15.73:0/1936556663 >> v2:172.21.15.120:3300/0 conn(0x7f218406f790 0x7f218406fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:44.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.044+0000 7f218ab5e640 1 -- 172.21.15.73:0/1936556663 >> v2:172.21.15.120:3300/0 conn(0x7f218406f790 msgr2=0x7f218406fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:44.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.044+0000 7f218ab5e640 1 --2- 172.21.15.73:0/1936556663 >> v2:172.21.15.120:3300/0 conn(0x7f218406f790 0x7f218406fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:44.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.044+0000 7f218ab5e640 1 -- 172.21.15.73:0/1936556663 >> v1:172.21.15.120:6789/0 conn(0x7f2184156460 legacy=0x7f21840cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:44.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.044+0000 7f218ab5e640 1 -- 172.21.15.73:0/1936556663 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21840cc4e0 con 0x7f2184072c50 2024-04-26T20:51:44.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.045+0000 7f218ab5e640 1 --2- 172.21.15.73:0/1936556663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2184072c50 0x7f2184073060 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f21840d0b90 tx=0x7f217c02c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdb4851ab64e1a96 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:44.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.045+0000 7f218935b640 1 -- 172.21.15.73:0/1936556663 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f217c009600 con 0x7f2184072c50 2024-04-26T20:51:44.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.045+0000 7f218935b640 1 -- 172.21.15.73:0/1936556663 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f217c0097a0 con 0x7f2184072c50 2024-04-26T20:51:44.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.045+0000 7f218935b640 1 -- 172.21.15.73:0/1936556663 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f217c009b70 con 0x7f2184072c50 2024-04-26T20:51:44.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.046+0000 7f218c5c1640 1 -- 172.21.15.73:0/1936556663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2184072c50 msgr2=0x7f2184073060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:44.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.046+0000 7f218c5c1640 1 --2- 172.21.15.73:0/1936556663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2184072c50 0x7f2184073060 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f21840d0b90 tx=0x7f217c02c900 comp rx=0 tx=0).stop 2024-04-26T20:51:44.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.046+0000 7f218c5c1640 1 -- 172.21.15.73:0/1936556663 shutdown_connections 2024-04-26T20:51:44.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.046+0000 7f218c5c1640 1 --2- 172.21.15.73:0/1936556663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2184072c50 0x7f2184073060 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:44.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.046+0000 7f218c5c1640 1 --2- 172.21.15.73:0/1936556663 >> v2:172.21.15.120:3300/0 conn(0x7f218406f790 0x7f218406fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:44.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.046+0000 7f218c5c1640 1 -- 172.21.15.73:0/1936556663 >> 172.21.15.73:0/1936556663 conn(0x7f21840d0be0 msgr2=0x7f21840d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:44.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.046+0000 7f218c5c1640 1 -- 172.21.15.73:0/1936556663 shutdown_connections 2024-04-26T20:51:44.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.046+0000 7f218c5c1640 1 -- 172.21.15.73:0/1936556663 wait complete. 2024-04-26T20:51:44.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.047+0000 7f218c5c1640 1 Processor -- start 2024-04-26T20:51:44.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.047+0000 7f218c5c1640 1 -- start start 2024-04-26T20:51:44.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.048+0000 7f218c5c1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f218406f790 0x7f21841f47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:44.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.048+0000 7f218c5c1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2184072c50 0x7f21841468f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:44.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.048+0000 7f218c5c1640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2184156460 0x7f2184146e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:44.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.048+0000 7f218c5c1640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f21840cbbe0 con 0x7f218406f790 2024-04-26T20:51:44.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.048+0000 7f218c5c1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2184072700 con 0x7f2184072c50 2024-04-26T20:51:44.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.048+0000 7f218c5c1640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f218406e260 con 0x7f2184156460 2024-04-26T20:51:44.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.048+0000 7f218a35d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f218406f790 0x7f21841f47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:44.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.048+0000 7f218a35d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f218406f790 0x7f21841f47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54492/0 (socket says 172.21.15.73:54492) 2024-04-26T20:51:44.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.048+0000 7f218a35d640 1 -- 172.21.15.73:0/60639400 learned_addr learned my addr 172.21.15.73:0/60639400 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:44.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.048+0000 7f2189b5c640 1 --2- 172.21.15.73:0/60639400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2184072c50 0x7f21841468f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:44.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.048+0000 7f218ab5e640 1 --2- 172.21.15.73:0/60639400 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2184156460 0x7f2184146e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:44.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.048+0000 7f218a35d640 1 -- 172.21.15.73:0/60639400 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2184156460 msgr2=0x7f2184146e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:44.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.048+0000 7f218a35d640 1 --2- 172.21.15.73:0/60639400 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2184156460 0x7f2184146e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:44.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.048+0000 7f218a35d640 1 -- 172.21.15.73:0/60639400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2184072c50 msgr2=0x7f21841468f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:44.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.048+0000 7f218a35d640 1 --2- 172.21.15.73:0/60639400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2184072c50 0x7f21841468f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:44.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.048+0000 7f218a35d640 1 -- 172.21.15.73:0/60639400 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21840c7cb0 con 0x7f218406f790 2024-04-26T20:51:44.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.048+0000 7f218a35d640 1 --2- 172.21.15.73:0/60639400 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f218406f790 0x7f21841f47a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f218000b3e0 tx=0x7f218000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:44.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.049+0000 7f217affd640 1 -- 172.21.15.73:0/60639400 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2180015020 con 0x7f218406f790 2024-04-26T20:51:44.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.049+0000 7f217affd640 1 -- 172.21.15.73:0/60639400 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2180011850 con 0x7f218406f790 2024-04-26T20:51:44.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.049+0000 7f218c5c1640 1 -- 172.21.15.73:0/60639400 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21840b00c0 con 0x7f218406f790 2024-04-26T20:51:44.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.049+0000 7f217affd640 1 -- 172.21.15.73:0/60639400 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2180011c40 con 0x7f218406f790 2024-04-26T20:51:44.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.049+0000 7f218c5c1640 1 -- 172.21.15.73:0/60639400 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2184144d90 con 0x7f218406f790 2024-04-26T20:51:44.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.050+0000 7f218c5c1640 1 -- 172.21.15.73:0/60639400 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21840694f0 con 0x7f218406f790 2024-04-26T20:51:44.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.050+0000 7f217affd640 1 -- 172.21.15.73:0/60639400 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2180018020 con 0x7f218406f790 2024-04-26T20:51:44.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.051+0000 7f217affd640 1 --2- 172.21.15.73:0/60639400 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f216806c8f0 0x7f216806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:44.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.051+0000 7f217affd640 1 -- 172.21.15.73:0/60639400 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5619+0+0 (secure 0 0 0) 0x7f218008b920 con 0x7f218406f790 2024-04-26T20:51:44.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.051+0000 7f2189b5c640 1 --2- 172.21.15.73:0/60639400 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f216806c8f0 0x7f216806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:44.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.051+0000 7f2189b5c640 1 --2- 172.21.15.73:0/60639400 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f216806c8f0 0x7f216806edb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f21841479f0 tx=0x7f2174008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:44.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.055+0000 7f217affd640 1 -- 172.21.15.73:0/60639400 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f218001f020 con 0x7f218406f790 2024-04-26T20:51:44.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.231+0000 7f218c5c1640 1 -- 172.21.15.73:0/60639400 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f2184152600 con 0x7f218406f790 2024-04-26T20:51:44.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.919+0000 7f217affd640 1 -- 172.21.15.73:0/60639400 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v37) ==== 170+0+0 (secure 0 0 0) 0x7f21840c7cb0 con 0x7f218406f790 2024-04-26T20:51:44.920 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data' created 2024-04-26T20:51:44.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.922+0000 7f218c5c1640 1 -- 172.21.15.73:0/60639400 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f216806c8f0 msgr2=0x7f216806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:44.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.922+0000 7f218c5c1640 1 --2- 172.21.15.73:0/60639400 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f216806c8f0 0x7f216806edb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f21841479f0 tx=0x7f2174008040 comp rx=0 tx=0).stop 2024-04-26T20:51:44.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.922+0000 7f218c5c1640 1 -- 172.21.15.73:0/60639400 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f218406f790 msgr2=0x7f21841f47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:44.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.922+0000 7f218c5c1640 1 --2- 172.21.15.73:0/60639400 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f218406f790 0x7f21841f47a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f218000b3e0 tx=0x7f218000b8a0 comp rx=0 tx=0).stop 2024-04-26T20:51:44.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.922+0000 7f218c5c1640 1 -- 172.21.15.73:0/60639400 shutdown_connections 2024-04-26T20:51:44.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.922+0000 7f218c5c1640 1 --2- 172.21.15.73:0/60639400 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f216806c8f0 0x7f216806edb0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:44.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.922+0000 7f218c5c1640 1 --2- 172.21.15.73:0/60639400 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2184156460 0x7f2184146e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:44.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.922+0000 7f218c5c1640 1 --2- 172.21.15.73:0/60639400 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2184072c50 0x7f21841468f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:44.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.922+0000 7f218c5c1640 1 --2- 172.21.15.73:0/60639400 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f218406f790 0x7f21841f47a0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:44.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.922+0000 7f218c5c1640 1 -- 172.21.15.73:0/60639400 >> 172.21.15.73:0/60639400 conn(0x7f21840d0be0 msgr2=0x7f218406edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:44.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.923+0000 7f218c5c1640 1 -- 172.21.15.73:0/60639400 shutdown_connections 2024-04-26T20:51:44.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:44.924+0000 7f218c5c1640 1 -- 172.21.15.73:0/60639400 wait complete. 2024-04-26T20:51:44.946 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T20:51:45.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.046+0000 7f679736a640 1 Processor -- start 2024-04-26T20:51:45.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.046+0000 7f679736a640 1 -- start start 2024-04-26T20:51:45.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.047+0000 7f679736a640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6790071260 0x7f679015e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:45.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.047+0000 7f679736a640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67900d2390 con 0x7f6790070350 2024-04-26T20:51:45.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.047+0000 7f679736a640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f679006d3f0 con 0x7f67900cfde0 2024-04-26T20:51:45.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.047+0000 7f679736a640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f67900afab0 con 0x7f6790071260 2024-04-26T20:51:45.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.047+0000 7f6795907640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6790071260 0x7f679015e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:45.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.047+0000 7f6794905640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f6790070350 0x7f6790070b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:41596/0 (socket says 172.21.15.73:41596) 2024-04-26T20:51:45.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.047+0000 7f6794905640 1 -- 172.21.15.73:0/4016981222 learned_addr learned my addr 172.21.15.73:0/4016981222 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:45.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.048+0000 7f6795907640 1 -- 172.21.15.73:0/4016981222 >> v1:172.21.15.120:6789/0 conn(0x7f67900cfde0 legacy=0x7f67900d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:45.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.048+0000 7f6795907640 1 -- 172.21.15.73:0/4016981222 >> v1:172.21.15.73:6789/0 conn(0x7f6790070350 legacy=0x7f6790070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:45.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.048+0000 7f6795907640 1 -- 172.21.15.73:0/4016981222 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f679006d6d0 con 0x7f6790071260 2024-04-26T20:51:45.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.048+0000 7f6787fff640 1 -- 172.21.15.73:0/4016981222 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2708144953 0 0) 0x7f67900d2390 con 0x7f6790070350 2024-04-26T20:51:45.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.048+0000 7f6795907640 1 --2- 172.21.15.73:0/4016981222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6790071260 0x7f679015e9f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f67880097a0 tx=0x7f678802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49286831530082cd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:45.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.048+0000 7f6787fff640 1 -- 172.21.15.73:0/4016981222 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6788037070 con 0x7f6790071260 2024-04-26T20:51:45.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.048+0000 7f6787fff640 1 -- 172.21.15.73:0/4016981222 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6788032440 con 0x7f6790071260 2024-04-26T20:51:45.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.048+0000 7f6787fff640 1 -- 172.21.15.73:0/4016981222 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6788032810 con 0x7f6790071260 2024-04-26T20:51:45.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.049+0000 7f679736a640 1 -- 172.21.15.73:0/4016981222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6790071260 msgr2=0x7f679015e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:45.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.049+0000 7f679736a640 1 --2- 172.21.15.73:0/4016981222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6790071260 0x7f679015e9f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f67880097a0 tx=0x7f678802a1a0 comp rx=0 tx=0).stop 2024-04-26T20:51:45.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.049+0000 7f679736a640 1 -- 172.21.15.73:0/4016981222 shutdown_connections 2024-04-26T20:51:45.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.049+0000 7f679736a640 1 --2- 172.21.15.73:0/4016981222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6790071260 0x7f679015e9f0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:45.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.049+0000 7f679736a640 1 -- 172.21.15.73:0/4016981222 >> 172.21.15.73:0/4016981222 conn(0x7f67900d2d80 msgr2=0x7f67900d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:45.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.049+0000 7f679736a640 1 -- 172.21.15.73:0/4016981222 shutdown_connections 2024-04-26T20:51:45.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.049+0000 7f679736a640 1 -- 172.21.15.73:0/4016981222 wait complete. 2024-04-26T20:51:45.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.050+0000 7f679736a640 1 Processor -- start 2024-04-26T20:51:45.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.050+0000 7f679736a640 1 -- start start 2024-04-26T20:51:45.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.051+0000 7f679736a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6790070350 0x7f67900b56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:45.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.051+0000 7f679736a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6790071260 0x7f67901f4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:45.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.051+0000 7f679736a640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f67900cfde0 0x7f679009d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:45.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.051+0000 7f679736a640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f679006f1c0 con 0x7f6790071260 2024-04-26T20:51:45.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.051+0000 7f679736a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f679006dd20 con 0x7f6790070350 2024-04-26T20:51:45.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.051+0000 7f679736a640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f679006f340 con 0x7f67900cfde0 2024-04-26T20:51:45.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.051+0000 7f6794905640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6790071260 0x7f67901f4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:45.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.051+0000 7f6794905640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6790071260 0x7f67901f4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54512/0 (socket says 172.21.15.73:54512) 2024-04-26T20:51:45.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.051+0000 7f6794905640 1 -- 172.21.15.73:0/784372921 learned_addr learned my addr 172.21.15.73:0/784372921 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:45.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.051+0000 7f6795106640 1 --2- 172.21.15.73:0/784372921 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6790070350 0x7f67900b56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:45.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.051+0000 7f6795907640 1 --2- 172.21.15.73:0/784372921 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f67900cfde0 0x7f679009d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:45.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.051+0000 7f6794905640 1 -- 172.21.15.73:0/784372921 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f67900cfde0 msgr2=0x7f679009d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:45.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.051+0000 7f6794905640 1 --2- 172.21.15.73:0/784372921 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f67900cfde0 0x7f679009d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:45.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.051+0000 7f6794905640 1 -- 172.21.15.73:0/784372921 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6790070350 msgr2=0x7f67900b56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:45.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.051+0000 7f6794905640 1 --2- 172.21.15.73:0/784372921 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6790070350 0x7f67900b56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:45.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.051+0000 7f6794905640 1 -- 172.21.15.73:0/784372921 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67901f5c60 con 0x7f6790071260 2024-04-26T20:51:45.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.051+0000 7f6795106640 1 --2- 172.21.15.73:0/784372921 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6790070350 0x7f67900b56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:45.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.051+0000 7f6794905640 1 --2- 172.21.15.73:0/784372921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6790071260 0x7f67901f4580 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f678000c420 tx=0x7f678000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:45.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.052+0000 7f6785ffb640 1 -- 172.21.15.73:0/784372921 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67800074e0 con 0x7f6790071260 2024-04-26T20:51:45.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.052+0000 7f6785ffb640 1 -- 172.21.15.73:0/784372921 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67800047e0 con 0x7f6790071260 2024-04-26T20:51:45.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.052+0000 7f679736a640 1 -- 172.21.15.73:0/784372921 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67901f4bc0 con 0x7f6790071260 2024-04-26T20:51:45.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.052+0000 7f6785ffb640 1 -- 172.21.15.73:0/784372921 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6780004c10 con 0x7f6790071260 2024-04-26T20:51:45.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.052+0000 7f679736a640 1 -- 172.21.15.73:0/784372921 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67901f5460 con 0x7f6790071260 2024-04-26T20:51:45.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.053+0000 7f679736a640 1 -- 172.21.15.73:0/784372921 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67900694f0 con 0x7f6790071260 2024-04-26T20:51:45.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.053+0000 7f6785ffb640 1 -- 172.21.15.73:0/784372921 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6780011070 con 0x7f6790071260 2024-04-26T20:51:45.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.053+0000 7f6785ffb640 1 --2- 172.21.15.73:0/784372921 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f677406c8f0 0x7f677406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:45.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.054+0000 7f6785ffb640 1 -- 172.21.15.73:0/784372921 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6001+0+0 (secure 0 0 0) 0x7f678008b8e0 con 0x7f6790071260 2024-04-26T20:51:45.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.054+0000 7f6795106640 1 --2- 172.21.15.73:0/784372921 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f677406c8f0 0x7f677406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:45.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.054+0000 7f6795106640 1 --2- 172.21.15.73:0/784372921 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f677406c8f0 0x7f677406edb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f678c0012e0 tx=0x7f678c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:45.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.058+0000 7f6785ffb640 1 -- 172.21.15.73:0/784372921 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f678000e280 con 0x7f6790071260 2024-04-26T20:51:45.273 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.272+0000 7f679736a640 1 -- 172.21.15.73:0/784372921 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f67900b8670 con 0x7f6790071260 2024-04-26T20:51:45.929 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.927+0000 7f6785ffb640 1 -- 172.21.15.73:0/784372921 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v13) ==== 172+0+0 (secure 0 0 0) 0x7f6780059be0 con 0x7f6790071260 2024-04-26T20:51:45.929 INFO:teuthology.orchestra.run.smithi073.stderr:new fs with metadata pool 5 and data pool 6 2024-04-26T20:51:45.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.934+0000 7f679736a640 1 -- 172.21.15.73:0/784372921 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f677406c8f0 msgr2=0x7f677406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:45.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.934+0000 7f679736a640 1 --2- 172.21.15.73:0/784372921 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f677406c8f0 0x7f677406edb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f678c0012e0 tx=0x7f678c009040 comp rx=0 tx=0).stop 2024-04-26T20:51:45.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.935+0000 7f679736a640 1 -- 172.21.15.73:0/784372921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6790071260 msgr2=0x7f67901f4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:45.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.935+0000 7f679736a640 1 --2- 172.21.15.73:0/784372921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6790071260 0x7f67901f4580 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f678000c420 tx=0x7f678000c8e0 comp rx=0 tx=0).stop 2024-04-26T20:51:45.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.940+0000 7f679736a640 1 -- 172.21.15.73:0/784372921 shutdown_connections 2024-04-26T20:51:45.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.940+0000 7f679736a640 1 --2- 172.21.15.73:0/784372921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6790071260 0x7f67901f4580 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:45.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.940+0000 7f679736a640 1 --2- 172.21.15.73:0/784372921 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f677406c8f0 0x7f677406edb0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:45.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.940+0000 7f679736a640 1 --2- 172.21.15.73:0/784372921 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f67900cfde0 0x7f679009d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:45.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.940+0000 7f679736a640 1 --2- 172.21.15.73:0/784372921 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6790070350 0x7f67900b56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:45.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.940+0000 7f679736a640 1 -- 172.21.15.73:0/784372921 >> 172.21.15.73:0/784372921 conn(0x7f67900d2d80 msgr2=0x7f67900b4060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:45.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.940+0000 7f679736a640 1 -- 172.21.15.73:0/784372921 shutdown_connections 2024-04-26T20:51:45.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:45.941+0000 7f679736a640 1 -- 172.21.15.73:0/784372921 wait complete. 2024-04-26T20:51:45.959 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:51:46.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.055+0000 7fc38ce69640 1 Processor -- start 2024-04-26T20:51:46.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.055+0000 7fc38ce69640 1 -- start start 2024-04-26T20:51:46.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.055+0000 7fc38ce69640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc38815f5a0 0x7fc3880d5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:46.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.055+0000 7fc38ce69640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fc3880d2830 0x7fc388162f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:46.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.055+0000 7fc38ce69640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3880d1a70 con 0x7fc3880d3c20 2024-04-26T20:51:46.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.055+0000 7fc38ce69640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fc388163d30 con 0x7fc3880d2830 2024-04-26T20:51:46.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.055+0000 7fc38ce69640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fc3880d1d50 con 0x7fc38815f5a0 2024-04-26T20:51:46.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.055+0000 7fc386d9d640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fc3880d3c20 0x7fc38815f990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:41610/0 (socket says 172.21.15.73:41610) 2024-04-26T20:51:46.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.056+0000 7fc386d9d640 1 -- 172.21.15.73:0/3177092690 learned_addr learned my addr 172.21.15.73:0/3177092690 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:46.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.056+0000 7fc385d9b640 1 -- 172.21.15.73:0/3177092690 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 224142276 0 0) 0x7fc3880d1a70 con 0x7fc3880d3c20 2024-04-26T20:51:46.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.056+0000 7fc385d9b640 1 -- 172.21.15.73:0/3177092690 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc374003520 con 0x7fc3880d3c20 2024-04-26T20:51:46.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.056+0000 7fc37ffff640 1 --2- 172.21.15.73:0/3177092690 >> v2:172.21.15.120:3300/0 conn(0x7fc3880d2830 0x7fc388162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:46.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.056+0000 7fc38659c640 1 --2- 172.21.15.73:0/3177092690 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc38815f5a0 0x7fc3880d5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:46.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.056+0000 7fc385d9b640 1 -- 172.21.15.73:0/3177092690 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3439508744 0 0) 0x7fc374003520 con 0x7fc3880d3c20 2024-04-26T20:51:46.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.056+0000 7fc385d9b640 1 -- 172.21.15.73:0/3177092690 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc38815f5a0 msgr2=0x7fc3880d5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:46.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.056+0000 7fc385d9b640 1 --2- 172.21.15.73:0/3177092690 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc38815f5a0 0x7fc3880d5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:46.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.056+0000 7fc385d9b640 1 -- 172.21.15.73:0/3177092690 >> v2:172.21.15.120:3300/0 conn(0x7fc3880d2830 msgr2=0x7fc388162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:46.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.056+0000 7fc385d9b640 1 --2- 172.21.15.73:0/3177092690 >> v2:172.21.15.120:3300/0 conn(0x7fc3880d2830 0x7fc388162f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:46.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.056+0000 7fc385d9b640 1 -- 172.21.15.73:0/3177092690 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3880d2030 con 0x7fc3880d3c20 2024-04-26T20:51:46.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.056+0000 7fc385d9b640 1 -- 172.21.15.73:0/3177092690 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7fc3800019f0 con 0x7fc3880d3c20 2024-04-26T20:51:46.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.056+0000 7fc385d9b640 1 -- 172.21.15.73:0/3177092690 >> v1:172.21.15.73:6789/0 conn(0x7fc3880d3c20 legacy=0x7fc38815f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:46.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.056+0000 7fc385d9b640 1 --2- 172.21.15.73:0/3177092690 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc3740039b0 0x7fc374003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:46.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.056+0000 7fc385d9b640 1 --2- 172.21.15.73:0/3177092690 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc374004310 0x7fc374004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:46.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.056+0000 7fc385d9b640 1 --2- 172.21.15.73:0/3177092690 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc374004f00 0x7fc37400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:46.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.056+0000 7fc385d9b640 1 -- 172.21.15.73:0/3177092690 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc3880d1d50 con 0x7fc374004f00 2024-04-26T20:51:46.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.056+0000 7fc385d9b640 1 -- 172.21.15.73:0/3177092690 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc37400bf00 con 0x7fc374004310 2024-04-26T20:51:46.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.056+0000 7fc385d9b640 1 -- 172.21.15.73:0/3177092690 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fc37400c080 con 0x7fc3740039b0 2024-04-26T20:51:46.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.056+0000 7fc386d9d640 1 --2- 172.21.15.73:0/3177092690 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc374004f00 0x7fc37400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:46.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.056+0000 7fc38659c640 1 --2- 172.21.15.73:0/3177092690 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc3740039b0 0x7fc374003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:46.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.056+0000 7fc37ffff640 1 --2- 172.21.15.73:0/3177092690 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc374004310 0x7fc374004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:46.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.056+0000 7fc386d9d640 1 -- 172.21.15.73:0/3177092690 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc3740039b0 msgr2=0x7fc374003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:46.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.056+0000 7fc386d9d640 1 --2- 172.21.15.73:0/3177092690 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc3740039b0 0x7fc374003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:46.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.057+0000 7fc386d9d640 1 -- 172.21.15.73:0/3177092690 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc374004310 msgr2=0x7fc374004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:46.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.057+0000 7fc386d9d640 1 --2- 172.21.15.73:0/3177092690 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc374004310 0x7fc374004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:46.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.057+0000 7fc386d9d640 1 -- 172.21.15.73:0/3177092690 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc374003520 con 0x7fc374004f00 2024-04-26T20:51:46.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.057+0000 7fc386d9d640 1 --2- 172.21.15.73:0/3177092690 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc374004f00 0x7fc37400b9c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc38000a870 tx=0x7fc374005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6ad14eb3fc00bae server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:46.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.057+0000 7fc385d9b640 1 -- 172.21.15.73:0/3177092690 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc38000a4c0 con 0x7fc374004f00 2024-04-26T20:51:46.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.057+0000 7fc385d9b640 1 -- 172.21.15.73:0/3177092690 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc38002f070 con 0x7fc374004f00 2024-04-26T20:51:46.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.057+0000 7fc385d9b640 1 -- 172.21.15.73:0/3177092690 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc38002bcc0 con 0x7fc374004f00 2024-04-26T20:51:46.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.058+0000 7fc38ce69640 1 -- 172.21.15.73:0/3177092690 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc374004f00 msgr2=0x7fc37400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:46.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.058+0000 7fc38ce69640 1 --2- 172.21.15.73:0/3177092690 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc374004f00 0x7fc37400b9c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc38000a870 tx=0x7fc374005830 comp rx=0 tx=0).stop 2024-04-26T20:51:46.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.058+0000 7fc386d9d640 1 -- 172.21.15.73:0/3177092690 reap_dead start 2024-04-26T20:51:46.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.058+0000 7fc38ce69640 1 -- 172.21.15.73:0/3177092690 shutdown_connections 2024-04-26T20:51:46.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.058+0000 7fc38ce69640 1 -- 172.21.15.73:0/3177092690 >> 172.21.15.73:0/3177092690 conn(0x7fc3880d5190 msgr2=0x7fc3880d55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:46.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.058+0000 7fc38ce69640 1 -- 172.21.15.73:0/3177092690 shutdown_connections 2024-04-26T20:51:46.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.058+0000 7fc38ce69640 1 -- 172.21.15.73:0/3177092690 wait complete. 2024-04-26T20:51:46.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.059+0000 7fc38ce69640 1 Processor -- start 2024-04-26T20:51:46.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.059+0000 7fc38ce69640 1 -- start start 2024-04-26T20:51:46.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.060+0000 7fc38ce69640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc374004f00 0x7fc388156960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:46.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.060+0000 7fc38ce69640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc388162d60 0x7fc3881631e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:46.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.060+0000 7fc38ce69640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc38815f5a0 0x7fc38815fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:46.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.060+0000 7fc38ce69640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc3880d5cb0 con 0x7fc374004f00 2024-04-26T20:51:46.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.060+0000 7fc38ce69640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc3880d4190 con 0x7fc388162d60 2024-04-26T20:51:46.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.060+0000 7fc38ce69640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fc3880d5e30 con 0x7fc38815f5a0 2024-04-26T20:51:46.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.060+0000 7fc38659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc374004f00 0x7fc388156960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:46.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.060+0000 7fc38659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc374004f00 0x7fc388156960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54528/0 (socket says 172.21.15.73:54528) 2024-04-26T20:51:46.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.060+0000 7fc38659c640 1 -- 172.21.15.73:0/723390583 learned_addr learned my addr 172.21.15.73:0/723390583 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:46.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.060+0000 7fc37ffff640 1 --2- 172.21.15.73:0/723390583 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc388162d60 0x7fc3881631e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:46.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.060+0000 7fc386d9d640 1 --2- 172.21.15.73:0/723390583 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc38815f5a0 0x7fc38815fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:46.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.060+0000 7fc38659c640 1 -- 172.21.15.73:0/723390583 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc38815f5a0 msgr2=0x7fc38815fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:46.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.060+0000 7fc38659c640 1 --2- 172.21.15.73:0/723390583 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc38815f5a0 0x7fc38815fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:46.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.060+0000 7fc38659c640 1 -- 172.21.15.73:0/723390583 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc388162d60 msgr2=0x7fc3881631e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:46.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.060+0000 7fc38659c640 1 --2- 172.21.15.73:0/723390583 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc388162d60 0x7fc3881631e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:46.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.060+0000 7fc38659c640 1 -- 172.21.15.73:0/723390583 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc38815ff90 con 0x7fc374004f00 2024-04-26T20:51:46.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.060+0000 7fc37ffff640 1 --2- 172.21.15.73:0/723390583 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc388162d60 0x7fc3881631e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:46.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.060+0000 7fc38659c640 1 --2- 172.21.15.73:0/723390583 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc374004f00 0x7fc388156960 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc37800b720 tx=0x7fc37800bbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:46.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.061+0000 7fc37effd640 1 -- 172.21.15.73:0/723390583 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc378018070 con 0x7fc374004f00 2024-04-26T20:51:46.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.061+0000 7fc37effd640 1 -- 172.21.15.73:0/723390583 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3780107d0 con 0x7fc374004f00 2024-04-26T20:51:46.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.061+0000 7fc38ce69640 1 -- 172.21.15.73:0/723390583 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc388160220 con 0x7fc374004f00 2024-04-26T20:51:46.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.061+0000 7fc37effd640 1 -- 172.21.15.73:0/723390583 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc378010ba0 con 0x7fc374004f00 2024-04-26T20:51:46.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.061+0000 7fc38ce69640 1 -- 172.21.15.73:0/723390583 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc388160af0 con 0x7fc374004f00 2024-04-26T20:51:46.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.062+0000 7fc38ce69640 1 -- 172.21.15.73:0/723390583 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc388160740 con 0x7fc374004f00 2024-04-26T20:51:46.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.062+0000 7fc37effd640 1 -- 172.21.15.73:0/723390583 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc37801b070 con 0x7fc374004f00 2024-04-26T20:51:46.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.062+0000 7fc37effd640 1 --2- 172.21.15.73:0/723390583 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc36806c8f0 0x7fc36806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:46.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.062+0000 7fc37effd640 1 -- 172.21.15.73:0/723390583 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fc37808c040 con 0x7fc374004f00 2024-04-26T20:51:46.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.062+0000 7fc37ffff640 1 --2- 172.21.15.73:0/723390583 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc36806c8f0 0x7fc36806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:46.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.063+0000 7fc37ffff640 1 --2- 172.21.15.73:0/723390583 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc36806c8f0 0x7fc36806edb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc3880cf4c0 tx=0x7fc370008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:46.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.067+0000 7fc37effd640 1 -- 172.21.15.73:0/723390583 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc37805a2e0 con 0x7fc374004f00 2024-04-26T20:51:46.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.241+0000 7fc38ce69640 1 -- 172.21.15.73:0/723390583 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc3880694f0 con 0x7fc374004f00 2024-04-26T20:51:46.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.242+0000 7fc37effd640 1 -- 172.21.15.73:0/723390583 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16955 (secure 0 0 0) 0x7fc378059be0 con 0x7fc374004f00 2024-04-26T20:51:46.243 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:51:46.243 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":38,"fsid":"fbd3f548-b547-4713-856b-09f98f5d1764","created":"2024-04-26T20:50:07.876786+0000","modified":"2024-04-26T20:51:45.273989+0000","last_up_change":"2024-04-26T20:50:19.082741+0000","last_in_change":"2024-04-26T20:50:10.018400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:50:19.993731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:51:43.110774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-26T20:51:44.235846+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d6f4f3-214a-4392-a02f-c33cbcab3c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6809","nonce":3461705124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6811","nonce":3461705124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6815","nonce":3461705124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6813","nonce":3461705124}]},"public_addr":"172.21.15.73:6809/3461705124","cluster_addr":"172.21.15.73:6811/3461705124","heartbeat_back_addr":"172.21.15.73:6815/3461705124","heartbeat_front_addr":"172.21.15.73:6813/3461705124","state":["exists","up"]},{"osd":1,"uuid":"786bf8ca-6ddc-4b3c-be6c-9152398bd5fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6817","nonce":2827619403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6819","nonce":2827619403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6825","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6827","nonce":2827619403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6821","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6823","nonce":2827619403}]},"public_addr":"172.21.15.73:6817/2827619403","cluster_addr":"172.21.15.73:6819/2827619403","heartbeat_back_addr":"172.21.15.73:6827/2827619403","heartbeat_front_addr":"172.21.15.73:6823/2827619403","state":["exists","up"]},{"osd":2,"uuid":"90963b1a-5437-4ef8-ad8b-6b74c534eeb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6801","nonce":2542681064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6803","nonce":2542681064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6808","nonce":2542681064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6805","nonce":2542681064}]},"public_addr":"172.21.15.73:6801/2542681064","cluster_addr":"172.21.15.73:6803/2542681064","heartbeat_back_addr":"172.21.15.73:6808/2542681064","heartbeat_front_addr":"172.21.15.73:6805/2542681064","state":["exists","up"]},{"osd":3,"uuid":"bd3d2148-6e38-41b4-a453-fe6e3b8b4ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6822","nonce":1604042718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6826","nonce":1604042718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6831","nonce":1604042718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6829","nonce":1604042718}]},"public_addr":"172.21.15.73:6822/1604042718","cluster_addr":"172.21.15.73:6826/1604042718","heartbeat_back_addr":"172.21.15.73:6831/1604042718","heartbeat_front_addr":"172.21.15.73:6829/1604042718","state":["exists","up"]},{"osd":4,"uuid":"e8668dc8-a23f-46d2-8311-cb882da08e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6817","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6819","nonce":3591055561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6824","nonce":3591055561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6831","nonce":3591055561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6828","nonce":3591055561}]},"public_addr":"172.21.15.120:6819/3591055561","cluster_addr":"172.21.15.120:6824/3591055561","heartbeat_back_addr":"172.21.15.120:6831/3591055561","heartbeat_front_addr":"172.21.15.120:6828/3591055561","state":["exists","up"]},{"osd":5,"uuid":"cbac0bdb-96bd-4852-bcce-e564495c3fcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6809","nonce":819450433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6811","nonce":819450433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6815","nonce":819450433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6813","nonce":819450433}]},"public_addr":"172.21.15.120:6809/819450433","cluster_addr":"172.21.15.120:6811/819450433","heartbeat_back_addr":"172.21.15.120:6815/819450433","heartbeat_front_addr":"172.21.15.120:6813/819450433","state":["exists","up"]},{"osd":6,"uuid":"1884b2a8-569f-4272-96a9-f25a09364dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6801","nonce":3946210727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6803","nonce":3946210727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6807","nonce":3946210727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6805","nonce":3946210727}]},"public_addr":"172.21.15.120:6801/3946210727","cluster_addr":"172.21.15.120:6803/3946210727","heartbeat_back_addr":"172.21.15.120:6807/3946210727","heartbeat_front_addr":"172.21.15.120:6805/3946210727","state":["exists","up"]},{"osd":7,"uuid":"c0c3bf6d-a4c5-4f1e-aa2d-330951830666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6818","nonce":2915574674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6821","nonce":2915574674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6827","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6829","nonce":2915574674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6823","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6825","nonce":2915574674}]},"public_addr":"172.21.15.120:6818/2915574674","cluster_addr":"172.21.15.120:6821/2915574674","heartbeat_back_addr":"172.21.15.120:6829/2915574674","heartbeat_front_addr":"172.21.15.120:6825/2915574674","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.218216+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.208710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.112705+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.182789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.164324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.174212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.225396+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.261559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:51:46.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.244+0000 7fc38ce69640 1 -- 172.21.15.73:0/723390583 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc36806c8f0 msgr2=0x7fc36806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:46.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.244+0000 7fc38ce69640 1 --2- 172.21.15.73:0/723390583 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc36806c8f0 0x7fc36806edb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc3880cf4c0 tx=0x7fc370008040 comp rx=0 tx=0).stop 2024-04-26T20:51:46.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.244+0000 7fc38ce69640 1 -- 172.21.15.73:0/723390583 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc374004f00 msgr2=0x7fc388156960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:46.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.244+0000 7fc38ce69640 1 --2- 172.21.15.73:0/723390583 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc374004f00 0x7fc388156960 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc37800b720 tx=0x7fc37800bbe0 comp rx=0 tx=0).stop 2024-04-26T20:51:46.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.244+0000 7fc38ce69640 1 -- 172.21.15.73:0/723390583 shutdown_connections 2024-04-26T20:51:46.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.244+0000 7fc38ce69640 1 --2- 172.21.15.73:0/723390583 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc36806c8f0 0x7fc36806edb0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:46.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.244+0000 7fc38ce69640 1 --2- 172.21.15.73:0/723390583 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc38815f5a0 0x7fc38815fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:46.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.244+0000 7fc38ce69640 1 --2- 172.21.15.73:0/723390583 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc388162d60 0x7fc3881631e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:46.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.244+0000 7fc38ce69640 1 --2- 172.21.15.73:0/723390583 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc374004f00 0x7fc388156960 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:46.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.244+0000 7fc38ce69640 1 -- 172.21.15.73:0/723390583 >> 172.21.15.73:0/723390583 conn(0x7fc3880d5190 msgr2=0x7fc38806e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:46.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.244+0000 7fc38ce69640 1 -- 172.21.15.73:0/723390583 shutdown_connections 2024-04-26T20:51:46.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.244+0000 7fc38ce69640 1 -- 172.21.15.73:0/723390583 wait complete. 2024-04-26T20:51:46.256 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:51:46.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.351+0000 7f72a39f9640 1 Processor -- start 2024-04-26T20:51:46.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.351+0000 7f72a39f9640 1 -- start start 2024-04-26T20:51:46.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.352+0000 7f72a39f9640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f729c15f3a0 0x7f729c0d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:46.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.352+0000 7f72a39f9640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f729c0d2330 0x7f729c0b2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:46.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.352+0000 7f72a39f9640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f729c0d0e20 con 0x7f729c0d1770 2024-04-26T20:51:46.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.352+0000 7f72a39f9640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f729c0b1d70 con 0x7f729c15f3a0 2024-04-26T20:51:46.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.352+0000 7f72a39f9640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f729c0b2050 con 0x7f729c0d2330 2024-04-26T20:51:46.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.352+0000 7f72a0f94640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f729c0d1770 0x7f729c0d1c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:41622/0 (socket says 172.21.15.73:41622) 2024-04-26T20:51:46.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.352+0000 7f72a0f94640 1 -- 172.21.15.73:0/1517835404 learned_addr learned my addr 172.21.15.73:0/1517835404 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:46.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.353+0000 7f72a1f96640 1 --2- 172.21.15.73:0/1517835404 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f729c0d2330 0x7f729c0b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:46.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.353+0000 7f72a1795640 1 --2- 172.21.15.73:0/1517835404 >> v2:172.21.15.120:3300/0 conn(0x7f729c15f3a0 0x7f729c0d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:46.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.353+0000 7f7293fff640 1 -- 172.21.15.73:0/1517835404 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3102999459 0 0) 0x7f729c0d0e20 con 0x7f729c0d1770 2024-04-26T20:51:46.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.353+0000 7f7293fff640 1 -- 172.21.15.73:0/1517835404 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7288003520 con 0x7f729c0d1770 2024-04-26T20:51:46.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.353+0000 7f7293fff640 1 -- 172.21.15.73:0/1517835404 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2466111340 0 0) 0x7f7288003520 con 0x7f729c0d1770 2024-04-26T20:51:46.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.353+0000 7f7293fff640 1 -- 172.21.15.73:0/1517835404 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f729c0d2330 msgr2=0x7f729c0b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:46.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.353+0000 7f7293fff640 1 --2- 172.21.15.73:0/1517835404 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f729c0d2330 0x7f729c0b2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:46.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.353+0000 7f7293fff640 1 -- 172.21.15.73:0/1517835404 >> v2:172.21.15.120:3300/0 conn(0x7f729c15f3a0 msgr2=0x7f729c0d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:46.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.353+0000 7f7293fff640 1 --2- 172.21.15.73:0/1517835404 >> v2:172.21.15.120:3300/0 conn(0x7f729c15f3a0 0x7f729c0d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:46.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.353+0000 7f7293fff640 1 -- 172.21.15.73:0/1517835404 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f729c0b2330 con 0x7f729c0d1770 2024-04-26T20:51:46.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.353+0000 7f7293fff640 1 -- 172.21.15.73:0/1517835404 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7f728c001670 con 0x7f729c0d1770 2024-04-26T20:51:46.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.353+0000 7f7293fff640 1 -- 172.21.15.73:0/1517835404 >> v1:172.21.15.73:6789/0 conn(0x7f729c0d1770 legacy=0x7f729c0d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:46.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.353+0000 7f7293fff640 1 --2- 172.21.15.73:0/1517835404 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f72880039b0 0x7f7288003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:46.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.353+0000 7f7293fff640 1 --2- 172.21.15.73:0/1517835404 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7288004800 0x7f7288004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:46.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.353+0000 7f7293fff640 1 --2- 172.21.15.73:0/1517835404 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f729c0d2330 0x7f72880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:46.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.353+0000 7f7293fff640 1 -- 172.21.15.73:0/1517835404 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f729c0b1d70 con 0x7f7288004800 2024-04-26T20:51:46.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.353+0000 7f7293fff640 1 -- 172.21.15.73:0/1517835404 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f729c0b2050 con 0x7f72880039b0 2024-04-26T20:51:46.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.353+0000 7f7293fff640 1 -- 172.21.15.73:0/1517835404 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f728800b9c0 con 0x7f729c0d2330 2024-04-26T20:51:46.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.353+0000 7f72a0f94640 1 --2- 172.21.15.73:0/1517835404 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7288004800 0x7f7288004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:46.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.353+0000 7f72a1795640 1 --2- 172.21.15.73:0/1517835404 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f72880039b0 0x7f7288003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:46.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.353+0000 7f72a1f96640 1 --2- 172.21.15.73:0/1517835404 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f729c0d2330 0x7f72880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:46.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.353+0000 7f72a0f94640 1 -- 172.21.15.73:0/1517835404 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f729c0d2330 msgr2=0x7f72880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:46.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.353+0000 7f72a0f94640 1 --2- 172.21.15.73:0/1517835404 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f729c0d2330 0x7f72880051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:46.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.353+0000 7f72a0f94640 1 -- 172.21.15.73:0/1517835404 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f72880039b0 msgr2=0x7f7288003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:46.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.353+0000 7f72a0f94640 1 --2- 172.21.15.73:0/1517835404 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f72880039b0 0x7f7288003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:46.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.353+0000 7f72a0f94640 1 -- 172.21.15.73:0/1517835404 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7288003520 con 0x7f7288004800 2024-04-26T20:51:46.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.354+0000 7f72a0f94640 1 --2- 172.21.15.73:0/1517835404 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7288004800 0x7f7288004c80 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f728c001050 tx=0x7f7288005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7eef8ef14799fa8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:46.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.354+0000 7f7293fff640 1 -- 172.21.15.73:0/1517835404 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728c02e070 con 0x7f7288004800 2024-04-26T20:51:46.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.354+0000 7f7293fff640 1 -- 172.21.15.73:0/1517835404 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f728c005af0 con 0x7f7288004800 2024-04-26T20:51:46.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.354+0000 7f7293fff640 1 -- 172.21.15.73:0/1517835404 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728c007d70 con 0x7f7288004800 2024-04-26T20:51:46.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.354+0000 7f72a39f9640 1 -- 172.21.15.73:0/1517835404 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7288004800 msgr2=0x7f7288004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:46.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.354+0000 7f72a39f9640 1 --2- 172.21.15.73:0/1517835404 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7288004800 0x7f7288004c80 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f728c001050 tx=0x7f7288005830 comp rx=0 tx=0).stop 2024-04-26T20:51:46.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.354+0000 7f72a1f96640 1 -- 172.21.15.73:0/1517835404 reap_dead start 2024-04-26T20:51:46.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.355+0000 7f72a39f9640 1 -- 172.21.15.73:0/1517835404 shutdown_connections 2024-04-26T20:51:46.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.355+0000 7f72a39f9640 1 -- 172.21.15.73:0/1517835404 >> 172.21.15.73:0/1517835404 conn(0x7f729c0d4f90 msgr2=0x7f729c0d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:46.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.355+0000 7f72a39f9640 1 -- 172.21.15.73:0/1517835404 shutdown_connections 2024-04-26T20:51:46.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.355+0000 7f72a39f9640 1 -- 172.21.15.73:0/1517835404 wait complete. 2024-04-26T20:51:46.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.356+0000 7f72a39f9640 1 Processor -- start 2024-04-26T20:51:46.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.356+0000 7f72a39f9640 1 -- start start 2024-04-26T20:51:46.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.356+0000 7f72a39f9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7288004800 0x7f729c159cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:46.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.356+0000 7f72a39f9640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f729c0cc060 0x7f729c0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:46.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.356+0000 7f72a39f9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f729c0cebb0 0x7f729c15ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:46.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.356+0000 7f72a39f9640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f729c0b29d0 con 0x7f729c0cebb0 2024-04-26T20:51:46.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.356+0000 7f72a39f9640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f729c0b2850 con 0x7f7288004800 2024-04-26T20:51:46.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.356+0000 7f72a39f9640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f729c0b2b50 con 0x7f729c0cc060 2024-04-26T20:51:46.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.357+0000 7f72a1f96640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f729c0cebb0 0x7f729c15ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:46.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.357+0000 7f72a1f96640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f729c0cebb0 0x7f729c15ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54540/0 (socket says 172.21.15.73:54540) 2024-04-26T20:51:46.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.357+0000 7f72a1f96640 1 -- 172.21.15.73:0/2077127320 learned_addr learned my addr 172.21.15.73:0/2077127320 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:46.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.357+0000 7f72a1f96640 1 -- 172.21.15.73:0/2077127320 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f729c0cc060 msgr2=0x7f729c0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:46.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.357+0000 7f72a1f96640 1 --2- 172.21.15.73:0/2077127320 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f729c0cc060 0x7f729c0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:46.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.357+0000 7f72a1f96640 1 -- 172.21.15.73:0/2077127320 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7288004800 msgr2=0x7f729c159cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:46.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.357+0000 7f72a1f96640 1 --2- 172.21.15.73:0/2077127320 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7288004800 0x7f729c159cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:46.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.357+0000 7f72a1f96640 1 -- 172.21.15.73:0/2077127320 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f729c15a200 con 0x7f729c0cebb0 2024-04-26T20:51:46.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.357+0000 7f72a1f96640 1 --2- 172.21.15.73:0/2077127320 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f729c0cebb0 0x7f729c15ad80 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f729c08e8d0 tx=0x7f72940115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:46.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.357+0000 7f7291ffb640 1 -- 172.21.15.73:0/2077127320 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7294011db0 con 0x7f729c0cebb0 2024-04-26T20:51:46.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.357+0000 7f7291ffb640 1 -- 172.21.15.73:0/2077127320 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f729401b070 con 0x7f729c0cebb0 2024-04-26T20:51:46.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.357+0000 7f72a39f9640 1 -- 172.21.15.73:0/2077127320 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f729c0cd1f0 con 0x7f729c0cebb0 2024-04-26T20:51:46.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.357+0000 7f7291ffb640 1 -- 172.21.15.73:0/2077127320 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7294003990 con 0x7f729c0cebb0 2024-04-26T20:51:46.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.357+0000 7f72a39f9640 1 -- 172.21.15.73:0/2077127320 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f729c0ccf10 con 0x7f729c0cebb0 2024-04-26T20:51:46.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.358+0000 7f72a39f9640 1 -- 172.21.15.73:0/2077127320 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f729c0694f0 con 0x7f729c0cebb0 2024-04-26T20:51:46.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.359+0000 7f7291ffb640 1 -- 172.21.15.73:0/2077127320 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7294005d10 con 0x7f729c0cebb0 2024-04-26T20:51:46.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.359+0000 7f7291ffb640 1 --2- 172.21.15.73:0/2077127320 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f728006c8f0 0x7f728006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:46.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.359+0000 7f7291ffb640 1 -- 172.21.15.73:0/2077127320 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f729408bf80 con 0x7f729c0cebb0 2024-04-26T20:51:46.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.359+0000 7f72a1795640 1 --2- 172.21.15.73:0/2077127320 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f728006c8f0 0x7f728006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:46.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.360+0000 7f72a1795640 1 --2- 172.21.15.73:0/2077127320 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f728006c8f0 0x7f728006edb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7298006b10 tx=0x7f7298006aa0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:46.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.364+0000 7f7291ffb640 1 -- 172.21.15.73:0/2077127320 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f729c15a200 con 0x7f729c0cebb0 2024-04-26T20:51:46.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.541+0000 7f72a39f9640 1 -- 172.21.15.73:0/2077127320 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f729c0b6860 con 0x7f729c0cebb0 2024-04-26T20:51:46.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.542+0000 7f7291ffb640 1 -- 172.21.15.73:0/2077127320 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16955 (secure 0 0 0) 0x7f729405a2b0 con 0x7f729c0cebb0 2024-04-26T20:51:46.543 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:51:46.543 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":38,"fsid":"fbd3f548-b547-4713-856b-09f98f5d1764","created":"2024-04-26T20:50:07.876786+0000","modified":"2024-04-26T20:51:45.273989+0000","last_up_change":"2024-04-26T20:50:19.082741+0000","last_in_change":"2024-04-26T20:50:10.018400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:50:19.993731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:51:43.110774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-26T20:51:44.235846+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d6f4f3-214a-4392-a02f-c33cbcab3c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6809","nonce":3461705124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6811","nonce":3461705124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6815","nonce":3461705124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6813","nonce":3461705124}]},"public_addr":"172.21.15.73:6809/3461705124","cluster_addr":"172.21.15.73:6811/3461705124","heartbeat_back_addr":"172.21.15.73:6815/3461705124","heartbeat_front_addr":"172.21.15.73:6813/3461705124","state":["exists","up"]},{"osd":1,"uuid":"786bf8ca-6ddc-4b3c-be6c-9152398bd5fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6817","nonce":2827619403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6819","nonce":2827619403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6825","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6827","nonce":2827619403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6821","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6823","nonce":2827619403}]},"public_addr":"172.21.15.73:6817/2827619403","cluster_addr":"172.21.15.73:6819/2827619403","heartbeat_back_addr":"172.21.15.73:6827/2827619403","heartbeat_front_addr":"172.21.15.73:6823/2827619403","state":["exists","up"]},{"osd":2,"uuid":"90963b1a-5437-4ef8-ad8b-6b74c534eeb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6801","nonce":2542681064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6803","nonce":2542681064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6808","nonce":2542681064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6805","nonce":2542681064}]},"public_addr":"172.21.15.73:6801/2542681064","cluster_addr":"172.21.15.73:6803/2542681064","heartbeat_back_addr":"172.21.15.73:6808/2542681064","heartbeat_front_addr":"172.21.15.73:6805/2542681064","state":["exists","up"]},{"osd":3,"uuid":"bd3d2148-6e38-41b4-a453-fe6e3b8b4ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6822","nonce":1604042718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6826","nonce":1604042718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6831","nonce":1604042718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6829","nonce":1604042718}]},"public_addr":"172.21.15.73:6822/1604042718","cluster_addr":"172.21.15.73:6826/1604042718","heartbeat_back_addr":"172.21.15.73:6831/1604042718","heartbeat_front_addr":"172.21.15.73:6829/1604042718","state":["exists","up"]},{"osd":4,"uuid":"e8668dc8-a23f-46d2-8311-cb882da08e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6817","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6819","nonce":3591055561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6824","nonce":3591055561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6831","nonce":3591055561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6828","nonce":3591055561}]},"public_addr":"172.21.15.120:6819/3591055561","cluster_addr":"172.21.15.120:6824/3591055561","heartbeat_back_addr":"172.21.15.120:6831/3591055561","heartbeat_front_addr":"172.21.15.120:6828/3591055561","state":["exists","up"]},{"osd":5,"uuid":"cbac0bdb-96bd-4852-bcce-e564495c3fcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6809","nonce":819450433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6811","nonce":819450433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6815","nonce":819450433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6813","nonce":819450433}]},"public_addr":"172.21.15.120:6809/819450433","cluster_addr":"172.21.15.120:6811/819450433","heartbeat_back_addr":"172.21.15.120:6815/819450433","heartbeat_front_addr":"172.21.15.120:6813/819450433","state":["exists","up"]},{"osd":6,"uuid":"1884b2a8-569f-4272-96a9-f25a09364dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6801","nonce":3946210727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6803","nonce":3946210727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6807","nonce":3946210727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6805","nonce":3946210727}]},"public_addr":"172.21.15.120:6801/3946210727","cluster_addr":"172.21.15.120:6803/3946210727","heartbeat_back_addr":"172.21.15.120:6807/3946210727","heartbeat_front_addr":"172.21.15.120:6805/3946210727","state":["exists","up"]},{"osd":7,"uuid":"c0c3bf6d-a4c5-4f1e-aa2d-330951830666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6818","nonce":2915574674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6821","nonce":2915574674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6827","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6829","nonce":2915574674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6823","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6825","nonce":2915574674}]},"public_addr":"172.21.15.120:6818/2915574674","cluster_addr":"172.21.15.120:6821/2915574674","heartbeat_back_addr":"172.21.15.120:6829/2915574674","heartbeat_front_addr":"172.21.15.120:6825/2915574674","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.218216+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.208710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.112705+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.182789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.164324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.174212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.225396+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.261559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:51:46.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.544+0000 7f72a39f9640 1 -- 172.21.15.73:0/2077127320 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f728006c8f0 msgr2=0x7f728006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:46.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.544+0000 7f72a39f9640 1 --2- 172.21.15.73:0/2077127320 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f728006c8f0 0x7f728006edb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7298006b10 tx=0x7f7298006aa0 comp rx=0 tx=0).stop 2024-04-26T20:51:46.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.544+0000 7f72a39f9640 1 -- 172.21.15.73:0/2077127320 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f729c0cebb0 msgr2=0x7f729c15ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:46.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.544+0000 7f72a39f9640 1 --2- 172.21.15.73:0/2077127320 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f729c0cebb0 0x7f729c15ad80 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f729c08e8d0 tx=0x7f72940115a0 comp rx=0 tx=0).stop 2024-04-26T20:51:46.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.544+0000 7f72a39f9640 1 -- 172.21.15.73:0/2077127320 shutdown_connections 2024-04-26T20:51:46.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.544+0000 7f72a39f9640 1 --2- 172.21.15.73:0/2077127320 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f729c0cebb0 0x7f729c15ad80 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:46.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.544+0000 7f72a39f9640 1 --2- 172.21.15.73:0/2077127320 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f728006c8f0 0x7f728006edb0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:46.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.544+0000 7f72a39f9640 1 --2- 172.21.15.73:0/2077127320 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f729c0cc060 0x7f729c0ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:46.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.544+0000 7f72a39f9640 1 --2- 172.21.15.73:0/2077127320 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7288004800 0x7f729c159cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:46.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.544+0000 7f72a39f9640 1 -- 172.21.15.73:0/2077127320 >> 172.21.15.73:0/2077127320 conn(0x7f729c0d4f90 msgr2=0x7f729c0d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:46.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.544+0000 7f72a39f9640 1 -- 172.21.15.73:0/2077127320 shutdown_connections 2024-04-26T20:51:46.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.544+0000 7f72a39f9640 1 -- 172.21.15.73:0/2077127320 wait complete. 2024-04-26T20:51:46.556 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T20:51:46.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.654+0000 7f12958ea640 1 Processor -- start 2024-04-26T20:51:46.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.654+0000 7f12958ea640 1 -- start start 2024-04-26T20:51:46.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.655+0000 7f12958ea640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f12900b0380 0x7f12900ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:46.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.655+0000 7f12958ea640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f129006c950 0x7f129015e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:46.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.655+0000 7f12958ea640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12900cfd80 con 0x7f12900b10b0 2024-04-26T20:51:46.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.655+0000 7f12958ea640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f12900af8e0 con 0x7f12900b0380 2024-04-26T20:51:46.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.655+0000 7f12958ea640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f12900d0020 con 0x7f129006c950 2024-04-26T20:51:46.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.655+0000 7f128f7fe640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f129006c950 0x7f129015e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:46.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.655+0000 7f128effd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f12900b0380 0x7f12900ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:46.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.655+0000 7f128effd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f12900b0380 0x7f12900ccd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.73:38806/0 (socket says 172.21.15.73:38806) 2024-04-26T20:51:46.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.655+0000 7f128f7fe640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f129006c950 0x7f129015e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.73:56652/0 (socket says 172.21.15.73:56652) 2024-04-26T20:51:46.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.655+0000 7f128effd640 1 -- 172.21.15.73:0/1470734388 learned_addr learned my addr 172.21.15.73:0/1470734388 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:46.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.656+0000 7f128effd640 1 -- 172.21.15.73:0/1470734388 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f129006c950 msgr2=0x7f129015e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:46.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.656+0000 7f128effd640 1 --2- 172.21.15.73:0/1470734388 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f129006c950 0x7f129015e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:46.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.656+0000 7f128effd640 1 -- 172.21.15.73:0/1470734388 >> v1:172.21.15.120:6789/0 conn(0x7f12900b10b0 legacy=0x7f129006c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:46.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.656+0000 7f128effd640 1 -- 172.21.15.73:0/1470734388 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12900d0300 con 0x7f12900b0380 2024-04-26T20:51:46.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.656+0000 7f128effd640 1 --2- 172.21.15.73:0/1470734388 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f12900b0380 0x7f12900ccd50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f12900afa60 tx=0x7f128002a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2973ec964dbb511 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:46.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.656+0000 7f128dffb640 1 -- 172.21.15.73:0/1470734388 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1280036070 con 0x7f12900b0380 2024-04-26T20:51:46.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.657+0000 7f128dffb640 1 -- 172.21.15.73:0/1470734388 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1280031600 con 0x7f12900b0380 2024-04-26T20:51:46.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.657+0000 7f128dffb640 1 -- 172.21.15.73:0/1470734388 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12800319d0 con 0x7f12900b0380 2024-04-26T20:51:46.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.657+0000 7f12958ea640 1 -- 172.21.15.73:0/1470734388 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f12900b0380 msgr2=0x7f12900ccd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:46.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.657+0000 7f12958ea640 1 --2- 172.21.15.73:0/1470734388 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f12900b0380 0x7f12900ccd50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f12900afa60 tx=0x7f128002a680 comp rx=0 tx=0).stop 2024-04-26T20:51:46.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.657+0000 7f12958ea640 1 -- 172.21.15.73:0/1470734388 shutdown_connections 2024-04-26T20:51:46.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.657+0000 7f12958ea640 1 --2- 172.21.15.73:0/1470734388 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f129006c950 0x7f129015e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:46.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.657+0000 7f12958ea640 1 --2- 172.21.15.73:0/1470734388 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f12900b0380 0x7f12900ccd50 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:46.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.657+0000 7f12958ea640 1 -- 172.21.15.73:0/1470734388 >> 172.21.15.73:0/1470734388 conn(0x7f12900d1cf0 msgr2=0x7f12900d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:46.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.657+0000 7f12958ea640 1 -- 172.21.15.73:0/1470734388 shutdown_connections 2024-04-26T20:51:46.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.657+0000 7f12958ea640 1 -- 172.21.15.73:0/1470734388 wait complete. 2024-04-26T20:51:46.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.658+0000 7f12958ea640 1 Processor -- start 2024-04-26T20:51:46.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.658+0000 7f12958ea640 1 -- start start 2024-04-26T20:51:46.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.659+0000 7f12958ea640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f129006c950 0x7f12900cb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:46.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.659+0000 7f12958ea640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12900b0380 0x7f12900b6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:46.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.659+0000 7f12958ea640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f12900b10b0 0x7f1290158780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:46.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.659+0000 7f12958ea640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f12900b0760 con 0x7f12900b0380 2024-04-26T20:51:46.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.659+0000 7f12958ea640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f12900d1980 con 0x7f12900b10b0 2024-04-26T20:51:46.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.659+0000 7f12958ea640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f12900d15c0 con 0x7f129006c950 2024-04-26T20:51:46.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.659+0000 7f128e7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12900b0380 0x7f12900b6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:46.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.659+0000 7f128e7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12900b0380 0x7f12900b6d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54544/0 (socket says 172.21.15.73:54544) 2024-04-26T20:51:46.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.659+0000 7f128e7fc640 1 -- 172.21.15.73:0/3429197170 learned_addr learned my addr 172.21.15.73:0/3429197170 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:46.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.659+0000 7f128effd640 1 --2- 172.21.15.73:0/3429197170 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f129006c950 0x7f12900cb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:46.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.659+0000 7f128f7fe640 1 --2- 172.21.15.73:0/3429197170 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f12900b10b0 0x7f1290158780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:46.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.659+0000 7f128e7fc640 1 -- 172.21.15.73:0/3429197170 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f129006c950 msgr2=0x7f12900cb8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:46.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.659+0000 7f128e7fc640 1 --2- 172.21.15.73:0/3429197170 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f129006c950 0x7f12900cb8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:46.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.659+0000 7f128e7fc640 1 -- 172.21.15.73:0/3429197170 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f12900b10b0 msgr2=0x7f1290158780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:46.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.659+0000 7f128e7fc640 1 --2- 172.21.15.73:0/3429197170 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f12900b10b0 0x7f1290158780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:46.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.659+0000 7f128e7fc640 1 -- 172.21.15.73:0/3429197170 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12900c9b40 con 0x7f12900b0380 2024-04-26T20:51:46.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.660+0000 7f128e7fc640 1 --2- 172.21.15.73:0/3429197170 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12900b0380 0x7f12900b6d30 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f128400c340 tx=0x7f128400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:46.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.660+0000 7f1277fff640 1 -- 172.21.15.73:0/3429197170 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1284017070 con 0x7f12900b0380 2024-04-26T20:51:46.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.660+0000 7f1277fff640 1 -- 172.21.15.73:0/3429197170 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12840052d0 con 0x7f12900b0380 2024-04-26T20:51:46.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.660+0000 7f12958ea640 1 -- 172.21.15.73:0/3429197170 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12900c9e20 con 0x7f12900b0380 2024-04-26T20:51:46.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.660+0000 7f1277fff640 1 -- 172.21.15.73:0/3429197170 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1284003850 con 0x7f12900b0380 2024-04-26T20:51:46.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.660+0000 7f12958ea640 1 -- 172.21.15.73:0/3429197170 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12900ca3e0 con 0x7f12900b0380 2024-04-26T20:51:46.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.661+0000 7f12958ea640 1 -- 172.21.15.73:0/3429197170 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12900694f0 con 0x7f12900b0380 2024-04-26T20:51:46.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.661+0000 7f1277fff640 1 -- 172.21.15.73:0/3429197170 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f128401a070 con 0x7f12900b0380 2024-04-26T20:51:46.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.661+0000 7f1277fff640 1 --2- 172.21.15.73:0/3429197170 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f127c06c920 0x7f127c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:46.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.662+0000 7f1277fff640 1 -- 172.21.15.73:0/3429197170 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f128408af80 con 0x7f12900b0380 2024-04-26T20:51:46.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.662+0000 7f128effd640 1 --2- 172.21.15.73:0/3429197170 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f127c06c920 0x7f127c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:46.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.662+0000 7f128effd640 1 --2- 172.21.15.73:0/3429197170 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f127c06c920 0x7f127c06ede0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1280033000 tx=0x7f1280009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:46.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.666+0000 7f1277fff640 1 -- 172.21.15.73:0/3429197170 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1284058b50 con 0x7f12900b0380 2024-04-26T20:51:46.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.877+0000 7f12958ea640 1 -- 172.21.15.73:0/3429197170 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f129006c490 con 0x7f12900b0380 2024-04-26T20:51:46.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.932+0000 7f1277fff640 1 -- 172.21.15.73:0/3429197170 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v14) ==== 118+0+0 (secure 0 0 0) 0x7f128405ce50 con 0x7f12900b0380 2024-04-26T20:51:46.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.934+0000 7f12958ea640 1 -- 172.21.15.73:0/3429197170 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f127c06c920 msgr2=0x7f127c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:46.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.934+0000 7f12958ea640 1 --2- 172.21.15.73:0/3429197170 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f127c06c920 0x7f127c06ede0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1280033000 tx=0x7f1280009000 comp rx=0 tx=0).stop 2024-04-26T20:51:46.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.934+0000 7f12958ea640 1 -- 172.21.15.73:0/3429197170 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12900b0380 msgr2=0x7f12900b6d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:46.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.934+0000 7f12958ea640 1 --2- 172.21.15.73:0/3429197170 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12900b0380 0x7f12900b6d30 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f128400c340 tx=0x7f128400c800 comp rx=0 tx=0).stop 2024-04-26T20:51:46.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.934+0000 7f12958ea640 1 -- 172.21.15.73:0/3429197170 shutdown_connections 2024-04-26T20:51:46.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.934+0000 7f12958ea640 1 --2- 172.21.15.73:0/3429197170 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12900b0380 0x7f12900b6d30 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:46.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.934+0000 7f12958ea640 1 --2- 172.21.15.73:0/3429197170 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f127c06c920 0x7f127c06ede0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:46.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.934+0000 7f12958ea640 1 --2- 172.21.15.73:0/3429197170 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f12900b10b0 0x7f1290158780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:46.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.934+0000 7f12958ea640 1 --2- 172.21.15.73:0/3429197170 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f129006c950 0x7f12900cb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:46.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.934+0000 7f12958ea640 1 -- 172.21.15.73:0/3429197170 >> 172.21.15.73:0/3429197170 conn(0x7f12900d1cf0 msgr2=0x7f12900b2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:46.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.935+0000 7f12958ea640 1 -- 172.21.15.73:0/3429197170 shutdown_connections 2024-04-26T20:51:46.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:46.935+0000 7f12958ea640 1 -- 172.21.15.73:0/3429197170 wait complete. 2024-04-26T20:51:46.947 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-26T20:51:46.947 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T20:51:47.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.046+0000 7f4c18cbd640 1 Processor -- start 2024-04-26T20:51:47.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.046+0000 7f4c18cbd640 1 -- start start 2024-04-26T20:51:47.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.047+0000 7f4c18cbd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f4c1415a9f0 0x7f4c140b4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:47.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.047+0000 7f4c18cbd640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4c140726e0 0x7f4c14072b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:47.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.047+0000 7f4c18cbd640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c140ce060 con 0x7f4c14161510 2024-04-26T20:51:47.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.047+0000 7f4c18cbd640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f4c140ce620 con 0x7f4c1415a9f0 2024-04-26T20:51:47.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.047+0000 7f4c18cbd640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f4c140ce340 con 0x7f4c140726e0 2024-04-26T20:51:47.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.047+0000 7f4c1259c640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f4c1415a9f0 0x7f4c140b4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:47.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.047+0000 7f4c12d9d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4c140726e0 0x7f4c14072b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:47.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.047+0000 7f4c11d9b640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f4c14161510 0x7f4c1415dbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.73:58060/0 (socket says 172.21.15.73:58060) 2024-04-26T20:51:47.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.047+0000 7f4c11d9b640 1 -- 172.21.15.73:0/3182570477 learned_addr learned my addr 172.21.15.73:0/3182570477 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:47.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.047+0000 7f4c1259c640 1 -- 172.21.15.73:0/3182570477 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4c140726e0 msgr2=0x7f4c14072b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:47.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.047+0000 7f4c1259c640 1 --2- 172.21.15.73:0/3182570477 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4c140726e0 0x7f4c14072b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:47.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.047+0000 7f4c1259c640 1 -- 172.21.15.73:0/3182570477 >> v1:172.21.15.120:6789/0 conn(0x7f4c14161510 legacy=0x7f4c1415dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:47.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.047+0000 7f4c1259c640 1 -- 172.21.15.73:0/3182570477 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c1415d6b0 con 0x7f4c1415a9f0 2024-04-26T20:51:47.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.048+0000 7f4c1259c640 1 --2- 172.21.15.73:0/3182570477 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c1415a9f0 0x7f4c140b4480 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4c040097a0 tx=0x7f4c0402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b7a6f70d86cd091 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:47.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.048+0000 7f4c1159a640 1 -- 172.21.15.73:0/3182570477 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c04037070 con 0x7f4c1415a9f0 2024-04-26T20:51:47.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.048+0000 7f4c1159a640 1 -- 172.21.15.73:0/3182570477 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c04032440 con 0x7f4c1415a9f0 2024-04-26T20:51:47.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.048+0000 7f4c1159a640 1 -- 172.21.15.73:0/3182570477 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c04032810 con 0x7f4c1415a9f0 2024-04-26T20:51:47.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.049+0000 7f4c18cbd640 1 -- 172.21.15.73:0/3182570477 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c1415a9f0 msgr2=0x7f4c140b4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:47.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.049+0000 7f4c18cbd640 1 --2- 172.21.15.73:0/3182570477 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c1415a9f0 0x7f4c140b4480 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4c040097a0 tx=0x7f4c0402a180 comp rx=0 tx=0).stop 2024-04-26T20:51:47.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.049+0000 7f4c18cbd640 1 -- 172.21.15.73:0/3182570477 shutdown_connections 2024-04-26T20:51:47.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.049+0000 7f4c18cbd640 1 --2- 172.21.15.73:0/3182570477 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4c140726e0 0x7f4c14072b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:47.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.049+0000 7f4c18cbd640 1 --2- 172.21.15.73:0/3182570477 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c1415a9f0 0x7f4c140b4480 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:47.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.049+0000 7f4c18cbd640 1 -- 172.21.15.73:0/3182570477 >> 172.21.15.73:0/3182570477 conn(0x7f4c140b3180 msgr2=0x7f4c140b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:47.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.049+0000 7f4c18cbd640 1 -- 172.21.15.73:0/3182570477 shutdown_connections 2024-04-26T20:51:47.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.049+0000 7f4c18cbd640 1 -- 172.21.15.73:0/3182570477 wait complete. 2024-04-26T20:51:47.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.050+0000 7f4c18cbd640 1 Processor -- start 2024-04-26T20:51:47.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.050+0000 7f4c18cbd640 1 -- start start 2024-04-26T20:51:47.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.051+0000 7f4c18cbd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c140726e0 0x7f4c14156000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:47.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.051+0000 7f4c18cbd640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4c1415a9f0 0x7f4c14143080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:47.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.051+0000 7f4c18cbd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c14161510 0x7f4c141e6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:47.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.051+0000 7f4c18cbd640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4c1415e2e0 con 0x7f4c14161510 2024-04-26T20:51:47.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.051+0000 7f4c18cbd640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4c1415add0 con 0x7f4c140726e0 2024-04-26T20:51:47.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.051+0000 7f4c18cbd640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f4c1415e460 con 0x7f4c1415a9f0 2024-04-26T20:51:47.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.051+0000 7f4c12d9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c14161510 0x7f4c141e6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:47.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.051+0000 7f4c1259c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c140726e0 0x7f4c14156000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:47.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.051+0000 7f4c12d9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c14161510 0x7f4c141e6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54546/0 (socket says 172.21.15.73:54546) 2024-04-26T20:51:47.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.051+0000 7f4c11d9b640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4c1415a9f0 0x7f4c14143080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:47.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.051+0000 7f4c12d9d640 1 -- 172.21.15.73:0/3515737839 learned_addr learned my addr 172.21.15.73:0/3515737839 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:47.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.051+0000 7f4c12d9d640 1 -- 172.21.15.73:0/3515737839 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4c1415a9f0 msgr2=0x7f4c14143080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:47.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.051+0000 7f4c12d9d640 1 --2- 172.21.15.73:0/3515737839 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4c1415a9f0 0x7f4c14143080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:47.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.051+0000 7f4c12d9d640 1 -- 172.21.15.73:0/3515737839 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c140726e0 msgr2=0x7f4c14156000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:47.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.051+0000 7f4c12d9d640 1 --2- 172.21.15.73:0/3515737839 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c140726e0 0x7f4c14156000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:47.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.051+0000 7f4c12d9d640 1 -- 172.21.15.73:0/3515737839 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c14154e40 con 0x7f4c14161510 2024-04-26T20:51:47.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.051+0000 7f4c1259c640 1 --2- 172.21.15.73:0/3515737839 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c140726e0 0x7f4c14156000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:51:47.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.052+0000 7f4c12d9d640 1 --2- 172.21.15.73:0/3515737839 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c14161510 0x7f4c141e6920 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c00b360 tx=0x7f4c0c00b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:47.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.052+0000 7f4c02ffd640 1 -- 172.21.15.73:0/3515737839 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c01e070 con 0x7f4c14161510 2024-04-26T20:51:47.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.052+0000 7f4c02ffd640 1 -- 172.21.15.73:0/3515737839 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c0c00be30 con 0x7f4c14161510 2024-04-26T20:51:47.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.052+0000 7f4c18cbd640 1 -- 172.21.15.73:0/3515737839 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c14155520 con 0x7f4c14161510 2024-04-26T20:51:47.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.052+0000 7f4c02ffd640 1 -- 172.21.15.73:0/3515737839 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c007740 con 0x7f4c14161510 2024-04-26T20:51:47.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.052+0000 7f4c18cbd640 1 -- 172.21.15.73:0/3515737839 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c14155ae0 con 0x7f4c14161510 2024-04-26T20:51:47.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.053+0000 7f4c18cbd640 1 -- 172.21.15.73:0/3515737839 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c140694f0 con 0x7f4c14161510 2024-04-26T20:51:47.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.053+0000 7f4c02ffd640 1 -- 172.21.15.73:0/3515737839 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4c0c021070 con 0x7f4c14161510 2024-04-26T20:51:47.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.053+0000 7f4c02ffd640 1 --2- 172.21.15.73:0/3515737839 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f4bf406c8f0 0x7f4bf406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:47.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.054+0000 7f4c02ffd640 1 -- 172.21.15.73:0/3515737839 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f4c0c08fcb0 con 0x7f4c14161510 2024-04-26T20:51:47.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.054+0000 7f4c1259c640 1 --2- 172.21.15.73:0/3515737839 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f4bf406c8f0 0x7f4bf406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:47.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.054+0000 7f4c1259c640 1 --2- 172.21.15.73:0/3515737839 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f4bf406c8f0 0x7f4bf406edb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4c04034000 tx=0x7f4c0402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:47.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.058+0000 7f4c02ffd640 1 -- 172.21.15.73:0/3515737839 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4c0c05d870 con 0x7f4c14161510 2024-04-26T20:51:47.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.269+0000 7f4c18cbd640 1 -- 172.21.15.73:0/3515737839 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f4c14069c20 con 0x7f4c14161510 2024-04-26T20:51:47.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.935+0000 7f4c02ffd640 1 -- 172.21.15.73:0/3515737839 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v15) ==== 122+0+0 (secure 0 0 0) 0x7f4c0c061b70 con 0x7f4c14161510 2024-04-26T20:51:47.938 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.937+0000 7f4c18cbd640 1 -- 172.21.15.73:0/3515737839 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f4bf406c8f0 msgr2=0x7f4bf406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:47.938 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.937+0000 7f4c18cbd640 1 --2- 172.21.15.73:0/3515737839 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f4bf406c8f0 0x7f4bf406edb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4c04034000 tx=0x7f4c0402e000 comp rx=0 tx=0).stop 2024-04-26T20:51:47.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.937+0000 7f4c18cbd640 1 -- 172.21.15.73:0/3515737839 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c14161510 msgr2=0x7f4c141e6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:47.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.937+0000 7f4c18cbd640 1 --2- 172.21.15.73:0/3515737839 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c14161510 0x7f4c141e6920 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c00b360 tx=0x7f4c0c00b820 comp rx=0 tx=0).stop 2024-04-26T20:51:47.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.937+0000 7f4c18cbd640 1 -- 172.21.15.73:0/3515737839 shutdown_connections 2024-04-26T20:51:47.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.937+0000 7f4c18cbd640 1 --2- 172.21.15.73:0/3515737839 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c14161510 0x7f4c141e6920 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:47.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.937+0000 7f4c18cbd640 1 --2- 172.21.15.73:0/3515737839 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f4bf406c8f0 0x7f4bf406edb0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:47.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.937+0000 7f4c18cbd640 1 --2- 172.21.15.73:0/3515737839 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4c1415a9f0 0x7f4c14143080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:47.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.937+0000 7f4c18cbd640 1 --2- 172.21.15.73:0/3515737839 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4c140726e0 0x7f4c14156000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:47.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.937+0000 7f4c18cbd640 1 -- 172.21.15.73:0/3515737839 >> 172.21.15.73:0/3515737839 conn(0x7f4c140b3180 msgr2=0x7f4c14073140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:47.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.937+0000 7f4c18cbd640 1 -- 172.21.15.73:0/3515737839 shutdown_connections 2024-04-26T20:51:47.939 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:47.937+0000 7f4c18cbd640 1 -- 172.21.15.73:0/3515737839 wait complete. 2024-04-26T20:51:47.950 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T20:51:47.951 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:51:48.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.048+0000 7f10e14d1640 1 Processor -- start 2024-04-26T20:51:48.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.048+0000 7f10e14d1640 1 -- start start 2024-04-26T20:51:48.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.049+0000 7f10e14d1640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f10dc0d0070 0x7f10dc0d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:48.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.049+0000 7f10e14d1640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f10dc06d560 0x7f10dc06db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:48.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.049+0000 7f10e14d1640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f10dc06e0a0 0x7f10dc162f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:48.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.049+0000 7f10e14d1640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f10dc162d00 con 0x7f10dc0d0070 2024-04-26T20:51:48.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.049+0000 7f10e14d1640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f10dc0d2d10 con 0x7f10dc06e0a0 2024-04-26T20:51:48.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.049+0000 7f10e14d1640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f10dc0d2420 con 0x7f10dc06d560 2024-04-26T20:51:48.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.049+0000 7f10daffd640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f10dc0d0070 0x7f10dc0d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:48.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.049+0000 7f10daffd640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f10dc0d0070 0x7f10dc0d1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54552/0 (socket says 172.21.15.73:54552) 2024-04-26T20:51:48.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.049+0000 7f10daffd640 1 -- 172.21.15.73:0/1317063500 learned_addr learned my addr 172.21.15.73:0/1317063500 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:48.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.050+0000 7f10db7fe640 1 --2- 172.21.15.73:0/1317063500 >> v2:172.21.15.120:3300/0 conn(0x7f10dc06e0a0 0x7f10dc162f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:48.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.050+0000 7f10d3fff640 1 --2- 172.21.15.73:0/1317063500 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f10dc06d560 0x7f10dc06db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:48.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.050+0000 7f10daffd640 1 -- 172.21.15.73:0/1317063500 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f10dc06d560 msgr2=0x7f10dc06db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:48.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.050+0000 7f10daffd640 1 --2- 172.21.15.73:0/1317063500 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f10dc06d560 0x7f10dc06db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:48.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.050+0000 7f10daffd640 1 -- 172.21.15.73:0/1317063500 >> v2:172.21.15.120:3300/0 conn(0x7f10dc06e0a0 msgr2=0x7f10dc162f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:48.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.050+0000 7f10daffd640 1 --2- 172.21.15.73:0/1317063500 >> v2:172.21.15.120:3300/0 conn(0x7f10dc06e0a0 0x7f10dc162f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:48.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.050+0000 7f10daffd640 1 -- 172.21.15.73:0/1317063500 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10dc0cd6b0 con 0x7f10dc0d0070 2024-04-26T20:51:48.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.050+0000 7f10db7fe640 1 --2- 172.21.15.73:0/1317063500 >> v2:172.21.15.120:3300/0 conn(0x7f10dc06e0a0 0x7f10dc162f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:48.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.050+0000 7f10daffd640 1 --2- 172.21.15.73:0/1317063500 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10dc0d0070 0x7f10dc0d1d40 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f10dc073370 tx=0x7f10cc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c0ca62f34b3cda9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:48.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.051+0000 7f10da7fc640 1 -- 172.21.15.73:0/1317063500 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10cc037070 con 0x7f10dc0d0070 2024-04-26T20:51:48.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.051+0000 7f10da7fc640 1 -- 172.21.15.73:0/1317063500 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10cc032440 con 0x7f10dc0d0070 2024-04-26T20:51:48.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.051+0000 7f10da7fc640 1 -- 172.21.15.73:0/1317063500 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10cc032810 con 0x7f10dc0d0070 2024-04-26T20:51:48.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.051+0000 7f10e14d1640 1 -- 172.21.15.73:0/1317063500 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10dc0d0070 msgr2=0x7f10dc0d1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:48.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.051+0000 7f10e14d1640 1 --2- 172.21.15.73:0/1317063500 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10dc0d0070 0x7f10dc0d1d40 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f10dc073370 tx=0x7f10cc02a270 comp rx=0 tx=0).stop 2024-04-26T20:51:48.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.051+0000 7f10e14d1640 1 -- 172.21.15.73:0/1317063500 shutdown_connections 2024-04-26T20:51:48.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.051+0000 7f10e14d1640 1 --2- 172.21.15.73:0/1317063500 >> v2:172.21.15.120:3300/0 conn(0x7f10dc06e0a0 0x7f10dc162f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:48.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.051+0000 7f10e14d1640 1 --2- 172.21.15.73:0/1317063500 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f10dc06d560 0x7f10dc06db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:48.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.051+0000 7f10e14d1640 1 --2- 172.21.15.73:0/1317063500 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10dc0d0070 0x7f10dc0d1d40 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:48.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.051+0000 7f10e14d1640 1 -- 172.21.15.73:0/1317063500 >> 172.21.15.73:0/1317063500 conn(0x7f10dc0d2ec0 msgr2=0x7f10dc0d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:48.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.051+0000 7f10e14d1640 1 -- 172.21.15.73:0/1317063500 shutdown_connections 2024-04-26T20:51:48.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.051+0000 7f10e14d1640 1 -- 172.21.15.73:0/1317063500 wait complete. 2024-04-26T20:51:48.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.052+0000 7f10e14d1640 1 Processor -- start 2024-04-26T20:51:48.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.052+0000 7f10e14d1640 1 -- start start 2024-04-26T20:51:48.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.053+0000 7f10e14d1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10dc06d560 0x7f10dc15f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:48.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.053+0000 7f10e14d1640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f10dc06e0a0 0x7f10dc09d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:48.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.053+0000 7f10e14d1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10dc0d0070 0x7f10dc09db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:48.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.053+0000 7f10e14d1640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f10dc163450 con 0x7f10dc0d0070 2024-04-26T20:51:48.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.053+0000 7f10e14d1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f10dc0cdc70 con 0x7f10dc06d560 2024-04-26T20:51:48.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.053+0000 7f10e14d1640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f10dc1635d0 con 0x7f10dc06e0a0 2024-04-26T20:51:48.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.053+0000 7f10db7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10dc0d0070 0x7f10dc09db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:48.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.053+0000 7f10db7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10dc0d0070 0x7f10dc09db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54564/0 (socket says 172.21.15.73:54564) 2024-04-26T20:51:48.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.053+0000 7f10db7fe640 1 -- 172.21.15.73:0/1626949721 learned_addr learned my addr 172.21.15.73:0/1626949721 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:48.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.053+0000 7f10daffd640 1 --2- 172.21.15.73:0/1626949721 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10dc06d560 0x7f10dc15f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:48.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.053+0000 7f10d3fff640 1 --2- 172.21.15.73:0/1626949721 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f10dc06e0a0 0x7f10dc09d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:48.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.053+0000 7f10db7fe640 1 -- 172.21.15.73:0/1626949721 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f10dc06e0a0 msgr2=0x7f10dc09d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:48.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.053+0000 7f10db7fe640 1 --2- 172.21.15.73:0/1626949721 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f10dc06e0a0 0x7f10dc09d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:48.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.053+0000 7f10db7fe640 1 -- 172.21.15.73:0/1626949721 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10dc06d560 msgr2=0x7f10dc15f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:48.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.053+0000 7f10db7fe640 1 --2- 172.21.15.73:0/1626949721 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10dc06d560 0x7f10dc15f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:48.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.053+0000 7f10db7fe640 1 -- 172.21.15.73:0/1626949721 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10dc0c85d0 con 0x7f10dc0d0070 2024-04-26T20:51:48.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.053+0000 7f10daffd640 1 --2- 172.21.15.73:0/1626949721 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10dc06d560 0x7f10dc15f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:48.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.054+0000 7f10db7fe640 1 --2- 172.21.15.73:0/1626949721 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10dc0d0070 0x7f10dc09db20 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f10d4009fa0 tx=0x7f10d400d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:48.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.054+0000 7f10d37fe640 1 -- 172.21.15.73:0/1626949721 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d401a070 con 0x7f10dc0d0070 2024-04-26T20:51:48.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.054+0000 7f10d37fe640 1 -- 172.21.15.73:0/1626949721 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10d4003bb0 con 0x7f10dc0d0070 2024-04-26T20:51:48.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.054+0000 7f10d37fe640 1 -- 172.21.15.73:0/1626949721 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d4004030 con 0x7f10dc0d0070 2024-04-26T20:51:48.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.054+0000 7f10e14d1640 1 -- 172.21.15.73:0/1626949721 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10dc0c88b0 con 0x7f10dc0d0070 2024-04-26T20:51:48.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.054+0000 7f10e14d1640 1 -- 172.21.15.73:0/1626949721 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10dc0c8e70 con 0x7f10dc0d0070 2024-04-26T20:51:48.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.055+0000 7f10e14d1640 1 -- 172.21.15.73:0/1626949721 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10dc0694f0 con 0x7f10dc0d0070 2024-04-26T20:51:48.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.055+0000 7f10d37fe640 1 -- 172.21.15.73:0/1626949721 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f10d401d070 con 0x7f10dc0d0070 2024-04-26T20:51:48.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.056+0000 7f10d37fe640 1 --2- 172.21.15.73:0/1626949721 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f10bc06c8f0 0x7f10bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:48.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.056+0000 7f10daffd640 1 --2- 172.21.15.73:0/1626949721 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f10bc06c8f0 0x7f10bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:48.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.056+0000 7f10d37fe640 1 -- 172.21.15.73:0/1626949721 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f10d408bf10 con 0x7f10dc0d0070 2024-04-26T20:51:48.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.056+0000 7f10daffd640 1 --2- 172.21.15.73:0/1626949721 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f10bc06c8f0 0x7f10bc06edb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f10cc034000 tx=0x7f10cc02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:48.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.060+0000 7f10d37fe640 1 -- 172.21.15.73:0/1626949721 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f10d4010070 con 0x7f10dc0d0070 2024-04-26T20:51:48.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.270+0000 7f10e14d1640 1 -- 172.21.15.73:0/1626949721 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f10dc0d0470 con 0x7f10dc0d0070 2024-04-26T20:51:48.272 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.271+0000 7f10d37fe640 1 -- 172.21.15.73:0/1626949721 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+5181 (secure 0 0 0) 0x7f10d405a240 con 0x7f10dc0d0070 2024-04-26T20:51:48.272 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:51:48.272 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4802,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/272662913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":272662913},{"type":"v1","addr":"172.21.15.73:6836","nonce":272662913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5047,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/1438858294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":1438858294},{"type":"v1","addr":"172.21.15.73:6837","nonce":1438858294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5050,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6834/3883540031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3883540031},{"type":"v1","addr":"172.21.15.120:6834","nonce":3883540031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:51:45.273953+0000","modified":"2024-04-26T20:51:47.270309+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5053":{"gid":5053,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.120:6835/761435580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6833","nonce":761435580},{"type":"v1","addr":"172.21.15.120:6835","nonce":761435580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5053,"qdb_cluster":[5053]},"id":2}]} 2024-04-26T20:51:48.272 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 15 2024-04-26T20:51:48.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.273+0000 7f10e14d1640 1 -- 172.21.15.73:0/1626949721 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f10bc06c8f0 msgr2=0x7f10bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:48.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.273+0000 7f10e14d1640 1 --2- 172.21.15.73:0/1626949721 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f10bc06c8f0 0x7f10bc06edb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f10cc034000 tx=0x7f10cc02e000 comp rx=0 tx=0).stop 2024-04-26T20:51:48.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.273+0000 7f10e14d1640 1 -- 172.21.15.73:0/1626949721 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10dc0d0070 msgr2=0x7f10dc09db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:48.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.273+0000 7f10e14d1640 1 --2- 172.21.15.73:0/1626949721 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10dc0d0070 0x7f10dc09db20 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f10d4009fa0 tx=0x7f10d400d560 comp rx=0 tx=0).stop 2024-04-26T20:51:48.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.273+0000 7f10e14d1640 1 -- 172.21.15.73:0/1626949721 shutdown_connections 2024-04-26T20:51:48.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.273+0000 7f10e14d1640 1 --2- 172.21.15.73:0/1626949721 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10dc0d0070 0x7f10dc09db20 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:48.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.273+0000 7f10e14d1640 1 --2- 172.21.15.73:0/1626949721 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f10bc06c8f0 0x7f10bc06edb0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:48.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.273+0000 7f10e14d1640 1 --2- 172.21.15.73:0/1626949721 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f10dc06e0a0 0x7f10dc09d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:48.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.273+0000 7f10e14d1640 1 --2- 172.21.15.73:0/1626949721 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f10dc06d560 0x7f10dc15f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:48.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.273+0000 7f10e14d1640 1 -- 172.21.15.73:0/1626949721 >> 172.21.15.73:0/1626949721 conn(0x7f10dc0d2ec0 msgr2=0x7f10dc0cc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:48.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.273+0000 7f10e14d1640 1 -- 172.21.15.73:0/1626949721 shutdown_connections 2024-04-26T20:51:48.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.273+0000 7f10e14d1640 1 -- 172.21.15.73:0/1626949721 wait complete. 2024-04-26T20:51:48.285 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:51:48.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.380+0000 7ff439f19640 1 Processor -- start 2024-04-26T20:51:48.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.380+0000 7ff439f19640 1 -- start start 2024-04-26T20:51:48.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.381+0000 7ff439f19640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff434162810 0x7ff434070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:48.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.381+0000 7ff439f19640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7ff4340b2340 0x7ff4340cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:48.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.381+0000 7ff439f19640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff4340cd930 0x7ff4340d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:48.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.381+0000 7ff439f19640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7ff4340d0750 con 0x7ff434162810 2024-04-26T20:51:48.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.381+0000 7ff439f19640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7ff4340d16a0 con 0x7ff4340b2340 2024-04-26T20:51:48.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.381+0000 7ff439f19640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7ff4340d10d0 con 0x7ff4340cd930 2024-04-26T20:51:48.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.381+0000 7ff4337fe640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff434162810 0x7ff434070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:48.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.381+0000 7ff4337fe640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff434162810 0x7ff434070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54576/0 (socket says 172.21.15.73:54576) 2024-04-26T20:51:48.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.381+0000 7ff4337fe640 1 -- 172.21.15.73:0/1112785240 learned_addr learned my addr 172.21.15.73:0/1112785240 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:48.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.381+0000 7ff432ffd640 1 --2- 172.21.15.73:0/1112785240 >> v2:172.21.15.120:3300/0 conn(0x7ff4340b2340 0x7ff4340cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:48.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.381+0000 7ff4337fe640 1 -- 172.21.15.73:0/1112785240 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff4340cd930 msgr2=0x7ff4340d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:48.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.381+0000 7ff433fff640 1 --2- 172.21.15.73:0/1112785240 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff4340cd930 0x7ff4340d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:48.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.381+0000 7ff4337fe640 1 --2- 172.21.15.73:0/1112785240 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff4340cd930 0x7ff4340d1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:48.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.381+0000 7ff4337fe640 1 -- 172.21.15.73:0/1112785240 >> v2:172.21.15.120:3300/0 conn(0x7ff4340b2340 msgr2=0x7ff4340cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:48.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.382+0000 7ff4337fe640 1 --2- 172.21.15.73:0/1112785240 >> v2:172.21.15.120:3300/0 conn(0x7ff4340b2340 0x7ff4340cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:48.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.382+0000 7ff4337fe640 1 -- 172.21.15.73:0/1112785240 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4340d1370 con 0x7ff434162810 2024-04-26T20:51:48.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.382+0000 7ff432ffd640 1 --2- 172.21.15.73:0/1112785240 >> v2:172.21.15.120:3300/0 conn(0x7ff4340b2340 0x7ff4340cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:48.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.382+0000 7ff433fff640 1 --2- 172.21.15.73:0/1112785240 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff4340cd930 0x7ff4340d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:48.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.382+0000 7ff4337fe640 1 --2- 172.21.15.73:0/1112785240 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff434162810 0x7ff434070350 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff4340cd0a0 tx=0x7ff42402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52df3dc8b126d10f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:48.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.382+0000 7ff4327fc640 1 -- 172.21.15.73:0/1112785240 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff424037070 con 0x7ff434162810 2024-04-26T20:51:48.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.382+0000 7ff4327fc640 1 -- 172.21.15.73:0/1112785240 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff424032440 con 0x7ff434162810 2024-04-26T20:51:48.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.382+0000 7ff4327fc640 1 -- 172.21.15.73:0/1112785240 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff424032810 con 0x7ff434162810 2024-04-26T20:51:48.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.383+0000 7ff439f19640 1 -- 172.21.15.73:0/1112785240 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff434162810 msgr2=0x7ff434070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:48.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.383+0000 7ff439f19640 1 --2- 172.21.15.73:0/1112785240 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff434162810 0x7ff434070350 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff4340cd0a0 tx=0x7ff42402a1e0 comp rx=0 tx=0).stop 2024-04-26T20:51:48.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.383+0000 7ff439f19640 1 -- 172.21.15.73:0/1112785240 shutdown_connections 2024-04-26T20:51:48.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.383+0000 7ff439f19640 1 --2- 172.21.15.73:0/1112785240 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff4340cd930 0x7ff4340d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:48.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.383+0000 7ff439f19640 1 --2- 172.21.15.73:0/1112785240 >> v2:172.21.15.120:3300/0 conn(0x7ff4340b2340 0x7ff4340cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:48.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.383+0000 7ff439f19640 1 --2- 172.21.15.73:0/1112785240 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff434162810 0x7ff434070350 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:48.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.383+0000 7ff439f19640 1 -- 172.21.15.73:0/1112785240 >> 172.21.15.73:0/1112785240 conn(0x7ff4340d2da0 msgr2=0x7ff4340d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:48.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.383+0000 7ff439f19640 1 -- 172.21.15.73:0/1112785240 shutdown_connections 2024-04-26T20:51:48.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.383+0000 7ff439f19640 1 -- 172.21.15.73:0/1112785240 wait complete. 2024-04-26T20:51:48.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.384+0000 7ff439f19640 1 Processor -- start 2024-04-26T20:51:48.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.384+0000 7ff439f19640 1 -- start start 2024-04-26T20:51:48.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.385+0000 7ff439f19640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4340b2340 0x7ff43406de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:48.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.385+0000 7ff439f19640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4340cd930 0x7ff4340c9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:48.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.385+0000 7ff439f19640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff434162810 0x7ff4340ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:48.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.385+0000 7ff439f19640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff434070890 con 0x7ff4340cd930 2024-04-26T20:51:48.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.385+0000 7ff439f19640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff4340d0470 con 0x7ff4340b2340 2024-04-26T20:51:48.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.385+0000 7ff439f19640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7ff4340d2070 con 0x7ff434162810 2024-04-26T20:51:48.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.385+0000 7ff432ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4340cd930 0x7ff4340c9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:48.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.385+0000 7ff432ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4340cd930 0x7ff4340c9e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54584/0 (socket says 172.21.15.73:54584) 2024-04-26T20:51:48.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.385+0000 7ff432ffd640 1 -- 172.21.15.73:0/1057461405 learned_addr learned my addr 172.21.15.73:0/1057461405 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:48.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.385+0000 7ff4337fe640 1 --2- 172.21.15.73:0/1057461405 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4340b2340 0x7ff43406de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:48.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.385+0000 7ff433fff640 1 --2- 172.21.15.73:0/1057461405 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff434162810 0x7ff4340ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:48.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.385+0000 7ff432ffd640 1 -- 172.21.15.73:0/1057461405 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff434162810 msgr2=0x7ff4340ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:48.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.385+0000 7ff432ffd640 1 --2- 172.21.15.73:0/1057461405 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff434162810 0x7ff4340ca3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:48.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.385+0000 7ff432ffd640 1 -- 172.21.15.73:0/1057461405 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4340b2340 msgr2=0x7ff43406de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:48.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.385+0000 7ff432ffd640 1 --2- 172.21.15.73:0/1057461405 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4340b2340 0x7ff43406de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:48.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.385+0000 7ff432ffd640 1 -- 172.21.15.73:0/1057461405 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4340ca8f0 con 0x7ff4340cd930 2024-04-26T20:51:48.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.385+0000 7ff4337fe640 1 --2- 172.21.15.73:0/1057461405 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4340b2340 0x7ff43406de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:48.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.385+0000 7ff433fff640 1 --2- 172.21.15.73:0/1057461405 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff434162810 0x7ff4340ca3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:48.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.385+0000 7ff432ffd640 1 --2- 172.21.15.73:0/1057461405 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4340cd930 0x7ff4340c9e70 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ff42800c7c0 tx=0x7ff42800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:48.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.386+0000 7ff41bfff640 1 -- 172.21.15.73:0/1057461405 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff428015070 con 0x7ff4340cd930 2024-04-26T20:51:48.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.386+0000 7ff41bfff640 1 -- 172.21.15.73:0/1057461405 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff428007a30 con 0x7ff4340cd930 2024-04-26T20:51:48.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.386+0000 7ff439f19640 1 -- 172.21.15.73:0/1057461405 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4340caba0 con 0x7ff4340cd930 2024-04-26T20:51:48.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.386+0000 7ff41bfff640 1 -- 172.21.15.73:0/1057461405 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff428007e00 con 0x7ff4340cd930 2024-04-26T20:51:48.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.386+0000 7ff439f19640 1 -- 172.21.15.73:0/1057461405 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4340cb160 con 0x7ff4340cd930 2024-04-26T20:51:48.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.387+0000 7ff439f19640 1 -- 172.21.15.73:0/1057461405 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4340694f0 con 0x7ff4340cd930 2024-04-26T20:51:48.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.387+0000 7ff41bfff640 1 -- 172.21.15.73:0/1057461405 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff428018070 con 0x7ff4340cd930 2024-04-26T20:51:48.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.387+0000 7ff41bfff640 1 --2- 172.21.15.73:0/1057461405 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff42006c8f0 0x7ff42006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:48.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.388+0000 7ff41bfff640 1 -- 172.21.15.73:0/1057461405 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7ff42808bc30 con 0x7ff4340cd930 2024-04-26T20:51:48.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.388+0000 7ff4337fe640 1 --2- 172.21.15.73:0/1057461405 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff42006c8f0 0x7ff42006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:48.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.388+0000 7ff4337fe640 1 --2- 172.21.15.73:0/1057461405 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff42006c8f0 0x7ff42006edb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff424034000 tx=0x7ff42402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:48.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.392+0000 7ff41bfff640 1 -- 172.21.15.73:0/1057461405 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff428059ed0 con 0x7ff4340cd930 2024-04-26T20:51:48.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.567+0000 7ff439f19640 1 -- 172.21.15.73:0/1057461405 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff43406ea60 con 0x7ff4340cd930 2024-04-26T20:51:48.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.568+0000 7ff41bfff640 1 -- 172.21.15.73:0/1057461405 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+16942 (secure 0 0 0) 0x7ff4280597d0 con 0x7ff4340cd930 2024-04-26T20:51:48.569 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:51:48.569 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":39,"fsid":"fbd3f548-b547-4713-856b-09f98f5d1764","created":"2024-04-26T20:50:07.876786+0000","modified":"2024-04-26T20:51:46.928774+0000","last_up_change":"2024-04-26T20:50:19.082741+0000","last_in_change":"2024-04-26T20:50:10.018400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:50:19.993731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:51:43.110774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-26T20:51:44.235846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d6f4f3-214a-4392-a02f-c33cbcab3c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6809","nonce":3461705124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6811","nonce":3461705124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6815","nonce":3461705124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6813","nonce":3461705124}]},"public_addr":"172.21.15.73:6809/3461705124","cluster_addr":"172.21.15.73:6811/3461705124","heartbeat_back_addr":"172.21.15.73:6815/3461705124","heartbeat_front_addr":"172.21.15.73:6813/3461705124","state":["exists","up"]},{"osd":1,"uuid":"786bf8ca-6ddc-4b3c-be6c-9152398bd5fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6817","nonce":2827619403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6819","nonce":2827619403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6825","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6827","nonce":2827619403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6821","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6823","nonce":2827619403}]},"public_addr":"172.21.15.73:6817/2827619403","cluster_addr":"172.21.15.73:6819/2827619403","heartbeat_back_addr":"172.21.15.73:6827/2827619403","heartbeat_front_addr":"172.21.15.73:6823/2827619403","state":["exists","up"]},{"osd":2,"uuid":"90963b1a-5437-4ef8-ad8b-6b74c534eeb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6801","nonce":2542681064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6803","nonce":2542681064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6808","nonce":2542681064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6805","nonce":2542681064}]},"public_addr":"172.21.15.73:6801/2542681064","cluster_addr":"172.21.15.73:6803/2542681064","heartbeat_back_addr":"172.21.15.73:6808/2542681064","heartbeat_front_addr":"172.21.15.73:6805/2542681064","state":["exists","up"]},{"osd":3,"uuid":"bd3d2148-6e38-41b4-a453-fe6e3b8b4ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6822","nonce":1604042718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6826","nonce":1604042718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6831","nonce":1604042718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6829","nonce":1604042718}]},"public_addr":"172.21.15.73:6822/1604042718","cluster_addr":"172.21.15.73:6826/1604042718","heartbeat_back_addr":"172.21.15.73:6831/1604042718","heartbeat_front_addr":"172.21.15.73:6829/1604042718","state":["exists","up"]},{"osd":4,"uuid":"e8668dc8-a23f-46d2-8311-cb882da08e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6817","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6819","nonce":3591055561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6824","nonce":3591055561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6831","nonce":3591055561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6828","nonce":3591055561}]},"public_addr":"172.21.15.120:6819/3591055561","cluster_addr":"172.21.15.120:6824/3591055561","heartbeat_back_addr":"172.21.15.120:6831/3591055561","heartbeat_front_addr":"172.21.15.120:6828/3591055561","state":["exists","up"]},{"osd":5,"uuid":"cbac0bdb-96bd-4852-bcce-e564495c3fcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6809","nonce":819450433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6811","nonce":819450433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6815","nonce":819450433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6813","nonce":819450433}]},"public_addr":"172.21.15.120:6809/819450433","cluster_addr":"172.21.15.120:6811/819450433","heartbeat_back_addr":"172.21.15.120:6815/819450433","heartbeat_front_addr":"172.21.15.120:6813/819450433","state":["exists","up"]},{"osd":6,"uuid":"1884b2a8-569f-4272-96a9-f25a09364dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6801","nonce":3946210727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6803","nonce":3946210727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6807","nonce":3946210727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6805","nonce":3946210727}]},"public_addr":"172.21.15.120:6801/3946210727","cluster_addr":"172.21.15.120:6803/3946210727","heartbeat_back_addr":"172.21.15.120:6807/3946210727","heartbeat_front_addr":"172.21.15.120:6805/3946210727","state":["exists","up"]},{"osd":7,"uuid":"c0c3bf6d-a4c5-4f1e-aa2d-330951830666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6818","nonce":2915574674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6821","nonce":2915574674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6827","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6829","nonce":2915574674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6823","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6825","nonce":2915574674}]},"public_addr":"172.21.15.120:6818/2915574674","cluster_addr":"172.21.15.120:6821/2915574674","heartbeat_back_addr":"172.21.15.120:6829/2915574674","heartbeat_front_addr":"172.21.15.120:6825/2915574674","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.218216+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.208710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.112705+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.182789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.164324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.174212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.225396+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.261559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:51:48.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.570+0000 7ff439f19640 1 -- 172.21.15.73:0/1057461405 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff42006c8f0 msgr2=0x7ff42006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:48.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.570+0000 7ff439f19640 1 --2- 172.21.15.73:0/1057461405 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff42006c8f0 0x7ff42006edb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff424034000 tx=0x7ff42402e000 comp rx=0 tx=0).stop 2024-04-26T20:51:48.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.570+0000 7ff439f19640 1 -- 172.21.15.73:0/1057461405 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4340cd930 msgr2=0x7ff4340c9e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:48.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.570+0000 7ff439f19640 1 --2- 172.21.15.73:0/1057461405 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4340cd930 0x7ff4340c9e70 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ff42800c7c0 tx=0x7ff42800cc80 comp rx=0 tx=0).stop 2024-04-26T20:51:48.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.570+0000 7ff439f19640 1 -- 172.21.15.73:0/1057461405 shutdown_connections 2024-04-26T20:51:48.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.570+0000 7ff439f19640 1 --2- 172.21.15.73:0/1057461405 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4340cd930 0x7ff4340c9e70 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:48.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.570+0000 7ff439f19640 1 --2- 172.21.15.73:0/1057461405 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff42006c8f0 0x7ff42006edb0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:48.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.570+0000 7ff439f19640 1 --2- 172.21.15.73:0/1057461405 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff434162810 0x7ff4340ca3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:48.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.570+0000 7ff439f19640 1 --2- 172.21.15.73:0/1057461405 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff4340b2340 0x7ff43406de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:48.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.570+0000 7ff439f19640 1 -- 172.21.15.73:0/1057461405 >> 172.21.15.73:0/1057461405 conn(0x7ff4340d2da0 msgr2=0x7ff434070b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:48.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.570+0000 7ff439f19640 1 -- 172.21.15.73:0/1057461405 shutdown_connections 2024-04-26T20:51:48.573 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.570+0000 7ff439f19640 1 -- 172.21.15.73:0/1057461405 wait complete. 2024-04-26T20:51:48.582 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T20:51:48.582 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:51:48.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.677+0000 7fc12c52b640 1 Processor -- start 2024-04-26T20:51:48.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.677+0000 7fc12c52b640 1 -- start start 2024-04-26T20:51:48.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.678+0000 7fc12c52b640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc12415f120 0x7fc12406f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:48.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.678+0000 7fc12c52b640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fc12406c950 0x7fc1240af030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:48.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.678+0000 7fc12c52b640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1240b49f0 con 0x7fc1240b4480 2024-04-26T20:51:48.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.678+0000 7fc12c52b640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fc12415f500 con 0x7fc12406c950 2024-04-26T20:51:48.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.678+0000 7fc12c52b640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fc1240ae8c0 con 0x7fc12415f120 2024-04-26T20:51:48.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.678+0000 7fc129ac6640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fc1240b4480 0x7fc12406c240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:41638/0 (socket says 172.21.15.73:41638) 2024-04-26T20:51:48.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.678+0000 7fc129ac6640 1 -- 172.21.15.73:0/1888754077 learned_addr learned my addr 172.21.15.73:0/1888754077 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:48.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.678+0000 7fc12a2c7640 1 --2- 172.21.15.73:0/1888754077 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc12415f120 0x7fc12406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:48.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.678+0000 7fc12aac8640 1 --2- 172.21.15.73:0/1888754077 >> v2:172.21.15.120:3300/0 conn(0x7fc12406c950 0x7fc1240af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:48.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.678+0000 7fc1292c5640 1 -- 172.21.15.73:0/1888754077 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3064383333 0 0) 0x7fc1240b49f0 con 0x7fc1240b4480 2024-04-26T20:51:48.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.678+0000 7fc1292c5640 1 -- 172.21.15.73:0/1888754077 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc110003520 con 0x7fc1240b4480 2024-04-26T20:51:48.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.678+0000 7fc1292c5640 1 -- 172.21.15.73:0/1888754077 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3949684166 0 0) 0x7fc110003520 con 0x7fc1240b4480 2024-04-26T20:51:48.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.678+0000 7fc1292c5640 1 -- 172.21.15.73:0/1888754077 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc12415f120 msgr2=0x7fc12406f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:48.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.678+0000 7fc1292c5640 1 --2- 172.21.15.73:0/1888754077 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc12415f120 0x7fc12406f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:48.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.678+0000 7fc1292c5640 1 -- 172.21.15.73:0/1888754077 >> v2:172.21.15.120:3300/0 conn(0x7fc12406c950 msgr2=0x7fc1240af030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:48.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.678+0000 7fc1292c5640 1 --2- 172.21.15.73:0/1888754077 >> v2:172.21.15.120:3300/0 conn(0x7fc12406c950 0x7fc1240af030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:48.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.678+0000 7fc1292c5640 1 -- 172.21.15.73:0/1888754077 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1240b4cd0 con 0x7fc1240b4480 2024-04-26T20:51:48.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.678+0000 7fc1292c5640 1 -- 172.21.15.73:0/1888754077 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7fc114001670 con 0x7fc1240b4480 2024-04-26T20:51:48.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.678+0000 7fc1292c5640 1 -- 172.21.15.73:0/1888754077 >> v1:172.21.15.73:6789/0 conn(0x7fc1240b4480 legacy=0x7fc12406c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:48.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.678+0000 7fc1292c5640 1 --2- 172.21.15.73:0/1888754077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1100039b0 0x7fc110003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:48.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.678+0000 7fc1292c5640 1 --2- 172.21.15.73:0/1888754077 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc110004800 0x7fc110004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:48.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.678+0000 7fc1292c5640 1 --2- 172.21.15.73:0/1888754077 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc12415f120 0x7fc1100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:48.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.678+0000 7fc1292c5640 1 -- 172.21.15.73:0/1888754077 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc12415f500 con 0x7fc1100039b0 2024-04-26T20:51:48.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.678+0000 7fc1292c5640 1 -- 172.21.15.73:0/1888754077 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc1240ae8c0 con 0x7fc110004800 2024-04-26T20:51:48.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.678+0000 7fc1292c5640 1 -- 172.21.15.73:0/1888754077 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fc11000b9c0 con 0x7fc12415f120 2024-04-26T20:51:48.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.679+0000 7fc12a2c7640 1 --2- 172.21.15.73:0/1888754077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1100039b0 0x7fc110003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:48.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.679+0000 7fc129ac6640 1 --2- 172.21.15.73:0/1888754077 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc110004800 0x7fc110004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:48.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.679+0000 7fc12aac8640 1 --2- 172.21.15.73:0/1888754077 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc12415f120 0x7fc1100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:48.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.679+0000 7fc12a2c7640 1 -- 172.21.15.73:0/1888754077 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc12415f120 msgr2=0x7fc1100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:48.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.679+0000 7fc12a2c7640 1 --2- 172.21.15.73:0/1888754077 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc12415f120 0x7fc1100051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:48.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.679+0000 7fc12a2c7640 1 -- 172.21.15.73:0/1888754077 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc110004800 msgr2=0x7fc110004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:48.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.679+0000 7fc12a2c7640 1 --2- 172.21.15.73:0/1888754077 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc110004800 0x7fc110004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:48.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.679+0000 7fc12a2c7640 1 -- 172.21.15.73:0/1888754077 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc110003520 con 0x7fc1100039b0 2024-04-26T20:51:48.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.679+0000 7fc12a2c7640 1 --2- 172.21.15.73:0/1888754077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1100039b0 0x7fc110003dd0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fc120009af0 tx=0x7fc110005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bcc4a00f2ee423b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:48.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.680+0000 7fc1292c5640 1 -- 172.21.15.73:0/1888754077 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc120036070 con 0x7fc1100039b0 2024-04-26T20:51:48.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.680+0000 7fc1292c5640 1 -- 172.21.15.73:0/1888754077 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc120031400 con 0x7fc1100039b0 2024-04-26T20:51:48.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.680+0000 7fc1292c5640 1 -- 172.21.15.73:0/1888754077 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc120009690 con 0x7fc1100039b0 2024-04-26T20:51:48.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.680+0000 7fc12c52b640 1 -- 172.21.15.73:0/1888754077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1100039b0 msgr2=0x7fc110003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:48.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.680+0000 7fc12c52b640 1 --2- 172.21.15.73:0/1888754077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1100039b0 0x7fc110003dd0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fc120009af0 tx=0x7fc110005830 comp rx=0 tx=0).stop 2024-04-26T20:51:48.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.680+0000 7fc12aac8640 1 -- 172.21.15.73:0/1888754077 reap_dead start 2024-04-26T20:51:48.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.680+0000 7fc12c52b640 1 -- 172.21.15.73:0/1888754077 shutdown_connections 2024-04-26T20:51:48.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.680+0000 7fc12c52b640 1 -- 172.21.15.73:0/1888754077 >> 172.21.15.73:0/1888754077 conn(0x7fc1240b3180 msgr2=0x7fc1240b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:48.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.680+0000 7fc12c52b640 1 -- 172.21.15.73:0/1888754077 shutdown_connections 2024-04-26T20:51:48.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.680+0000 7fc12c52b640 1 -- 172.21.15.73:0/1888754077 wait complete. 2024-04-26T20:51:48.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.681+0000 7fc12c52b640 1 Processor -- start 2024-04-26T20:51:48.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.681+0000 7fc12c52b640 1 -- start start 2024-04-26T20:51:48.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.682+0000 7fc12c52b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1100039b0 0x7fc1240c8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:48.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.682+0000 7fc12c52b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc12414eb80 0x7fc12414f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:48.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.682+0000 7fc12c52b640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc1241f4dc0 0x7fc1241f5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:48.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.682+0000 7fc12c52b640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc12406f110 con 0x7fc1100039b0 2024-04-26T20:51:48.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.682+0000 7fc12c52b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc1240aeba0 con 0x7fc12414eb80 2024-04-26T20:51:48.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.682+0000 7fc12c52b640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fc12406f290 con 0x7fc1241f4dc0 2024-04-26T20:51:48.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.682+0000 7fc12a2c7640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1100039b0 0x7fc1240c8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:48.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.682+0000 7fc12a2c7640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1100039b0 0x7fc1240c8640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54604/0 (socket says 172.21.15.73:54604) 2024-04-26T20:51:48.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.682+0000 7fc12a2c7640 1 -- 172.21.15.73:0/784122876 learned_addr learned my addr 172.21.15.73:0/784122876 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:48.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.682+0000 7fc12a2c7640 1 -- 172.21.15.73:0/784122876 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc1241f4dc0 msgr2=0x7fc1241f5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:48.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.682+0000 7fc12a2c7640 1 --2- 172.21.15.73:0/784122876 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc1241f4dc0 0x7fc1241f5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:48.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.682+0000 7fc129ac6640 1 --2- 172.21.15.73:0/784122876 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc12414eb80 0x7fc12414f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:48.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.682+0000 7fc12a2c7640 1 -- 172.21.15.73:0/784122876 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc12414eb80 msgr2=0x7fc12414f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:48.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.682+0000 7fc12a2c7640 1 --2- 172.21.15.73:0/784122876 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc12414eb80 0x7fc12414f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:48.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.682+0000 7fc12a2c7640 1 -- 172.21.15.73:0/784122876 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc12415a6b0 con 0x7fc1100039b0 2024-04-26T20:51:48.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.682+0000 7fc12a2c7640 1 --2- 172.21.15.73:0/784122876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1100039b0 0x7fc1240c8640 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fc120034040 tx=0x7fc120004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:48.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.683+0000 7fc11affd640 1 -- 172.21.15.73:0/784122876 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc120040020 con 0x7fc1100039b0 2024-04-26T20:51:48.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.683+0000 7fc11affd640 1 -- 172.21.15.73:0/784122876 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc120003ac0 con 0x7fc1100039b0 2024-04-26T20:51:48.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.683+0000 7fc11affd640 1 -- 172.21.15.73:0/784122876 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc120036040 con 0x7fc1100039b0 2024-04-26T20:51:48.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.683+0000 7fc12c52b640 1 -- 172.21.15.73:0/784122876 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc12415a950 con 0x7fc1100039b0 2024-04-26T20:51:48.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.683+0000 7fc12c52b640 1 -- 172.21.15.73:0/784122876 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc12415af10 con 0x7fc1100039b0 2024-04-26T20:51:48.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.684+0000 7fc12c52b640 1 -- 172.21.15.73:0/784122876 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1240694f0 con 0x7fc1100039b0 2024-04-26T20:51:48.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.684+0000 7fc11affd640 1 -- 172.21.15.73:0/784122876 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc120043020 con 0x7fc1100039b0 2024-04-26T20:51:48.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.684+0000 7fc11affd640 1 --2- 172.21.15.73:0/784122876 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc10806c8f0 0x7fc10806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:48.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.684+0000 7fc11affd640 1 -- 172.21.15.73:0/784122876 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fc1200abe80 con 0x7fc1100039b0 2024-04-26T20:51:48.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.684+0000 7fc129ac6640 1 --2- 172.21.15.73:0/784122876 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc10806c8f0 0x7fc10806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:48.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.685+0000 7fc129ac6640 1 --2- 172.21.15.73:0/784122876 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc10806c8f0 0x7fc10806edb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc1240b3810 tx=0x7fc114009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:48.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.689+0000 7fc11affd640 1 -- 172.21.15.73:0/784122876 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc12004a070 con 0x7fc1100039b0 2024-04-26T20:51:48.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.860+0000 7fc12c52b640 1 -- 172.21.15.73:0/784122876 --> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc12414f540 con 0x7fc10806c8f0 2024-04-26T20:51:48.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.866+0000 7fc11affd640 1 -- 172.21.15.73:0/784122876 <== mgr.4107 v2:172.21.15.73:6832/666905549 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246635 (secure 0 0 0) 0x7fc12414f540 con 0x7fc10806c8f0 2024-04-26T20:51:48.869 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:51:48.871 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-26T20:51:48.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.869+0000 7fc12c52b640 1 -- 172.21.15.73:0/784122876 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc10806c8f0 msgr2=0x7fc10806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:48.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.869+0000 7fc12c52b640 1 --2- 172.21.15.73:0/784122876 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc10806c8f0 0x7fc10806edb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc1240b3810 tx=0x7fc114009040 comp rx=0 tx=0).stop 2024-04-26T20:51:48.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.869+0000 7fc12c52b640 1 -- 172.21.15.73:0/784122876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1100039b0 msgr2=0x7fc1240c8640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:48.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.869+0000 7fc12c52b640 1 --2- 172.21.15.73:0/784122876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1100039b0 0x7fc1240c8640 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fc120034040 tx=0x7fc120004730 comp rx=0 tx=0).stop 2024-04-26T20:51:48.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.869+0000 7fc12c52b640 1 -- 172.21.15.73:0/784122876 shutdown_connections 2024-04-26T20:51:48.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.869+0000 7fc12c52b640 1 --2- 172.21.15.73:0/784122876 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc10806c8f0 0x7fc10806edb0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:48.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.869+0000 7fc12c52b640 1 --2- 172.21.15.73:0/784122876 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc1241f4dc0 0x7fc1241f5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:48.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.869+0000 7fc12c52b640 1 --2- 172.21.15.73:0/784122876 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc12414eb80 0x7fc12414f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:48.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.869+0000 7fc12c52b640 1 --2- 172.21.15.73:0/784122876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1100039b0 0x7fc1240c8640 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:48.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.869+0000 7fc12c52b640 1 -- 172.21.15.73:0/784122876 >> 172.21.15.73:0/784122876 conn(0x7fc1240b3180 msgr2=0x7fc1240b1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:48.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.869+0000 7fc12c52b640 1 -- 172.21.15.73:0/784122876 shutdown_connections 2024-04-26T20:51:48.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:48.869+0000 7fc12c52b640 1 -- 172.21.15.73:0/784122876 wait complete. 2024-04-26T20:51:48.880 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":76,"stamp":"2024-04-26T20:51:47.997420+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220552,"kb_used_data":4392,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754168,"statfs":{"total":773094113280,"available":772868268032,"internally_reserved":0,"allocated":4497408,"data_stored":2593784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.078798"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.927134+0000","last_change":"2024-04-26T20:51:44.927134+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.927134+0000","last_undegraded":"2024-04-26T20:51:44.927134+0000","last_fullsized":"2024-04-26T20:51:44.927134+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.927081+0000","last_change":"2024-04-26T20:51:44.927081+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.927081+0000","last_undegraded":"2024-04-26T20:51:44.927081+0000","last_fullsized":"2024-04-26T20:51:44.927081+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.930409+0000","last_change":"2024-04-26T20:51:44.930409+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.930409+0000","last_undegraded":"2024-04-26T20:51:44.930409+0000","last_fullsized":"2024-04-26T20:51:44.930409+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.926209+0000","last_change":"2024-04-26T20:51:44.926209+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.926209+0000","last_undegraded":"2024-04-26T20:51:44.926209+0000","last_fullsized":"2024-04-26T20:51:44.926209+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.925479+0000","last_change":"2024-04-26T20:51:44.925479+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.925479+0000","last_undegraded":"2024-04-26T20:51:44.925479+0000","last_fullsized":"2024-04-26T20:51:44.925479+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.934070+0000","last_change":"2024-04-26T20:51:44.934070+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.934070+0000","last_undegraded":"2024-04-26T20:51:44.934070+0000","last_fullsized":"2024-04-26T20:51:44.934070+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.930809+0000","last_change":"2024-04-26T20:51:44.930809+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.930809+0000","last_undegraded":"2024-04-26T20:51:44.930809+0000","last_fullsized":"2024-04-26T20:51:44.930809+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.926467+0000","last_change":"2024-04-26T20:51:44.926467+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.926467+0000","last_undegraded":"2024-04-26T20:51:44.926467+0000","last_fullsized":"2024-04-26T20:51:44.926467+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.929217+0000","last_change":"2024-04-26T20:51:44.929217+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.929217+0000","last_undegraded":"2024-04-26T20:51:44.929217+0000","last_fullsized":"2024-04-26T20:51:44.929217+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.930277+0000","last_change":"2024-04-26T20:51:44.930277+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.930277+0000","last_undegraded":"2024-04-26T20:51:44.930277+0000","last_fullsized":"2024-04-26T20:51:44.930277+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.930633+0000","last_change":"2024-04-26T20:51:44.930633+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.930633+0000","last_undegraded":"2024-04-26T20:51:44.930633+0000","last_fullsized":"2024-04-26T20:51:44.930633+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.929572+0000","last_change":"2024-04-26T20:51:44.929572+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.929572+0000","last_undegraded":"2024-04-26T20:51:44.929572+0000","last_fullsized":"2024-04-26T20:51:44.929572+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.929667+0000","last_change":"2024-04-26T20:51:44.929667+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.929667+0000","last_undegraded":"2024-04-26T20:51:44.929667+0000","last_fullsized":"2024-04-26T20:51:44.929667+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.930603+0000","last_change":"2024-04-26T20:51:44.930603+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.930603+0000","last_undegraded":"2024-04-26T20:51:44.930603+0000","last_fullsized":"2024-04-26T20:51:44.930603+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.929584+0000","last_change":"2024-04-26T20:51:44.929584+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.929584+0000","last_undegraded":"2024-04-26T20:51:44.929584+0000","last_fullsized":"2024-04-26T20:51:44.929584+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.934714+0000","last_change":"2024-04-26T20:51:44.934714+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.934714+0000","last_undegraded":"2024-04-26T20:51:44.934714+0000","last_fullsized":"2024-04-26T20:51:44.934714+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.933252+0000","last_change":"2024-04-26T20:51:44.933252+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.933252+0000","last_undegraded":"2024-04-26T20:51:44.933252+0000","last_fullsized":"2024-04-26T20:51:44.933252+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.923957+0000","last_change":"2024-04-26T20:51:44.923957+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.923957+0000","last_undegraded":"2024-04-26T20:51:44.923957+0000","last_fullsized":"2024-04-26T20:51:44.923957+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.926243+0000","last_change":"2024-04-26T20:51:44.926243+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.926243+0000","last_undegraded":"2024-04-26T20:51:44.926243+0000","last_fullsized":"2024-04-26T20:51:44.926243+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.924107+0000","last_change":"2024-04-26T20:51:44.924107+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.924107+0000","last_undegraded":"2024-04-26T20:51:44.924107+0000","last_fullsized":"2024-04-26T20:51:44.924107+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.927282+0000","last_change":"2024-04-26T20:51:44.927282+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.927282+0000","last_undegraded":"2024-04-26T20:51:44.927282+0000","last_fullsized":"2024-04-26T20:51:44.927282+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.928722+0000","last_change":"2024-04-26T20:51:44.928722+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.928722+0000","last_undegraded":"2024-04-26T20:51:44.928722+0000","last_fullsized":"2024-04-26T20:51:44.928722+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.934479+0000","last_change":"2024-04-26T20:51:44.934479+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.934479+0000","last_undegraded":"2024-04-26T20:51:44.934479+0000","last_fullsized":"2024-04-26T20:51:44.934479+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.925511+0000","last_change":"2024-04-26T20:51:44.925511+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.925511+0000","last_undegraded":"2024-04-26T20:51:44.925511+0000","last_fullsized":"2024-04-26T20:51:44.925511+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.923419+0000","last_change":"2024-04-26T20:51:44.923419+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.923419+0000","last_undegraded":"2024-04-26T20:51:44.923419+0000","last_fullsized":"2024-04-26T20:51:44.923419+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.930697+0000","last_change":"2024-04-26T20:51:44.930697+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.930697+0000","last_undegraded":"2024-04-26T20:51:44.930697+0000","last_fullsized":"2024-04-26T20:51:44.930697+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.923397+0000","last_change":"2024-04-26T20:51:44.923397+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.923397+0000","last_undegraded":"2024-04-26T20:51:44.923397+0000","last_fullsized":"2024-04-26T20:51:44.923397+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.926829+0000","last_change":"2024-04-26T20:51:44.926829+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.926829+0000","last_undegraded":"2024-04-26T20:51:44.926829+0000","last_fullsized":"2024-04-26T20:51:44.926829+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.923542+0000","last_change":"2024-04-26T20:51:44.923542+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.923542+0000","last_undegraded":"2024-04-26T20:51:44.923542+0000","last_fullsized":"2024-04-26T20:51:44.923542+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.923673+0000","last_change":"2024-04-26T20:51:44.923673+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.923673+0000","last_undegraded":"2024-04-26T20:51:44.923673+0000","last_fullsized":"2024-04-26T20:51:44.923673+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.932216+0000","last_change":"2024-04-26T20:51:44.932216+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.932216+0000","last_undegraded":"2024-04-26T20:51:44.932216+0000","last_fullsized":"2024-04-26T20:51:44.932216+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.930954+0000","last_change":"2024-04-26T20:51:44.930954+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.930954+0000","last_undegraded":"2024-04-26T20:51:44.930954+0000","last_fullsized":"2024-04-26T20:51:44.930954+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.932301+0000","last_change":"2024-04-26T20:51:44.932301+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.932301+0000","last_undegraded":"2024-04-26T20:51:44.932301+0000","last_fullsized":"2024-04-26T20:51:44.932301+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.936717+0000","last_change":"2024-04-26T20:51:44.935237+0000","last_active":"2024-04-26T20:51:44.936717+0000","last_peered":"2024-04-26T20:51:44.936717+0000","last_clean":"2024-04-26T20:51:44.936717+0000","last_became_active":"2024-04-26T20:51:44.934964+0000","last_became_peered":"2024-04-26T20:51:44.934964+0000","last_unstale":"2024-04-26T20:51:44.936717+0000","last_undegraded":"2024-04-26T20:51:44.936717+0000","last_fullsized":"2024-04-26T20:51:44.936717+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:22.556226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.934004+0000","last_change":"2024-04-26T20:51:44.934004+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.934004+0000","last_undegraded":"2024-04-26T20:51:44.934004+0000","last_fullsized":"2024-04-26T20:51:44.934004+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.940096+0000","last_change":"2024-04-26T20:51:44.937990+0000","last_active":"2024-04-26T20:51:44.940096+0000","last_peered":"2024-04-26T20:51:44.940096+0000","last_clean":"2024-04-26T20:51:44.940096+0000","last_became_active":"2024-04-26T20:51:44.937723+0000","last_became_peered":"2024-04-26T20:51:44.937723+0000","last_unstale":"2024-04-26T20:51:44.940096+0000","last_undegraded":"2024-04-26T20:51:44.940096+0000","last_fullsized":"2024-04-26T20:51:44.940096+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:48:23.398632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.928437+0000","last_change":"2024-04-26T20:51:44.928437+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.928437+0000","last_undegraded":"2024-04-26T20:51:44.928437+0000","last_fullsized":"2024-04-26T20:51:44.928437+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.943035+0000","last_change":"2024-04-26T20:51:44.939632+0000","last_active":"2024-04-26T20:51:44.943035+0000","last_peered":"2024-04-26T20:51:44.943035+0000","last_clean":"2024-04-26T20:51:44.943035+0000","last_became_active":"2024-04-26T20:51:44.939299+0000","last_became_peered":"2024-04-26T20:51:44.939299+0000","last_unstale":"2024-04-26T20:51:44.943035+0000","last_undegraded":"2024-04-26T20:51:44.943035+0000","last_fullsized":"2024-04-26T20:51:44.943035+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:35.270443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.925851+0000","last_change":"2024-04-26T20:51:44.925851+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.925851+0000","last_undegraded":"2024-04-26T20:51:44.925851+0000","last_fullsized":"2024-04-26T20:51:44.925851+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.939345+0000","last_change":"2024-04-26T20:51:44.937707+0000","last_active":"2024-04-26T20:51:44.939345+0000","last_peered":"2024-04-26T20:51:44.939345+0000","last_clean":"2024-04-26T20:51:44.939345+0000","last_became_active":"2024-04-26T20:51:44.937394+0000","last_became_peered":"2024-04-26T20:51:44.937394+0000","last_unstale":"2024-04-26T20:51:44.939345+0000","last_undegraded":"2024-04-26T20:51:44.939345+0000","last_fullsized":"2024-04-26T20:51:44.939345+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:14:24.515988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.930643+0000","last_change":"2024-04-26T20:51:44.930643+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.930643+0000","last_undegraded":"2024-04-26T20:51:44.930643+0000","last_fullsized":"2024-04-26T20:51:44.930643+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.939732+0000","last_change":"2024-04-26T20:51:44.937929+0000","last_active":"2024-04-26T20:51:44.939732+0000","last_peered":"2024-04-26T20:51:44.939732+0000","last_clean":"2024-04-26T20:51:44.939732+0000","last_became_active":"2024-04-26T20:51:44.937626+0000","last_became_peered":"2024-04-26T20:51:44.937626+0000","last_unstale":"2024-04-26T20:51:44.939732+0000","last_undegraded":"2024-04-26T20:51:44.939732+0000","last_fullsized":"2024-04-26T20:51:44.939732+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:48:23.398632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.926384+0000","last_change":"2024-04-26T20:51:44.926384+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.926384+0000","last_undegraded":"2024-04-26T20:51:44.926384+0000","last_fullsized":"2024-04-26T20:51:44.926384+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.945361+0000","last_change":"2024-04-26T20:51:44.944114+0000","last_active":"2024-04-26T20:51:44.945361+0000","last_peered":"2024-04-26T20:51:44.945361+0000","last_clean":"2024-04-26T20:51:44.945361+0000","last_became_active":"2024-04-26T20:51:44.943821+0000","last_became_peered":"2024-04-26T20:51:44.943821+0000","last_unstale":"2024-04-26T20:51:44.945361+0000","last_undegraded":"2024-04-26T20:51:44.945361+0000","last_fullsized":"2024-04-26T20:51:44.945361+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:28:21.811767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.927913+0000","last_change":"2024-04-26T20:51:44.927913+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.927913+0000","last_undegraded":"2024-04-26T20:51:44.927913+0000","last_fullsized":"2024-04-26T20:51:44.927913+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.943098+0000","last_change":"2024-04-26T20:51:44.942003+0000","last_active":"2024-04-26T20:51:44.943098+0000","last_peered":"2024-04-26T20:51:44.943098+0000","last_clean":"2024-04-26T20:51:44.943098+0000","last_became_active":"2024-04-26T20:51:44.941646+0000","last_became_peered":"2024-04-26T20:51:44.941646+0000","last_unstale":"2024-04-26T20:51:44.943098+0000","last_undegraded":"2024-04-26T20:51:44.943098+0000","last_fullsized":"2024-04-26T20:51:44.943098+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:14:24.515988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.926719+0000","last_change":"2024-04-26T20:51:44.926719+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.926719+0000","last_undegraded":"2024-04-26T20:51:44.926719+0000","last_fullsized":"2024-04-26T20:51:44.926719+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.936220+0000","last_change":"2024-04-26T20:51:44.934491+0000","last_active":"2024-04-26T20:51:44.936220+0000","last_peered":"2024-04-26T20:51:44.936220+0000","last_clean":"2024-04-26T20:51:44.936220+0000","last_became_active":"2024-04-26T20:51:44.934145+0000","last_became_peered":"2024-04-26T20:51:44.934145+0000","last_unstale":"2024-04-26T20:51:44.936220+0000","last_undegraded":"2024-04-26T20:51:44.936220+0000","last_fullsized":"2024-04-26T20:51:44.936220+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:48:23.398632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.925518+0000","last_change":"2024-04-26T20:51:44.925518+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.925518+0000","last_undegraded":"2024-04-26T20:51:44.925518+0000","last_fullsized":"2024-04-26T20:51:44.925518+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.941952+0000","last_change":"2024-04-26T20:51:44.938418+0000","last_active":"2024-04-26T20:51:44.941952+0000","last_peered":"2024-04-26T20:51:44.941952+0000","last_clean":"2024-04-26T20:51:44.941952+0000","last_became_active":"2024-04-26T20:51:44.938146+0000","last_became_peered":"2024-04-26T20:51:44.938146+0000","last_unstale":"2024-04-26T20:51:44.941952+0000","last_undegraded":"2024-04-26T20:51:44.941952+0000","last_fullsized":"2024-04-26T20:51:44.941952+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:44.088012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.940376+0000","last_change":"2024-04-26T20:51:44.940376+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.940376+0000","last_undegraded":"2024-04-26T20:51:44.940376+0000","last_fullsized":"2024-04-26T20:51:44.940376+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.941993+0000","last_change":"2024-04-26T20:51:44.941093+0000","last_active":"2024-04-26T20:51:44.941993+0000","last_peered":"2024-04-26T20:51:44.941993+0000","last_clean":"2024-04-26T20:51:44.941993+0000","last_became_active":"2024-04-26T20:51:44.940805+0000","last_became_peered":"2024-04-26T20:51:44.940805+0000","last_unstale":"2024-04-26T20:51:44.941993+0000","last_undegraded":"2024-04-26T20:51:44.941993+0000","last_fullsized":"2024-04-26T20:51:44.941993+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:10:56.604114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.930862+0000","last_change":"2024-04-26T20:51:44.930862+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.930862+0000","last_undegraded":"2024-04-26T20:51:44.930862+0000","last_fullsized":"2024-04-26T20:51:44.930862+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.939847+0000","last_change":"2024-04-26T20:51:44.939214+0000","last_active":"2024-04-26T20:51:44.939847+0000","last_peered":"2024-04-26T20:51:44.939847+0000","last_clean":"2024-04-26T20:51:44.939847+0000","last_became_active":"2024-04-26T20:51:44.938907+0000","last_became_peered":"2024-04-26T20:51:44.938907+0000","last_unstale":"2024-04-26T20:51:44.939847+0000","last_undegraded":"2024-04-26T20:51:44.939847+0000","last_fullsized":"2024-04-26T20:51:44.939847+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:17.661527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.930135+0000","last_change":"2024-04-26T20:51:44.930135+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.930135+0000","last_undegraded":"2024-04-26T20:51:44.930135+0000","last_fullsized":"2024-04-26T20:51:44.930135+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.942052+0000","last_change":"2024-04-26T20:51:44.939371+0000","last_active":"2024-04-26T20:51:44.942052+0000","last_peered":"2024-04-26T20:51:44.942052+0000","last_clean":"2024-04-26T20:51:44.942052+0000","last_became_active":"2024-04-26T20:51:44.939090+0000","last_became_peered":"2024-04-26T20:51:44.939090+0000","last_unstale":"2024-04-26T20:51:44.942052+0000","last_undegraded":"2024-04-26T20:51:44.942052+0000","last_fullsized":"2024-04-26T20:51:44.942052+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:40:59.156357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.930280+0000","last_change":"2024-04-26T20:51:44.930280+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.930280+0000","last_undegraded":"2024-04-26T20:51:44.930280+0000","last_fullsized":"2024-04-26T20:51:44.930280+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.942877+0000","last_change":"2024-04-26T20:51:44.940477+0000","last_active":"2024-04-26T20:51:44.942877+0000","last_peered":"2024-04-26T20:51:44.942877+0000","last_clean":"2024-04-26T20:51:44.942877+0000","last_became_active":"2024-04-26T20:51:44.940192+0000","last_became_peered":"2024-04-26T20:51:44.940192+0000","last_unstale":"2024-04-26T20:51:44.942877+0000","last_undegraded":"2024-04-26T20:51:44.942877+0000","last_fullsized":"2024-04-26T20:51:44.942877+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:38:31.038452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.926418+0000","last_change":"2024-04-26T20:51:44.926418+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.926418+0000","last_undegraded":"2024-04-26T20:51:44.926418+0000","last_fullsized":"2024-04-26T20:51:44.926418+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.943067+0000","last_change":"2024-04-26T20:51:44.940382+0000","last_active":"2024-04-26T20:51:44.943067+0000","last_peered":"2024-04-26T20:51:44.943067+0000","last_clean":"2024-04-26T20:51:44.943067+0000","last_became_active":"2024-04-26T20:51:44.940109+0000","last_became_peered":"2024-04-26T20:51:44.940109+0000","last_unstale":"2024-04-26T20:51:44.943067+0000","last_undegraded":"2024-04-26T20:51:44.943067+0000","last_fullsized":"2024-04-26T20:51:44.943067+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:48:23.398632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.932149+0000","last_change":"2024-04-26T20:51:44.932149+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.932149+0000","last_undegraded":"2024-04-26T20:51:44.932149+0000","last_fullsized":"2024-04-26T20:51:44.932149+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.936667+0000","last_change":"2024-04-26T20:51:44.934857+0000","last_active":"2024-04-26T20:51:44.936667+0000","last_peered":"2024-04-26T20:51:44.936667+0000","last_clean":"2024-04-26T20:51:44.936667+0000","last_became_active":"2024-04-26T20:51:44.934556+0000","last_became_peered":"2024-04-26T20:51:44.934556+0000","last_unstale":"2024-04-26T20:51:44.936667+0000","last_undegraded":"2024-04-26T20:51:44.936667+0000","last_fullsized":"2024-04-26T20:51:44.936667+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:04:31.671170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.925836+0000","last_change":"2024-04-26T20:51:44.925836+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.925836+0000","last_undegraded":"2024-04-26T20:51:44.925836+0000","last_fullsized":"2024-04-26T20:51:44.925836+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.944907+0000","last_change":"2024-04-26T20:51:44.943632+0000","last_active":"2024-04-26T20:51:44.944907+0000","last_peered":"2024-04-26T20:51:44.944907+0000","last_clean":"2024-04-26T20:51:44.944907+0000","last_became_active":"2024-04-26T20:51:44.943278+0000","last_became_peered":"2024-04-26T20:51:44.943278+0000","last_unstale":"2024-04-26T20:51:44.944907+0000","last_undegraded":"2024-04-26T20:51:44.944907+0000","last_fullsized":"2024-04-26T20:51:44.944907+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:59:50.250119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.926462+0000","last_change":"2024-04-26T20:51:44.926462+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.926462+0000","last_undegraded":"2024-04-26T20:51:44.926462+0000","last_fullsized":"2024-04-26T20:51:44.926462+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.937919+0000","last_change":"2024-04-26T20:51:44.935150+0000","last_active":"2024-04-26T20:51:44.937919+0000","last_peered":"2024-04-26T20:51:44.937919+0000","last_clean":"2024-04-26T20:51:44.937919+0000","last_became_active":"2024-04-26T20:51:44.934078+0000","last_became_peered":"2024-04-26T20:51:44.934078+0000","last_unstale":"2024-04-26T20:51:44.937919+0000","last_undegraded":"2024-04-26T20:51:44.937919+0000","last_fullsized":"2024-04-26T20:51:44.937919+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:35.270443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.926645+0000","last_change":"2024-04-26T20:51:44.926645+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.926645+0000","last_undegraded":"2024-04-26T20:51:44.926645+0000","last_fullsized":"2024-04-26T20:51:44.926645+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.934725+0000","last_change":"2024-04-26T20:51:44.932407+0000","last_active":"2024-04-26T20:51:44.934725+0000","last_peered":"2024-04-26T20:51:44.934725+0000","last_clean":"2024-04-26T20:51:44.934725+0000","last_became_active":"2024-04-26T20:51:44.931987+0000","last_became_peered":"2024-04-26T20:51:44.931987+0000","last_unstale":"2024-04-26T20:51:44.934725+0000","last_undegraded":"2024-04-26T20:51:44.934725+0000","last_fullsized":"2024-04-26T20:51:44.934725+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:36:55.165610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.929390+0000","last_change":"2024-04-26T20:51:44.929390+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.929390+0000","last_undegraded":"2024-04-26T20:51:44.929390+0000","last_fullsized":"2024-04-26T20:51:44.929390+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.943664+0000","last_change":"2024-04-26T20:51:44.941836+0000","last_active":"2024-04-26T20:51:44.943664+0000","last_peered":"2024-04-26T20:51:44.943664+0000","last_clean":"2024-04-26T20:51:44.943664+0000","last_became_active":"2024-04-26T20:51:44.941581+0000","last_became_peered":"2024-04-26T20:51:44.941581+0000","last_unstale":"2024-04-26T20:51:44.943664+0000","last_undegraded":"2024-04-26T20:51:44.943664+0000","last_fullsized":"2024-04-26T20:51:44.943664+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:30:26.145171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.924912+0000","last_change":"2024-04-26T20:51:44.924912+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.924912+0000","last_undegraded":"2024-04-26T20:51:44.924912+0000","last_fullsized":"2024-04-26T20:51:44.924912+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.942287+0000","last_change":"2024-04-26T20:51:44.938146+0000","last_active":"2024-04-26T20:51:44.942287+0000","last_peered":"2024-04-26T20:51:44.942287+0000","last_clean":"2024-04-26T20:51:44.942287+0000","last_became_active":"2024-04-26T20:51:44.937861+0000","last_became_peered":"2024-04-26T20:51:44.937861+0000","last_unstale":"2024-04-26T20:51:44.942287+0000","last_undegraded":"2024-04-26T20:51:44.942287+0000","last_fullsized":"2024-04-26T20:51:44.942287+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:36:55.165610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.932596+0000","last_change":"2024-04-26T20:51:44.932596+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.932596+0000","last_undegraded":"2024-04-26T20:51:44.932596+0000","last_fullsized":"2024-04-26T20:51:44.932596+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.941851+0000","last_change":"2024-04-26T20:51:44.940796+0000","last_active":"2024-04-26T20:51:44.941851+0000","last_peered":"2024-04-26T20:51:44.941851+0000","last_clean":"2024-04-26T20:51:44.941851+0000","last_became_active":"2024-04-26T20:51:44.940524+0000","last_became_peered":"2024-04-26T20:51:44.940524+0000","last_unstale":"2024-04-26T20:51:44.941851+0000","last_undegraded":"2024-04-26T20:51:44.941851+0000","last_fullsized":"2024-04-26T20:51:44.941851+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:31:06.346238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.929053+0000","last_change":"2024-04-26T20:51:44.929053+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.929053+0000","last_undegraded":"2024-04-26T20:51:44.929053+0000","last_fullsized":"2024-04-26T20:51:44.929053+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.939800+0000","last_change":"2024-04-26T20:51:44.934985+0000","last_active":"2024-04-26T20:51:44.939800+0000","last_peered":"2024-04-26T20:51:44.939800+0000","last_clean":"2024-04-26T20:51:44.939800+0000","last_became_active":"2024-04-26T20:51:44.934690+0000","last_became_peered":"2024-04-26T20:51:44.934690+0000","last_unstale":"2024-04-26T20:51:44.939800+0000","last_undegraded":"2024-04-26T20:51:44.939800+0000","last_fullsized":"2024-04-26T20:51:44.939800+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:02.518303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.926386+0000","last_change":"2024-04-26T20:51:44.926386+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.926386+0000","last_undegraded":"2024-04-26T20:51:44.926386+0000","last_fullsized":"2024-04-26T20:51:44.926386+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.945415+0000","last_change":"2024-04-26T20:51:44.944446+0000","last_active":"2024-04-26T20:51:44.945415+0000","last_peered":"2024-04-26T20:51:44.945415+0000","last_clean":"2024-04-26T20:51:44.945415+0000","last_became_active":"2024-04-26T20:51:44.944147+0000","last_became_peered":"2024-04-26T20:51:44.944147+0000","last_unstale":"2024-04-26T20:51:44.945415+0000","last_undegraded":"2024-04-26T20:51:44.945415+0000","last_fullsized":"2024-04-26T20:51:44.945415+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:16:58.774484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.926312+0000","last_change":"2024-04-26T20:51:44.926312+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.926312+0000","last_undegraded":"2024-04-26T20:51:44.926312+0000","last_fullsized":"2024-04-26T20:51:44.926312+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.939834+0000","last_change":"2024-04-26T20:51:44.932932+0000","last_active":"2024-04-26T20:51:44.939834+0000","last_peered":"2024-04-26T20:51:44.939834+0000","last_clean":"2024-04-26T20:51:44.939834+0000","last_became_active":"2024-04-26T20:51:44.932657+0000","last_became_peered":"2024-04-26T20:51:44.932657+0000","last_unstale":"2024-04-26T20:51:44.939834+0000","last_undegraded":"2024-04-26T20:51:44.939834+0000","last_fullsized":"2024-04-26T20:51:44.939834+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:14:17.119338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.931512+0000","last_change":"2024-04-26T20:51:44.931512+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.931512+0000","last_undegraded":"2024-04-26T20:51:44.931512+0000","last_fullsized":"2024-04-26T20:51:44.931512+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.939843+0000","last_change":"2024-04-26T20:51:44.938761+0000","last_active":"2024-04-26T20:51:44.939843+0000","last_peered":"2024-04-26T20:51:44.939843+0000","last_clean":"2024-04-26T20:51:44.939843+0000","last_became_active":"2024-04-26T20:51:44.938483+0000","last_became_peered":"2024-04-26T20:51:44.938483+0000","last_unstale":"2024-04-26T20:51:44.939843+0000","last_undegraded":"2024-04-26T20:51:44.939843+0000","last_fullsized":"2024-04-26T20:51:44.939843+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:14:24.515988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.938640+0000","last_change":"2024-04-26T20:51:44.938640+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.938640+0000","last_undegraded":"2024-04-26T20:51:44.938640+0000","last_fullsized":"2024-04-26T20:51:44.938640+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.939747+0000","last_change":"2024-04-26T20:51:44.932513+0000","last_active":"2024-04-26T20:51:44.939747+0000","last_peered":"2024-04-26T20:51:44.939747+0000","last_clean":"2024-04-26T20:51:44.939747+0000","last_became_active":"2024-04-26T20:51:44.932223+0000","last_became_peered":"2024-04-26T20:51:44.932223+0000","last_unstale":"2024-04-26T20:51:44.939747+0000","last_undegraded":"2024-04-26T20:51:44.939747+0000","last_fullsized":"2024-04-26T20:51:44.939747+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:28:26.304960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.926513+0000","last_change":"2024-04-26T20:51:44.926513+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.926513+0000","last_undegraded":"2024-04-26T20:51:44.926513+0000","last_fullsized":"2024-04-26T20:51:44.926513+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.940605+0000","last_change":"2024-04-26T20:51:44.938966+0000","last_active":"2024-04-26T20:51:44.940605+0000","last_peered":"2024-04-26T20:51:44.940605+0000","last_clean":"2024-04-26T20:51:44.940605+0000","last_became_active":"2024-04-26T20:51:44.938662+0000","last_became_peered":"2024-04-26T20:51:44.938662+0000","last_unstale":"2024-04-26T20:51:44.940605+0000","last_undegraded":"2024-04-26T20:51:44.940605+0000","last_fullsized":"2024-04-26T20:51:44.940605+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:14:24.515988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.929938+0000","last_change":"2024-04-26T20:51:44.929938+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.929938+0000","last_undegraded":"2024-04-26T20:51:44.929938+0000","last_fullsized":"2024-04-26T20:51:44.929938+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.939612+0000","last_change":"2024-04-26T20:51:44.935952+0000","last_active":"2024-04-26T20:51:44.939612+0000","last_peered":"2024-04-26T20:51:44.939612+0000","last_clean":"2024-04-26T20:51:44.939612+0000","last_became_active":"2024-04-26T20:51:44.935682+0000","last_became_peered":"2024-04-26T20:51:44.935682+0000","last_unstale":"2024-04-26T20:51:44.939612+0000","last_undegraded":"2024-04-26T20:51:44.939612+0000","last_fullsized":"2024-04-26T20:51:44.939612+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:35.270443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.933638+0000","last_change":"2024-04-26T20:51:44.933638+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.933638+0000","last_undegraded":"2024-04-26T20:51:44.933638+0000","last_fullsized":"2024-04-26T20:51:44.933638+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.933704+0000","last_change":"2024-04-26T20:51:44.932688+0000","last_active":"2024-04-26T20:51:44.933704+0000","last_peered":"2024-04-26T20:51:44.933704+0000","last_clean":"2024-04-26T20:51:44.933704+0000","last_became_active":"2024-04-26T20:51:44.932417+0000","last_became_peered":"2024-04-26T20:51:44.932417+0000","last_unstale":"2024-04-26T20:51:44.933704+0000","last_undegraded":"2024-04-26T20:51:44.933704+0000","last_fullsized":"2024-04-26T20:51:44.933704+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:55:14.475567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.930210+0000","last_change":"2024-04-26T20:51:44.930210+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.930210+0000","last_undegraded":"2024-04-26T20:51:44.930210+0000","last_fullsized":"2024-04-26T20:51:44.930210+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.937168+0000","last_change":"2024-04-26T20:51:44.934242+0000","last_active":"2024-04-26T20:51:44.937168+0000","last_peered":"2024-04-26T20:51:44.937168+0000","last_clean":"2024-04-26T20:51:44.937168+0000","last_became_active":"2024-04-26T20:51:44.933449+0000","last_became_peered":"2024-04-26T20:51:44.933449+0000","last_unstale":"2024-04-26T20:51:44.937168+0000","last_undegraded":"2024-04-26T20:51:44.937168+0000","last_fullsized":"2024-04-26T20:51:44.937168+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:35.270443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.932718+0000","last_change":"2024-04-26T20:51:44.932718+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.932718+0000","last_undegraded":"2024-04-26T20:51:44.932718+0000","last_fullsized":"2024-04-26T20:51:44.932718+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.940384+0000","last_change":"2024-04-26T20:51:44.934065+0000","last_active":"2024-04-26T20:51:44.940384+0000","last_peered":"2024-04-26T20:51:44.940384+0000","last_clean":"2024-04-26T20:51:44.940384+0000","last_became_active":"2024-04-26T20:51:44.933763+0000","last_became_peered":"2024-04-26T20:51:44.933763+0000","last_unstale":"2024-04-26T20:51:44.940384+0000","last_undegraded":"2024-04-26T20:51:44.940384+0000","last_fullsized":"2024-04-26T20:51:44.940384+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:46:52.657100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.927313+0000","last_change":"2024-04-26T20:51:44.927313+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.927313+0000","last_undegraded":"2024-04-26T20:51:44.927313+0000","last_fullsized":"2024-04-26T20:51:44.927313+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.946013+0000","last_change":"2024-04-26T20:51:44.944770+0000","last_active":"2024-04-26T20:51:44.946013+0000","last_peered":"2024-04-26T20:51:44.946013+0000","last_clean":"2024-04-26T20:51:44.946013+0000","last_became_active":"2024-04-26T20:51:44.944478+0000","last_became_peered":"2024-04-26T20:51:44.944478+0000","last_unstale":"2024-04-26T20:51:44.946013+0000","last_undegraded":"2024-04-26T20:51:44.946013+0000","last_fullsized":"2024-04-26T20:51:44.946013+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:37.193221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":199,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.941219+0000","last_change":"2024-04-26T20:50:21.150132+0000","last_active":"2024-04-26T20:51:44.941219+0000","last_peered":"2024-04-26T20:51:44.941219+0000","last_clean":"2024-04-26T20:51:44.941219+0000","last_became_active":"2024-04-26T20:50:21.147439+0000","last_became_peered":"2024-04-26T20:50:21.147439+0000","last_unstale":"2024-04-26T20:51:44.941219+0000","last_undegraded":"2024-04-26T20:51:44.941219+0000","last_fullsized":"2024-04-26T20:51:44.941219+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:20.084499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:20.084499+0000","last_clean_scrub_stamp":"2024-04-26T20:50:20.084499+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:19:29.395477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":1007616,"data_stored":766999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":1007616,"data_stored":766999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:51:48.885 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:51:49.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.016+0000 7f9881c37640 1 Processor -- start 2024-04-26T20:51:49.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.016+0000 7f9881c37640 1 -- start start 2024-04-26T20:51:49.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.017+0000 7f9881c37640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f987c0b2340 0x7f987c0cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:49.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.017+0000 7f9881c37640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f987c0cd930 0x7f987c0d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:49.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.017+0000 7f9881c37640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f987c0d10d0 con 0x7f987c162810 2024-04-26T20:51:49.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.017+0000 7f9881c37640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f987c0d0750 con 0x7f987c0cd930 2024-04-26T20:51:49.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.017+0000 7f9881c37640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f987c0d16a0 con 0x7f987c0b2340 2024-04-26T20:51:49.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.017+0000 7f987b7fe640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f987c162810 0x7f987c070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:41640/0 (socket says 172.21.15.73:41640) 2024-04-26T20:51:49.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.017+0000 7f987b7fe640 1 -- 172.21.15.73:0/864719463 learned_addr learned my addr 172.21.15.73:0/864719463 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:49.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.017+0000 7f987bfff640 1 --2- 172.21.15.73:0/864719463 >> v2:172.21.15.120:3300/0 conn(0x7f987c0cd930 0x7f987c0d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:49.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.017+0000 7f987affd640 1 --2- 172.21.15.73:0/864719463 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f987c0b2340 0x7f987c0cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:49.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.017+0000 7f987a7fc640 1 -- 172.21.15.73:0/864719463 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4101560658 0 0) 0x7f987c0d10d0 con 0x7f987c162810 2024-04-26T20:51:49.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.017+0000 7f987a7fc640 1 -- 172.21.15.73:0/864719463 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9864003520 con 0x7f987c162810 2024-04-26T20:51:49.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.017+0000 7f987a7fc640 1 -- 172.21.15.73:0/864719463 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 567357531 0 0) 0x7f9864003520 con 0x7f987c162810 2024-04-26T20:51:49.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.017+0000 7f987a7fc640 1 -- 172.21.15.73:0/864719463 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f987c0b2340 msgr2=0x7f987c0cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:49.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.017+0000 7f987a7fc640 1 --2- 172.21.15.73:0/864719463 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f987c0b2340 0x7f987c0cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:49.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.017+0000 7f987a7fc640 1 -- 172.21.15.73:0/864719463 >> v2:172.21.15.120:3300/0 conn(0x7f987c0cd930 msgr2=0x7f987c0d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:49.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.017+0000 7f987a7fc640 1 --2- 172.21.15.73:0/864719463 >> v2:172.21.15.120:3300/0 conn(0x7f987c0cd930 0x7f987c0d1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:49.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.017+0000 7f987a7fc640 1 -- 172.21.15.73:0/864719463 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f987c0d1370 con 0x7f987c162810 2024-04-26T20:51:49.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.017+0000 7f987a7fc640 1 -- 172.21.15.73:0/864719463 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7f986c001670 con 0x7f987c162810 2024-04-26T20:51:49.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.017+0000 7f987affd640 1 --2- 172.21.15.73:0/864719463 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f987c0b2340 0x7f987c0cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:51:49.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.017+0000 7f987bfff640 1 --2- 172.21.15.73:0/864719463 >> v2:172.21.15.120:3300/0 conn(0x7f987c0cd930 0x7f987c0d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:51:49.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.017+0000 7f987a7fc640 1 -- 172.21.15.73:0/864719463 >> v1:172.21.15.73:6789/0 conn(0x7f987c162810 legacy=0x7f987c070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:49.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.017+0000 7f987a7fc640 1 --2- 172.21.15.73:0/864719463 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98640039b0 0x7f9864003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:49.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.018+0000 7f987a7fc640 1 --2- 172.21.15.73:0/864719463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9864004800 0x7f9864004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:49.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.018+0000 7f987a7fc640 1 --2- 172.21.15.73:0/864719463 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f987c0b2340 0x7f98640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:49.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.018+0000 7f987a7fc640 1 -- 172.21.15.73:0/864719463 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f987c0d0750 con 0x7f9864004800 2024-04-26T20:51:49.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.018+0000 7f987a7fc640 1 -- 172.21.15.73:0/864719463 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f987c0d16a0 con 0x7f98640039b0 2024-04-26T20:51:49.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.018+0000 7f987a7fc640 1 -- 172.21.15.73:0/864719463 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f986400b9c0 con 0x7f987c0b2340 2024-04-26T20:51:49.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.018+0000 7f987affd640 1 --2- 172.21.15.73:0/864719463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9864004800 0x7f9864004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:49.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.018+0000 7f987b7fe640 1 --2- 172.21.15.73:0/864719463 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98640039b0 0x7f9864003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:49.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.018+0000 7f987bfff640 1 --2- 172.21.15.73:0/864719463 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f987c0b2340 0x7f98640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:49.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.018+0000 7f987affd640 1 -- 172.21.15.73:0/864719463 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f987c0b2340 msgr2=0x7f98640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:49.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.018+0000 7f987affd640 1 --2- 172.21.15.73:0/864719463 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f987c0b2340 0x7f98640051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:49.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.018+0000 7f987affd640 1 -- 172.21.15.73:0/864719463 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98640039b0 msgr2=0x7f9864003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:49.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.018+0000 7f987affd640 1 --2- 172.21.15.73:0/864719463 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98640039b0 0x7f9864003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:49.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.018+0000 7f987affd640 1 -- 172.21.15.73:0/864719463 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9864003520 con 0x7f9864004800 2024-04-26T20:51:49.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.019+0000 7f987affd640 1 --2- 172.21.15.73:0/864719463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9864004800 0x7f9864004c80 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f9870009c90 tx=0x7f9864005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4328f76f60b2a05 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:49.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.019+0000 7f987a7fc640 1 -- 172.21.15.73:0/864719463 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9870036070 con 0x7f9864004800 2024-04-26T20:51:49.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.019+0000 7f987a7fc640 1 -- 172.21.15.73:0/864719463 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9870031400 con 0x7f9864004800 2024-04-26T20:51:49.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.019+0000 7f987a7fc640 1 -- 172.21.15.73:0/864719463 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9870009860 con 0x7f9864004800 2024-04-26T20:51:49.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.019+0000 7f9881c37640 1 -- 172.21.15.73:0/864719463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9864004800 msgr2=0x7f9864004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:49.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.019+0000 7f9881c37640 1 --2- 172.21.15.73:0/864719463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9864004800 0x7f9864004c80 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f9870009c90 tx=0x7f9864005830 comp rx=0 tx=0).stop 2024-04-26T20:51:49.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.019+0000 7f987bfff640 1 -- 172.21.15.73:0/864719463 reap_dead start 2024-04-26T20:51:49.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.019+0000 7f9881c37640 1 -- 172.21.15.73:0/864719463 shutdown_connections 2024-04-26T20:51:49.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.019+0000 7f9881c37640 1 -- 172.21.15.73:0/864719463 >> 172.21.15.73:0/864719463 conn(0x7f987c0d2da0 msgr2=0x7f987c0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:49.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.019+0000 7f9881c37640 1 -- 172.21.15.73:0/864719463 shutdown_connections 2024-04-26T20:51:49.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.019+0000 7f9881c37640 1 -- 172.21.15.73:0/864719463 wait complete. 2024-04-26T20:51:49.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.020+0000 7f9881c37640 1 Processor -- start 2024-04-26T20:51:49.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.021+0000 7f9881c37640 1 -- start start 2024-04-26T20:51:49.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.021+0000 7f9881c37640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9864004800 0x7f987c152450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:49.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.021+0000 7f9881c37640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f987c0c8cf0 0x7f987c0c9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:49.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.021+0000 7f9881c37640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f987c0cb8b0 0x7f987c0cbd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:49.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.021+0000 7f9881c37640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f987c0cd3f0 con 0x7f9864004800 2024-04-26T20:51:49.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.021+0000 7f9881c37640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f987c0d0470 con 0x7f987c0c8cf0 2024-04-26T20:51:49.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.021+0000 7f9881c37640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f987c0cd570 con 0x7f987c0cb8b0 2024-04-26T20:51:49.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.021+0000 7f987b7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9864004800 0x7f987c152450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:49.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.021+0000 7f987b7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9864004800 0x7f987c152450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54622/0 (socket says 172.21.15.73:54622) 2024-04-26T20:51:49.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.021+0000 7f987b7fe640 1 -- 172.21.15.73:0/1487832067 learned_addr learned my addr 172.21.15.73:0/1487832067 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:49.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.022+0000 7f987affd640 1 --2- 172.21.15.73:0/1487832067 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f987c0c8cf0 0x7f987c0c9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:49.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.022+0000 7f987bfff640 1 --2- 172.21.15.73:0/1487832067 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f987c0cb8b0 0x7f987c0cbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:49.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.022+0000 7f987b7fe640 1 -- 172.21.15.73:0/1487832067 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f987c0cb8b0 msgr2=0x7f987c0cbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:49.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.022+0000 7f987b7fe640 1 --2- 172.21.15.73:0/1487832067 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f987c0cb8b0 0x7f987c0cbd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:49.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.022+0000 7f987b7fe640 1 -- 172.21.15.73:0/1487832067 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f987c0c8cf0 msgr2=0x7f987c0c9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:49.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.022+0000 7f987b7fe640 1 --2- 172.21.15.73:0/1487832067 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f987c0c8cf0 0x7f987c0c9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:49.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.022+0000 7f987b7fe640 1 -- 172.21.15.73:0/1487832067 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f987c0c74f0 con 0x7f9864004800 2024-04-26T20:51:49.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.022+0000 7f987bfff640 1 --2- 172.21.15.73:0/1487832067 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f987c0cb8b0 0x7f987c0cbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:49.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.022+0000 7f987b7fe640 1 --2- 172.21.15.73:0/1487832067 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9864004800 0x7f987c152450 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f986c007340 tx=0x7f986c007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:49.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.022+0000 7f9863fff640 1 -- 172.21.15.73:0/1487832067 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986c00f8c0 con 0x7f9864004800 2024-04-26T20:51:49.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.022+0000 7f9863fff640 1 -- 172.21.15.73:0/1487832067 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f986c018070 con 0x7f9864004800 2024-04-26T20:51:49.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.022+0000 7f9863fff640 1 -- 172.21.15.73:0/1487832067 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986c00fcc0 con 0x7f9864004800 2024-04-26T20:51:49.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.022+0000 7f9881c37640 1 -- 172.21.15.73:0/1487832067 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f987c0c77d0 con 0x7f9864004800 2024-04-26T20:51:49.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.022+0000 7f9881c37640 1 -- 172.21.15.73:0/1487832067 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f987c0c7d90 con 0x7f9864004800 2024-04-26T20:51:49.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.023+0000 7f9881c37640 1 -- 172.21.15.73:0/1487832067 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f987c0694f0 con 0x7f9864004800 2024-04-26T20:51:49.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.023+0000 7f9863fff640 1 -- 172.21.15.73:0/1487832067 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f986c01d460 con 0x7f9864004800 2024-04-26T20:51:49.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.024+0000 7f9863fff640 1 --2- 172.21.15.73:0/1487832067 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f986806c920 0x7f986806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:49.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.024+0000 7f9863fff640 1 -- 172.21.15.73:0/1487832067 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f986c08bd30 con 0x7f9864004800 2024-04-26T20:51:49.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.024+0000 7f987affd640 1 --2- 172.21.15.73:0/1487832067 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f986806c920 0x7f986806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:49.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.024+0000 7f987affd640 1 --2- 172.21.15.73:0/1487832067 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f986806c920 0x7f986806ede0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9870033000 tx=0x7f9870008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:49.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.028+0000 7f9863fff640 1 -- 172.21.15.73:0/1487832067 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f986c00fe60 con 0x7f9864004800 2024-04-26T20:51:49.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.198+0000 7f9881c37640 1 -- 172.21.15.73:0/1487832067 --> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f987c0c9b60 con 0x7f986806c920 2024-04-26T20:51:49.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.204+0000 7f9863fff640 1 -- 172.21.15.73:0/1487832067 <== mgr.4107 v2:172.21.15.73:6832/666905549 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246635 (secure 0 0 0) 0x7f987c0c9b60 con 0x7f986806c920 2024-04-26T20:51:49.208 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:51:49.209 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-26T20:51:49.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.207+0000 7f9881c37640 1 -- 172.21.15.73:0/1487832067 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f986806c920 msgr2=0x7f986806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:49.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.207+0000 7f9881c37640 1 --2- 172.21.15.73:0/1487832067 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f986806c920 0x7f986806ede0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9870033000 tx=0x7f9870008030 comp rx=0 tx=0).stop 2024-04-26T20:51:49.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.207+0000 7f9881c37640 1 -- 172.21.15.73:0/1487832067 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9864004800 msgr2=0x7f987c152450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:49.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.207+0000 7f9881c37640 1 --2- 172.21.15.73:0/1487832067 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9864004800 0x7f987c152450 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f986c007340 tx=0x7f986c007800 comp rx=0 tx=0).stop 2024-04-26T20:51:49.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.207+0000 7f9881c37640 1 -- 172.21.15.73:0/1487832067 shutdown_connections 2024-04-26T20:51:49.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.207+0000 7f9881c37640 1 --2- 172.21.15.73:0/1487832067 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f986806c920 0x7f986806ede0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:49.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.207+0000 7f9881c37640 1 --2- 172.21.15.73:0/1487832067 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f987c0cb8b0 0x7f987c0cbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:49.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.207+0000 7f9881c37640 1 --2- 172.21.15.73:0/1487832067 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f987c0c8cf0 0x7f987c0c9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:49.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.207+0000 7f9881c37640 1 --2- 172.21.15.73:0/1487832067 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9864004800 0x7f987c152450 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:49.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.207+0000 7f9881c37640 1 -- 172.21.15.73:0/1487832067 >> 172.21.15.73:0/1487832067 conn(0x7f987c0d2da0 msgr2=0x7f987c0b1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:49.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.207+0000 7f9881c37640 1 -- 172.21.15.73:0/1487832067 shutdown_connections 2024-04-26T20:51:49.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.207+0000 7f9881c37640 1 -- 172.21.15.73:0/1487832067 wait complete. 2024-04-26T20:51:49.219 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":76,"stamp":"2024-04-26T20:51:47.997420+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220552,"kb_used_data":4392,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754168,"statfs":{"total":773094113280,"available":772868268032,"internally_reserved":0,"allocated":4497408,"data_stored":2593784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.078798"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.927134+0000","last_change":"2024-04-26T20:51:44.927134+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.927134+0000","last_undegraded":"2024-04-26T20:51:44.927134+0000","last_fullsized":"2024-04-26T20:51:44.927134+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.927081+0000","last_change":"2024-04-26T20:51:44.927081+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.927081+0000","last_undegraded":"2024-04-26T20:51:44.927081+0000","last_fullsized":"2024-04-26T20:51:44.927081+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.930409+0000","last_change":"2024-04-26T20:51:44.930409+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.930409+0000","last_undegraded":"2024-04-26T20:51:44.930409+0000","last_fullsized":"2024-04-26T20:51:44.930409+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.926209+0000","last_change":"2024-04-26T20:51:44.926209+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.926209+0000","last_undegraded":"2024-04-26T20:51:44.926209+0000","last_fullsized":"2024-04-26T20:51:44.926209+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.925479+0000","last_change":"2024-04-26T20:51:44.925479+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.925479+0000","last_undegraded":"2024-04-26T20:51:44.925479+0000","last_fullsized":"2024-04-26T20:51:44.925479+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.934070+0000","last_change":"2024-04-26T20:51:44.934070+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.934070+0000","last_undegraded":"2024-04-26T20:51:44.934070+0000","last_fullsized":"2024-04-26T20:51:44.934070+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.930809+0000","last_change":"2024-04-26T20:51:44.930809+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.930809+0000","last_undegraded":"2024-04-26T20:51:44.930809+0000","last_fullsized":"2024-04-26T20:51:44.930809+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.926467+0000","last_change":"2024-04-26T20:51:44.926467+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.926467+0000","last_undegraded":"2024-04-26T20:51:44.926467+0000","last_fullsized":"2024-04-26T20:51:44.926467+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.929217+0000","last_change":"2024-04-26T20:51:44.929217+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.929217+0000","last_undegraded":"2024-04-26T20:51:44.929217+0000","last_fullsized":"2024-04-26T20:51:44.929217+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.930277+0000","last_change":"2024-04-26T20:51:44.930277+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.930277+0000","last_undegraded":"2024-04-26T20:51:44.930277+0000","last_fullsized":"2024-04-26T20:51:44.930277+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.930633+0000","last_change":"2024-04-26T20:51:44.930633+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.930633+0000","last_undegraded":"2024-04-26T20:51:44.930633+0000","last_fullsized":"2024-04-26T20:51:44.930633+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.929572+0000","last_change":"2024-04-26T20:51:44.929572+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.929572+0000","last_undegraded":"2024-04-26T20:51:44.929572+0000","last_fullsized":"2024-04-26T20:51:44.929572+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.929667+0000","last_change":"2024-04-26T20:51:44.929667+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.929667+0000","last_undegraded":"2024-04-26T20:51:44.929667+0000","last_fullsized":"2024-04-26T20:51:44.929667+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.930603+0000","last_change":"2024-04-26T20:51:44.930603+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.930603+0000","last_undegraded":"2024-04-26T20:51:44.930603+0000","last_fullsized":"2024-04-26T20:51:44.930603+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.929584+0000","last_change":"2024-04-26T20:51:44.929584+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.929584+0000","last_undegraded":"2024-04-26T20:51:44.929584+0000","last_fullsized":"2024-04-26T20:51:44.929584+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.934714+0000","last_change":"2024-04-26T20:51:44.934714+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.934714+0000","last_undegraded":"2024-04-26T20:51:44.934714+0000","last_fullsized":"2024-04-26T20:51:44.934714+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.933252+0000","last_change":"2024-04-26T20:51:44.933252+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.933252+0000","last_undegraded":"2024-04-26T20:51:44.933252+0000","last_fullsized":"2024-04-26T20:51:44.933252+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.923957+0000","last_change":"2024-04-26T20:51:44.923957+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.923957+0000","last_undegraded":"2024-04-26T20:51:44.923957+0000","last_fullsized":"2024-04-26T20:51:44.923957+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.926243+0000","last_change":"2024-04-26T20:51:44.926243+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.926243+0000","last_undegraded":"2024-04-26T20:51:44.926243+0000","last_fullsized":"2024-04-26T20:51:44.926243+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.924107+0000","last_change":"2024-04-26T20:51:44.924107+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.924107+0000","last_undegraded":"2024-04-26T20:51:44.924107+0000","last_fullsized":"2024-04-26T20:51:44.924107+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.927282+0000","last_change":"2024-04-26T20:51:44.927282+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.927282+0000","last_undegraded":"2024-04-26T20:51:44.927282+0000","last_fullsized":"2024-04-26T20:51:44.927282+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.928722+0000","last_change":"2024-04-26T20:51:44.928722+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.928722+0000","last_undegraded":"2024-04-26T20:51:44.928722+0000","last_fullsized":"2024-04-26T20:51:44.928722+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.934479+0000","last_change":"2024-04-26T20:51:44.934479+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.934479+0000","last_undegraded":"2024-04-26T20:51:44.934479+0000","last_fullsized":"2024-04-26T20:51:44.934479+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.925511+0000","last_change":"2024-04-26T20:51:44.925511+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.925511+0000","last_undegraded":"2024-04-26T20:51:44.925511+0000","last_fullsized":"2024-04-26T20:51:44.925511+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.923419+0000","last_change":"2024-04-26T20:51:44.923419+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.923419+0000","last_undegraded":"2024-04-26T20:51:44.923419+0000","last_fullsized":"2024-04-26T20:51:44.923419+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.930697+0000","last_change":"2024-04-26T20:51:44.930697+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.930697+0000","last_undegraded":"2024-04-26T20:51:44.930697+0000","last_fullsized":"2024-04-26T20:51:44.930697+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.923397+0000","last_change":"2024-04-26T20:51:44.923397+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.923397+0000","last_undegraded":"2024-04-26T20:51:44.923397+0000","last_fullsized":"2024-04-26T20:51:44.923397+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.926829+0000","last_change":"2024-04-26T20:51:44.926829+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.926829+0000","last_undegraded":"2024-04-26T20:51:44.926829+0000","last_fullsized":"2024-04-26T20:51:44.926829+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.923542+0000","last_change":"2024-04-26T20:51:44.923542+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.923542+0000","last_undegraded":"2024-04-26T20:51:44.923542+0000","last_fullsized":"2024-04-26T20:51:44.923542+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.923673+0000","last_change":"2024-04-26T20:51:44.923673+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.923673+0000","last_undegraded":"2024-04-26T20:51:44.923673+0000","last_fullsized":"2024-04-26T20:51:44.923673+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.932216+0000","last_change":"2024-04-26T20:51:44.932216+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.932216+0000","last_undegraded":"2024-04-26T20:51:44.932216+0000","last_fullsized":"2024-04-26T20:51:44.932216+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.930954+0000","last_change":"2024-04-26T20:51:44.930954+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.930954+0000","last_undegraded":"2024-04-26T20:51:44.930954+0000","last_fullsized":"2024-04-26T20:51:44.930954+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.932301+0000","last_change":"2024-04-26T20:51:44.932301+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.932301+0000","last_undegraded":"2024-04-26T20:51:44.932301+0000","last_fullsized":"2024-04-26T20:51:44.932301+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.936717+0000","last_change":"2024-04-26T20:51:44.935237+0000","last_active":"2024-04-26T20:51:44.936717+0000","last_peered":"2024-04-26T20:51:44.936717+0000","last_clean":"2024-04-26T20:51:44.936717+0000","last_became_active":"2024-04-26T20:51:44.934964+0000","last_became_peered":"2024-04-26T20:51:44.934964+0000","last_unstale":"2024-04-26T20:51:44.936717+0000","last_undegraded":"2024-04-26T20:51:44.936717+0000","last_fullsized":"2024-04-26T20:51:44.936717+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:22.556226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.934004+0000","last_change":"2024-04-26T20:51:44.934004+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.934004+0000","last_undegraded":"2024-04-26T20:51:44.934004+0000","last_fullsized":"2024-04-26T20:51:44.934004+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.940096+0000","last_change":"2024-04-26T20:51:44.937990+0000","last_active":"2024-04-26T20:51:44.940096+0000","last_peered":"2024-04-26T20:51:44.940096+0000","last_clean":"2024-04-26T20:51:44.940096+0000","last_became_active":"2024-04-26T20:51:44.937723+0000","last_became_peered":"2024-04-26T20:51:44.937723+0000","last_unstale":"2024-04-26T20:51:44.940096+0000","last_undegraded":"2024-04-26T20:51:44.940096+0000","last_fullsized":"2024-04-26T20:51:44.940096+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:48:23.398632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.928437+0000","last_change":"2024-04-26T20:51:44.928437+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.928437+0000","last_undegraded":"2024-04-26T20:51:44.928437+0000","last_fullsized":"2024-04-26T20:51:44.928437+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.943035+0000","last_change":"2024-04-26T20:51:44.939632+0000","last_active":"2024-04-26T20:51:44.943035+0000","last_peered":"2024-04-26T20:51:44.943035+0000","last_clean":"2024-04-26T20:51:44.943035+0000","last_became_active":"2024-04-26T20:51:44.939299+0000","last_became_peered":"2024-04-26T20:51:44.939299+0000","last_unstale":"2024-04-26T20:51:44.943035+0000","last_undegraded":"2024-04-26T20:51:44.943035+0000","last_fullsized":"2024-04-26T20:51:44.943035+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:35.270443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.925851+0000","last_change":"2024-04-26T20:51:44.925851+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.925851+0000","last_undegraded":"2024-04-26T20:51:44.925851+0000","last_fullsized":"2024-04-26T20:51:44.925851+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.939345+0000","last_change":"2024-04-26T20:51:44.937707+0000","last_active":"2024-04-26T20:51:44.939345+0000","last_peered":"2024-04-26T20:51:44.939345+0000","last_clean":"2024-04-26T20:51:44.939345+0000","last_became_active":"2024-04-26T20:51:44.937394+0000","last_became_peered":"2024-04-26T20:51:44.937394+0000","last_unstale":"2024-04-26T20:51:44.939345+0000","last_undegraded":"2024-04-26T20:51:44.939345+0000","last_fullsized":"2024-04-26T20:51:44.939345+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:14:24.515988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.930643+0000","last_change":"2024-04-26T20:51:44.930643+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.930643+0000","last_undegraded":"2024-04-26T20:51:44.930643+0000","last_fullsized":"2024-04-26T20:51:44.930643+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.939732+0000","last_change":"2024-04-26T20:51:44.937929+0000","last_active":"2024-04-26T20:51:44.939732+0000","last_peered":"2024-04-26T20:51:44.939732+0000","last_clean":"2024-04-26T20:51:44.939732+0000","last_became_active":"2024-04-26T20:51:44.937626+0000","last_became_peered":"2024-04-26T20:51:44.937626+0000","last_unstale":"2024-04-26T20:51:44.939732+0000","last_undegraded":"2024-04-26T20:51:44.939732+0000","last_fullsized":"2024-04-26T20:51:44.939732+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:48:23.398632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.926384+0000","last_change":"2024-04-26T20:51:44.926384+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.926384+0000","last_undegraded":"2024-04-26T20:51:44.926384+0000","last_fullsized":"2024-04-26T20:51:44.926384+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.945361+0000","last_change":"2024-04-26T20:51:44.944114+0000","last_active":"2024-04-26T20:51:44.945361+0000","last_peered":"2024-04-26T20:51:44.945361+0000","last_clean":"2024-04-26T20:51:44.945361+0000","last_became_active":"2024-04-26T20:51:44.943821+0000","last_became_peered":"2024-04-26T20:51:44.943821+0000","last_unstale":"2024-04-26T20:51:44.945361+0000","last_undegraded":"2024-04-26T20:51:44.945361+0000","last_fullsized":"2024-04-26T20:51:44.945361+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:28:21.811767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.927913+0000","last_change":"2024-04-26T20:51:44.927913+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.927913+0000","last_undegraded":"2024-04-26T20:51:44.927913+0000","last_fullsized":"2024-04-26T20:51:44.927913+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.943098+0000","last_change":"2024-04-26T20:51:44.942003+0000","last_active":"2024-04-26T20:51:44.943098+0000","last_peered":"2024-04-26T20:51:44.943098+0000","last_clean":"2024-04-26T20:51:44.943098+0000","last_became_active":"2024-04-26T20:51:44.941646+0000","last_became_peered":"2024-04-26T20:51:44.941646+0000","last_unstale":"2024-04-26T20:51:44.943098+0000","last_undegraded":"2024-04-26T20:51:44.943098+0000","last_fullsized":"2024-04-26T20:51:44.943098+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:14:24.515988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.926719+0000","last_change":"2024-04-26T20:51:44.926719+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.926719+0000","last_undegraded":"2024-04-26T20:51:44.926719+0000","last_fullsized":"2024-04-26T20:51:44.926719+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.936220+0000","last_change":"2024-04-26T20:51:44.934491+0000","last_active":"2024-04-26T20:51:44.936220+0000","last_peered":"2024-04-26T20:51:44.936220+0000","last_clean":"2024-04-26T20:51:44.936220+0000","last_became_active":"2024-04-26T20:51:44.934145+0000","last_became_peered":"2024-04-26T20:51:44.934145+0000","last_unstale":"2024-04-26T20:51:44.936220+0000","last_undegraded":"2024-04-26T20:51:44.936220+0000","last_fullsized":"2024-04-26T20:51:44.936220+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:48:23.398632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.925518+0000","last_change":"2024-04-26T20:51:44.925518+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.925518+0000","last_undegraded":"2024-04-26T20:51:44.925518+0000","last_fullsized":"2024-04-26T20:51:44.925518+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.941952+0000","last_change":"2024-04-26T20:51:44.938418+0000","last_active":"2024-04-26T20:51:44.941952+0000","last_peered":"2024-04-26T20:51:44.941952+0000","last_clean":"2024-04-26T20:51:44.941952+0000","last_became_active":"2024-04-26T20:51:44.938146+0000","last_became_peered":"2024-04-26T20:51:44.938146+0000","last_unstale":"2024-04-26T20:51:44.941952+0000","last_undegraded":"2024-04-26T20:51:44.941952+0000","last_fullsized":"2024-04-26T20:51:44.941952+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:44.088012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.940376+0000","last_change":"2024-04-26T20:51:44.940376+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.940376+0000","last_undegraded":"2024-04-26T20:51:44.940376+0000","last_fullsized":"2024-04-26T20:51:44.940376+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.941993+0000","last_change":"2024-04-26T20:51:44.941093+0000","last_active":"2024-04-26T20:51:44.941993+0000","last_peered":"2024-04-26T20:51:44.941993+0000","last_clean":"2024-04-26T20:51:44.941993+0000","last_became_active":"2024-04-26T20:51:44.940805+0000","last_became_peered":"2024-04-26T20:51:44.940805+0000","last_unstale":"2024-04-26T20:51:44.941993+0000","last_undegraded":"2024-04-26T20:51:44.941993+0000","last_fullsized":"2024-04-26T20:51:44.941993+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:10:56.604114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.930862+0000","last_change":"2024-04-26T20:51:44.930862+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.930862+0000","last_undegraded":"2024-04-26T20:51:44.930862+0000","last_fullsized":"2024-04-26T20:51:44.930862+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.939847+0000","last_change":"2024-04-26T20:51:44.939214+0000","last_active":"2024-04-26T20:51:44.939847+0000","last_peered":"2024-04-26T20:51:44.939847+0000","last_clean":"2024-04-26T20:51:44.939847+0000","last_became_active":"2024-04-26T20:51:44.938907+0000","last_became_peered":"2024-04-26T20:51:44.938907+0000","last_unstale":"2024-04-26T20:51:44.939847+0000","last_undegraded":"2024-04-26T20:51:44.939847+0000","last_fullsized":"2024-04-26T20:51:44.939847+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:17.661527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.930135+0000","last_change":"2024-04-26T20:51:44.930135+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.930135+0000","last_undegraded":"2024-04-26T20:51:44.930135+0000","last_fullsized":"2024-04-26T20:51:44.930135+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.942052+0000","last_change":"2024-04-26T20:51:44.939371+0000","last_active":"2024-04-26T20:51:44.942052+0000","last_peered":"2024-04-26T20:51:44.942052+0000","last_clean":"2024-04-26T20:51:44.942052+0000","last_became_active":"2024-04-26T20:51:44.939090+0000","last_became_peered":"2024-04-26T20:51:44.939090+0000","last_unstale":"2024-04-26T20:51:44.942052+0000","last_undegraded":"2024-04-26T20:51:44.942052+0000","last_fullsized":"2024-04-26T20:51:44.942052+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:40:59.156357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.930280+0000","last_change":"2024-04-26T20:51:44.930280+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.930280+0000","last_undegraded":"2024-04-26T20:51:44.930280+0000","last_fullsized":"2024-04-26T20:51:44.930280+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.942877+0000","last_change":"2024-04-26T20:51:44.940477+0000","last_active":"2024-04-26T20:51:44.942877+0000","last_peered":"2024-04-26T20:51:44.942877+0000","last_clean":"2024-04-26T20:51:44.942877+0000","last_became_active":"2024-04-26T20:51:44.940192+0000","last_became_peered":"2024-04-26T20:51:44.940192+0000","last_unstale":"2024-04-26T20:51:44.942877+0000","last_undegraded":"2024-04-26T20:51:44.942877+0000","last_fullsized":"2024-04-26T20:51:44.942877+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:38:31.038452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.926418+0000","last_change":"2024-04-26T20:51:44.926418+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.926418+0000","last_undegraded":"2024-04-26T20:51:44.926418+0000","last_fullsized":"2024-04-26T20:51:44.926418+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.943067+0000","last_change":"2024-04-26T20:51:44.940382+0000","last_active":"2024-04-26T20:51:44.943067+0000","last_peered":"2024-04-26T20:51:44.943067+0000","last_clean":"2024-04-26T20:51:44.943067+0000","last_became_active":"2024-04-26T20:51:44.940109+0000","last_became_peered":"2024-04-26T20:51:44.940109+0000","last_unstale":"2024-04-26T20:51:44.943067+0000","last_undegraded":"2024-04-26T20:51:44.943067+0000","last_fullsized":"2024-04-26T20:51:44.943067+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:48:23.398632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.932149+0000","last_change":"2024-04-26T20:51:44.932149+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.932149+0000","last_undegraded":"2024-04-26T20:51:44.932149+0000","last_fullsized":"2024-04-26T20:51:44.932149+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.936667+0000","last_change":"2024-04-26T20:51:44.934857+0000","last_active":"2024-04-26T20:51:44.936667+0000","last_peered":"2024-04-26T20:51:44.936667+0000","last_clean":"2024-04-26T20:51:44.936667+0000","last_became_active":"2024-04-26T20:51:44.934556+0000","last_became_peered":"2024-04-26T20:51:44.934556+0000","last_unstale":"2024-04-26T20:51:44.936667+0000","last_undegraded":"2024-04-26T20:51:44.936667+0000","last_fullsized":"2024-04-26T20:51:44.936667+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:04:31.671170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.925836+0000","last_change":"2024-04-26T20:51:44.925836+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.925836+0000","last_undegraded":"2024-04-26T20:51:44.925836+0000","last_fullsized":"2024-04-26T20:51:44.925836+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.944907+0000","last_change":"2024-04-26T20:51:44.943632+0000","last_active":"2024-04-26T20:51:44.944907+0000","last_peered":"2024-04-26T20:51:44.944907+0000","last_clean":"2024-04-26T20:51:44.944907+0000","last_became_active":"2024-04-26T20:51:44.943278+0000","last_became_peered":"2024-04-26T20:51:44.943278+0000","last_unstale":"2024-04-26T20:51:44.944907+0000","last_undegraded":"2024-04-26T20:51:44.944907+0000","last_fullsized":"2024-04-26T20:51:44.944907+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:59:50.250119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.926462+0000","last_change":"2024-04-26T20:51:44.926462+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.926462+0000","last_undegraded":"2024-04-26T20:51:44.926462+0000","last_fullsized":"2024-04-26T20:51:44.926462+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.937919+0000","last_change":"2024-04-26T20:51:44.935150+0000","last_active":"2024-04-26T20:51:44.937919+0000","last_peered":"2024-04-26T20:51:44.937919+0000","last_clean":"2024-04-26T20:51:44.937919+0000","last_became_active":"2024-04-26T20:51:44.934078+0000","last_became_peered":"2024-04-26T20:51:44.934078+0000","last_unstale":"2024-04-26T20:51:44.937919+0000","last_undegraded":"2024-04-26T20:51:44.937919+0000","last_fullsized":"2024-04-26T20:51:44.937919+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:35.270443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.926645+0000","last_change":"2024-04-26T20:51:44.926645+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.926645+0000","last_undegraded":"2024-04-26T20:51:44.926645+0000","last_fullsized":"2024-04-26T20:51:44.926645+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.934725+0000","last_change":"2024-04-26T20:51:44.932407+0000","last_active":"2024-04-26T20:51:44.934725+0000","last_peered":"2024-04-26T20:51:44.934725+0000","last_clean":"2024-04-26T20:51:44.934725+0000","last_became_active":"2024-04-26T20:51:44.931987+0000","last_became_peered":"2024-04-26T20:51:44.931987+0000","last_unstale":"2024-04-26T20:51:44.934725+0000","last_undegraded":"2024-04-26T20:51:44.934725+0000","last_fullsized":"2024-04-26T20:51:44.934725+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:36:55.165610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.929390+0000","last_change":"2024-04-26T20:51:44.929390+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.929390+0000","last_undegraded":"2024-04-26T20:51:44.929390+0000","last_fullsized":"2024-04-26T20:51:44.929390+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.943664+0000","last_change":"2024-04-26T20:51:44.941836+0000","last_active":"2024-04-26T20:51:44.943664+0000","last_peered":"2024-04-26T20:51:44.943664+0000","last_clean":"2024-04-26T20:51:44.943664+0000","last_became_active":"2024-04-26T20:51:44.941581+0000","last_became_peered":"2024-04-26T20:51:44.941581+0000","last_unstale":"2024-04-26T20:51:44.943664+0000","last_undegraded":"2024-04-26T20:51:44.943664+0000","last_fullsized":"2024-04-26T20:51:44.943664+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:30:26.145171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.924912+0000","last_change":"2024-04-26T20:51:44.924912+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.924912+0000","last_undegraded":"2024-04-26T20:51:44.924912+0000","last_fullsized":"2024-04-26T20:51:44.924912+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.942287+0000","last_change":"2024-04-26T20:51:44.938146+0000","last_active":"2024-04-26T20:51:44.942287+0000","last_peered":"2024-04-26T20:51:44.942287+0000","last_clean":"2024-04-26T20:51:44.942287+0000","last_became_active":"2024-04-26T20:51:44.937861+0000","last_became_peered":"2024-04-26T20:51:44.937861+0000","last_unstale":"2024-04-26T20:51:44.942287+0000","last_undegraded":"2024-04-26T20:51:44.942287+0000","last_fullsized":"2024-04-26T20:51:44.942287+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:36:55.165610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.932596+0000","last_change":"2024-04-26T20:51:44.932596+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.932596+0000","last_undegraded":"2024-04-26T20:51:44.932596+0000","last_fullsized":"2024-04-26T20:51:44.932596+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.941851+0000","last_change":"2024-04-26T20:51:44.940796+0000","last_active":"2024-04-26T20:51:44.941851+0000","last_peered":"2024-04-26T20:51:44.941851+0000","last_clean":"2024-04-26T20:51:44.941851+0000","last_became_active":"2024-04-26T20:51:44.940524+0000","last_became_peered":"2024-04-26T20:51:44.940524+0000","last_unstale":"2024-04-26T20:51:44.941851+0000","last_undegraded":"2024-04-26T20:51:44.941851+0000","last_fullsized":"2024-04-26T20:51:44.941851+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:31:06.346238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.929053+0000","last_change":"2024-04-26T20:51:44.929053+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.929053+0000","last_undegraded":"2024-04-26T20:51:44.929053+0000","last_fullsized":"2024-04-26T20:51:44.929053+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.939800+0000","last_change":"2024-04-26T20:51:44.934985+0000","last_active":"2024-04-26T20:51:44.939800+0000","last_peered":"2024-04-26T20:51:44.939800+0000","last_clean":"2024-04-26T20:51:44.939800+0000","last_became_active":"2024-04-26T20:51:44.934690+0000","last_became_peered":"2024-04-26T20:51:44.934690+0000","last_unstale":"2024-04-26T20:51:44.939800+0000","last_undegraded":"2024-04-26T20:51:44.939800+0000","last_fullsized":"2024-04-26T20:51:44.939800+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:02.518303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.926386+0000","last_change":"2024-04-26T20:51:44.926386+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.926386+0000","last_undegraded":"2024-04-26T20:51:44.926386+0000","last_fullsized":"2024-04-26T20:51:44.926386+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.945415+0000","last_change":"2024-04-26T20:51:44.944446+0000","last_active":"2024-04-26T20:51:44.945415+0000","last_peered":"2024-04-26T20:51:44.945415+0000","last_clean":"2024-04-26T20:51:44.945415+0000","last_became_active":"2024-04-26T20:51:44.944147+0000","last_became_peered":"2024-04-26T20:51:44.944147+0000","last_unstale":"2024-04-26T20:51:44.945415+0000","last_undegraded":"2024-04-26T20:51:44.945415+0000","last_fullsized":"2024-04-26T20:51:44.945415+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:16:58.774484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.926312+0000","last_change":"2024-04-26T20:51:44.926312+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.926312+0000","last_undegraded":"2024-04-26T20:51:44.926312+0000","last_fullsized":"2024-04-26T20:51:44.926312+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.939834+0000","last_change":"2024-04-26T20:51:44.932932+0000","last_active":"2024-04-26T20:51:44.939834+0000","last_peered":"2024-04-26T20:51:44.939834+0000","last_clean":"2024-04-26T20:51:44.939834+0000","last_became_active":"2024-04-26T20:51:44.932657+0000","last_became_peered":"2024-04-26T20:51:44.932657+0000","last_unstale":"2024-04-26T20:51:44.939834+0000","last_undegraded":"2024-04-26T20:51:44.939834+0000","last_fullsized":"2024-04-26T20:51:44.939834+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:14:17.119338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.931512+0000","last_change":"2024-04-26T20:51:44.931512+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.931512+0000","last_undegraded":"2024-04-26T20:51:44.931512+0000","last_fullsized":"2024-04-26T20:51:44.931512+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.939843+0000","last_change":"2024-04-26T20:51:44.938761+0000","last_active":"2024-04-26T20:51:44.939843+0000","last_peered":"2024-04-26T20:51:44.939843+0000","last_clean":"2024-04-26T20:51:44.939843+0000","last_became_active":"2024-04-26T20:51:44.938483+0000","last_became_peered":"2024-04-26T20:51:44.938483+0000","last_unstale":"2024-04-26T20:51:44.939843+0000","last_undegraded":"2024-04-26T20:51:44.939843+0000","last_fullsized":"2024-04-26T20:51:44.939843+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:14:24.515988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.938640+0000","last_change":"2024-04-26T20:51:44.938640+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.938640+0000","last_undegraded":"2024-04-26T20:51:44.938640+0000","last_fullsized":"2024-04-26T20:51:44.938640+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.939747+0000","last_change":"2024-04-26T20:51:44.932513+0000","last_active":"2024-04-26T20:51:44.939747+0000","last_peered":"2024-04-26T20:51:44.939747+0000","last_clean":"2024-04-26T20:51:44.939747+0000","last_became_active":"2024-04-26T20:51:44.932223+0000","last_became_peered":"2024-04-26T20:51:44.932223+0000","last_unstale":"2024-04-26T20:51:44.939747+0000","last_undegraded":"2024-04-26T20:51:44.939747+0000","last_fullsized":"2024-04-26T20:51:44.939747+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:28:26.304960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.926513+0000","last_change":"2024-04-26T20:51:44.926513+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.926513+0000","last_undegraded":"2024-04-26T20:51:44.926513+0000","last_fullsized":"2024-04-26T20:51:44.926513+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.940605+0000","last_change":"2024-04-26T20:51:44.938966+0000","last_active":"2024-04-26T20:51:44.940605+0000","last_peered":"2024-04-26T20:51:44.940605+0000","last_clean":"2024-04-26T20:51:44.940605+0000","last_became_active":"2024-04-26T20:51:44.938662+0000","last_became_peered":"2024-04-26T20:51:44.938662+0000","last_unstale":"2024-04-26T20:51:44.940605+0000","last_undegraded":"2024-04-26T20:51:44.940605+0000","last_fullsized":"2024-04-26T20:51:44.940605+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:14:24.515988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.929938+0000","last_change":"2024-04-26T20:51:44.929938+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.929938+0000","last_undegraded":"2024-04-26T20:51:44.929938+0000","last_fullsized":"2024-04-26T20:51:44.929938+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.939612+0000","last_change":"2024-04-26T20:51:44.935952+0000","last_active":"2024-04-26T20:51:44.939612+0000","last_peered":"2024-04-26T20:51:44.939612+0000","last_clean":"2024-04-26T20:51:44.939612+0000","last_became_active":"2024-04-26T20:51:44.935682+0000","last_became_peered":"2024-04-26T20:51:44.935682+0000","last_unstale":"2024-04-26T20:51:44.939612+0000","last_undegraded":"2024-04-26T20:51:44.939612+0000","last_fullsized":"2024-04-26T20:51:44.939612+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:35.270443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.933638+0000","last_change":"2024-04-26T20:51:44.933638+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.933638+0000","last_undegraded":"2024-04-26T20:51:44.933638+0000","last_fullsized":"2024-04-26T20:51:44.933638+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.933704+0000","last_change":"2024-04-26T20:51:44.932688+0000","last_active":"2024-04-26T20:51:44.933704+0000","last_peered":"2024-04-26T20:51:44.933704+0000","last_clean":"2024-04-26T20:51:44.933704+0000","last_became_active":"2024-04-26T20:51:44.932417+0000","last_became_peered":"2024-04-26T20:51:44.932417+0000","last_unstale":"2024-04-26T20:51:44.933704+0000","last_undegraded":"2024-04-26T20:51:44.933704+0000","last_fullsized":"2024-04-26T20:51:44.933704+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:55:14.475567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.930210+0000","last_change":"2024-04-26T20:51:44.930210+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.930210+0000","last_undegraded":"2024-04-26T20:51:44.930210+0000","last_fullsized":"2024-04-26T20:51:44.930210+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.937168+0000","last_change":"2024-04-26T20:51:44.934242+0000","last_active":"2024-04-26T20:51:44.937168+0000","last_peered":"2024-04-26T20:51:44.937168+0000","last_clean":"2024-04-26T20:51:44.937168+0000","last_became_active":"2024-04-26T20:51:44.933449+0000","last_became_peered":"2024-04-26T20:51:44.933449+0000","last_unstale":"2024-04-26T20:51:44.937168+0000","last_undegraded":"2024-04-26T20:51:44.937168+0000","last_fullsized":"2024-04-26T20:51:44.937168+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:35.270443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.932718+0000","last_change":"2024-04-26T20:51:44.932718+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.932718+0000","last_undegraded":"2024-04-26T20:51:44.932718+0000","last_fullsized":"2024-04-26T20:51:44.932718+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.940384+0000","last_change":"2024-04-26T20:51:44.934065+0000","last_active":"2024-04-26T20:51:44.940384+0000","last_peered":"2024-04-26T20:51:44.940384+0000","last_clean":"2024-04-26T20:51:44.940384+0000","last_became_active":"2024-04-26T20:51:44.933763+0000","last_became_peered":"2024-04-26T20:51:44.933763+0000","last_unstale":"2024-04-26T20:51:44.940384+0000","last_undegraded":"2024-04-26T20:51:44.940384+0000","last_fullsized":"2024-04-26T20:51:44.940384+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:46:52.657100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.927313+0000","last_change":"2024-04-26T20:51:44.927313+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.927313+0000","last_undegraded":"2024-04-26T20:51:44.927313+0000","last_fullsized":"2024-04-26T20:51:44.927313+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.946013+0000","last_change":"2024-04-26T20:51:44.944770+0000","last_active":"2024-04-26T20:51:44.946013+0000","last_peered":"2024-04-26T20:51:44.946013+0000","last_clean":"2024-04-26T20:51:44.946013+0000","last_became_active":"2024-04-26T20:51:44.944478+0000","last_became_peered":"2024-04-26T20:51:44.944478+0000","last_unstale":"2024-04-26T20:51:44.946013+0000","last_undegraded":"2024-04-26T20:51:44.946013+0000","last_fullsized":"2024-04-26T20:51:44.946013+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:37.193221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":199,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.941219+0000","last_change":"2024-04-26T20:50:21.150132+0000","last_active":"2024-04-26T20:51:44.941219+0000","last_peered":"2024-04-26T20:51:44.941219+0000","last_clean":"2024-04-26T20:51:44.941219+0000","last_became_active":"2024-04-26T20:50:21.147439+0000","last_became_peered":"2024-04-26T20:50:21.147439+0000","last_unstale":"2024-04-26T20:51:44.941219+0000","last_undegraded":"2024-04-26T20:51:44.941219+0000","last_fullsized":"2024-04-26T20:51:44.941219+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:20.084499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:20.084499+0000","last_clean_scrub_stamp":"2024-04-26T20:50:20.084499+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:19:29.395477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":1007616,"data_stored":766999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":1007616,"data_stored":766999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:51:49.222 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:51:49.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.355+0000 7f4ecadc6640 1 Processor -- start 2024-04-26T20:51:49.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.355+0000 7f4ecadc6640 1 -- start start 2024-04-26T20:51:49.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.355+0000 7f4ecadc6640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f4ec41608f0 0x7f4ec40d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:49.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.355+0000 7f4ecadc6640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4ec40cffa0 0x7f4ec40b2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:49.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.355+0000 7f4ecadc6640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ec40cde20 con 0x7f4ec40d0a90 2024-04-26T20:51:49.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.355+0000 7f4ecadc6640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f4ec40cea30 con 0x7f4ec41608f0 2024-04-26T20:51:49.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.355+0000 7f4ecadc6640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f4ec40ce100 con 0x7f4ec40cffa0 2024-04-26T20:51:49.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.356+0000 7f4ec8b62640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f4ec41608f0 0x7f4ec40d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:49.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.356+0000 7f4ec9363640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4ec40cffa0 0x7f4ec40b2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:49.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.356+0000 7f4ec3fff640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f4ec40d0a90 0x7f4ec40cf890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.73:58066/0 (socket says 172.21.15.73:58066) 2024-04-26T20:51:49.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.356+0000 7f4ec8b62640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f4ec41608f0 0x7f4ec40d1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.73:38950/0 (socket says 172.21.15.73:38950) 2024-04-26T20:51:49.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.356+0000 7f4ec3fff640 1 -- 172.21.15.73:0/3704178696 learned_addr learned my addr 172.21.15.73:0/3704178696 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:49.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.356+0000 7f4ec9363640 1 -- 172.21.15.73:0/3704178696 >> v2:172.21.15.120:3300/0 conn(0x7f4ec41608f0 msgr2=0x7f4ec40d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:49.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.356+0000 7f4ec9363640 1 --2- 172.21.15.73:0/3704178696 >> v2:172.21.15.120:3300/0 conn(0x7f4ec41608f0 0x7f4ec40d1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:49.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.356+0000 7f4ec9363640 1 -- 172.21.15.73:0/3704178696 >> v1:172.21.15.120:6789/0 conn(0x7f4ec40d0a90 legacy=0x7f4ec40cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:49.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.356+0000 7f4ec9363640 1 -- 172.21.15.73:0/3704178696 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ec40ce3e0 con 0x7f4ec40cffa0 2024-04-26T20:51:49.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.357+0000 7f4ec9363640 1 --2- 172.21.15.73:0/3704178696 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4ec40cffa0 0x7f4ec40b2300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc009d90 tx=0x7f4ebc02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6878df058d11fb7f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:49.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.357+0000 7f4ec37fe640 1 -- 172.21.15.73:0/3704178696 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc007740 con 0x7f4ec40cffa0 2024-04-26T20:51:49.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.357+0000 7f4ec37fe640 1 -- 172.21.15.73:0/3704178696 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ebc0078e0 con 0x7f4ec40cffa0 2024-04-26T20:51:49.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.357+0000 7f4ec37fe640 1 -- 172.21.15.73:0/3704178696 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc007cb0 con 0x7f4ec40cffa0 2024-04-26T20:51:49.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.358+0000 7f4ecadc6640 1 -- 172.21.15.73:0/3704178696 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4ec40cffa0 msgr2=0x7f4ec40b2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:49.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.358+0000 7f4ecadc6640 1 --2- 172.21.15.73:0/3704178696 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4ec40cffa0 0x7f4ec40b2300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc009d90 tx=0x7f4ebc02a660 comp rx=0 tx=0).stop 2024-04-26T20:51:49.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.358+0000 7f4ecadc6640 1 -- 172.21.15.73:0/3704178696 shutdown_connections 2024-04-26T20:51:49.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.358+0000 7f4ecadc6640 1 --2- 172.21.15.73:0/3704178696 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4ec40cffa0 0x7f4ec40b2300 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:49.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.358+0000 7f4ecadc6640 1 --2- 172.21.15.73:0/3704178696 >> v2:172.21.15.120:3300/0 conn(0x7f4ec41608f0 0x7f4ec40d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:49.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.358+0000 7f4ecadc6640 1 -- 172.21.15.73:0/3704178696 >> 172.21.15.73:0/3704178696 conn(0x7f4ec40d2ec0 msgr2=0x7f4ec40d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:49.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.358+0000 7f4ecadc6640 1 -- 172.21.15.73:0/3704178696 shutdown_connections 2024-04-26T20:51:49.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.358+0000 7f4ecadc6640 1 -- 172.21.15.73:0/3704178696 wait complete. 2024-04-26T20:51:49.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.359+0000 7f4ecadc6640 1 Processor -- start 2024-04-26T20:51:49.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.359+0000 7f4ecadc6640 1 -- start start 2024-04-26T20:51:49.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.360+0000 7f4ecadc6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ec40cffa0 0x7f4ec41f3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:49.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.360+0000 7f4ecadc6640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4ec40d0a90 0x7f4ec415b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:49.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.360+0000 7f4ecadc6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4ec41608f0 0x7f4ec415b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:49.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.360+0000 7f4ecadc6640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec40ceec0 con 0x7f4ec40cffa0 2024-04-26T20:51:49.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.360+0000 7f4ecadc6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec40d0ef0 con 0x7f4ec41608f0 2024-04-26T20:51:49.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.360+0000 7f4ecadc6640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f4ec40cda40 con 0x7f4ec40d0a90 2024-04-26T20:51:49.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.360+0000 7f4ec8b62640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ec40cffa0 0x7f4ec41f3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:49.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.360+0000 7f4ec8b62640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ec40cffa0 0x7f4ec41f3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54630/0 (socket says 172.21.15.73:54630) 2024-04-26T20:51:49.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.360+0000 7f4ec8b62640 1 -- 172.21.15.73:0/2081712417 learned_addr learned my addr 172.21.15.73:0/2081712417 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:49.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.360+0000 7f4ec9363640 1 --2- 172.21.15.73:0/2081712417 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4ec41608f0 0x7f4ec415b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:49.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.360+0000 7f4ec3fff640 1 --2- 172.21.15.73:0/2081712417 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4ec40d0a90 0x7f4ec415b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:49.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.360+0000 7f4ec8b62640 1 -- 172.21.15.73:0/2081712417 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4ec40d0a90 msgr2=0x7f4ec415b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:49.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.360+0000 7f4ec8b62640 1 --2- 172.21.15.73:0/2081712417 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4ec40d0a90 0x7f4ec415b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:49.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.360+0000 7f4ec8b62640 1 -- 172.21.15.73:0/2081712417 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4ec41608f0 msgr2=0x7f4ec415b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:49.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.360+0000 7f4ec8b62640 1 --2- 172.21.15.73:0/2081712417 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4ec41608f0 0x7f4ec415b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:49.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.360+0000 7f4ec8b62640 1 -- 172.21.15.73:0/2081712417 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ec41f6940 con 0x7f4ec40cffa0 2024-04-26T20:51:49.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.360+0000 7f4ec8b62640 1 --2- 172.21.15.73:0/2081712417 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ec40cffa0 0x7f4ec41f3800 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f4eb800e9e0 tx=0x7f4eb800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:49.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.360+0000 7f4ec17fa640 1 -- 172.21.15.73:0/2081712417 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb8017070 con 0x7f4ec40cffa0 2024-04-26T20:51:49.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.360+0000 7f4ec17fa640 1 -- 172.21.15.73:0/2081712417 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4eb800c9d0 con 0x7f4ec40cffa0 2024-04-26T20:51:49.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.360+0000 7f4ecadc6640 1 -- 172.21.15.73:0/2081712417 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ec41f6c20 con 0x7f4ec40cffa0 2024-04-26T20:51:49.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.360+0000 7f4ec17fa640 1 -- 172.21.15.73:0/2081712417 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb800cdc0 con 0x7f4ec40cffa0 2024-04-26T20:51:49.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.360+0000 7f4ecadc6640 1 -- 172.21.15.73:0/2081712417 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ec41f71e0 con 0x7f4ec40cffa0 2024-04-26T20:51:49.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.361+0000 7f4ecadc6640 1 -- 172.21.15.73:0/2081712417 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ec40694f0 con 0x7f4ec40cffa0 2024-04-26T20:51:49.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.362+0000 7f4ec17fa640 1 -- 172.21.15.73:0/2081712417 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4eb801a070 con 0x7f4ec40cffa0 2024-04-26T20:51:49.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.362+0000 7f4ec17fa640 1 --2- 172.21.15.73:0/2081712417 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f4ea806c8f0 0x7f4ea806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:49.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.362+0000 7f4ec17fa640 1 -- 172.21.15.73:0/2081712417 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f4eb808c170 con 0x7f4ec40cffa0 2024-04-26T20:51:49.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.362+0000 7f4ec3fff640 1 --2- 172.21.15.73:0/2081712417 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f4ea806c8f0 0x7f4ea806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:49.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.363+0000 7f4ec3fff640 1 --2- 172.21.15.73:0/2081712417 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f4ea806c8f0 0x7f4ea806edb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4158da0 tx=0x7f4eb0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:49.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.366+0000 7f4ec17fa640 1 -- 172.21.15.73:0/2081712417 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4eb805a390 con 0x7f4ec40cffa0 2024-04-26T20:51:49.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.538+0000 7f4ecadc6640 1 -- 172.21.15.73:0/2081712417 --> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4ec40d0470 con 0x7f4ea806c8f0 2024-04-26T20:51:49.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.544+0000 7f4ec17fa640 1 -- 172.21.15.73:0/2081712417 <== mgr.4107 v2:172.21.15.73:6832/666905549 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246635 (secure 0 0 0) 0x7f4ec40d0470 con 0x7f4ea806c8f0 2024-04-26T20:51:49.548 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:51:49.549 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-26T20:51:49.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.547+0000 7f4ecadc6640 1 -- 172.21.15.73:0/2081712417 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f4ea806c8f0 msgr2=0x7f4ea806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:49.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.547+0000 7f4ecadc6640 1 --2- 172.21.15.73:0/2081712417 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f4ea806c8f0 0x7f4ea806edb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4158da0 tx=0x7f4eb0009040 comp rx=0 tx=0).stop 2024-04-26T20:51:49.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.547+0000 7f4ecadc6640 1 -- 172.21.15.73:0/2081712417 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ec40cffa0 msgr2=0x7f4ec41f3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:49.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.547+0000 7f4ecadc6640 1 --2- 172.21.15.73:0/2081712417 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ec40cffa0 0x7f4ec41f3800 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f4eb800e9e0 tx=0x7f4eb800eea0 comp rx=0 tx=0).stop 2024-04-26T20:51:49.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.547+0000 7f4ecadc6640 1 -- 172.21.15.73:0/2081712417 shutdown_connections 2024-04-26T20:51:49.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.547+0000 7f4ecadc6640 1 --2- 172.21.15.73:0/2081712417 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f4ea806c8f0 0x7f4ea806edb0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:49.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.547+0000 7f4ecadc6640 1 --2- 172.21.15.73:0/2081712417 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4ec41608f0 0x7f4ec415b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:49.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.547+0000 7f4ecadc6640 1 --2- 172.21.15.73:0/2081712417 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4ec40d0a90 0x7f4ec415b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:49.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.547+0000 7f4ecadc6640 1 --2- 172.21.15.73:0/2081712417 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ec40cffa0 0x7f4ec41f3800 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:49.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.547+0000 7f4ecadc6640 1 -- 172.21.15.73:0/2081712417 >> 172.21.15.73:0/2081712417 conn(0x7f4ec40d2ec0 msgr2=0x7f4ec40d1070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:49.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.547+0000 7f4ecadc6640 1 -- 172.21.15.73:0/2081712417 shutdown_connections 2024-04-26T20:51:49.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.547+0000 7f4ecadc6640 1 -- 172.21.15.73:0/2081712417 wait complete. 2024-04-26T20:51:49.559 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":76,"stamp":"2024-04-26T20:51:47.997420+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220552,"kb_used_data":4392,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754168,"statfs":{"total":773094113280,"available":772868268032,"internally_reserved":0,"allocated":4497408,"data_stored":2593784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.078798"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.927134+0000","last_change":"2024-04-26T20:51:44.927134+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.927134+0000","last_undegraded":"2024-04-26T20:51:44.927134+0000","last_fullsized":"2024-04-26T20:51:44.927134+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.927081+0000","last_change":"2024-04-26T20:51:44.927081+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.927081+0000","last_undegraded":"2024-04-26T20:51:44.927081+0000","last_fullsized":"2024-04-26T20:51:44.927081+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.930409+0000","last_change":"2024-04-26T20:51:44.930409+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.930409+0000","last_undegraded":"2024-04-26T20:51:44.930409+0000","last_fullsized":"2024-04-26T20:51:44.930409+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.926209+0000","last_change":"2024-04-26T20:51:44.926209+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.926209+0000","last_undegraded":"2024-04-26T20:51:44.926209+0000","last_fullsized":"2024-04-26T20:51:44.926209+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.925479+0000","last_change":"2024-04-26T20:51:44.925479+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.925479+0000","last_undegraded":"2024-04-26T20:51:44.925479+0000","last_fullsized":"2024-04-26T20:51:44.925479+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.934070+0000","last_change":"2024-04-26T20:51:44.934070+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.934070+0000","last_undegraded":"2024-04-26T20:51:44.934070+0000","last_fullsized":"2024-04-26T20:51:44.934070+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.930809+0000","last_change":"2024-04-26T20:51:44.930809+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.930809+0000","last_undegraded":"2024-04-26T20:51:44.930809+0000","last_fullsized":"2024-04-26T20:51:44.930809+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.926467+0000","last_change":"2024-04-26T20:51:44.926467+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.926467+0000","last_undegraded":"2024-04-26T20:51:44.926467+0000","last_fullsized":"2024-04-26T20:51:44.926467+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.929217+0000","last_change":"2024-04-26T20:51:44.929217+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.929217+0000","last_undegraded":"2024-04-26T20:51:44.929217+0000","last_fullsized":"2024-04-26T20:51:44.929217+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.930277+0000","last_change":"2024-04-26T20:51:44.930277+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.930277+0000","last_undegraded":"2024-04-26T20:51:44.930277+0000","last_fullsized":"2024-04-26T20:51:44.930277+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.930633+0000","last_change":"2024-04-26T20:51:44.930633+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.930633+0000","last_undegraded":"2024-04-26T20:51:44.930633+0000","last_fullsized":"2024-04-26T20:51:44.930633+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.929572+0000","last_change":"2024-04-26T20:51:44.929572+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.929572+0000","last_undegraded":"2024-04-26T20:51:44.929572+0000","last_fullsized":"2024-04-26T20:51:44.929572+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.929667+0000","last_change":"2024-04-26T20:51:44.929667+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.929667+0000","last_undegraded":"2024-04-26T20:51:44.929667+0000","last_fullsized":"2024-04-26T20:51:44.929667+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.930603+0000","last_change":"2024-04-26T20:51:44.930603+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.930603+0000","last_undegraded":"2024-04-26T20:51:44.930603+0000","last_fullsized":"2024-04-26T20:51:44.930603+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.929584+0000","last_change":"2024-04-26T20:51:44.929584+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.929584+0000","last_undegraded":"2024-04-26T20:51:44.929584+0000","last_fullsized":"2024-04-26T20:51:44.929584+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.934714+0000","last_change":"2024-04-26T20:51:44.934714+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.934714+0000","last_undegraded":"2024-04-26T20:51:44.934714+0000","last_fullsized":"2024-04-26T20:51:44.934714+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.933252+0000","last_change":"2024-04-26T20:51:44.933252+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.933252+0000","last_undegraded":"2024-04-26T20:51:44.933252+0000","last_fullsized":"2024-04-26T20:51:44.933252+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.923957+0000","last_change":"2024-04-26T20:51:44.923957+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.923957+0000","last_undegraded":"2024-04-26T20:51:44.923957+0000","last_fullsized":"2024-04-26T20:51:44.923957+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.926243+0000","last_change":"2024-04-26T20:51:44.926243+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.926243+0000","last_undegraded":"2024-04-26T20:51:44.926243+0000","last_fullsized":"2024-04-26T20:51:44.926243+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.924107+0000","last_change":"2024-04-26T20:51:44.924107+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.924107+0000","last_undegraded":"2024-04-26T20:51:44.924107+0000","last_fullsized":"2024-04-26T20:51:44.924107+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.927282+0000","last_change":"2024-04-26T20:51:44.927282+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.927282+0000","last_undegraded":"2024-04-26T20:51:44.927282+0000","last_fullsized":"2024-04-26T20:51:44.927282+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.928722+0000","last_change":"2024-04-26T20:51:44.928722+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.928722+0000","last_undegraded":"2024-04-26T20:51:44.928722+0000","last_fullsized":"2024-04-26T20:51:44.928722+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.934479+0000","last_change":"2024-04-26T20:51:44.934479+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.934479+0000","last_undegraded":"2024-04-26T20:51:44.934479+0000","last_fullsized":"2024-04-26T20:51:44.934479+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.925511+0000","last_change":"2024-04-26T20:51:44.925511+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.925511+0000","last_undegraded":"2024-04-26T20:51:44.925511+0000","last_fullsized":"2024-04-26T20:51:44.925511+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.923419+0000","last_change":"2024-04-26T20:51:44.923419+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.923419+0000","last_undegraded":"2024-04-26T20:51:44.923419+0000","last_fullsized":"2024-04-26T20:51:44.923419+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.930697+0000","last_change":"2024-04-26T20:51:44.930697+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.930697+0000","last_undegraded":"2024-04-26T20:51:44.930697+0000","last_fullsized":"2024-04-26T20:51:44.930697+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.923397+0000","last_change":"2024-04-26T20:51:44.923397+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.923397+0000","last_undegraded":"2024-04-26T20:51:44.923397+0000","last_fullsized":"2024-04-26T20:51:44.923397+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.926829+0000","last_change":"2024-04-26T20:51:44.926829+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.926829+0000","last_undegraded":"2024-04-26T20:51:44.926829+0000","last_fullsized":"2024-04-26T20:51:44.926829+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.923542+0000","last_change":"2024-04-26T20:51:44.923542+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.923542+0000","last_undegraded":"2024-04-26T20:51:44.923542+0000","last_fullsized":"2024-04-26T20:51:44.923542+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.923673+0000","last_change":"2024-04-26T20:51:44.923673+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.923673+0000","last_undegraded":"2024-04-26T20:51:44.923673+0000","last_fullsized":"2024-04-26T20:51:44.923673+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.932216+0000","last_change":"2024-04-26T20:51:44.932216+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.932216+0000","last_undegraded":"2024-04-26T20:51:44.932216+0000","last_fullsized":"2024-04-26T20:51:44.932216+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.930954+0000","last_change":"2024-04-26T20:51:44.930954+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.930954+0000","last_undegraded":"2024-04-26T20:51:44.930954+0000","last_fullsized":"2024-04-26T20:51:44.930954+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.932301+0000","last_change":"2024-04-26T20:51:44.932301+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.932301+0000","last_undegraded":"2024-04-26T20:51:44.932301+0000","last_fullsized":"2024-04-26T20:51:44.932301+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.936717+0000","last_change":"2024-04-26T20:51:44.935237+0000","last_active":"2024-04-26T20:51:44.936717+0000","last_peered":"2024-04-26T20:51:44.936717+0000","last_clean":"2024-04-26T20:51:44.936717+0000","last_became_active":"2024-04-26T20:51:44.934964+0000","last_became_peered":"2024-04-26T20:51:44.934964+0000","last_unstale":"2024-04-26T20:51:44.936717+0000","last_undegraded":"2024-04-26T20:51:44.936717+0000","last_fullsized":"2024-04-26T20:51:44.936717+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:22.556226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.934004+0000","last_change":"2024-04-26T20:51:44.934004+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.934004+0000","last_undegraded":"2024-04-26T20:51:44.934004+0000","last_fullsized":"2024-04-26T20:51:44.934004+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.940096+0000","last_change":"2024-04-26T20:51:44.937990+0000","last_active":"2024-04-26T20:51:44.940096+0000","last_peered":"2024-04-26T20:51:44.940096+0000","last_clean":"2024-04-26T20:51:44.940096+0000","last_became_active":"2024-04-26T20:51:44.937723+0000","last_became_peered":"2024-04-26T20:51:44.937723+0000","last_unstale":"2024-04-26T20:51:44.940096+0000","last_undegraded":"2024-04-26T20:51:44.940096+0000","last_fullsized":"2024-04-26T20:51:44.940096+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:48:23.398632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.928437+0000","last_change":"2024-04-26T20:51:44.928437+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.928437+0000","last_undegraded":"2024-04-26T20:51:44.928437+0000","last_fullsized":"2024-04-26T20:51:44.928437+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.943035+0000","last_change":"2024-04-26T20:51:44.939632+0000","last_active":"2024-04-26T20:51:44.943035+0000","last_peered":"2024-04-26T20:51:44.943035+0000","last_clean":"2024-04-26T20:51:44.943035+0000","last_became_active":"2024-04-26T20:51:44.939299+0000","last_became_peered":"2024-04-26T20:51:44.939299+0000","last_unstale":"2024-04-26T20:51:44.943035+0000","last_undegraded":"2024-04-26T20:51:44.943035+0000","last_fullsized":"2024-04-26T20:51:44.943035+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:35.270443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.925851+0000","last_change":"2024-04-26T20:51:44.925851+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.925851+0000","last_undegraded":"2024-04-26T20:51:44.925851+0000","last_fullsized":"2024-04-26T20:51:44.925851+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.939345+0000","last_change":"2024-04-26T20:51:44.937707+0000","last_active":"2024-04-26T20:51:44.939345+0000","last_peered":"2024-04-26T20:51:44.939345+0000","last_clean":"2024-04-26T20:51:44.939345+0000","last_became_active":"2024-04-26T20:51:44.937394+0000","last_became_peered":"2024-04-26T20:51:44.937394+0000","last_unstale":"2024-04-26T20:51:44.939345+0000","last_undegraded":"2024-04-26T20:51:44.939345+0000","last_fullsized":"2024-04-26T20:51:44.939345+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:14:24.515988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.930643+0000","last_change":"2024-04-26T20:51:44.930643+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.930643+0000","last_undegraded":"2024-04-26T20:51:44.930643+0000","last_fullsized":"2024-04-26T20:51:44.930643+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.939732+0000","last_change":"2024-04-26T20:51:44.937929+0000","last_active":"2024-04-26T20:51:44.939732+0000","last_peered":"2024-04-26T20:51:44.939732+0000","last_clean":"2024-04-26T20:51:44.939732+0000","last_became_active":"2024-04-26T20:51:44.937626+0000","last_became_peered":"2024-04-26T20:51:44.937626+0000","last_unstale":"2024-04-26T20:51:44.939732+0000","last_undegraded":"2024-04-26T20:51:44.939732+0000","last_fullsized":"2024-04-26T20:51:44.939732+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:48:23.398632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.926384+0000","last_change":"2024-04-26T20:51:44.926384+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.926384+0000","last_undegraded":"2024-04-26T20:51:44.926384+0000","last_fullsized":"2024-04-26T20:51:44.926384+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.945361+0000","last_change":"2024-04-26T20:51:44.944114+0000","last_active":"2024-04-26T20:51:44.945361+0000","last_peered":"2024-04-26T20:51:44.945361+0000","last_clean":"2024-04-26T20:51:44.945361+0000","last_became_active":"2024-04-26T20:51:44.943821+0000","last_became_peered":"2024-04-26T20:51:44.943821+0000","last_unstale":"2024-04-26T20:51:44.945361+0000","last_undegraded":"2024-04-26T20:51:44.945361+0000","last_fullsized":"2024-04-26T20:51:44.945361+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:28:21.811767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.927913+0000","last_change":"2024-04-26T20:51:44.927913+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.927913+0000","last_undegraded":"2024-04-26T20:51:44.927913+0000","last_fullsized":"2024-04-26T20:51:44.927913+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.943098+0000","last_change":"2024-04-26T20:51:44.942003+0000","last_active":"2024-04-26T20:51:44.943098+0000","last_peered":"2024-04-26T20:51:44.943098+0000","last_clean":"2024-04-26T20:51:44.943098+0000","last_became_active":"2024-04-26T20:51:44.941646+0000","last_became_peered":"2024-04-26T20:51:44.941646+0000","last_unstale":"2024-04-26T20:51:44.943098+0000","last_undegraded":"2024-04-26T20:51:44.943098+0000","last_fullsized":"2024-04-26T20:51:44.943098+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:14:24.515988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.926719+0000","last_change":"2024-04-26T20:51:44.926719+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.926719+0000","last_undegraded":"2024-04-26T20:51:44.926719+0000","last_fullsized":"2024-04-26T20:51:44.926719+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.936220+0000","last_change":"2024-04-26T20:51:44.934491+0000","last_active":"2024-04-26T20:51:44.936220+0000","last_peered":"2024-04-26T20:51:44.936220+0000","last_clean":"2024-04-26T20:51:44.936220+0000","last_became_active":"2024-04-26T20:51:44.934145+0000","last_became_peered":"2024-04-26T20:51:44.934145+0000","last_unstale":"2024-04-26T20:51:44.936220+0000","last_undegraded":"2024-04-26T20:51:44.936220+0000","last_fullsized":"2024-04-26T20:51:44.936220+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:48:23.398632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.925518+0000","last_change":"2024-04-26T20:51:44.925518+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.925518+0000","last_undegraded":"2024-04-26T20:51:44.925518+0000","last_fullsized":"2024-04-26T20:51:44.925518+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.941952+0000","last_change":"2024-04-26T20:51:44.938418+0000","last_active":"2024-04-26T20:51:44.941952+0000","last_peered":"2024-04-26T20:51:44.941952+0000","last_clean":"2024-04-26T20:51:44.941952+0000","last_became_active":"2024-04-26T20:51:44.938146+0000","last_became_peered":"2024-04-26T20:51:44.938146+0000","last_unstale":"2024-04-26T20:51:44.941952+0000","last_undegraded":"2024-04-26T20:51:44.941952+0000","last_fullsized":"2024-04-26T20:51:44.941952+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:44.088012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.940376+0000","last_change":"2024-04-26T20:51:44.940376+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.940376+0000","last_undegraded":"2024-04-26T20:51:44.940376+0000","last_fullsized":"2024-04-26T20:51:44.940376+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.941993+0000","last_change":"2024-04-26T20:51:44.941093+0000","last_active":"2024-04-26T20:51:44.941993+0000","last_peered":"2024-04-26T20:51:44.941993+0000","last_clean":"2024-04-26T20:51:44.941993+0000","last_became_active":"2024-04-26T20:51:44.940805+0000","last_became_peered":"2024-04-26T20:51:44.940805+0000","last_unstale":"2024-04-26T20:51:44.941993+0000","last_undegraded":"2024-04-26T20:51:44.941993+0000","last_fullsized":"2024-04-26T20:51:44.941993+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:10:56.604114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.930862+0000","last_change":"2024-04-26T20:51:44.930862+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.930862+0000","last_undegraded":"2024-04-26T20:51:44.930862+0000","last_fullsized":"2024-04-26T20:51:44.930862+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.939847+0000","last_change":"2024-04-26T20:51:44.939214+0000","last_active":"2024-04-26T20:51:44.939847+0000","last_peered":"2024-04-26T20:51:44.939847+0000","last_clean":"2024-04-26T20:51:44.939847+0000","last_became_active":"2024-04-26T20:51:44.938907+0000","last_became_peered":"2024-04-26T20:51:44.938907+0000","last_unstale":"2024-04-26T20:51:44.939847+0000","last_undegraded":"2024-04-26T20:51:44.939847+0000","last_fullsized":"2024-04-26T20:51:44.939847+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:17.661527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.930135+0000","last_change":"2024-04-26T20:51:44.930135+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.930135+0000","last_undegraded":"2024-04-26T20:51:44.930135+0000","last_fullsized":"2024-04-26T20:51:44.930135+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.942052+0000","last_change":"2024-04-26T20:51:44.939371+0000","last_active":"2024-04-26T20:51:44.942052+0000","last_peered":"2024-04-26T20:51:44.942052+0000","last_clean":"2024-04-26T20:51:44.942052+0000","last_became_active":"2024-04-26T20:51:44.939090+0000","last_became_peered":"2024-04-26T20:51:44.939090+0000","last_unstale":"2024-04-26T20:51:44.942052+0000","last_undegraded":"2024-04-26T20:51:44.942052+0000","last_fullsized":"2024-04-26T20:51:44.942052+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:40:59.156357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.930280+0000","last_change":"2024-04-26T20:51:44.930280+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.930280+0000","last_undegraded":"2024-04-26T20:51:44.930280+0000","last_fullsized":"2024-04-26T20:51:44.930280+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.942877+0000","last_change":"2024-04-26T20:51:44.940477+0000","last_active":"2024-04-26T20:51:44.942877+0000","last_peered":"2024-04-26T20:51:44.942877+0000","last_clean":"2024-04-26T20:51:44.942877+0000","last_became_active":"2024-04-26T20:51:44.940192+0000","last_became_peered":"2024-04-26T20:51:44.940192+0000","last_unstale":"2024-04-26T20:51:44.942877+0000","last_undegraded":"2024-04-26T20:51:44.942877+0000","last_fullsized":"2024-04-26T20:51:44.942877+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:38:31.038452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.926418+0000","last_change":"2024-04-26T20:51:44.926418+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.926418+0000","last_undegraded":"2024-04-26T20:51:44.926418+0000","last_fullsized":"2024-04-26T20:51:44.926418+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.943067+0000","last_change":"2024-04-26T20:51:44.940382+0000","last_active":"2024-04-26T20:51:44.943067+0000","last_peered":"2024-04-26T20:51:44.943067+0000","last_clean":"2024-04-26T20:51:44.943067+0000","last_became_active":"2024-04-26T20:51:44.940109+0000","last_became_peered":"2024-04-26T20:51:44.940109+0000","last_unstale":"2024-04-26T20:51:44.943067+0000","last_undegraded":"2024-04-26T20:51:44.943067+0000","last_fullsized":"2024-04-26T20:51:44.943067+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:48:23.398632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.932149+0000","last_change":"2024-04-26T20:51:44.932149+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.932149+0000","last_undegraded":"2024-04-26T20:51:44.932149+0000","last_fullsized":"2024-04-26T20:51:44.932149+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.936667+0000","last_change":"2024-04-26T20:51:44.934857+0000","last_active":"2024-04-26T20:51:44.936667+0000","last_peered":"2024-04-26T20:51:44.936667+0000","last_clean":"2024-04-26T20:51:44.936667+0000","last_became_active":"2024-04-26T20:51:44.934556+0000","last_became_peered":"2024-04-26T20:51:44.934556+0000","last_unstale":"2024-04-26T20:51:44.936667+0000","last_undegraded":"2024-04-26T20:51:44.936667+0000","last_fullsized":"2024-04-26T20:51:44.936667+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:04:31.671170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.925836+0000","last_change":"2024-04-26T20:51:44.925836+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.925836+0000","last_undegraded":"2024-04-26T20:51:44.925836+0000","last_fullsized":"2024-04-26T20:51:44.925836+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.944907+0000","last_change":"2024-04-26T20:51:44.943632+0000","last_active":"2024-04-26T20:51:44.944907+0000","last_peered":"2024-04-26T20:51:44.944907+0000","last_clean":"2024-04-26T20:51:44.944907+0000","last_became_active":"2024-04-26T20:51:44.943278+0000","last_became_peered":"2024-04-26T20:51:44.943278+0000","last_unstale":"2024-04-26T20:51:44.944907+0000","last_undegraded":"2024-04-26T20:51:44.944907+0000","last_fullsized":"2024-04-26T20:51:44.944907+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:59:50.250119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.926462+0000","last_change":"2024-04-26T20:51:44.926462+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.926462+0000","last_undegraded":"2024-04-26T20:51:44.926462+0000","last_fullsized":"2024-04-26T20:51:44.926462+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.937919+0000","last_change":"2024-04-26T20:51:44.935150+0000","last_active":"2024-04-26T20:51:44.937919+0000","last_peered":"2024-04-26T20:51:44.937919+0000","last_clean":"2024-04-26T20:51:44.937919+0000","last_became_active":"2024-04-26T20:51:44.934078+0000","last_became_peered":"2024-04-26T20:51:44.934078+0000","last_unstale":"2024-04-26T20:51:44.937919+0000","last_undegraded":"2024-04-26T20:51:44.937919+0000","last_fullsized":"2024-04-26T20:51:44.937919+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:35.270443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.926645+0000","last_change":"2024-04-26T20:51:44.926645+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.926645+0000","last_undegraded":"2024-04-26T20:51:44.926645+0000","last_fullsized":"2024-04-26T20:51:44.926645+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.934725+0000","last_change":"2024-04-26T20:51:44.932407+0000","last_active":"2024-04-26T20:51:44.934725+0000","last_peered":"2024-04-26T20:51:44.934725+0000","last_clean":"2024-04-26T20:51:44.934725+0000","last_became_active":"2024-04-26T20:51:44.931987+0000","last_became_peered":"2024-04-26T20:51:44.931987+0000","last_unstale":"2024-04-26T20:51:44.934725+0000","last_undegraded":"2024-04-26T20:51:44.934725+0000","last_fullsized":"2024-04-26T20:51:44.934725+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:36:55.165610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.929390+0000","last_change":"2024-04-26T20:51:44.929390+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.929390+0000","last_undegraded":"2024-04-26T20:51:44.929390+0000","last_fullsized":"2024-04-26T20:51:44.929390+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.943664+0000","last_change":"2024-04-26T20:51:44.941836+0000","last_active":"2024-04-26T20:51:44.943664+0000","last_peered":"2024-04-26T20:51:44.943664+0000","last_clean":"2024-04-26T20:51:44.943664+0000","last_became_active":"2024-04-26T20:51:44.941581+0000","last_became_peered":"2024-04-26T20:51:44.941581+0000","last_unstale":"2024-04-26T20:51:44.943664+0000","last_undegraded":"2024-04-26T20:51:44.943664+0000","last_fullsized":"2024-04-26T20:51:44.943664+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:30:26.145171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.924912+0000","last_change":"2024-04-26T20:51:44.924912+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.924912+0000","last_undegraded":"2024-04-26T20:51:44.924912+0000","last_fullsized":"2024-04-26T20:51:44.924912+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.942287+0000","last_change":"2024-04-26T20:51:44.938146+0000","last_active":"2024-04-26T20:51:44.942287+0000","last_peered":"2024-04-26T20:51:44.942287+0000","last_clean":"2024-04-26T20:51:44.942287+0000","last_became_active":"2024-04-26T20:51:44.937861+0000","last_became_peered":"2024-04-26T20:51:44.937861+0000","last_unstale":"2024-04-26T20:51:44.942287+0000","last_undegraded":"2024-04-26T20:51:44.942287+0000","last_fullsized":"2024-04-26T20:51:44.942287+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:36:55.165610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.932596+0000","last_change":"2024-04-26T20:51:44.932596+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.932596+0000","last_undegraded":"2024-04-26T20:51:44.932596+0000","last_fullsized":"2024-04-26T20:51:44.932596+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.941851+0000","last_change":"2024-04-26T20:51:44.940796+0000","last_active":"2024-04-26T20:51:44.941851+0000","last_peered":"2024-04-26T20:51:44.941851+0000","last_clean":"2024-04-26T20:51:44.941851+0000","last_became_active":"2024-04-26T20:51:44.940524+0000","last_became_peered":"2024-04-26T20:51:44.940524+0000","last_unstale":"2024-04-26T20:51:44.941851+0000","last_undegraded":"2024-04-26T20:51:44.941851+0000","last_fullsized":"2024-04-26T20:51:44.941851+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:31:06.346238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.929053+0000","last_change":"2024-04-26T20:51:44.929053+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.929053+0000","last_undegraded":"2024-04-26T20:51:44.929053+0000","last_fullsized":"2024-04-26T20:51:44.929053+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.939800+0000","last_change":"2024-04-26T20:51:44.934985+0000","last_active":"2024-04-26T20:51:44.939800+0000","last_peered":"2024-04-26T20:51:44.939800+0000","last_clean":"2024-04-26T20:51:44.939800+0000","last_became_active":"2024-04-26T20:51:44.934690+0000","last_became_peered":"2024-04-26T20:51:44.934690+0000","last_unstale":"2024-04-26T20:51:44.939800+0000","last_undegraded":"2024-04-26T20:51:44.939800+0000","last_fullsized":"2024-04-26T20:51:44.939800+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:02.518303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.926386+0000","last_change":"2024-04-26T20:51:44.926386+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.926386+0000","last_undegraded":"2024-04-26T20:51:44.926386+0000","last_fullsized":"2024-04-26T20:51:44.926386+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.945415+0000","last_change":"2024-04-26T20:51:44.944446+0000","last_active":"2024-04-26T20:51:44.945415+0000","last_peered":"2024-04-26T20:51:44.945415+0000","last_clean":"2024-04-26T20:51:44.945415+0000","last_became_active":"2024-04-26T20:51:44.944147+0000","last_became_peered":"2024-04-26T20:51:44.944147+0000","last_unstale":"2024-04-26T20:51:44.945415+0000","last_undegraded":"2024-04-26T20:51:44.945415+0000","last_fullsized":"2024-04-26T20:51:44.945415+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:16:58.774484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.926312+0000","last_change":"2024-04-26T20:51:44.926312+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.926312+0000","last_undegraded":"2024-04-26T20:51:44.926312+0000","last_fullsized":"2024-04-26T20:51:44.926312+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.939834+0000","last_change":"2024-04-26T20:51:44.932932+0000","last_active":"2024-04-26T20:51:44.939834+0000","last_peered":"2024-04-26T20:51:44.939834+0000","last_clean":"2024-04-26T20:51:44.939834+0000","last_became_active":"2024-04-26T20:51:44.932657+0000","last_became_peered":"2024-04-26T20:51:44.932657+0000","last_unstale":"2024-04-26T20:51:44.939834+0000","last_undegraded":"2024-04-26T20:51:44.939834+0000","last_fullsized":"2024-04-26T20:51:44.939834+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:14:17.119338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.931512+0000","last_change":"2024-04-26T20:51:44.931512+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.931512+0000","last_undegraded":"2024-04-26T20:51:44.931512+0000","last_fullsized":"2024-04-26T20:51:44.931512+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.939843+0000","last_change":"2024-04-26T20:51:44.938761+0000","last_active":"2024-04-26T20:51:44.939843+0000","last_peered":"2024-04-26T20:51:44.939843+0000","last_clean":"2024-04-26T20:51:44.939843+0000","last_became_active":"2024-04-26T20:51:44.938483+0000","last_became_peered":"2024-04-26T20:51:44.938483+0000","last_unstale":"2024-04-26T20:51:44.939843+0000","last_undegraded":"2024-04-26T20:51:44.939843+0000","last_fullsized":"2024-04-26T20:51:44.939843+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:14:24.515988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.938640+0000","last_change":"2024-04-26T20:51:44.938640+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.938640+0000","last_undegraded":"2024-04-26T20:51:44.938640+0000","last_fullsized":"2024-04-26T20:51:44.938640+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.939747+0000","last_change":"2024-04-26T20:51:44.932513+0000","last_active":"2024-04-26T20:51:44.939747+0000","last_peered":"2024-04-26T20:51:44.939747+0000","last_clean":"2024-04-26T20:51:44.939747+0000","last_became_active":"2024-04-26T20:51:44.932223+0000","last_became_peered":"2024-04-26T20:51:44.932223+0000","last_unstale":"2024-04-26T20:51:44.939747+0000","last_undegraded":"2024-04-26T20:51:44.939747+0000","last_fullsized":"2024-04-26T20:51:44.939747+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:28:26.304960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.926513+0000","last_change":"2024-04-26T20:51:44.926513+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.926513+0000","last_undegraded":"2024-04-26T20:51:44.926513+0000","last_fullsized":"2024-04-26T20:51:44.926513+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.940605+0000","last_change":"2024-04-26T20:51:44.938966+0000","last_active":"2024-04-26T20:51:44.940605+0000","last_peered":"2024-04-26T20:51:44.940605+0000","last_clean":"2024-04-26T20:51:44.940605+0000","last_became_active":"2024-04-26T20:51:44.938662+0000","last_became_peered":"2024-04-26T20:51:44.938662+0000","last_unstale":"2024-04-26T20:51:44.940605+0000","last_undegraded":"2024-04-26T20:51:44.940605+0000","last_fullsized":"2024-04-26T20:51:44.940605+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:14:24.515988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.929938+0000","last_change":"2024-04-26T20:51:44.929938+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.929938+0000","last_undegraded":"2024-04-26T20:51:44.929938+0000","last_fullsized":"2024-04-26T20:51:44.929938+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.939612+0000","last_change":"2024-04-26T20:51:44.935952+0000","last_active":"2024-04-26T20:51:44.939612+0000","last_peered":"2024-04-26T20:51:44.939612+0000","last_clean":"2024-04-26T20:51:44.939612+0000","last_became_active":"2024-04-26T20:51:44.935682+0000","last_became_peered":"2024-04-26T20:51:44.935682+0000","last_unstale":"2024-04-26T20:51:44.939612+0000","last_undegraded":"2024-04-26T20:51:44.939612+0000","last_fullsized":"2024-04-26T20:51:44.939612+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:35.270443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.933638+0000","last_change":"2024-04-26T20:51:44.933638+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.933638+0000","last_undegraded":"2024-04-26T20:51:44.933638+0000","last_fullsized":"2024-04-26T20:51:44.933638+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.933704+0000","last_change":"2024-04-26T20:51:44.932688+0000","last_active":"2024-04-26T20:51:44.933704+0000","last_peered":"2024-04-26T20:51:44.933704+0000","last_clean":"2024-04-26T20:51:44.933704+0000","last_became_active":"2024-04-26T20:51:44.932417+0000","last_became_peered":"2024-04-26T20:51:44.932417+0000","last_unstale":"2024-04-26T20:51:44.933704+0000","last_undegraded":"2024-04-26T20:51:44.933704+0000","last_fullsized":"2024-04-26T20:51:44.933704+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:55:14.475567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.930210+0000","last_change":"2024-04-26T20:51:44.930210+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.930210+0000","last_undegraded":"2024-04-26T20:51:44.930210+0000","last_fullsized":"2024-04-26T20:51:44.930210+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.937168+0000","last_change":"2024-04-26T20:51:44.934242+0000","last_active":"2024-04-26T20:51:44.937168+0000","last_peered":"2024-04-26T20:51:44.937168+0000","last_clean":"2024-04-26T20:51:44.937168+0000","last_became_active":"2024-04-26T20:51:44.933449+0000","last_became_peered":"2024-04-26T20:51:44.933449+0000","last_unstale":"2024-04-26T20:51:44.937168+0000","last_undegraded":"2024-04-26T20:51:44.937168+0000","last_fullsized":"2024-04-26T20:51:44.937168+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:35.270443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.932718+0000","last_change":"2024-04-26T20:51:44.932718+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.932718+0000","last_undegraded":"2024-04-26T20:51:44.932718+0000","last_fullsized":"2024-04-26T20:51:44.932718+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.940384+0000","last_change":"2024-04-26T20:51:44.934065+0000","last_active":"2024-04-26T20:51:44.940384+0000","last_peered":"2024-04-26T20:51:44.940384+0000","last_clean":"2024-04-26T20:51:44.940384+0000","last_became_active":"2024-04-26T20:51:44.933763+0000","last_became_peered":"2024-04-26T20:51:44.933763+0000","last_unstale":"2024-04-26T20:51:44.940384+0000","last_undegraded":"2024-04-26T20:51:44.940384+0000","last_fullsized":"2024-04-26T20:51:44.940384+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:46:52.657100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.927313+0000","last_change":"2024-04-26T20:51:44.927313+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.927313+0000","last_undegraded":"2024-04-26T20:51:44.927313+0000","last_fullsized":"2024-04-26T20:51:44.927313+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.946013+0000","last_change":"2024-04-26T20:51:44.944770+0000","last_active":"2024-04-26T20:51:44.946013+0000","last_peered":"2024-04-26T20:51:44.946013+0000","last_clean":"2024-04-26T20:51:44.946013+0000","last_became_active":"2024-04-26T20:51:44.944478+0000","last_became_peered":"2024-04-26T20:51:44.944478+0000","last_unstale":"2024-04-26T20:51:44.946013+0000","last_undegraded":"2024-04-26T20:51:44.946013+0000","last_fullsized":"2024-04-26T20:51:44.946013+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:37.193221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":199,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.941219+0000","last_change":"2024-04-26T20:50:21.150132+0000","last_active":"2024-04-26T20:51:44.941219+0000","last_peered":"2024-04-26T20:51:44.941219+0000","last_clean":"2024-04-26T20:51:44.941219+0000","last_became_active":"2024-04-26T20:50:21.147439+0000","last_became_peered":"2024-04-26T20:50:21.147439+0000","last_unstale":"2024-04-26T20:51:44.941219+0000","last_undegraded":"2024-04-26T20:51:44.941219+0000","last_fullsized":"2024-04-26T20:51:44.941219+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:20.084499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:20.084499+0000","last_clean_scrub_stamp":"2024-04-26T20:50:20.084499+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:19:29.395477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":1007616,"data_stored":766999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":1007616,"data_stored":766999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:51:49.562 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3f is not active+clean 2024-04-26T20:51:49.562 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.927134+0000', 'last_change': '2024-04-26T20:51:44.927134+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.927134+0000', 'last_undegraded': '2024-04-26T20:51:44.927134+0000', 'last_fullsized': '2024-04-26T20:51:44.927134+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T20:51:49.562 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3e is not active+clean 2024-04-26T20:51:49.562 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.927081+0000', 'last_change': '2024-04-26T20:51:44.927081+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.927081+0000', 'last_undegraded': '2024-04-26T20:51:44.927081+0000', 'last_fullsized': '2024-04-26T20:51:44.927081+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T20:51:49.562 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3d is not active+clean 2024-04-26T20:51:49.563 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.930409+0000', 'last_change': '2024-04-26T20:51:44.930409+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.930409+0000', 'last_undegraded': '2024-04-26T20:51:44.930409+0000', 'last_fullsized': '2024-04-26T20:51:44.930409+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T20:51:49.563 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3c is not active+clean 2024-04-26T20:51:49.563 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.926209+0000', 'last_change': '2024-04-26T20:51:44.926209+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.926209+0000', 'last_undegraded': '2024-04-26T20:51:44.926209+0000', 'last_fullsized': '2024-04-26T20:51:44.926209+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T20:51:49.563 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3b is not active+clean 2024-04-26T20:51:49.563 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.925479+0000', 'last_change': '2024-04-26T20:51:44.925479+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.925479+0000', 'last_undegraded': '2024-04-26T20:51:44.925479+0000', 'last_fullsized': '2024-04-26T20:51:44.925479+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T20:51:49.563 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3a is not active+clean 2024-04-26T20:51:49.563 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.934070+0000', 'last_change': '2024-04-26T20:51:44.934070+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.934070+0000', 'last_undegraded': '2024-04-26T20:51:44.934070+0000', 'last_fullsized': '2024-04-26T20:51:44.934070+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T20:51:49.563 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.39 is not active+clean 2024-04-26T20:51:49.563 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.930809+0000', 'last_change': '2024-04-26T20:51:44.930809+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.930809+0000', 'last_undegraded': '2024-04-26T20:51:44.930809+0000', 'last_fullsized': '2024-04-26T20:51:44.930809+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T20:51:49.563 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.38 is not active+clean 2024-04-26T20:51:49.563 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.926467+0000', 'last_change': '2024-04-26T20:51:44.926467+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.926467+0000', 'last_undegraded': '2024-04-26T20:51:44.926467+0000', 'last_fullsized': '2024-04-26T20:51:44.926467+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T20:51:49.564 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.37 is not active+clean 2024-04-26T20:51:49.564 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.929217+0000', 'last_change': '2024-04-26T20:51:44.929217+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.929217+0000', 'last_undegraded': '2024-04-26T20:51:44.929217+0000', 'last_fullsized': '2024-04-26T20:51:44.929217+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T20:51:49.564 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.36 is not active+clean 2024-04-26T20:51:49.564 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.930277+0000', 'last_change': '2024-04-26T20:51:44.930277+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.930277+0000', 'last_undegraded': '2024-04-26T20:51:44.930277+0000', 'last_fullsized': '2024-04-26T20:51:44.930277+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T20:51:49.564 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.35 is not active+clean 2024-04-26T20:51:49.564 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.930633+0000', 'last_change': '2024-04-26T20:51:44.930633+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.930633+0000', 'last_undegraded': '2024-04-26T20:51:44.930633+0000', 'last_fullsized': '2024-04-26T20:51:44.930633+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T20:51:49.564 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.34 is not active+clean 2024-04-26T20:51:49.564 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.929572+0000', 'last_change': '2024-04-26T20:51:44.929572+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.929572+0000', 'last_undegraded': '2024-04-26T20:51:44.929572+0000', 'last_fullsized': '2024-04-26T20:51:44.929572+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T20:51:49.564 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.33 is not active+clean 2024-04-26T20:51:49.564 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.929667+0000', 'last_change': '2024-04-26T20:51:44.929667+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.929667+0000', 'last_undegraded': '2024-04-26T20:51:44.929667+0000', 'last_fullsized': '2024-04-26T20:51:44.929667+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T20:51:49.564 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.32 is not active+clean 2024-04-26T20:51:49.564 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.930603+0000', 'last_change': '2024-04-26T20:51:44.930603+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.930603+0000', 'last_undegraded': '2024-04-26T20:51:44.930603+0000', 'last_fullsized': '2024-04-26T20:51:44.930603+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T20:51:49.565 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.31 is not active+clean 2024-04-26T20:51:49.565 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.929584+0000', 'last_change': '2024-04-26T20:51:44.929584+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.929584+0000', 'last_undegraded': '2024-04-26T20:51:44.929584+0000', 'last_fullsized': '2024-04-26T20:51:44.929584+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T20:51:49.565 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.30 is not active+clean 2024-04-26T20:51:49.565 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.934714+0000', 'last_change': '2024-04-26T20:51:44.934714+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.934714+0000', 'last_undegraded': '2024-04-26T20:51:44.934714+0000', 'last_fullsized': '2024-04-26T20:51:44.934714+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T20:51:49.565 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2f is not active+clean 2024-04-26T20:51:49.565 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.933252+0000', 'last_change': '2024-04-26T20:51:44.933252+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.933252+0000', 'last_undegraded': '2024-04-26T20:51:44.933252+0000', 'last_fullsized': '2024-04-26T20:51:44.933252+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T20:51:49.565 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2e is not active+clean 2024-04-26T20:51:49.565 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.923957+0000', 'last_change': '2024-04-26T20:51:44.923957+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.923957+0000', 'last_undegraded': '2024-04-26T20:51:44.923957+0000', 'last_fullsized': '2024-04-26T20:51:44.923957+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T20:51:49.565 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2d is not active+clean 2024-04-26T20:51:49.565 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.926243+0000', 'last_change': '2024-04-26T20:51:44.926243+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.926243+0000', 'last_undegraded': '2024-04-26T20:51:44.926243+0000', 'last_fullsized': '2024-04-26T20:51:44.926243+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T20:51:49.565 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2c is not active+clean 2024-04-26T20:51:49.565 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.924107+0000', 'last_change': '2024-04-26T20:51:44.924107+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.924107+0000', 'last_undegraded': '2024-04-26T20:51:44.924107+0000', 'last_fullsized': '2024-04-26T20:51:44.924107+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T20:51:49.566 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2b is not active+clean 2024-04-26T20:51:49.566 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.927282+0000', 'last_change': '2024-04-26T20:51:44.927282+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.927282+0000', 'last_undegraded': '2024-04-26T20:51:44.927282+0000', 'last_fullsized': '2024-04-26T20:51:44.927282+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T20:51:49.566 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2a is not active+clean 2024-04-26T20:51:49.566 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.928722+0000', 'last_change': '2024-04-26T20:51:44.928722+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.928722+0000', 'last_undegraded': '2024-04-26T20:51:44.928722+0000', 'last_fullsized': '2024-04-26T20:51:44.928722+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T20:51:49.566 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.29 is not active+clean 2024-04-26T20:51:49.566 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.934479+0000', 'last_change': '2024-04-26T20:51:44.934479+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.934479+0000', 'last_undegraded': '2024-04-26T20:51:44.934479+0000', 'last_fullsized': '2024-04-26T20:51:44.934479+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T20:51:49.566 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.28 is not active+clean 2024-04-26T20:51:49.566 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.925511+0000', 'last_change': '2024-04-26T20:51:44.925511+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.925511+0000', 'last_undegraded': '2024-04-26T20:51:44.925511+0000', 'last_fullsized': '2024-04-26T20:51:44.925511+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T20:51:49.566 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.27 is not active+clean 2024-04-26T20:51:49.566 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.923419+0000', 'last_change': '2024-04-26T20:51:44.923419+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.923419+0000', 'last_undegraded': '2024-04-26T20:51:44.923419+0000', 'last_fullsized': '2024-04-26T20:51:44.923419+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T20:51:49.566 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.26 is not active+clean 2024-04-26T20:51:49.566 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.930697+0000', 'last_change': '2024-04-26T20:51:44.930697+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.930697+0000', 'last_undegraded': '2024-04-26T20:51:44.930697+0000', 'last_fullsized': '2024-04-26T20:51:44.930697+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T20:51:49.567 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.25 is not active+clean 2024-04-26T20:51:49.567 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.923397+0000', 'last_change': '2024-04-26T20:51:44.923397+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.923397+0000', 'last_undegraded': '2024-04-26T20:51:44.923397+0000', 'last_fullsized': '2024-04-26T20:51:44.923397+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T20:51:49.567 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.24 is not active+clean 2024-04-26T20:51:49.567 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.926829+0000', 'last_change': '2024-04-26T20:51:44.926829+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.926829+0000', 'last_undegraded': '2024-04-26T20:51:44.926829+0000', 'last_fullsized': '2024-04-26T20:51:44.926829+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T20:51:49.567 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.23 is not active+clean 2024-04-26T20:51:49.567 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.923542+0000', 'last_change': '2024-04-26T20:51:44.923542+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.923542+0000', 'last_undegraded': '2024-04-26T20:51:44.923542+0000', 'last_fullsized': '2024-04-26T20:51:44.923542+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T20:51:49.567 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.22 is not active+clean 2024-04-26T20:51:49.567 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.923673+0000', 'last_change': '2024-04-26T20:51:44.923673+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.923673+0000', 'last_undegraded': '2024-04-26T20:51:44.923673+0000', 'last_fullsized': '2024-04-26T20:51:44.923673+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T20:51:49.567 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.21 is not active+clean 2024-04-26T20:51:49.567 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.932216+0000', 'last_change': '2024-04-26T20:51:44.932216+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.932216+0000', 'last_undegraded': '2024-04-26T20:51:44.932216+0000', 'last_fullsized': '2024-04-26T20:51:44.932216+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T20:51:49.567 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.20 is not active+clean 2024-04-26T20:51:49.567 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.930954+0000', 'last_change': '2024-04-26T20:51:44.930954+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.930954+0000', 'last_undegraded': '2024-04-26T20:51:44.930954+0000', 'last_fullsized': '2024-04-26T20:51:44.930954+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T20:51:49.568 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1c is not active+clean 2024-04-26T20:51:49.568 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.932301+0000', 'last_change': '2024-04-26T20:51:44.932301+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.932301+0000', 'last_undegraded': '2024-04-26T20:51:44.932301+0000', 'last_fullsized': '2024-04-26T20:51:44.932301+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T20:51:49.568 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1d is not active+clean 2024-04-26T20:51:49.568 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.934004+0000', 'last_change': '2024-04-26T20:51:44.934004+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.934004+0000', 'last_undegraded': '2024-04-26T20:51:44.934004+0000', 'last_fullsized': '2024-04-26T20:51:44.934004+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T20:51:49.568 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1e is not active+clean 2024-04-26T20:51:49.568 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.928437+0000', 'last_change': '2024-04-26T20:51:44.928437+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.928437+0000', 'last_undegraded': '2024-04-26T20:51:44.928437+0000', 'last_fullsized': '2024-04-26T20:51:44.928437+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T20:51:49.568 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1f is not active+clean 2024-04-26T20:51:49.568 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.925851+0000', 'last_change': '2024-04-26T20:51:44.925851+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.925851+0000', 'last_undegraded': '2024-04-26T20:51:44.925851+0000', 'last_fullsized': '2024-04-26T20:51:44.925851+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T20:51:49.568 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.18 is not active+clean 2024-04-26T20:51:49.568 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.930643+0000', 'last_change': '2024-04-26T20:51:44.930643+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.930643+0000', 'last_undegraded': '2024-04-26T20:51:44.930643+0000', 'last_fullsized': '2024-04-26T20:51:44.930643+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T20:51:49.568 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.19 is not active+clean 2024-04-26T20:51:49.568 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.926384+0000', 'last_change': '2024-04-26T20:51:44.926384+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.926384+0000', 'last_undegraded': '2024-04-26T20:51:44.926384+0000', 'last_fullsized': '2024-04-26T20:51:44.926384+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T20:51:49.569 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1a is not active+clean 2024-04-26T20:51:49.569 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.927913+0000', 'last_change': '2024-04-26T20:51:44.927913+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.927913+0000', 'last_undegraded': '2024-04-26T20:51:44.927913+0000', 'last_fullsized': '2024-04-26T20:51:44.927913+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T20:51:49.569 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1b is not active+clean 2024-04-26T20:51:49.569 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.926719+0000', 'last_change': '2024-04-26T20:51:44.926719+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.926719+0000', 'last_undegraded': '2024-04-26T20:51:44.926719+0000', 'last_fullsized': '2024-04-26T20:51:44.926719+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T20:51:49.569 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.14 is not active+clean 2024-04-26T20:51:49.569 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.925518+0000', 'last_change': '2024-04-26T20:51:44.925518+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.925518+0000', 'last_undegraded': '2024-04-26T20:51:44.925518+0000', 'last_fullsized': '2024-04-26T20:51:44.925518+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T20:51:49.569 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.15 is not active+clean 2024-04-26T20:51:49.569 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.940376+0000', 'last_change': '2024-04-26T20:51:44.940376+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.940376+0000', 'last_undegraded': '2024-04-26T20:51:44.940376+0000', 'last_fullsized': '2024-04-26T20:51:44.940376+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T20:51:49.569 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.16 is not active+clean 2024-04-26T20:51:49.569 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.930862+0000', 'last_change': '2024-04-26T20:51:44.930862+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.930862+0000', 'last_undegraded': '2024-04-26T20:51:44.930862+0000', 'last_fullsized': '2024-04-26T20:51:44.930862+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T20:51:49.569 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.17 is not active+clean 2024-04-26T20:51:49.569 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.930135+0000', 'last_change': '2024-04-26T20:51:44.930135+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.930135+0000', 'last_undegraded': '2024-04-26T20:51:44.930135+0000', 'last_fullsized': '2024-04-26T20:51:44.930135+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T20:51:49.570 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.10 is not active+clean 2024-04-26T20:51:49.570 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.930280+0000', 'last_change': '2024-04-26T20:51:44.930280+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.930280+0000', 'last_undegraded': '2024-04-26T20:51:44.930280+0000', 'last_fullsized': '2024-04-26T20:51:44.930280+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T20:51:49.570 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.11 is not active+clean 2024-04-26T20:51:49.570 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.926418+0000', 'last_change': '2024-04-26T20:51:44.926418+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.926418+0000', 'last_undegraded': '2024-04-26T20:51:44.926418+0000', 'last_fullsized': '2024-04-26T20:51:44.926418+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T20:51:49.570 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.12 is not active+clean 2024-04-26T20:51:49.570 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.932149+0000', 'last_change': '2024-04-26T20:51:44.932149+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.932149+0000', 'last_undegraded': '2024-04-26T20:51:44.932149+0000', 'last_fullsized': '2024-04-26T20:51:44.932149+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T20:51:49.570 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.13 is not active+clean 2024-04-26T20:51:49.570 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.925836+0000', 'last_change': '2024-04-26T20:51:44.925836+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.925836+0000', 'last_undegraded': '2024-04-26T20:51:44.925836+0000', 'last_fullsized': '2024-04-26T20:51:44.925836+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T20:51:49.570 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.c is not active+clean 2024-04-26T20:51:49.570 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.926462+0000', 'last_change': '2024-04-26T20:51:44.926462+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.926462+0000', 'last_undegraded': '2024-04-26T20:51:44.926462+0000', 'last_fullsized': '2024-04-26T20:51:44.926462+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T20:51:49.570 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.d is not active+clean 2024-04-26T20:51:49.571 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.926645+0000', 'last_change': '2024-04-26T20:51:44.926645+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.926645+0000', 'last_undegraded': '2024-04-26T20:51:44.926645+0000', 'last_fullsized': '2024-04-26T20:51:44.926645+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T20:51:49.571 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.e is not active+clean 2024-04-26T20:51:49.571 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.929390+0000', 'last_change': '2024-04-26T20:51:44.929390+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.929390+0000', 'last_undegraded': '2024-04-26T20:51:44.929390+0000', 'last_fullsized': '2024-04-26T20:51:44.929390+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T20:51:49.571 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.f is not active+clean 2024-04-26T20:51:49.571 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.924912+0000', 'last_change': '2024-04-26T20:51:44.924912+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.924912+0000', 'last_undegraded': '2024-04-26T20:51:44.924912+0000', 'last_fullsized': '2024-04-26T20:51:44.924912+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T20:51:49.571 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.8 is not active+clean 2024-04-26T20:51:49.571 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.932596+0000', 'last_change': '2024-04-26T20:51:44.932596+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.932596+0000', 'last_undegraded': '2024-04-26T20:51:44.932596+0000', 'last_fullsized': '2024-04-26T20:51:44.932596+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T20:51:49.571 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.9 is not active+clean 2024-04-26T20:51:49.571 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.929053+0000', 'last_change': '2024-04-26T20:51:44.929053+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.929053+0000', 'last_undegraded': '2024-04-26T20:51:44.929053+0000', 'last_fullsized': '2024-04-26T20:51:44.929053+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T20:51:49.571 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.a is not active+clean 2024-04-26T20:51:49.571 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.926386+0000', 'last_change': '2024-04-26T20:51:44.926386+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.926386+0000', 'last_undegraded': '2024-04-26T20:51:44.926386+0000', 'last_fullsized': '2024-04-26T20:51:44.926386+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T20:51:49.571 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.b is not active+clean 2024-04-26T20:51:49.571 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.926312+0000', 'last_change': '2024-04-26T20:51:44.926312+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.926312+0000', 'last_undegraded': '2024-04-26T20:51:44.926312+0000', 'last_fullsized': '2024-04-26T20:51:44.926312+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T20:51:49.572 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.4 is not active+clean 2024-04-26T20:51:49.572 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.931512+0000', 'last_change': '2024-04-26T20:51:44.931512+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.931512+0000', 'last_undegraded': '2024-04-26T20:51:44.931512+0000', 'last_fullsized': '2024-04-26T20:51:44.931512+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T20:51:49.572 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.5 is not active+clean 2024-04-26T20:51:49.572 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.938640+0000', 'last_change': '2024-04-26T20:51:44.938640+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.938640+0000', 'last_undegraded': '2024-04-26T20:51:44.938640+0000', 'last_fullsized': '2024-04-26T20:51:44.938640+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T20:51:49.572 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.6 is not active+clean 2024-04-26T20:51:49.572 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.926513+0000', 'last_change': '2024-04-26T20:51:44.926513+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.926513+0000', 'last_undegraded': '2024-04-26T20:51:44.926513+0000', 'last_fullsized': '2024-04-26T20:51:44.926513+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T20:51:49.572 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.0 is not active+clean 2024-04-26T20:51:49.572 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.929938+0000', 'last_change': '2024-04-26T20:51:44.929938+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.929938+0000', 'last_undegraded': '2024-04-26T20:51:44.929938+0000', 'last_fullsized': '2024-04-26T20:51:44.929938+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T20:51:49.572 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1 is not active+clean 2024-04-26T20:51:49.572 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.933638+0000', 'last_change': '2024-04-26T20:51:44.933638+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.933638+0000', 'last_undegraded': '2024-04-26T20:51:44.933638+0000', 'last_fullsized': '2024-04-26T20:51:44.933638+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T20:51:49.572 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2 is not active+clean 2024-04-26T20:51:49.572 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.930210+0000', 'last_change': '2024-04-26T20:51:44.930210+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.930210+0000', 'last_undegraded': '2024-04-26T20:51:44.930210+0000', 'last_fullsized': '2024-04-26T20:51:44.930210+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T20:51:49.573 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3 is not active+clean 2024-04-26T20:51:49.573 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.932718+0000', 'last_change': '2024-04-26T20:51:44.932718+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.932718+0000', 'last_undegraded': '2024-04-26T20:51:44.932718+0000', 'last_fullsized': '2024-04-26T20:51:44.932718+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T20:51:49.573 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.7 is not active+clean 2024-04-26T20:51:49.573 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:51:44.927313+0000', 'last_change': '2024-04-26T20:51:44.927313+0000', 'last_active': '2024-04-26T20:51:44.916026+0000', 'last_peered': '2024-04-26T20:51:44.916026+0000', 'last_clean': '2024-04-26T20:51:44.916026+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:51:44.927313+0000', 'last_undegraded': '2024-04-26T20:51:44.927313+0000', 'last_fullsized': '2024-04-26T20:51:44.927313+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'last_clean_scrub_stamp': '2024-04-26T20:51:44.916026+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T20:51:49.573 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T20:51:49.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.694+0000 7fa0ec0ea640 1 Processor -- start 2024-04-26T20:51:49.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.694+0000 7fa0ec0ea640 1 -- start start 2024-04-26T20:51:49.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.695+0000 7fa0ec0ea640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa0e415f3a0 0x7fa0e40d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:49.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.695+0000 7fa0ec0ea640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa0e40d1770 0x7fa0e40d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:49.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.695+0000 7fa0ec0ea640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fa0e40d2130 0x7fa0e40d2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:49.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.695+0000 7fa0ec0ea640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7fa0e40d25a0 con 0x7fa0e40d1770 2024-04-26T20:51:49.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.695+0000 7fa0ec0ea640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fa0e40d0e20 con 0x7fa0e40d2130 2024-04-26T20:51:49.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.695+0000 7fa0ec0ea640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fa0e40b1d70 con 0x7fa0e415f3a0 2024-04-26T20:51:49.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.695+0000 7fa0e9685640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa0e40d1770 0x7fa0e40d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:49.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.695+0000 7fa0e9685640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa0e40d1770 0x7fa0e40d1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40994/0 (socket says 172.21.15.73:40994) 2024-04-26T20:51:49.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.695+0000 7fa0e9685640 1 -- 172.21.15.73:0/3057777060 learned_addr learned my addr 172.21.15.73:0/3057777060 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:49.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.695+0000 7fa0ea687640 1 --2- 172.21.15.73:0/3057777060 >> v2:172.21.15.120:3300/0 conn(0x7fa0e40d2130 0x7fa0e40d2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:49.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.696+0000 7fa0e9e86640 1 --2- 172.21.15.73:0/3057777060 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa0e415f3a0 0x7fa0e40d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:49.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.696+0000 7fa0e9685640 1 -- 172.21.15.73:0/3057777060 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa0e415f3a0 msgr2=0x7fa0e40d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:49.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.696+0000 7fa0e9685640 1 --2- 172.21.15.73:0/3057777060 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa0e415f3a0 0x7fa0e40d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:49.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.696+0000 7fa0e9685640 1 -- 172.21.15.73:0/3057777060 >> v2:172.21.15.120:3300/0 conn(0x7fa0e40d2130 msgr2=0x7fa0e40d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:49.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.696+0000 7fa0e9685640 1 --2- 172.21.15.73:0/3057777060 >> v2:172.21.15.120:3300/0 conn(0x7fa0e40d2130 0x7fa0e40d2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:49.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.696+0000 7fa0e9685640 1 -- 172.21.15.73:0/3057777060 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0e40b2050 con 0x7fa0e40d1770 2024-04-26T20:51:49.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.696+0000 7fa0e9e86640 1 --2- 172.21.15.73:0/3057777060 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa0e415f3a0 0x7fa0e40d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:49.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.696+0000 7fa0e9685640 1 --2- 172.21.15.73:0/3057777060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0e40d1770 0x7fa0e40d1bf0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fa0e408e8d0 tx=0x7fa0d402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59ff6e1ab92f07b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:49.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.696+0000 7fa0e8e84640 1 -- 172.21.15.73:0/3057777060 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d4037070 con 0x7fa0e40d1770 2024-04-26T20:51:49.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.697+0000 7fa0e8e84640 1 -- 172.21.15.73:0/3057777060 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0d4032440 con 0x7fa0e40d1770 2024-04-26T20:51:49.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.697+0000 7fa0e8e84640 1 -- 172.21.15.73:0/3057777060 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d4032810 con 0x7fa0e40d1770 2024-04-26T20:51:49.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.697+0000 7fa0ec0ea640 1 -- 172.21.15.73:0/3057777060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0e40d1770 msgr2=0x7fa0e40d1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:49.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.697+0000 7fa0ec0ea640 1 --2- 172.21.15.73:0/3057777060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0e40d1770 0x7fa0e40d1bf0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fa0e408e8d0 tx=0x7fa0d402a1e0 comp rx=0 tx=0).stop 2024-04-26T20:51:49.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.697+0000 7fa0ec0ea640 1 -- 172.21.15.73:0/3057777060 shutdown_connections 2024-04-26T20:51:49.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.697+0000 7fa0ec0ea640 1 --2- 172.21.15.73:0/3057777060 >> v2:172.21.15.120:3300/0 conn(0x7fa0e40d2130 0x7fa0e40d2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:49.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.697+0000 7fa0ec0ea640 1 --2- 172.21.15.73:0/3057777060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0e40d1770 0x7fa0e40d1bf0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:49.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.697+0000 7fa0ec0ea640 1 --2- 172.21.15.73:0/3057777060 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa0e415f3a0 0x7fa0e40d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:49.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.697+0000 7fa0ec0ea640 1 -- 172.21.15.73:0/3057777060 >> 172.21.15.73:0/3057777060 conn(0x7fa0e40d4f90 msgr2=0x7fa0e40d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:49.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.697+0000 7fa0ec0ea640 1 -- 172.21.15.73:0/3057777060 shutdown_connections 2024-04-26T20:51:49.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.697+0000 7fa0ec0ea640 1 -- 172.21.15.73:0/3057777060 wait complete. 2024-04-26T20:51:49.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.698+0000 7fa0ec0ea640 1 Processor -- start 2024-04-26T20:51:49.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.698+0000 7fa0ec0ea640 1 -- start start 2024-04-26T20:51:49.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.699+0000 7fa0ec0ea640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0e40d1770 0x7fa0e4158b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:49.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.699+0000 7fa0ec0ea640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0e40d2130 0x7fa0e41f4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:49.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.699+0000 7fa0ec0ea640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa0e415f3a0 0x7fa0e40b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:49.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.699+0000 7fa0ec0ea640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e415f900 con 0x7fa0e40d1770 2024-04-26T20:51:49.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.699+0000 7fa0ec0ea640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e415f780 con 0x7fa0e40d2130 2024-04-26T20:51:49.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.699+0000 7fa0ec0ea640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fa0e40d3390 con 0x7fa0e415f3a0 2024-04-26T20:51:49.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.699+0000 7fa0e9e86640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0e40d1770 0x7fa0e4158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:49.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.699+0000 7fa0e9e86640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0e40d1770 0x7fa0e4158b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41002/0 (socket says 172.21.15.73:41002) 2024-04-26T20:51:49.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.699+0000 7fa0e9e86640 1 -- 172.21.15.73:0/3434814543 learned_addr learned my addr 172.21.15.73:0/3434814543 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:49.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.699+0000 7fa0e9685640 1 --2- 172.21.15.73:0/3434814543 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0e40d2130 0x7fa0e41f4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:49.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.699+0000 7fa0ea687640 1 --2- 172.21.15.73:0/3434814543 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa0e415f3a0 0x7fa0e40b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:49.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.699+0000 7fa0e9e86640 1 -- 172.21.15.73:0/3434814543 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa0e415f3a0 msgr2=0x7fa0e40b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:49.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.699+0000 7fa0e9e86640 1 --2- 172.21.15.73:0/3434814543 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa0e415f3a0 0x7fa0e40b4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:49.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.699+0000 7fa0e9e86640 1 -- 172.21.15.73:0/3434814543 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0e40d2130 msgr2=0x7fa0e41f4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:49.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.699+0000 7fa0e9e86640 1 --2- 172.21.15.73:0/3434814543 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0e40d2130 0x7fa0e41f4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:49.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.699+0000 7fa0e9e86640 1 -- 172.21.15.73:0/3434814543 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0e41f5950 con 0x7fa0e40d1770 2024-04-26T20:51:49.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.699+0000 7fa0e9685640 1 --2- 172.21.15.73:0/3434814543 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0e40d2130 0x7fa0e41f4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:49.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.699+0000 7fa0e9e86640 1 --2- 172.21.15.73:0/3434814543 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0e40d1770 0x7fa0e4158b80 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0007f20 tx=0x7fa0e000d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:49.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.700+0000 7fa0da7fc640 1 -- 172.21.15.73:0/3434814543 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e001a070 con 0x7fa0e40d1770 2024-04-26T20:51:49.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.700+0000 7fa0da7fc640 1 -- 172.21.15.73:0/3434814543 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0e0003b90 con 0x7fa0e40d1770 2024-04-26T20:51:49.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.700+0000 7fa0ec0ea640 1 -- 172.21.15.73:0/3434814543 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0e41f5c30 con 0x7fa0e40d1770 2024-04-26T20:51:49.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.700+0000 7fa0da7fc640 1 -- 172.21.15.73:0/3434814543 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e0004030 con 0x7fa0e40d1770 2024-04-26T20:51:49.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.700+0000 7fa0ec0ea640 1 -- 172.21.15.73:0/3434814543 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0e41f61f0 con 0x7fa0e40d1770 2024-04-26T20:51:49.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.701+0000 7fa0ec0ea640 1 -- 172.21.15.73:0/3434814543 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0e40d1590 con 0x7fa0e40d1770 2024-04-26T20:51:49.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.701+0000 7fa0da7fc640 1 -- 172.21.15.73:0/3434814543 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa0e001d070 con 0x7fa0e40d1770 2024-04-26T20:51:49.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.702+0000 7fa0da7fc640 1 --2- 172.21.15.73:0/3434814543 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa0c806c8f0 0x7fa0c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:49.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.702+0000 7fa0da7fc640 1 -- 172.21.15.73:0/3434814543 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fa0e008be90 con 0x7fa0e40d1770 2024-04-26T20:51:49.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.702+0000 7fa0e9685640 1 --2- 172.21.15.73:0/3434814543 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa0c806c8f0 0x7fa0c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:49.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.702+0000 7fa0e9685640 1 --2- 172.21.15.73:0/3434814543 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa0c806c8f0 0x7fa0c806edb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa0d4034000 tx=0x7fa0d402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:49.706 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.706+0000 7fa0da7fc640 1 -- 172.21.15.73:0/3434814543 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa0e40d1590 con 0x7fa0e40d1770 2024-04-26T20:51:49.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.948+0000 7fa0ec0ea640 1 -- 172.21.15.73:0/3434814543 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fa0e40694f0 con 0x7fa0e40d1770 2024-04-26T20:51:49.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.949+0000 7fa0da7fc640 1 -- 172.21.15.73:0/3434814543 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1195 (secure 0 0 0) 0x7fa0e0010070 con 0x7fa0e40d1770 2024-04-26T20:51:49.950 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:51:49.950 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"fbd3f548-b547-4713-856b-09f98f5d1764","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":102,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":39,"num_osds":8,"num_up_osds":8,"osd_up_since":1714164619,"num_in_osds":8,"osd_in_since":1714164610,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":225845248,"bytes_avail":772868268032,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":15,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"d","status":"up:active","gid":5053}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-04-26T20:51:27.993985+0000","services":{"mds":{"daemons":{"summary":"","c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-26T20:51:49.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.951+0000 7fa0ec0ea640 1 -- 172.21.15.73:0/3434814543 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa0c806c8f0 msgr2=0x7fa0c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:49.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.951+0000 7fa0ec0ea640 1 --2- 172.21.15.73:0/3434814543 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa0c806c8f0 0x7fa0c806edb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa0d4034000 tx=0x7fa0d402e000 comp rx=0 tx=0).stop 2024-04-26T20:51:49.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.951+0000 7fa0ec0ea640 1 -- 172.21.15.73:0/3434814543 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0e40d1770 msgr2=0x7fa0e4158b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:49.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.951+0000 7fa0ec0ea640 1 --2- 172.21.15.73:0/3434814543 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0e40d1770 0x7fa0e4158b80 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0007f20 tx=0x7fa0e000d560 comp rx=0 tx=0).stop 2024-04-26T20:51:49.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.951+0000 7fa0ec0ea640 1 -- 172.21.15.73:0/3434814543 shutdown_connections 2024-04-26T20:51:49.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.951+0000 7fa0ec0ea640 1 --2- 172.21.15.73:0/3434814543 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa0c806c8f0 0x7fa0c806edb0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:49.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.951+0000 7fa0ec0ea640 1 --2- 172.21.15.73:0/3434814543 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa0e415f3a0 0x7fa0e40b4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:49.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.951+0000 7fa0ec0ea640 1 --2- 172.21.15.73:0/3434814543 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0e40d2130 0x7fa0e41f4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:49.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.951+0000 7fa0ec0ea640 1 --2- 172.21.15.73:0/3434814543 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0e40d1770 0x7fa0e4158b80 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:49.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.951+0000 7fa0ec0ea640 1 -- 172.21.15.73:0/3434814543 >> 172.21.15.73:0/3434814543 conn(0x7fa0e40d4f90 msgr2=0x7fa0e40d42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:49.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.951+0000 7fa0ec0ea640 1 -- 172.21.15.73:0/3434814543 shutdown_connections 2024-04-26T20:51:49.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:49.951+0000 7fa0ec0ea640 1 -- 172.21.15.73:0/3434814543 wait complete. 2024-04-26T20:51:49.962 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-26T20:51:49.963 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:51:50.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.058+0000 7f78f4c08640 1 Processor -- start 2024-04-26T20:51:50.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.058+0000 7f78f4c08640 1 -- start start 2024-04-26T20:51:50.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.059+0000 7f78f4c08640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f78f01608f0 0x7f78f00d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:50.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.059+0000 7f78f4c08640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78f00cde20 con 0x7f78f00d0a90 2024-04-26T20:51:50.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.059+0000 7f78f4c08640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78f00ce100 con 0x7f78f00cffa0 2024-04-26T20:51:50.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.059+0000 7f78f4c08640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f78f00cea30 con 0x7f78f01608f0 2024-04-26T20:51:50.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.059+0000 7f78ee59c640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f78f01608f0 0x7f78f00d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:50.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.059+0000 7f78edd9b640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f78f00d0a90 0x7f78f00cf890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40540/0 (socket says 172.21.15.73:40540) 2024-04-26T20:51:50.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.059+0000 7f78edd9b640 1 -- 172.21.15.73:0/3577508036 learned_addr learned my addr 172.21.15.73:0/3577508036 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:50.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.060+0000 7f78ee59c640 1 -- 172.21.15.73:0/3577508036 >> v1:172.21.15.120:6789/0 conn(0x7f78f00cffa0 legacy=0x7f78f00b2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:50.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.060+0000 7f78ee59c640 1 -- 172.21.15.73:0/3577508036 >> v1:172.21.15.73:6789/0 conn(0x7f78f00d0a90 legacy=0x7f78f00cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:50.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.060+0000 7f78ee59c640 1 -- 172.21.15.73:0/3577508036 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78f00ce3e0 con 0x7f78f01608f0 2024-04-26T20:51:50.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.060+0000 7f78ed59a640 1 -- 172.21.15.73:0/3577508036 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4058674585 0 0) 0x7f78f00cde20 con 0x7f78f00d0a90 2024-04-26T20:51:50.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.060+0000 7f78ee59c640 1 --2- 172.21.15.73:0/3577508036 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78f01608f0 0x7f78f00d1bf0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f78e8009d90 tx=0x7f78e802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa226af6cab1ca75 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:50.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.061+0000 7f78ed59a640 1 -- 172.21.15.73:0/3577508036 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e8007740 con 0x7f78f01608f0 2024-04-26T20:51:50.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.061+0000 7f78ed59a640 1 -- 172.21.15.73:0/3577508036 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78e80078e0 con 0x7f78f01608f0 2024-04-26T20:51:50.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.061+0000 7f78ed59a640 1 -- 172.21.15.73:0/3577508036 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e8007cb0 con 0x7f78f01608f0 2024-04-26T20:51:50.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.061+0000 7f78f4c08640 1 -- 172.21.15.73:0/3577508036 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78f01608f0 msgr2=0x7f78f00d1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:50.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.061+0000 7f78f4c08640 1 --2- 172.21.15.73:0/3577508036 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78f01608f0 0x7f78f00d1bf0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f78e8009d90 tx=0x7f78e802a6f0 comp rx=0 tx=0).stop 2024-04-26T20:51:50.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.061+0000 7f78f4c08640 1 -- 172.21.15.73:0/3577508036 shutdown_connections 2024-04-26T20:51:50.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.061+0000 7f78f4c08640 1 --2- 172.21.15.73:0/3577508036 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78f01608f0 0x7f78f00d1bf0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:50.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.061+0000 7f78f4c08640 1 -- 172.21.15.73:0/3577508036 >> 172.21.15.73:0/3577508036 conn(0x7f78f00d2ec0 msgr2=0x7f78f00d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:50.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.061+0000 7f78f4c08640 1 -- 172.21.15.73:0/3577508036 shutdown_connections 2024-04-26T20:51:50.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.061+0000 7f78f4c08640 1 -- 172.21.15.73:0/3577508036 wait complete. 2024-04-26T20:51:50.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.062+0000 7f78f4c08640 1 Processor -- start 2024-04-26T20:51:50.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.063+0000 7f78f4c08640 1 -- start start 2024-04-26T20:51:50.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.063+0000 7f78f4c08640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f78f00cffa0 0x7f78f0158330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:50.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.063+0000 7f78f4c08640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78f00d0a90 0x7f78f0157c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:50.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.063+0000 7f78f4c08640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78f01608f0 0x7f78f00cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:50.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.063+0000 7f78f4c08640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f78f00ceec0 con 0x7f78f01608f0 2024-04-26T20:51:50.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.063+0000 7f78f4c08640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f78f00d0ef0 con 0x7f78f00d0a90 2024-04-26T20:51:50.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.063+0000 7f78f4c08640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f78f00cda40 con 0x7f78f00cffa0 2024-04-26T20:51:50.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.063+0000 7f78eed9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78f01608f0 0x7f78f00cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:50.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.063+0000 7f78eed9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78f01608f0 0x7f78f00cafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41026/0 (socket says 172.21.15.73:41026) 2024-04-26T20:51:50.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.063+0000 7f78eed9d640 1 -- 172.21.15.73:0/4000806483 learned_addr learned my addr 172.21.15.73:0/4000806483 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:50.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.064+0000 7f78edd9b640 1 --2- 172.21.15.73:0/4000806483 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78f00d0a90 0x7f78f0157c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:50.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.064+0000 7f78eed9d640 1 -- 172.21.15.73:0/4000806483 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f78f00cffa0 msgr2=0x7f78f0158330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:50.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.064+0000 7f78eed9d640 1 --2- 172.21.15.73:0/4000806483 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f78f00cffa0 0x7f78f0158330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:50.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.064+0000 7f78eed9d640 1 -- 172.21.15.73:0/4000806483 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78f00d0a90 msgr2=0x7f78f0157c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:50.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.064+0000 7f78eed9d640 1 --2- 172.21.15.73:0/4000806483 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78f00d0a90 0x7f78f0157c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:50.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.064+0000 7f78eed9d640 1 -- 172.21.15.73:0/4000806483 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78f015d770 con 0x7f78f01608f0 2024-04-26T20:51:50.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.064+0000 7f78edd9b640 1 --2- 172.21.15.73:0/4000806483 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78f00d0a90 0x7f78f0157c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:50.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.064+0000 7f78eed9d640 1 --2- 172.21.15.73:0/4000806483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78f01608f0 0x7f78f00cafe0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f78e000c3f0 tx=0x7f78e000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:50.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.064+0000 7f78e6ffd640 1 -- 172.21.15.73:0/4000806483 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e0017070 con 0x7f78f01608f0 2024-04-26T20:51:50.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.064+0000 7f78e6ffd640 1 -- 172.21.15.73:0/4000806483 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78e00052d0 con 0x7f78f01608f0 2024-04-26T20:51:50.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.064+0000 7f78e6ffd640 1 -- 172.21.15.73:0/4000806483 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e0003830 con 0x7f78f01608f0 2024-04-26T20:51:50.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.064+0000 7f78f4c08640 1 -- 172.21.15.73:0/4000806483 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78f00cb750 con 0x7f78f01608f0 2024-04-26T20:51:50.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.064+0000 7f78f4c08640 1 -- 172.21.15.73:0/4000806483 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78f00cbd10 con 0x7f78f01608f0 2024-04-26T20:51:50.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.065+0000 7f78f4c08640 1 -- 172.21.15.73:0/4000806483 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78f00cbf50 con 0x7f78f01608f0 2024-04-26T20:51:50.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.066+0000 7f78e6ffd640 1 -- 172.21.15.73:0/4000806483 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f78e001a070 con 0x7f78f01608f0 2024-04-26T20:51:50.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.066+0000 7f78e6ffd640 1 --2- 172.21.15.73:0/4000806483 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f78d006c920 0x7f78d006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:50.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.066+0000 7f78ee59c640 1 --2- 172.21.15.73:0/4000806483 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f78d006c920 0x7f78d006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:50.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.066+0000 7f78e6ffd640 1 -- 172.21.15.73:0/4000806483 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f78e008af60 con 0x7f78f01608f0 2024-04-26T20:51:50.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.066+0000 7f78ee59c640 1 --2- 172.21.15.73:0/4000806483 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f78d006c920 0x7f78d006ede0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f78e8034000 tx=0x7f78e802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:50.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.070+0000 7f78e6ffd640 1 -- 172.21.15.73:0/4000806483 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f78e00592c0 con 0x7f78f01608f0 2024-04-26T20:51:50.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.242+0000 7f78f4c08640 1 -- 172.21.15.73:0/4000806483 --> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f78f00694f0 con 0x7f78d006c920 2024-04-26T20:51:50.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.248+0000 7f78e6ffd640 1 -- 172.21.15.73:0/4000806483 <== mgr.4107 v2:172.21.15.73:6832/666905549 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246635 (secure 0 0 0) 0x7f78f00694f0 con 0x7f78d006c920 2024-04-26T20:51:50.251 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:51:50.252 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-26T20:51:50.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.251+0000 7f78f4c08640 1 -- 172.21.15.73:0/4000806483 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f78d006c920 msgr2=0x7f78d006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:50.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.251+0000 7f78f4c08640 1 --2- 172.21.15.73:0/4000806483 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f78d006c920 0x7f78d006ede0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f78e8034000 tx=0x7f78e802e000 comp rx=0 tx=0).stop 2024-04-26T20:51:50.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.251+0000 7f78f4c08640 1 -- 172.21.15.73:0/4000806483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78f01608f0 msgr2=0x7f78f00cafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:50.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.251+0000 7f78f4c08640 1 --2- 172.21.15.73:0/4000806483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78f01608f0 0x7f78f00cafe0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f78e000c3f0 tx=0x7f78e000c7f0 comp rx=0 tx=0).stop 2024-04-26T20:51:50.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.251+0000 7f78f4c08640 1 -- 172.21.15.73:0/4000806483 shutdown_connections 2024-04-26T20:51:50.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.251+0000 7f78f4c08640 1 --2- 172.21.15.73:0/4000806483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78f01608f0 0x7f78f00cafe0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:50.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.251+0000 7f78f4c08640 1 --2- 172.21.15.73:0/4000806483 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f78d006c920 0x7f78d006ede0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:50.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.251+0000 7f78f4c08640 1 --2- 172.21.15.73:0/4000806483 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78f00d0a90 0x7f78f0157c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:50.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.251+0000 7f78f4c08640 1 --2- 172.21.15.73:0/4000806483 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f78f00cffa0 0x7f78f0158330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:50.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.251+0000 7f78f4c08640 1 -- 172.21.15.73:0/4000806483 >> 172.21.15.73:0/4000806483 conn(0x7f78f00d2ec0 msgr2=0x7f78f00b6170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:50.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.251+0000 7f78f4c08640 1 -- 172.21.15.73:0/4000806483 shutdown_connections 2024-04-26T20:51:50.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:50.251+0000 7f78f4c08640 1 -- 172.21.15.73:0/4000806483 wait complete. 2024-04-26T20:51:50.262 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":77,"stamp":"2024-04-26T20:51:49.997717+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220552,"kb_used_data":4392,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754168,"statfs":{"total":773094113280,"available":772868268032,"internally_reserved":0,"allocated":4497408,"data_stored":2593784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000825"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.927134+0000","last_change":"2024-04-26T20:51:44.927134+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.927134+0000","last_undegraded":"2024-04-26T20:51:44.927134+0000","last_fullsized":"2024-04-26T20:51:44.927134+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.927081+0000","last_change":"2024-04-26T20:51:44.927081+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.927081+0000","last_undegraded":"2024-04-26T20:51:44.927081+0000","last_fullsized":"2024-04-26T20:51:44.927081+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.930409+0000","last_change":"2024-04-26T20:51:44.930409+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.930409+0000","last_undegraded":"2024-04-26T20:51:44.930409+0000","last_fullsized":"2024-04-26T20:51:44.930409+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.926209+0000","last_change":"2024-04-26T20:51:44.926209+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.926209+0000","last_undegraded":"2024-04-26T20:51:44.926209+0000","last_fullsized":"2024-04-26T20:51:44.926209+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.925479+0000","last_change":"2024-04-26T20:51:44.925479+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.925479+0000","last_undegraded":"2024-04-26T20:51:44.925479+0000","last_fullsized":"2024-04-26T20:51:44.925479+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.934070+0000","last_change":"2024-04-26T20:51:44.934070+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.934070+0000","last_undegraded":"2024-04-26T20:51:44.934070+0000","last_fullsized":"2024-04-26T20:51:44.934070+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.930809+0000","last_change":"2024-04-26T20:51:44.930809+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.930809+0000","last_undegraded":"2024-04-26T20:51:44.930809+0000","last_fullsized":"2024-04-26T20:51:44.930809+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.926467+0000","last_change":"2024-04-26T20:51:44.926467+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.926467+0000","last_undegraded":"2024-04-26T20:51:44.926467+0000","last_fullsized":"2024-04-26T20:51:44.926467+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.929217+0000","last_change":"2024-04-26T20:51:44.929217+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.929217+0000","last_undegraded":"2024-04-26T20:51:44.929217+0000","last_fullsized":"2024-04-26T20:51:44.929217+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.930277+0000","last_change":"2024-04-26T20:51:44.930277+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.930277+0000","last_undegraded":"2024-04-26T20:51:44.930277+0000","last_fullsized":"2024-04-26T20:51:44.930277+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.930633+0000","last_change":"2024-04-26T20:51:44.930633+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.930633+0000","last_undegraded":"2024-04-26T20:51:44.930633+0000","last_fullsized":"2024-04-26T20:51:44.930633+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.929572+0000","last_change":"2024-04-26T20:51:44.929572+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.929572+0000","last_undegraded":"2024-04-26T20:51:44.929572+0000","last_fullsized":"2024-04-26T20:51:44.929572+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.929667+0000","last_change":"2024-04-26T20:51:44.929667+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.929667+0000","last_undegraded":"2024-04-26T20:51:44.929667+0000","last_fullsized":"2024-04-26T20:51:44.929667+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.930603+0000","last_change":"2024-04-26T20:51:44.930603+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.930603+0000","last_undegraded":"2024-04-26T20:51:44.930603+0000","last_fullsized":"2024-04-26T20:51:44.930603+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.929584+0000","last_change":"2024-04-26T20:51:44.929584+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.929584+0000","last_undegraded":"2024-04-26T20:51:44.929584+0000","last_fullsized":"2024-04-26T20:51:44.929584+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.934714+0000","last_change":"2024-04-26T20:51:44.934714+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.934714+0000","last_undegraded":"2024-04-26T20:51:44.934714+0000","last_fullsized":"2024-04-26T20:51:44.934714+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.933252+0000","last_change":"2024-04-26T20:51:44.933252+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.933252+0000","last_undegraded":"2024-04-26T20:51:44.933252+0000","last_fullsized":"2024-04-26T20:51:44.933252+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.923957+0000","last_change":"2024-04-26T20:51:44.923957+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.923957+0000","last_undegraded":"2024-04-26T20:51:44.923957+0000","last_fullsized":"2024-04-26T20:51:44.923957+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.926243+0000","last_change":"2024-04-26T20:51:44.926243+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.926243+0000","last_undegraded":"2024-04-26T20:51:44.926243+0000","last_fullsized":"2024-04-26T20:51:44.926243+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.924107+0000","last_change":"2024-04-26T20:51:44.924107+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.924107+0000","last_undegraded":"2024-04-26T20:51:44.924107+0000","last_fullsized":"2024-04-26T20:51:44.924107+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.927282+0000","last_change":"2024-04-26T20:51:44.927282+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.927282+0000","last_undegraded":"2024-04-26T20:51:44.927282+0000","last_fullsized":"2024-04-26T20:51:44.927282+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.928722+0000","last_change":"2024-04-26T20:51:44.928722+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.928722+0000","last_undegraded":"2024-04-26T20:51:44.928722+0000","last_fullsized":"2024-04-26T20:51:44.928722+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.934479+0000","last_change":"2024-04-26T20:51:44.934479+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.934479+0000","last_undegraded":"2024-04-26T20:51:44.934479+0000","last_fullsized":"2024-04-26T20:51:44.934479+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.925511+0000","last_change":"2024-04-26T20:51:44.925511+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.925511+0000","last_undegraded":"2024-04-26T20:51:44.925511+0000","last_fullsized":"2024-04-26T20:51:44.925511+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.923419+0000","last_change":"2024-04-26T20:51:44.923419+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.923419+0000","last_undegraded":"2024-04-26T20:51:44.923419+0000","last_fullsized":"2024-04-26T20:51:44.923419+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.930697+0000","last_change":"2024-04-26T20:51:44.930697+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.930697+0000","last_undegraded":"2024-04-26T20:51:44.930697+0000","last_fullsized":"2024-04-26T20:51:44.930697+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.923397+0000","last_change":"2024-04-26T20:51:44.923397+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.923397+0000","last_undegraded":"2024-04-26T20:51:44.923397+0000","last_fullsized":"2024-04-26T20:51:44.923397+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.926829+0000","last_change":"2024-04-26T20:51:44.926829+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.926829+0000","last_undegraded":"2024-04-26T20:51:44.926829+0000","last_fullsized":"2024-04-26T20:51:44.926829+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.923542+0000","last_change":"2024-04-26T20:51:44.923542+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.923542+0000","last_undegraded":"2024-04-26T20:51:44.923542+0000","last_fullsized":"2024-04-26T20:51:44.923542+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.923673+0000","last_change":"2024-04-26T20:51:44.923673+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.923673+0000","last_undegraded":"2024-04-26T20:51:44.923673+0000","last_fullsized":"2024-04-26T20:51:44.923673+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.932216+0000","last_change":"2024-04-26T20:51:44.932216+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.932216+0000","last_undegraded":"2024-04-26T20:51:44.932216+0000","last_fullsized":"2024-04-26T20:51:44.932216+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.930954+0000","last_change":"2024-04-26T20:51:44.930954+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.930954+0000","last_undegraded":"2024-04-26T20:51:44.930954+0000","last_fullsized":"2024-04-26T20:51:44.930954+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.932301+0000","last_change":"2024-04-26T20:51:44.932301+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.932301+0000","last_undegraded":"2024-04-26T20:51:44.932301+0000","last_fullsized":"2024-04-26T20:51:44.932301+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.936717+0000","last_change":"2024-04-26T20:51:44.935237+0000","last_active":"2024-04-26T20:51:44.936717+0000","last_peered":"2024-04-26T20:51:44.936717+0000","last_clean":"2024-04-26T20:51:44.936717+0000","last_became_active":"2024-04-26T20:51:44.934964+0000","last_became_peered":"2024-04-26T20:51:44.934964+0000","last_unstale":"2024-04-26T20:51:44.936717+0000","last_undegraded":"2024-04-26T20:51:44.936717+0000","last_fullsized":"2024-04-26T20:51:44.936717+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:22.556226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.934004+0000","last_change":"2024-04-26T20:51:44.934004+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.934004+0000","last_undegraded":"2024-04-26T20:51:44.934004+0000","last_fullsized":"2024-04-26T20:51:44.934004+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.940096+0000","last_change":"2024-04-26T20:51:44.937990+0000","last_active":"2024-04-26T20:51:44.940096+0000","last_peered":"2024-04-26T20:51:44.940096+0000","last_clean":"2024-04-26T20:51:44.940096+0000","last_became_active":"2024-04-26T20:51:44.937723+0000","last_became_peered":"2024-04-26T20:51:44.937723+0000","last_unstale":"2024-04-26T20:51:44.940096+0000","last_undegraded":"2024-04-26T20:51:44.940096+0000","last_fullsized":"2024-04-26T20:51:44.940096+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:48:23.398632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.928437+0000","last_change":"2024-04-26T20:51:44.928437+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.928437+0000","last_undegraded":"2024-04-26T20:51:44.928437+0000","last_fullsized":"2024-04-26T20:51:44.928437+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.943035+0000","last_change":"2024-04-26T20:51:44.939632+0000","last_active":"2024-04-26T20:51:44.943035+0000","last_peered":"2024-04-26T20:51:44.943035+0000","last_clean":"2024-04-26T20:51:44.943035+0000","last_became_active":"2024-04-26T20:51:44.939299+0000","last_became_peered":"2024-04-26T20:51:44.939299+0000","last_unstale":"2024-04-26T20:51:44.943035+0000","last_undegraded":"2024-04-26T20:51:44.943035+0000","last_fullsized":"2024-04-26T20:51:44.943035+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:35.270443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.925851+0000","last_change":"2024-04-26T20:51:44.925851+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.925851+0000","last_undegraded":"2024-04-26T20:51:44.925851+0000","last_fullsized":"2024-04-26T20:51:44.925851+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.939345+0000","last_change":"2024-04-26T20:51:44.937707+0000","last_active":"2024-04-26T20:51:44.939345+0000","last_peered":"2024-04-26T20:51:44.939345+0000","last_clean":"2024-04-26T20:51:44.939345+0000","last_became_active":"2024-04-26T20:51:44.937394+0000","last_became_peered":"2024-04-26T20:51:44.937394+0000","last_unstale":"2024-04-26T20:51:44.939345+0000","last_undegraded":"2024-04-26T20:51:44.939345+0000","last_fullsized":"2024-04-26T20:51:44.939345+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:14:24.515988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.930643+0000","last_change":"2024-04-26T20:51:44.930643+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.930643+0000","last_undegraded":"2024-04-26T20:51:44.930643+0000","last_fullsized":"2024-04-26T20:51:44.930643+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.939732+0000","last_change":"2024-04-26T20:51:44.937929+0000","last_active":"2024-04-26T20:51:44.939732+0000","last_peered":"2024-04-26T20:51:44.939732+0000","last_clean":"2024-04-26T20:51:44.939732+0000","last_became_active":"2024-04-26T20:51:44.937626+0000","last_became_peered":"2024-04-26T20:51:44.937626+0000","last_unstale":"2024-04-26T20:51:44.939732+0000","last_undegraded":"2024-04-26T20:51:44.939732+0000","last_fullsized":"2024-04-26T20:51:44.939732+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:48:23.398632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.926384+0000","last_change":"2024-04-26T20:51:44.926384+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.926384+0000","last_undegraded":"2024-04-26T20:51:44.926384+0000","last_fullsized":"2024-04-26T20:51:44.926384+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.945361+0000","last_change":"2024-04-26T20:51:44.944114+0000","last_active":"2024-04-26T20:51:44.945361+0000","last_peered":"2024-04-26T20:51:44.945361+0000","last_clean":"2024-04-26T20:51:44.945361+0000","last_became_active":"2024-04-26T20:51:44.943821+0000","last_became_peered":"2024-04-26T20:51:44.943821+0000","last_unstale":"2024-04-26T20:51:44.945361+0000","last_undegraded":"2024-04-26T20:51:44.945361+0000","last_fullsized":"2024-04-26T20:51:44.945361+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:28:21.811767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.927913+0000","last_change":"2024-04-26T20:51:44.927913+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.927913+0000","last_undegraded":"2024-04-26T20:51:44.927913+0000","last_fullsized":"2024-04-26T20:51:44.927913+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.943098+0000","last_change":"2024-04-26T20:51:44.942003+0000","last_active":"2024-04-26T20:51:44.943098+0000","last_peered":"2024-04-26T20:51:44.943098+0000","last_clean":"2024-04-26T20:51:44.943098+0000","last_became_active":"2024-04-26T20:51:44.941646+0000","last_became_peered":"2024-04-26T20:51:44.941646+0000","last_unstale":"2024-04-26T20:51:44.943098+0000","last_undegraded":"2024-04-26T20:51:44.943098+0000","last_fullsized":"2024-04-26T20:51:44.943098+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:14:24.515988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.926719+0000","last_change":"2024-04-26T20:51:44.926719+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.926719+0000","last_undegraded":"2024-04-26T20:51:44.926719+0000","last_fullsized":"2024-04-26T20:51:44.926719+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.936220+0000","last_change":"2024-04-26T20:51:44.934491+0000","last_active":"2024-04-26T20:51:44.936220+0000","last_peered":"2024-04-26T20:51:44.936220+0000","last_clean":"2024-04-26T20:51:44.936220+0000","last_became_active":"2024-04-26T20:51:44.934145+0000","last_became_peered":"2024-04-26T20:51:44.934145+0000","last_unstale":"2024-04-26T20:51:44.936220+0000","last_undegraded":"2024-04-26T20:51:44.936220+0000","last_fullsized":"2024-04-26T20:51:44.936220+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:48:23.398632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.925518+0000","last_change":"2024-04-26T20:51:44.925518+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.925518+0000","last_undegraded":"2024-04-26T20:51:44.925518+0000","last_fullsized":"2024-04-26T20:51:44.925518+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.941952+0000","last_change":"2024-04-26T20:51:44.938418+0000","last_active":"2024-04-26T20:51:44.941952+0000","last_peered":"2024-04-26T20:51:44.941952+0000","last_clean":"2024-04-26T20:51:44.941952+0000","last_became_active":"2024-04-26T20:51:44.938146+0000","last_became_peered":"2024-04-26T20:51:44.938146+0000","last_unstale":"2024-04-26T20:51:44.941952+0000","last_undegraded":"2024-04-26T20:51:44.941952+0000","last_fullsized":"2024-04-26T20:51:44.941952+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:04:44.088012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.940376+0000","last_change":"2024-04-26T20:51:44.940376+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.940376+0000","last_undegraded":"2024-04-26T20:51:44.940376+0000","last_fullsized":"2024-04-26T20:51:44.940376+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.941993+0000","last_change":"2024-04-26T20:51:44.941093+0000","last_active":"2024-04-26T20:51:44.941993+0000","last_peered":"2024-04-26T20:51:44.941993+0000","last_clean":"2024-04-26T20:51:44.941993+0000","last_became_active":"2024-04-26T20:51:44.940805+0000","last_became_peered":"2024-04-26T20:51:44.940805+0000","last_unstale":"2024-04-26T20:51:44.941993+0000","last_undegraded":"2024-04-26T20:51:44.941993+0000","last_fullsized":"2024-04-26T20:51:44.941993+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:10:56.604114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.930862+0000","last_change":"2024-04-26T20:51:44.930862+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.930862+0000","last_undegraded":"2024-04-26T20:51:44.930862+0000","last_fullsized":"2024-04-26T20:51:44.930862+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.939847+0000","last_change":"2024-04-26T20:51:44.939214+0000","last_active":"2024-04-26T20:51:44.939847+0000","last_peered":"2024-04-26T20:51:44.939847+0000","last_clean":"2024-04-26T20:51:44.939847+0000","last_became_active":"2024-04-26T20:51:44.938907+0000","last_became_peered":"2024-04-26T20:51:44.938907+0000","last_unstale":"2024-04-26T20:51:44.939847+0000","last_undegraded":"2024-04-26T20:51:44.939847+0000","last_fullsized":"2024-04-26T20:51:44.939847+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:17.661527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.930135+0000","last_change":"2024-04-26T20:51:44.930135+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.930135+0000","last_undegraded":"2024-04-26T20:51:44.930135+0000","last_fullsized":"2024-04-26T20:51:44.930135+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.942052+0000","last_change":"2024-04-26T20:51:44.939371+0000","last_active":"2024-04-26T20:51:44.942052+0000","last_peered":"2024-04-26T20:51:44.942052+0000","last_clean":"2024-04-26T20:51:44.942052+0000","last_became_active":"2024-04-26T20:51:44.939090+0000","last_became_peered":"2024-04-26T20:51:44.939090+0000","last_unstale":"2024-04-26T20:51:44.942052+0000","last_undegraded":"2024-04-26T20:51:44.942052+0000","last_fullsized":"2024-04-26T20:51:44.942052+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:40:59.156357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.930280+0000","last_change":"2024-04-26T20:51:44.930280+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.930280+0000","last_undegraded":"2024-04-26T20:51:44.930280+0000","last_fullsized":"2024-04-26T20:51:44.930280+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.942877+0000","last_change":"2024-04-26T20:51:44.940477+0000","last_active":"2024-04-26T20:51:44.942877+0000","last_peered":"2024-04-26T20:51:44.942877+0000","last_clean":"2024-04-26T20:51:44.942877+0000","last_became_active":"2024-04-26T20:51:44.940192+0000","last_became_peered":"2024-04-26T20:51:44.940192+0000","last_unstale":"2024-04-26T20:51:44.942877+0000","last_undegraded":"2024-04-26T20:51:44.942877+0000","last_fullsized":"2024-04-26T20:51:44.942877+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:38:31.038452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.926418+0000","last_change":"2024-04-26T20:51:44.926418+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.926418+0000","last_undegraded":"2024-04-26T20:51:44.926418+0000","last_fullsized":"2024-04-26T20:51:44.926418+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.943067+0000","last_change":"2024-04-26T20:51:44.940382+0000","last_active":"2024-04-26T20:51:44.943067+0000","last_peered":"2024-04-26T20:51:44.943067+0000","last_clean":"2024-04-26T20:51:44.943067+0000","last_became_active":"2024-04-26T20:51:44.940109+0000","last_became_peered":"2024-04-26T20:51:44.940109+0000","last_unstale":"2024-04-26T20:51:44.943067+0000","last_undegraded":"2024-04-26T20:51:44.943067+0000","last_fullsized":"2024-04-26T20:51:44.943067+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:48:23.398632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.932149+0000","last_change":"2024-04-26T20:51:44.932149+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.932149+0000","last_undegraded":"2024-04-26T20:51:44.932149+0000","last_fullsized":"2024-04-26T20:51:44.932149+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.936667+0000","last_change":"2024-04-26T20:51:44.934857+0000","last_active":"2024-04-26T20:51:44.936667+0000","last_peered":"2024-04-26T20:51:44.936667+0000","last_clean":"2024-04-26T20:51:44.936667+0000","last_became_active":"2024-04-26T20:51:44.934556+0000","last_became_peered":"2024-04-26T20:51:44.934556+0000","last_unstale":"2024-04-26T20:51:44.936667+0000","last_undegraded":"2024-04-26T20:51:44.936667+0000","last_fullsized":"2024-04-26T20:51:44.936667+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:04:31.671170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.925836+0000","last_change":"2024-04-26T20:51:44.925836+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.925836+0000","last_undegraded":"2024-04-26T20:51:44.925836+0000","last_fullsized":"2024-04-26T20:51:44.925836+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.944907+0000","last_change":"2024-04-26T20:51:44.943632+0000","last_active":"2024-04-26T20:51:44.944907+0000","last_peered":"2024-04-26T20:51:44.944907+0000","last_clean":"2024-04-26T20:51:44.944907+0000","last_became_active":"2024-04-26T20:51:44.943278+0000","last_became_peered":"2024-04-26T20:51:44.943278+0000","last_unstale":"2024-04-26T20:51:44.944907+0000","last_undegraded":"2024-04-26T20:51:44.944907+0000","last_fullsized":"2024-04-26T20:51:44.944907+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:59:50.250119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.926462+0000","last_change":"2024-04-26T20:51:44.926462+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.926462+0000","last_undegraded":"2024-04-26T20:51:44.926462+0000","last_fullsized":"2024-04-26T20:51:44.926462+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.937919+0000","last_change":"2024-04-26T20:51:44.935150+0000","last_active":"2024-04-26T20:51:44.937919+0000","last_peered":"2024-04-26T20:51:44.937919+0000","last_clean":"2024-04-26T20:51:44.937919+0000","last_became_active":"2024-04-26T20:51:44.934078+0000","last_became_peered":"2024-04-26T20:51:44.934078+0000","last_unstale":"2024-04-26T20:51:44.937919+0000","last_undegraded":"2024-04-26T20:51:44.937919+0000","last_fullsized":"2024-04-26T20:51:44.937919+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:35.270443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.926645+0000","last_change":"2024-04-26T20:51:44.926645+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.926645+0000","last_undegraded":"2024-04-26T20:51:44.926645+0000","last_fullsized":"2024-04-26T20:51:44.926645+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.934725+0000","last_change":"2024-04-26T20:51:44.932407+0000","last_active":"2024-04-26T20:51:44.934725+0000","last_peered":"2024-04-26T20:51:44.934725+0000","last_clean":"2024-04-26T20:51:44.934725+0000","last_became_active":"2024-04-26T20:51:44.931987+0000","last_became_peered":"2024-04-26T20:51:44.931987+0000","last_unstale":"2024-04-26T20:51:44.934725+0000","last_undegraded":"2024-04-26T20:51:44.934725+0000","last_fullsized":"2024-04-26T20:51:44.934725+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:36:55.165610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.929390+0000","last_change":"2024-04-26T20:51:44.929390+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.929390+0000","last_undegraded":"2024-04-26T20:51:44.929390+0000","last_fullsized":"2024-04-26T20:51:44.929390+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.943664+0000","last_change":"2024-04-26T20:51:44.941836+0000","last_active":"2024-04-26T20:51:44.943664+0000","last_peered":"2024-04-26T20:51:44.943664+0000","last_clean":"2024-04-26T20:51:44.943664+0000","last_became_active":"2024-04-26T20:51:44.941581+0000","last_became_peered":"2024-04-26T20:51:44.941581+0000","last_unstale":"2024-04-26T20:51:44.943664+0000","last_undegraded":"2024-04-26T20:51:44.943664+0000","last_fullsized":"2024-04-26T20:51:44.943664+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:30:26.145171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.924912+0000","last_change":"2024-04-26T20:51:44.924912+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.924912+0000","last_undegraded":"2024-04-26T20:51:44.924912+0000","last_fullsized":"2024-04-26T20:51:44.924912+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.942287+0000","last_change":"2024-04-26T20:51:44.938146+0000","last_active":"2024-04-26T20:51:44.942287+0000","last_peered":"2024-04-26T20:51:44.942287+0000","last_clean":"2024-04-26T20:51:44.942287+0000","last_became_active":"2024-04-26T20:51:44.937861+0000","last_became_peered":"2024-04-26T20:51:44.937861+0000","last_unstale":"2024-04-26T20:51:44.942287+0000","last_undegraded":"2024-04-26T20:51:44.942287+0000","last_fullsized":"2024-04-26T20:51:44.942287+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:36:55.165610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.932596+0000","last_change":"2024-04-26T20:51:44.932596+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.932596+0000","last_undegraded":"2024-04-26T20:51:44.932596+0000","last_fullsized":"2024-04-26T20:51:44.932596+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.941851+0000","last_change":"2024-04-26T20:51:44.940796+0000","last_active":"2024-04-26T20:51:44.941851+0000","last_peered":"2024-04-26T20:51:44.941851+0000","last_clean":"2024-04-26T20:51:44.941851+0000","last_became_active":"2024-04-26T20:51:44.940524+0000","last_became_peered":"2024-04-26T20:51:44.940524+0000","last_unstale":"2024-04-26T20:51:44.941851+0000","last_undegraded":"2024-04-26T20:51:44.941851+0000","last_fullsized":"2024-04-26T20:51:44.941851+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:31:06.346238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.929053+0000","last_change":"2024-04-26T20:51:44.929053+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.929053+0000","last_undegraded":"2024-04-26T20:51:44.929053+0000","last_fullsized":"2024-04-26T20:51:44.929053+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.939800+0000","last_change":"2024-04-26T20:51:44.934985+0000","last_active":"2024-04-26T20:51:44.939800+0000","last_peered":"2024-04-26T20:51:44.939800+0000","last_clean":"2024-04-26T20:51:44.939800+0000","last_became_active":"2024-04-26T20:51:44.934690+0000","last_became_peered":"2024-04-26T20:51:44.934690+0000","last_unstale":"2024-04-26T20:51:44.939800+0000","last_undegraded":"2024-04-26T20:51:44.939800+0000","last_fullsized":"2024-04-26T20:51:44.939800+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:02.518303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.926386+0000","last_change":"2024-04-26T20:51:44.926386+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.926386+0000","last_undegraded":"2024-04-26T20:51:44.926386+0000","last_fullsized":"2024-04-26T20:51:44.926386+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.945415+0000","last_change":"2024-04-26T20:51:44.944446+0000","last_active":"2024-04-26T20:51:44.945415+0000","last_peered":"2024-04-26T20:51:44.945415+0000","last_clean":"2024-04-26T20:51:44.945415+0000","last_became_active":"2024-04-26T20:51:44.944147+0000","last_became_peered":"2024-04-26T20:51:44.944147+0000","last_unstale":"2024-04-26T20:51:44.945415+0000","last_undegraded":"2024-04-26T20:51:44.945415+0000","last_fullsized":"2024-04-26T20:51:44.945415+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:16:58.774484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.926312+0000","last_change":"2024-04-26T20:51:44.926312+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.926312+0000","last_undegraded":"2024-04-26T20:51:44.926312+0000","last_fullsized":"2024-04-26T20:51:44.926312+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.939834+0000","last_change":"2024-04-26T20:51:44.932932+0000","last_active":"2024-04-26T20:51:44.939834+0000","last_peered":"2024-04-26T20:51:44.939834+0000","last_clean":"2024-04-26T20:51:44.939834+0000","last_became_active":"2024-04-26T20:51:44.932657+0000","last_became_peered":"2024-04-26T20:51:44.932657+0000","last_unstale":"2024-04-26T20:51:44.939834+0000","last_undegraded":"2024-04-26T20:51:44.939834+0000","last_fullsized":"2024-04-26T20:51:44.939834+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:14:17.119338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.931512+0000","last_change":"2024-04-26T20:51:44.931512+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.931512+0000","last_undegraded":"2024-04-26T20:51:44.931512+0000","last_fullsized":"2024-04-26T20:51:44.931512+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.939843+0000","last_change":"2024-04-26T20:51:44.938761+0000","last_active":"2024-04-26T20:51:44.939843+0000","last_peered":"2024-04-26T20:51:44.939843+0000","last_clean":"2024-04-26T20:51:44.939843+0000","last_became_active":"2024-04-26T20:51:44.938483+0000","last_became_peered":"2024-04-26T20:51:44.938483+0000","last_unstale":"2024-04-26T20:51:44.939843+0000","last_undegraded":"2024-04-26T20:51:44.939843+0000","last_fullsized":"2024-04-26T20:51:44.939843+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:14:24.515988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.938640+0000","last_change":"2024-04-26T20:51:44.938640+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.938640+0000","last_undegraded":"2024-04-26T20:51:44.938640+0000","last_fullsized":"2024-04-26T20:51:44.938640+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.939747+0000","last_change":"2024-04-26T20:51:44.932513+0000","last_active":"2024-04-26T20:51:44.939747+0000","last_peered":"2024-04-26T20:51:44.939747+0000","last_clean":"2024-04-26T20:51:44.939747+0000","last_became_active":"2024-04-26T20:51:44.932223+0000","last_became_peered":"2024-04-26T20:51:44.932223+0000","last_unstale":"2024-04-26T20:51:44.939747+0000","last_undegraded":"2024-04-26T20:51:44.939747+0000","last_fullsized":"2024-04-26T20:51:44.939747+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:28:26.304960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.926513+0000","last_change":"2024-04-26T20:51:44.926513+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.926513+0000","last_undegraded":"2024-04-26T20:51:44.926513+0000","last_fullsized":"2024-04-26T20:51:44.926513+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.940605+0000","last_change":"2024-04-26T20:51:44.938966+0000","last_active":"2024-04-26T20:51:44.940605+0000","last_peered":"2024-04-26T20:51:44.940605+0000","last_clean":"2024-04-26T20:51:44.940605+0000","last_became_active":"2024-04-26T20:51:44.938662+0000","last_became_peered":"2024-04-26T20:51:44.938662+0000","last_unstale":"2024-04-26T20:51:44.940605+0000","last_undegraded":"2024-04-26T20:51:44.940605+0000","last_fullsized":"2024-04-26T20:51:44.940605+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:14:24.515988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.929938+0000","last_change":"2024-04-26T20:51:44.929938+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.929938+0000","last_undegraded":"2024-04-26T20:51:44.929938+0000","last_fullsized":"2024-04-26T20:51:44.929938+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.939612+0000","last_change":"2024-04-26T20:51:44.935952+0000","last_active":"2024-04-26T20:51:44.939612+0000","last_peered":"2024-04-26T20:51:44.939612+0000","last_clean":"2024-04-26T20:51:44.939612+0000","last_became_active":"2024-04-26T20:51:44.935682+0000","last_became_peered":"2024-04-26T20:51:44.935682+0000","last_unstale":"2024-04-26T20:51:44.939612+0000","last_undegraded":"2024-04-26T20:51:44.939612+0000","last_fullsized":"2024-04-26T20:51:44.939612+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:35.270443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.933638+0000","last_change":"2024-04-26T20:51:44.933638+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.933638+0000","last_undegraded":"2024-04-26T20:51:44.933638+0000","last_fullsized":"2024-04-26T20:51:44.933638+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.933704+0000","last_change":"2024-04-26T20:51:44.932688+0000","last_active":"2024-04-26T20:51:44.933704+0000","last_peered":"2024-04-26T20:51:44.933704+0000","last_clean":"2024-04-26T20:51:44.933704+0000","last_became_active":"2024-04-26T20:51:44.932417+0000","last_became_peered":"2024-04-26T20:51:44.932417+0000","last_unstale":"2024-04-26T20:51:44.933704+0000","last_undegraded":"2024-04-26T20:51:44.933704+0000","last_fullsized":"2024-04-26T20:51:44.933704+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:55:14.475567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.930210+0000","last_change":"2024-04-26T20:51:44.930210+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.930210+0000","last_undegraded":"2024-04-26T20:51:44.930210+0000","last_fullsized":"2024-04-26T20:51:44.930210+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.937168+0000","last_change":"2024-04-26T20:51:44.934242+0000","last_active":"2024-04-26T20:51:44.937168+0000","last_peered":"2024-04-26T20:51:44.937168+0000","last_clean":"2024-04-26T20:51:44.937168+0000","last_became_active":"2024-04-26T20:51:44.933449+0000","last_became_peered":"2024-04-26T20:51:44.933449+0000","last_unstale":"2024-04-26T20:51:44.937168+0000","last_undegraded":"2024-04-26T20:51:44.937168+0000","last_fullsized":"2024-04-26T20:51:44.937168+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:35.270443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.932718+0000","last_change":"2024-04-26T20:51:44.932718+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.932718+0000","last_undegraded":"2024-04-26T20:51:44.932718+0000","last_fullsized":"2024-04-26T20:51:44.932718+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.940384+0000","last_change":"2024-04-26T20:51:44.934065+0000","last_active":"2024-04-26T20:51:44.940384+0000","last_peered":"2024-04-26T20:51:44.940384+0000","last_clean":"2024-04-26T20:51:44.940384+0000","last_became_active":"2024-04-26T20:51:44.933763+0000","last_became_peered":"2024-04-26T20:51:44.933763+0000","last_unstale":"2024-04-26T20:51:44.940384+0000","last_undegraded":"2024-04-26T20:51:44.940384+0000","last_fullsized":"2024-04-26T20:51:44.940384+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:46:52.657100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-26T20:51:44.927313+0000","last_change":"2024-04-26T20:51:44.927313+0000","last_active":"2024-04-26T20:51:44.916026+0000","last_peered":"2024-04-26T20:51:44.916026+0000","last_clean":"2024-04-26T20:51:44.916026+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:51:44.927313+0000","last_undegraded":"2024-04-26T20:51:44.927313+0000","last_fullsized":"2024-04-26T20:51:44.927313+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.946013+0000","last_change":"2024-04-26T20:51:44.944770+0000","last_active":"2024-04-26T20:51:44.946013+0000","last_peered":"2024-04-26T20:51:44.946013+0000","last_clean":"2024-04-26T20:51:44.946013+0000","last_became_active":"2024-04-26T20:51:44.944478+0000","last_became_peered":"2024-04-26T20:51:44.944478+0000","last_unstale":"2024-04-26T20:51:44.946013+0000","last_undegraded":"2024-04-26T20:51:44.946013+0000","last_fullsized":"2024-04-26T20:51:44.946013+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:07:37.193221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":199,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-26T20:51:44.941219+0000","last_change":"2024-04-26T20:50:21.150132+0000","last_active":"2024-04-26T20:51:44.941219+0000","last_peered":"2024-04-26T20:51:44.941219+0000","last_clean":"2024-04-26T20:51:44.941219+0000","last_became_active":"2024-04-26T20:50:21.147439+0000","last_became_peered":"2024-04-26T20:50:21.147439+0000","last_unstale":"2024-04-26T20:51:44.941219+0000","last_undegraded":"2024-04-26T20:51:44.941219+0000","last_fullsized":"2024-04-26T20:51:44.941219+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:20.084499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:20.084499+0000","last_clean_scrub_stamp":"2024-04-26T20:50:20.084499+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:19:29.395477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":1007616,"data_stored":766999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":1007616,"data_stored":766999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:51:50.311 INFO:tasks.ceph.mds.d.smithi120.stderr:2024-04-26T20:51:50.310+0000 7f01c8c1b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T20:51:52.150 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:51:52.153 DEBUG:teuthology.orchestra.run.smithi120:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:51:53.267 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:51:53.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.380+0000 7fa0763e2640 1 Processor -- start 2024-04-26T20:51:53.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.380+0000 7fa0763e2640 1 -- start start 2024-04-26T20:51:53.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.381+0000 7fa0763e2640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa0700cc5b0 0x7fa070072240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:53.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.381+0000 7fa0763e2640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa07006e240 con 0x7fa070071e60 2024-04-26T20:51:53.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.381+0000 7fa0763e2640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa07006fc30 con 0x7fa070072780 2024-04-26T20:51:53.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.381+0000 7fa0763e2640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fa0700701f0 con 0x7fa0700cc5b0 2024-04-26T20:51:53.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.381+0000 7fa06ffff640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fa070071e60 0x7fa07015df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40550/0 (socket says 172.21.15.73:40550) 2024-04-26T20:51:53.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.381+0000 7fa06ffff640 1 -- 172.21.15.73:0/3153583911 learned_addr learned my addr 172.21.15.73:0/3153583911 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:53.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.382+0000 7fa06f7fe640 1 --2- 172.21.15.73:0/3153583911 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa0700cc5b0 0x7fa070072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:53.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.382+0000 7fa06effd640 1 -- 172.21.15.73:0/3153583911 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2367153091 0 0) 0x7fa07006e240 con 0x7fa070071e60 2024-04-26T20:51:53.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.382+0000 7fa06effd640 1 -- 172.21.15.73:0/3153583911 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa058003520 con 0x7fa070071e60 2024-04-26T20:51:53.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.382+0000 7fa06effd640 1 -- 172.21.15.73:0/3153583911 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3193182318 0 0) 0x7fa07006fc30 con 0x7fa070072780 2024-04-26T20:51:53.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.382+0000 7fa06effd640 1 -- 172.21.15.73:0/3153583911 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa07006e240 con 0x7fa070072780 2024-04-26T20:51:53.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.382+0000 7fa06effd640 1 -- 172.21.15.73:0/3153583911 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4057651905 0 0) 0x7fa058003520 con 0x7fa070071e60 2024-04-26T20:51:53.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.382+0000 7fa06effd640 1 -- 172.21.15.73:0/3153583911 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa0700cc5b0 msgr2=0x7fa070072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:53.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.382+0000 7fa06effd640 1 --2- 172.21.15.73:0/3153583911 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa0700cc5b0 0x7fa070072240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:53.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.382+0000 7fa06effd640 1 -- 172.21.15.73:0/3153583911 >> v1:172.21.15.120:6789/0 conn(0x7fa070072780 legacy=0x7fa070166b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:53.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.382+0000 7fa06effd640 1 -- 172.21.15.73:0/3153583911 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa07006ff10 con 0x7fa070071e60 2024-04-26T20:51:53.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.382+0000 7fa06effd640 1 -- 172.21.15.73:0/3153583911 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7fa060001670 con 0x7fa070071e60 2024-04-26T20:51:53.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.382+0000 7fa06effd640 1 -- 172.21.15.73:0/3153583911 >> v1:172.21.15.73:6789/0 conn(0x7fa070071e60 legacy=0x7fa07015df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:53.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.382+0000 7fa06effd640 1 --2- 172.21.15.73:0/3153583911 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa058003b70 0x7fa058003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:53.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.382+0000 7fa06effd640 1 --2- 172.21.15.73:0/3153583911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa058004800 0x7fa058004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:53.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.382+0000 7fa06effd640 1 --2- 172.21.15.73:0/3153583911 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa0700cc5b0 0x7fa0580051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:53.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.382+0000 7fa06effd640 1 -- 172.21.15.73:0/3153583911 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa0700701f0 con 0x7fa058004800 2024-04-26T20:51:53.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.382+0000 7fa06effd640 1 -- 172.21.15.73:0/3153583911 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa05800bad0 con 0x7fa058003b70 2024-04-26T20:51:53.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.382+0000 7fa06effd640 1 -- 172.21.15.73:0/3153583911 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fa05800bcd0 con 0x7fa0700cc5b0 2024-04-26T20:51:53.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.382+0000 7fa06f7fe640 1 --2- 172.21.15.73:0/3153583911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa058004800 0x7fa058004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:53.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.382+0000 7fa06ffff640 1 --2- 172.21.15.73:0/3153583911 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa058003b70 0x7fa058003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:53.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.382+0000 7fa07497f640 1 --2- 172.21.15.73:0/3153583911 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa0700cc5b0 0x7fa0580051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:53.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.382+0000 7fa06f7fe640 1 -- 172.21.15.73:0/3153583911 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa0700cc5b0 msgr2=0x7fa0580051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:53.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.382+0000 7fa06f7fe640 1 --2- 172.21.15.73:0/3153583911 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa0700cc5b0 0x7fa0580051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:53.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.382+0000 7fa06f7fe640 1 -- 172.21.15.73:0/3153583911 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa058003b70 msgr2=0x7fa058003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:53.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.382+0000 7fa06f7fe640 1 --2- 172.21.15.73:0/3153583911 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa058003b70 0x7fa058003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:53.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.383+0000 7fa06f7fe640 1 -- 172.21.15.73:0/3153583911 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa058003520 con 0x7fa058004800 2024-04-26T20:51:53.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.383+0000 7fa07497f640 1 --2- 172.21.15.73:0/3153583911 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa0700cc5b0 0x7fa0580051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:53.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.383+0000 7fa06f7fe640 1 --2- 172.21.15.73:0/3153583911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa058004800 0x7fa058004c80 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa064009ac0 tx=0x7fa058005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c69e07264759df34 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:53.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.383+0000 7fa06effd640 1 -- 172.21.15.73:0/3153583911 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa064036070 con 0x7fa058004800 2024-04-26T20:51:53.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.383+0000 7fa06effd640 1 -- 172.21.15.73:0/3153583911 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa064031400 con 0x7fa058004800 2024-04-26T20:51:53.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.383+0000 7fa06effd640 1 -- 172.21.15.73:0/3153583911 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa064009690 con 0x7fa058004800 2024-04-26T20:51:53.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.384+0000 7fa0763e2640 1 -- 172.21.15.73:0/3153583911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa058004800 msgr2=0x7fa058004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:53.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.384+0000 7fa0763e2640 1 --2- 172.21.15.73:0/3153583911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa058004800 0x7fa058004c80 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa064009ac0 tx=0x7fa058005830 comp rx=0 tx=0).stop 2024-04-26T20:51:53.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.384+0000 7fa07497f640 1 -- 172.21.15.73:0/3153583911 reap_dead start 2024-04-26T20:51:53.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.384+0000 7fa0763e2640 1 -- 172.21.15.73:0/3153583911 shutdown_connections 2024-04-26T20:51:53.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.384+0000 7fa0763e2640 1 -- 172.21.15.73:0/3153583911 >> 172.21.15.73:0/3153583911 conn(0x7fa0700d0be0 msgr2=0x7fa0700d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:53.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.384+0000 7fa0763e2640 1 -- 172.21.15.73:0/3153583911 shutdown_connections 2024-04-26T20:51:53.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.384+0000 7fa0763e2640 1 -- 172.21.15.73:0/3153583911 wait complete. 2024-04-26T20:51:53.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.385+0000 7fa0763e2640 1 Processor -- start 2024-04-26T20:51:53.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.385+0000 7fa0763e2640 1 -- start start 2024-04-26T20:51:53.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.386+0000 7fa0763e2640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa058004800 0x7fa070149e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:53.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.386+0000 7fa0763e2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0700c9480 0x7fa070157420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:53.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.386+0000 7fa0763e2640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa0700ca440 0x7fa070157960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:53.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.386+0000 7fa0763e2640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa07006f8e0 con 0x7fa058004800 2024-04-26T20:51:53.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.386+0000 7fa0763e2640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa070072c10 con 0x7fa0700c9480 2024-04-26T20:51:53.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.386+0000 7fa0763e2640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fa07006ddf0 con 0x7fa0700ca440 2024-04-26T20:51:53.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.386+0000 7fa06ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa058004800 0x7fa070149e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:53.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.386+0000 7fa06ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa058004800 0x7fa070149e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41038/0 (socket says 172.21.15.73:41038) 2024-04-26T20:51:53.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.386+0000 7fa06ffff640 1 -- 172.21.15.73:0/4023121695 learned_addr learned my addr 172.21.15.73:0/4023121695 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:53.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.387+0000 7fa06f7fe640 1 --2- 172.21.15.73:0/4023121695 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0700c9480 0x7fa070157420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:53.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.387+0000 7fa07497f640 1 --2- 172.21.15.73:0/4023121695 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa0700ca440 0x7fa070157960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:53.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.387+0000 7fa06ffff640 1 -- 172.21.15.73:0/4023121695 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa0700ca440 msgr2=0x7fa070157960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:53.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.387+0000 7fa06ffff640 1 --2- 172.21.15.73:0/4023121695 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa0700ca440 0x7fa070157960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:53.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.387+0000 7fa06ffff640 1 -- 172.21.15.73:0/4023121695 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0700c9480 msgr2=0x7fa070157420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:53.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.387+0000 7fa06ffff640 1 --2- 172.21.15.73:0/4023121695 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0700c9480 0x7fa070157420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:53.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.387+0000 7fa06ffff640 1 -- 172.21.15.73:0/4023121695 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0700ca9b0 con 0x7fa058004800 2024-04-26T20:51:53.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.387+0000 7fa07497f640 1 --2- 172.21.15.73:0/4023121695 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa0700ca440 0x7fa070157960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:53.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.387+0000 7fa06f7fe640 1 --2- 172.21.15.73:0/4023121695 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0700c9480 0x7fa070157420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:53.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.387+0000 7fa06ffff640 1 --2- 172.21.15.73:0/4023121695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa058004800 0x7fa070149e50 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa060007340 tx=0x7fa060007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:53.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.387+0000 7fa06cff9640 1 -- 172.21.15.73:0/4023121695 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa06000f860 con 0x7fa058004800 2024-04-26T20:51:53.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.387+0000 7fa06cff9640 1 -- 172.21.15.73:0/4023121695 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa060018070 con 0x7fa058004800 2024-04-26T20:51:53.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.387+0000 7fa0763e2640 1 -- 172.21.15.73:0/4023121695 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0700c86c0 con 0x7fa058004800 2024-04-26T20:51:53.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.387+0000 7fa06cff9640 1 -- 172.21.15.73:0/4023121695 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa06000fc60 con 0x7fa058004800 2024-04-26T20:51:53.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.387+0000 7fa0763e2640 1 -- 172.21.15.73:0/4023121695 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0700c8c80 con 0x7fa058004800 2024-04-26T20:51:53.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.388+0000 7fa0763e2640 1 -- 172.21.15.73:0/4023121695 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0700694f0 con 0x7fa058004800 2024-04-26T20:51:53.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.389+0000 7fa06cff9640 1 -- 172.21.15.73:0/4023121695 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa06001d460 con 0x7fa058004800 2024-04-26T20:51:53.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.389+0000 7fa06cff9640 1 --2- 172.21.15.73:0/4023121695 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa05c06c8f0 0x7fa05c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:53.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.389+0000 7fa06f7fe640 1 --2- 172.21.15.73:0/4023121695 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa05c06c8f0 0x7fa05c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:53.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.389+0000 7fa06cff9640 1 -- 172.21.15.73:0/4023121695 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fa06008bd60 con 0x7fa058004800 2024-04-26T20:51:53.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.390+0000 7fa06f7fe640 1 --2- 172.21.15.73:0/4023121695 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa05c06c8f0 0x7fa05c06edb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa064033000 tx=0x7fa064008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:53.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.394+0000 7fa06cff9640 1 -- 172.21.15.73:0/4023121695 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa06005a000 con 0x7fa058004800 2024-04-26T20:51:53.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.565+0000 7fa0763e2640 1 -- 172.21.15.73:0/4023121695 --> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa07006ce60 con 0x7fa05c06c8f0 2024-04-26T20:51:53.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.571+0000 7fa06cff9640 1 -- 172.21.15.73:0/4023121695 <== mgr.4107 v2:172.21.15.73:6832/666905549 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251959 (secure 0 0 0) 0x7fa07006ce60 con 0x7fa05c06c8f0 2024-04-26T20:51:53.577 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:51:53.579 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-26T20:51:53.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.574+0000 7fa0763e2640 1 -- 172.21.15.73:0/4023121695 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa05c06c8f0 msgr2=0x7fa05c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:53.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.574+0000 7fa0763e2640 1 --2- 172.21.15.73:0/4023121695 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa05c06c8f0 0x7fa05c06edb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa064033000 tx=0x7fa064008030 comp rx=0 tx=0).stop 2024-04-26T20:51:53.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.574+0000 7fa0763e2640 1 -- 172.21.15.73:0/4023121695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa058004800 msgr2=0x7fa070149e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:53.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.574+0000 7fa0763e2640 1 --2- 172.21.15.73:0/4023121695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa058004800 0x7fa070149e50 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa060007340 tx=0x7fa060007800 comp rx=0 tx=0).stop 2024-04-26T20:51:53.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.574+0000 7fa0763e2640 1 -- 172.21.15.73:0/4023121695 shutdown_connections 2024-04-26T20:51:53.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.574+0000 7fa0763e2640 1 --2- 172.21.15.73:0/4023121695 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa05c06c8f0 0x7fa05c06edb0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:53.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.574+0000 7fa0763e2640 1 --2- 172.21.15.73:0/4023121695 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa0700ca440 0x7fa070157960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:53.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.574+0000 7fa0763e2640 1 --2- 172.21.15.73:0/4023121695 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa0700c9480 0x7fa070157420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:53.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.574+0000 7fa0763e2640 1 --2- 172.21.15.73:0/4023121695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa058004800 0x7fa070149e50 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:53.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.574+0000 7fa0763e2640 1 -- 172.21.15.73:0/4023121695 >> 172.21.15.73:0/4023121695 conn(0x7fa0700d0be0 msgr2=0x7fa0701600e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:53.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.575+0000 7fa0763e2640 1 -- 172.21.15.73:0/4023121695 shutdown_connections 2024-04-26T20:51:53.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.575+0000 7fa0763e2640 1 -- 172.21.15.73:0/4023121695 wait complete. 2024-04-26T20:51:53.587 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":78,"stamp":"2024-04-26T20:51:51.998041+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":175,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220952,"kb_used_data":4744,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753768,"statfs":{"total":773094113280,"available":772867858432,"internally_reserved":0,"allocated":4857856,"data_stored":2762812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.077155"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:47.393639+0000","last_change":"2024-04-26T20:51:45.940184+0000","last_active":"2024-04-26T20:51:47.393639+0000","last_peered":"2024-04-26T20:51:47.393639+0000","last_clean":"2024-04-26T20:51:47.393639+0000","last_became_active":"2024-04-26T20:51:45.939728+0000","last_became_peered":"2024-04-26T20:51:45.939728+0000","last_unstale":"2024-04-26T20:51:47.393639+0000","last_undegraded":"2024-04-26T20:51:47.393639+0000","last_fullsized":"2024-04-26T20:51:47.393639+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:47:41.260126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:47.393303+0000","last_change":"2024-04-26T20:51:45.940142+0000","last_active":"2024-04-26T20:51:47.393303+0000","last_peered":"2024-04-26T20:51:47.393303+0000","last_clean":"2024-04-26T20:51:47.393303+0000","last_became_active":"2024-04-26T20:51:45.939745+0000","last_became_peered":"2024-04-26T20:51:45.939745+0000","last_unstale":"2024-04-26T20:51:47.393303+0000","last_undegraded":"2024-04-26T20:51:47.393303+0000","last_fullsized":"2024-04-26T20:51:47.393303+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:21:11.332420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:47.393186+0000","last_change":"2024-04-26T20:51:45.939611+0000","last_active":"2024-04-26T20:51:47.393186+0000","last_peered":"2024-04-26T20:51:47.393186+0000","last_clean":"2024-04-26T20:51:47.393186+0000","last_became_active":"2024-04-26T20:51:45.939276+0000","last_became_peered":"2024-04-26T20:51:45.939276+0000","last_unstale":"2024-04-26T20:51:47.393186+0000","last_undegraded":"2024-04-26T20:51:47.393186+0000","last_fullsized":"2024-04-26T20:51:47.393186+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:31:37.196512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:46.943608+0000","last_change":"2024-04-26T20:51:45.934679+0000","last_active":"2024-04-26T20:51:46.943608+0000","last_peered":"2024-04-26T20:51:46.943608+0000","last_clean":"2024-04-26T20:51:46.943608+0000","last_became_active":"2024-04-26T20:51:45.933948+0000","last_became_peered":"2024-04-26T20:51:45.933948+0000","last_unstale":"2024-04-26T20:51:46.943608+0000","last_undegraded":"2024-04-26T20:51:46.943608+0000","last_fullsized":"2024-04-26T20:51:46.943608+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:04:32.673340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:46.943411+0000","last_change":"2024-04-26T20:51:45.929389+0000","last_active":"2024-04-26T20:51:46.943411+0000","last_peered":"2024-04-26T20:51:46.943411+0000","last_clean":"2024-04-26T20:51:46.943411+0000","last_became_active":"2024-04-26T20:51:45.929130+0000","last_became_peered":"2024-04-26T20:51:45.929130+0000","last_unstale":"2024-04-26T20:51:46.943411+0000","last_undegraded":"2024-04-26T20:51:46.943411+0000","last_fullsized":"2024-04-26T20:51:46.943411+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:39:26.197554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:46.935208+0000","last_change":"2024-04-26T20:51:45.936832+0000","last_active":"2024-04-26T20:51:46.935208+0000","last_peered":"2024-04-26T20:51:46.935208+0000","last_clean":"2024-04-26T20:51:46.935208+0000","last_became_active":"2024-04-26T20:51:45.936581+0000","last_became_peered":"2024-04-26T20:51:45.936581+0000","last_unstale":"2024-04-26T20:51:46.935208+0000","last_undegraded":"2024-04-26T20:51:46.935208+0000","last_fullsized":"2024-04-26T20:51:46.935208+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:31:37.196512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:46.945502+0000","last_change":"2024-04-26T20:51:45.951959+0000","last_active":"2024-04-26T20:51:46.945502+0000","last_peered":"2024-04-26T20:51:46.945502+0000","last_clean":"2024-04-26T20:51:46.945502+0000","last_became_active":"2024-04-26T20:51:45.951700+0000","last_became_peered":"2024-04-26T20:51:45.951700+0000","last_unstale":"2024-04-26T20:51:46.945502+0000","last_undegraded":"2024-04-26T20:51:46.945502+0000","last_fullsized":"2024-04-26T20:51:46.945502+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:49:57.120389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:47.393054+0000","last_change":"2024-04-26T20:51:45.939555+0000","last_active":"2024-04-26T20:51:47.393054+0000","last_peered":"2024-04-26T20:51:47.393054+0000","last_clean":"2024-04-26T20:51:47.393054+0000","last_became_active":"2024-04-26T20:51:45.938781+0000","last_became_peered":"2024-04-26T20:51:45.938781+0000","last_unstale":"2024-04-26T20:51:47.393054+0000","last_undegraded":"2024-04-26T20:51:47.393054+0000","last_fullsized":"2024-04-26T20:51:47.393054+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:34:19.975832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:47.393609+0000","last_change":"2024-04-26T20:51:45.954633+0000","last_active":"2024-04-26T20:51:47.393609+0000","last_peered":"2024-04-26T20:51:47.393609+0000","last_clean":"2024-04-26T20:51:47.393609+0000","last_became_active":"2024-04-26T20:51:45.954378+0000","last_became_peered":"2024-04-26T20:51:45.954378+0000","last_unstale":"2024-04-26T20:51:47.393609+0000","last_undegraded":"2024-04-26T20:51:47.393609+0000","last_fullsized":"2024-04-26T20:51:47.393609+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:06:29.255667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:46.945971+0000","last_change":"2024-04-26T20:51:45.952849+0000","last_active":"2024-04-26T20:51:46.945971+0000","last_peered":"2024-04-26T20:51:46.945971+0000","last_clean":"2024-04-26T20:51:46.945971+0000","last_became_active":"2024-04-26T20:51:45.952524+0000","last_became_peered":"2024-04-26T20:51:45.952524+0000","last_unstale":"2024-04-26T20:51:46.945971+0000","last_undegraded":"2024-04-26T20:51:46.945971+0000","last_fullsized":"2024-04-26T20:51:46.945971+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:21:11.332420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:46.945796+0000","last_change":"2024-04-26T20:51:45.947844+0000","last_active":"2024-04-26T20:51:46.945796+0000","last_peered":"2024-04-26T20:51:46.945796+0000","last_clean":"2024-04-26T20:51:46.945796+0000","last_became_active":"2024-04-26T20:51:45.947570+0000","last_became_peered":"2024-04-26T20:51:45.947570+0000","last_unstale":"2024-04-26T20:51:46.945796+0000","last_undegraded":"2024-04-26T20:51:46.945796+0000","last_fullsized":"2024-04-26T20:51:46.945796+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:34:19.975832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:47.393251+0000","last_change":"2024-04-26T20:51:45.940246+0000","last_active":"2024-04-26T20:51:47.393251+0000","last_peered":"2024-04-26T20:51:47.393251+0000","last_clean":"2024-04-26T20:51:47.393251+0000","last_became_active":"2024-04-26T20:51:45.939969+0000","last_became_peered":"2024-04-26T20:51:45.939969+0000","last_unstale":"2024-04-26T20:51:47.393251+0000","last_undegraded":"2024-04-26T20:51:47.393251+0000","last_fullsized":"2024-04-26T20:51:47.393251+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:53:41.498373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:46.945662+0000","last_change":"2024-04-26T20:51:45.952880+0000","last_active":"2024-04-26T20:51:46.945662+0000","last_peered":"2024-04-26T20:51:46.945662+0000","last_clean":"2024-04-26T20:51:46.945662+0000","last_became_active":"2024-04-26T20:51:45.952621+0000","last_became_peered":"2024-04-26T20:51:45.952621+0000","last_unstale":"2024-04-26T20:51:46.945662+0000","last_undegraded":"2024-04-26T20:51:46.945662+0000","last_fullsized":"2024-04-26T20:51:46.945662+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:35:34.359004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:46.938253+0000","last_change":"2024-04-26T20:51:45.943388+0000","last_active":"2024-04-26T20:51:46.938253+0000","last_peered":"2024-04-26T20:51:46.938253+0000","last_clean":"2024-04-26T20:51:46.938253+0000","last_became_active":"2024-04-26T20:51:45.943077+0000","last_became_peered":"2024-04-26T20:51:45.943077+0000","last_unstale":"2024-04-26T20:51:46.938253+0000","last_undegraded":"2024-04-26T20:51:46.938253+0000","last_fullsized":"2024-04-26T20:51:46.938253+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:06:47.440852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:46.945845+0000","last_change":"2024-04-26T20:51:45.941980+0000","last_active":"2024-04-26T20:51:46.945845+0000","last_peered":"2024-04-26T20:51:46.945845+0000","last_clean":"2024-04-26T20:51:46.945845+0000","last_became_active":"2024-04-26T20:51:45.941532+0000","last_became_peered":"2024-04-26T20:51:45.941532+0000","last_unstale":"2024-04-26T20:51:46.945845+0000","last_undegraded":"2024-04-26T20:51:46.945845+0000","last_fullsized":"2024-04-26T20:51:46.945845+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:13:56.735997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:47.393064+0000","last_change":"2024-04-26T20:51:45.954734+0000","last_active":"2024-04-26T20:51:47.393064+0000","last_peered":"2024-04-26T20:51:47.393064+0000","last_clean":"2024-04-26T20:51:47.393064+0000","last_became_active":"2024-04-26T20:51:45.954476+0000","last_became_peered":"2024-04-26T20:51:45.954476+0000","last_unstale":"2024-04-26T20:51:47.393064+0000","last_undegraded":"2024-04-26T20:51:47.393064+0000","last_fullsized":"2024-04-26T20:51:47.393064+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:26:13.701304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:47.393588+0000","last_change":"2024-04-26T20:51:45.955071+0000","last_active":"2024-04-26T20:51:47.393588+0000","last_peered":"2024-04-26T20:51:47.393588+0000","last_clean":"2024-04-26T20:51:47.393588+0000","last_became_active":"2024-04-26T20:51:45.954286+0000","last_became_peered":"2024-04-26T20:51:45.954286+0000","last_unstale":"2024-04-26T20:51:47.393588+0000","last_undegraded":"2024-04-26T20:51:47.393588+0000","last_fullsized":"2024-04-26T20:51:47.393588+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:00:12.557858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:47.226538+0000","last_change":"2024-04-26T20:51:45.943226+0000","last_active":"2024-04-26T20:51:47.226538+0000","last_peered":"2024-04-26T20:51:47.226538+0000","last_clean":"2024-04-26T20:51:47.226538+0000","last_became_active":"2024-04-26T20:51:45.942948+0000","last_became_peered":"2024-04-26T20:51:45.942948+0000","last_unstale":"2024-04-26T20:51:47.226538+0000","last_undegraded":"2024-04-26T20:51:47.226538+0000","last_fullsized":"2024-04-26T20:51:47.226538+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:10:47.642612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:46.943735+0000","last_change":"2024-04-26T20:51:45.932900+0000","last_active":"2024-04-26T20:51:46.943735+0000","last_peered":"2024-04-26T20:51:46.943735+0000","last_clean":"2024-04-26T20:51:46.943735+0000","last_became_active":"2024-04-26T20:51:45.932643+0000","last_became_peered":"2024-04-26T20:51:45.932643+0000","last_unstale":"2024-04-26T20:51:46.943735+0000","last_undegraded":"2024-04-26T20:51:46.943735+0000","last_fullsized":"2024-04-26T20:51:46.943735+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:50:28.819280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:47.226349+0000","last_change":"2024-04-26T20:51:45.943720+0000","last_active":"2024-04-26T20:51:47.226349+0000","last_peered":"2024-04-26T20:51:47.226349+0000","last_clean":"2024-04-26T20:51:47.226349+0000","last_became_active":"2024-04-26T20:51:45.943414+0000","last_became_peered":"2024-04-26T20:51:45.943414+0000","last_unstale":"2024-04-26T20:51:47.226349+0000","last_undegraded":"2024-04-26T20:51:47.226349+0000","last_fullsized":"2024-04-26T20:51:47.226349+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:47:44.122606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:46.945671+0000","last_change":"2024-04-26T20:51:45.956229+0000","last_active":"2024-04-26T20:51:46.945671+0000","last_peered":"2024-04-26T20:51:46.945671+0000","last_clean":"2024-04-26T20:51:46.945671+0000","last_became_active":"2024-04-26T20:51:45.955984+0000","last_became_peered":"2024-04-26T20:51:45.955984+0000","last_unstale":"2024-04-26T20:51:46.945671+0000","last_undegraded":"2024-04-26T20:51:46.945671+0000","last_fullsized":"2024-04-26T20:51:46.945671+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:58:56.461112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:46.938577+0000","last_change":"2024-04-26T20:51:45.935394+0000","last_active":"2024-04-26T20:51:46.938577+0000","last_peered":"2024-04-26T20:51:46.938577+0000","last_clean":"2024-04-26T20:51:46.938577+0000","last_became_active":"2024-04-26T20:51:45.935088+0000","last_became_peered":"2024-04-26T20:51:45.935088+0000","last_unstale":"2024-04-26T20:51:46.938577+0000","last_undegraded":"2024-04-26T20:51:46.938577+0000","last_fullsized":"2024-04-26T20:51:46.938577+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:41.996127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:47.393142+0000","last_change":"2024-04-26T20:51:45.946260+0000","last_active":"2024-04-26T20:51:47.393142+0000","last_peered":"2024-04-26T20:51:47.393142+0000","last_clean":"2024-04-26T20:51:47.393142+0000","last_became_active":"2024-04-26T20:51:45.945940+0000","last_became_peered":"2024-04-26T20:51:45.945940+0000","last_unstale":"2024-04-26T20:51:47.393142+0000","last_undegraded":"2024-04-26T20:51:47.393142+0000","last_fullsized":"2024-04-26T20:51:47.393142+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:53:36.700235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:46.943348+0000","last_change":"2024-04-26T20:51:45.946584+0000","last_active":"2024-04-26T20:51:46.943348+0000","last_peered":"2024-04-26T20:51:46.943348+0000","last_clean":"2024-04-26T20:51:46.943348+0000","last_became_active":"2024-04-26T20:51:45.946256+0000","last_became_peered":"2024-04-26T20:51:45.946256+0000","last_unstale":"2024-04-26T20:51:46.943348+0000","last_undegraded":"2024-04-26T20:51:46.943348+0000","last_fullsized":"2024-04-26T20:51:46.943348+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:38:38.973527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:47.226569+0000","last_change":"2024-04-26T20:51:45.943362+0000","last_active":"2024-04-26T20:51:47.226569+0000","last_peered":"2024-04-26T20:51:47.226569+0000","last_clean":"2024-04-26T20:51:47.226569+0000","last_became_active":"2024-04-26T20:51:45.943083+0000","last_became_peered":"2024-04-26T20:51:45.943083+0000","last_unstale":"2024-04-26T20:51:47.226569+0000","last_undegraded":"2024-04-26T20:51:47.226569+0000","last_fullsized":"2024-04-26T20:51:47.226569+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:46:53.659271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:46.935526+0000","last_change":"2024-04-26T20:51:45.955365+0000","last_active":"2024-04-26T20:51:46.935526+0000","last_peered":"2024-04-26T20:51:46.935526+0000","last_clean":"2024-04-26T20:51:46.935526+0000","last_became_active":"2024-04-26T20:51:45.955073+0000","last_became_peered":"2024-04-26T20:51:45.955073+0000","last_unstale":"2024-04-26T20:51:46.935526+0000","last_undegraded":"2024-04-26T20:51:46.935526+0000","last_fullsized":"2024-04-26T20:51:46.935526+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:53:41.498373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:47.226467+0000","last_change":"2024-04-26T20:51:45.943969+0000","last_active":"2024-04-26T20:51:47.226467+0000","last_peered":"2024-04-26T20:51:47.226467+0000","last_clean":"2024-04-26T20:51:47.226467+0000","last_became_active":"2024-04-26T20:51:45.943712+0000","last_became_peered":"2024-04-26T20:51:45.943712+0000","last_unstale":"2024-04-26T20:51:47.226467+0000","last_undegraded":"2024-04-26T20:51:47.226467+0000","last_fullsized":"2024-04-26T20:51:47.226467+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:39.837579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:46.945911+0000","last_change":"2024-04-26T20:51:45.942287+0000","last_active":"2024-04-26T20:51:46.945911+0000","last_peered":"2024-04-26T20:51:46.945911+0000","last_clean":"2024-04-26T20:51:46.945911+0000","last_became_active":"2024-04-26T20:51:45.941983+0000","last_became_peered":"2024-04-26T20:51:45.941983+0000","last_unstale":"2024-04-26T20:51:46.945911+0000","last_undegraded":"2024-04-26T20:51:46.945911+0000","last_fullsized":"2024-04-26T20:51:46.945911+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:10:36.220753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:47.226328+0000","last_change":"2024-04-26T20:51:45.937339+0000","last_active":"2024-04-26T20:51:47.226328+0000","last_peered":"2024-04-26T20:51:47.226328+0000","last_clean":"2024-04-26T20:51:47.226328+0000","last_became_active":"2024-04-26T20:51:45.936667+0000","last_became_peered":"2024-04-26T20:51:45.936667+0000","last_unstale":"2024-04-26T20:51:47.226328+0000","last_undegraded":"2024-04-26T20:51:47.226328+0000","last_fullsized":"2024-04-26T20:51:47.226328+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:54:41.096080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:47.226282+0000","last_change":"2024-04-26T20:51:45.939065+0000","last_active":"2024-04-26T20:51:47.226282+0000","last_peered":"2024-04-26T20:51:47.226282+0000","last_clean":"2024-04-26T20:51:47.226282+0000","last_became_active":"2024-04-26T20:51:45.938792+0000","last_became_peered":"2024-04-26T20:51:45.938792+0000","last_unstale":"2024-04-26T20:51:47.226282+0000","last_undegraded":"2024-04-26T20:51:47.226282+0000","last_fullsized":"2024-04-26T20:51:47.226282+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:10:57.606284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:47.393307+0000","last_change":"2024-04-26T20:51:45.946533+0000","last_active":"2024-04-26T20:51:47.393307+0000","last_peered":"2024-04-26T20:51:47.393307+0000","last_clean":"2024-04-26T20:51:47.393307+0000","last_became_active":"2024-04-26T20:51:45.945237+0000","last_became_peered":"2024-04-26T20:51:45.945237+0000","last_unstale":"2024-04-26T20:51:47.393307+0000","last_undegraded":"2024-04-26T20:51:47.393307+0000","last_fullsized":"2024-04-26T20:51:47.393307+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:06:47.440852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:46.935117+0000","last_change":"2024-04-26T20:51:45.935403+0000","last_active":"2024-04-26T20:51:46.935117+0000","last_peered":"2024-04-26T20:51:46.935117+0000","last_clean":"2024-04-26T20:51:46.935117+0000","last_became_active":"2024-04-26T20:51:45.935144+0000","last_became_peered":"2024-04-26T20:51:45.935144+0000","last_unstale":"2024-04-26T20:51:46.935117+0000","last_undegraded":"2024-04-26T20:51:46.935117+0000","last_fullsized":"2024-04-26T20:51:46.935117+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:19:15.464982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:47.393416+0000","last_change":"2024-04-26T20:51:45.942814+0000","last_active":"2024-04-26T20:51:47.393416+0000","last_peered":"2024-04-26T20:51:47.393416+0000","last_clean":"2024-04-26T20:51:47.393416+0000","last_became_active":"2024-04-26T20:51:45.942527+0000","last_became_peered":"2024-04-26T20:51:45.942527+0000","last_unstale":"2024-04-26T20:51:47.393416+0000","last_undegraded":"2024-04-26T20:51:47.393416+0000","last_fullsized":"2024-04-26T20:51:47.393416+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:41.996127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":18,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:47.226534+0000","last_change":"2024-04-26T20:51:44.935237+0000","last_active":"2024-04-26T20:51:47.226534+0000","last_peered":"2024-04-26T20:51:47.226534+0000","last_clean":"2024-04-26T20:51:47.226534+0000","last_became_active":"2024-04-26T20:51:44.934964+0000","last_became_peered":"2024-04-26T20:51:44.934964+0000","last_unstale":"2024-04-26T20:51:47.226534+0000","last_undegraded":"2024-04-26T20:51:47.226534+0000","last_fullsized":"2024-04-26T20:51:47.226534+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:28:21.811767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:46.935626+0000","last_change":"2024-04-26T20:51:45.939501+0000","last_active":"2024-04-26T20:51:46.935626+0000","last_peered":"2024-04-26T20:51:46.935626+0000","last_clean":"2024-04-26T20:51:46.935626+0000","last_became_active":"2024-04-26T20:51:45.939237+0000","last_became_peered":"2024-04-26T20:51:45.939237+0000","last_unstale":"2024-04-26T20:51:46.935626+0000","last_undegraded":"2024-04-26T20:51:46.935626+0000","last_fullsized":"2024-04-26T20:51:46.935626+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:19:18.478019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:47.393278+0000","last_change":"2024-04-26T20:51:44.937990+0000","last_active":"2024-04-26T20:51:47.393278+0000","last_peered":"2024-04-26T20:51:47.393278+0000","last_clean":"2024-04-26T20:51:47.393278+0000","last_became_active":"2024-04-26T20:51:44.937723+0000","last_became_peered":"2024-04-26T20:51:44.937723+0000","last_unstale":"2024-04-26T20:51:47.393278+0000","last_undegraded":"2024-04-26T20:51:47.393278+0000","last_fullsized":"2024-04-26T20:51:47.393278+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:47:43.120436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:46.938794+0000","last_change":"2024-04-26T20:51:45.936126+0000","last_active":"2024-04-26T20:51:46.938794+0000","last_peered":"2024-04-26T20:51:46.938794+0000","last_clean":"2024-04-26T20:51:46.938794+0000","last_became_active":"2024-04-26T20:51:45.935847+0000","last_became_peered":"2024-04-26T20:51:45.935847+0000","last_unstale":"2024-04-26T20:51:46.938794+0000","last_undegraded":"2024-04-26T20:51:46.938794+0000","last_fullsized":"2024-04-26T20:51:46.938794+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:19:18.478019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:46.935703+0000","last_change":"2024-04-26T20:51:44.939632+0000","last_active":"2024-04-26T20:51:46.935703+0000","last_peered":"2024-04-26T20:51:46.935703+0000","last_clean":"2024-04-26T20:51:46.935703+0000","last_became_active":"2024-04-26T20:51:44.939299+0000","last_became_peered":"2024-04-26T20:51:44.939299+0000","last_unstale":"2024-04-26T20:51:46.935703+0000","last_undegraded":"2024-04-26T20:51:46.935703+0000","last_fullsized":"2024-04-26T20:51:46.935703+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:54:40.093910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:47.393341+0000","last_change":"2024-04-26T20:51:45.944130+0000","last_active":"2024-04-26T20:51:47.393341+0000","last_peered":"2024-04-26T20:51:47.393341+0000","last_clean":"2024-04-26T20:51:47.393341+0000","last_became_active":"2024-04-26T20:51:45.943641+0000","last_became_peered":"2024-04-26T20:51:45.943641+0000","last_unstale":"2024-04-26T20:51:47.393341+0000","last_undegraded":"2024-04-26T20:51:47.393341+0000","last_fullsized":"2024-04-26T20:51:47.393341+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:10:36.220753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:46.938692+0000","last_change":"2024-04-26T20:51:44.937707+0000","last_active":"2024-04-26T20:51:46.938692+0000","last_peered":"2024-04-26T20:51:46.938692+0000","last_clean":"2024-04-26T20:51:46.938692+0000","last_became_active":"2024-04-26T20:51:44.937394+0000","last_became_peered":"2024-04-26T20:51:44.937394+0000","last_unstale":"2024-04-26T20:51:46.938692+0000","last_undegraded":"2024-04-26T20:51:46.938692+0000","last_fullsized":"2024-04-26T20:51:46.938692+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:38.835409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:47.393054+0000","last_change":"2024-04-26T20:51:45.936803+0000","last_active":"2024-04-26T20:51:47.393054+0000","last_peered":"2024-04-26T20:51:47.393054+0000","last_clean":"2024-04-26T20:51:47.393054+0000","last_became_active":"2024-04-26T20:51:45.936537+0000","last_became_peered":"2024-04-26T20:51:45.936537+0000","last_unstale":"2024-04-26T20:51:47.393054+0000","last_undegraded":"2024-04-26T20:51:47.393054+0000","last_fullsized":"2024-04-26T20:51:47.393054+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:19:15.464982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:46.945900+0000","last_change":"2024-04-26T20:51:44.937929+0000","last_active":"2024-04-26T20:51:46.945900+0000","last_peered":"2024-04-26T20:51:46.945900+0000","last_clean":"2024-04-26T20:51:46.945900+0000","last_became_active":"2024-04-26T20:51:44.937626+0000","last_became_peered":"2024-04-26T20:51:44.937626+0000","last_unstale":"2024-04-26T20:51:46.945900+0000","last_undegraded":"2024-04-26T20:51:46.945900+0000","last_fullsized":"2024-04-26T20:51:46.945900+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:46:52.657100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:46.945737+0000","last_change":"2024-04-26T20:51:45.940844+0000","last_active":"2024-04-26T20:51:46.945737+0000","last_peered":"2024-04-26T20:51:46.945737+0000","last_clean":"2024-04-26T20:51:46.945737+0000","last_became_active":"2024-04-26T20:51:45.940491+0000","last_became_peered":"2024-04-26T20:51:45.940491+0000","last_unstale":"2024-04-26T20:51:46.945737+0000","last_undegraded":"2024-04-26T20:51:46.945737+0000","last_fullsized":"2024-04-26T20:51:46.945737+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:58:03.456652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:47.393161+0000","last_change":"2024-04-26T20:51:44.944114+0000","last_active":"2024-04-26T20:51:47.393161+0000","last_peered":"2024-04-26T20:51:47.393161+0000","last_clean":"2024-04-26T20:51:47.393161+0000","last_became_active":"2024-04-26T20:51:44.943821+0000","last_became_peered":"2024-04-26T20:51:44.943821+0000","last_unstale":"2024-04-26T20:51:47.393161+0000","last_undegraded":"2024-04-26T20:51:47.393161+0000","last_fullsized":"2024-04-26T20:51:47.393161+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:36:55.165610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:46.938871+0000","last_change":"2024-04-26T20:51:45.941933+0000","last_active":"2024-04-26T20:51:46.938871+0000","last_peered":"2024-04-26T20:51:46.938871+0000","last_clean":"2024-04-26T20:51:46.938871+0000","last_became_active":"2024-04-26T20:51:45.941594+0000","last_became_peered":"2024-04-26T20:51:45.941594+0000","last_unstale":"2024-04-26T20:51:46.938871+0000","last_undegraded":"2024-04-26T20:51:46.938871+0000","last_fullsized":"2024-04-26T20:51:46.938871+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:06:29.255667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:46.935159+0000","last_change":"2024-04-26T20:51:44.942003+0000","last_active":"2024-04-26T20:51:46.935159+0000","last_peered":"2024-04-26T20:51:46.935159+0000","last_clean":"2024-04-26T20:51:46.935159+0000","last_became_active":"2024-04-26T20:51:44.941646+0000","last_became_peered":"2024-04-26T20:51:44.941646+0000","last_unstale":"2024-04-26T20:51:46.935159+0000","last_undegraded":"2024-04-26T20:51:46.935159+0000","last_fullsized":"2024-04-26T20:51:46.935159+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:10:56.604114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:47.393478+0000","last_change":"2024-04-26T20:51:45.947436+0000","last_active":"2024-04-26T20:51:47.393478+0000","last_peered":"2024-04-26T20:51:47.393478+0000","last_clean":"2024-04-26T20:51:47.393478+0000","last_became_active":"2024-04-26T20:51:45.947210+0000","last_became_peered":"2024-04-26T20:51:45.947210+0000","last_unstale":"2024-04-26T20:51:47.393478+0000","last_undegraded":"2024-04-26T20:51:47.393478+0000","last_fullsized":"2024-04-26T20:51:47.393478+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:44:45.080071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:46.938135+0000","last_change":"2024-04-26T20:51:44.934491+0000","last_active":"2024-04-26T20:51:46.938135+0000","last_peered":"2024-04-26T20:51:46.938135+0000","last_clean":"2024-04-26T20:51:46.938135+0000","last_became_active":"2024-04-26T20:51:44.934145+0000","last_became_peered":"2024-04-26T20:51:44.934145+0000","last_unstale":"2024-04-26T20:51:46.938135+0000","last_undegraded":"2024-04-26T20:51:46.938135+0000","last_fullsized":"2024-04-26T20:51:46.938135+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:10:56.604114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:46.943627+0000","last_change":"2024-04-26T20:51:45.930015+0000","last_active":"2024-04-26T20:51:46.943627+0000","last_peered":"2024-04-26T20:51:46.943627+0000","last_clean":"2024-04-26T20:51:46.943627+0000","last_became_active":"2024-04-26T20:51:45.929767+0000","last_became_peered":"2024-04-26T20:51:45.929767+0000","last_unstale":"2024-04-26T20:51:46.943627+0000","last_undegraded":"2024-04-26T20:51:46.943627+0000","last_fullsized":"2024-04-26T20:51:46.943627+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:23.558396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:47.393696+0000","last_change":"2024-04-26T20:51:44.938418+0000","last_active":"2024-04-26T20:51:47.393696+0000","last_peered":"2024-04-26T20:51:47.393696+0000","last_clean":"2024-04-26T20:51:47.393696+0000","last_became_active":"2024-04-26T20:51:44.938146+0000","last_became_peered":"2024-04-26T20:51:44.938146+0000","last_unstale":"2024-04-26T20:51:47.393696+0000","last_undegraded":"2024-04-26T20:51:47.393696+0000","last_fullsized":"2024-04-26T20:51:47.393696+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:19:14.462811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:47.393408+0000","last_change":"2024-04-26T20:51:45.948986+0000","last_active":"2024-04-26T20:51:47.393408+0000","last_peered":"2024-04-26T20:51:47.393408+0000","last_clean":"2024-04-26T20:51:47.393408+0000","last_became_active":"2024-04-26T20:51:45.948727+0000","last_became_peered":"2024-04-26T20:51:45.948727+0000","last_unstale":"2024-04-26T20:51:47.393408+0000","last_undegraded":"2024-04-26T20:51:47.393408+0000","last_fullsized":"2024-04-26T20:51:47.393408+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:53:41.498373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:46.946137+0000","last_change":"2024-04-26T20:51:44.941093+0000","last_active":"2024-04-26T20:51:46.946137+0000","last_peered":"2024-04-26T20:51:46.946137+0000","last_clean":"2024-04-26T20:51:46.946137+0000","last_became_active":"2024-04-26T20:51:44.940805+0000","last_became_peered":"2024-04-26T20:51:44.940805+0000","last_unstale":"2024-04-26T20:51:46.946137+0000","last_undegraded":"2024-04-26T20:51:46.946137+0000","last_fullsized":"2024-04-26T20:51:46.946137+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:40:59.156357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:47.393180+0000","last_change":"2024-04-26T20:51:45.944434+0000","last_active":"2024-04-26T20:51:47.393180+0000","last_peered":"2024-04-26T20:51:47.393180+0000","last_clean":"2024-04-26T20:51:47.393180+0000","last_became_active":"2024-04-26T20:51:45.944181+0000","last_became_peered":"2024-04-26T20:51:45.944181+0000","last_unstale":"2024-04-26T20:51:47.393180+0000","last_undegraded":"2024-04-26T20:51:47.393180+0000","last_fullsized":"2024-04-26T20:51:47.393180+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:06:47.440852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:46.945945+0000","last_change":"2024-04-26T20:51:44.939214+0000","last_active":"2024-04-26T20:51:46.945945+0000","last_peered":"2024-04-26T20:51:46.945945+0000","last_clean":"2024-04-26T20:51:46.945945+0000","last_became_active":"2024-04-26T20:51:44.938907+0000","last_became_peered":"2024-04-26T20:51:44.938907+0000","last_unstale":"2024-04-26T20:51:46.945945+0000","last_undegraded":"2024-04-26T20:51:46.945945+0000","last_fullsized":"2024-04-26T20:51:46.945945+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:38.835409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:46.938703+0000","last_change":"2024-04-26T20:51:45.935297+0000","last_active":"2024-04-26T20:51:46.938703+0000","last_peered":"2024-04-26T20:51:46.938703+0000","last_clean":"2024-04-26T20:51:46.938703+0000","last_became_active":"2024-04-26T20:51:45.935002+0000","last_became_peered":"2024-04-26T20:51:45.935002+0000","last_unstale":"2024-04-26T20:51:46.938703+0000","last_undegraded":"2024-04-26T20:51:46.938703+0000","last_fullsized":"2024-04-26T20:51:46.938703+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:53:41.498373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":20,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:47.393469+0000","last_change":"2024-04-26T20:51:44.939371+0000","last_active":"2024-04-26T20:51:47.393469+0000","last_peered":"2024-04-26T20:51:47.393469+0000","last_clean":"2024-04-26T20:51:47.393469+0000","last_became_active":"2024-04-26T20:51:44.939090+0000","last_became_peered":"2024-04-26T20:51:44.939090+0000","last_unstale":"2024-04-26T20:51:47.393469+0000","last_undegraded":"2024-04-26T20:51:47.393469+0000","last_fullsized":"2024-04-26T20:51:47.393469+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:40.993957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:47.393581+0000","last_change":"2024-04-26T20:51:45.938599+0000","last_active":"2024-04-26T20:51:47.393581+0000","last_peered":"2024-04-26T20:51:47.393581+0000","last_clean":"2024-04-26T20:51:47.393581+0000","last_became_active":"2024-04-26T20:51:45.938356+0000","last_became_peered":"2024-04-26T20:51:45.938356+0000","last_unstale":"2024-04-26T20:51:47.393581+0000","last_undegraded":"2024-04-26T20:51:47.393581+0000","last_fullsized":"2024-04-26T20:51:47.393581+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:00:12.557858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:47.393589+0000","last_change":"2024-04-26T20:51:44.940477+0000","last_active":"2024-04-26T20:51:47.393589+0000","last_peered":"2024-04-26T20:51:47.393589+0000","last_clean":"2024-04-26T20:51:47.393589+0000","last_became_active":"2024-04-26T20:51:44.940192+0000","last_became_peered":"2024-04-26T20:51:44.940192+0000","last_unstale":"2024-04-26T20:51:47.393589+0000","last_undegraded":"2024-04-26T20:51:47.393589+0000","last_fullsized":"2024-04-26T20:51:47.393589+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:53:35.698065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:47.393308+0000","last_change":"2024-04-26T20:51:45.939421+0000","last_active":"2024-04-26T20:51:47.393308+0000","last_peered":"2024-04-26T20:51:47.393308+0000","last_clean":"2024-04-26T20:51:47.393308+0000","last_became_active":"2024-04-26T20:51:45.938997+0000","last_became_peered":"2024-04-26T20:51:45.938997+0000","last_unstale":"2024-04-26T20:51:47.393308+0000","last_undegraded":"2024-04-26T20:51:47.393308+0000","last_fullsized":"2024-04-26T20:51:47.393308+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:20:45.610758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:46.935182+0000","last_change":"2024-04-26T20:51:44.940382+0000","last_active":"2024-04-26T20:51:46.935182+0000","last_peered":"2024-04-26T20:51:46.935182+0000","last_clean":"2024-04-26T20:51:46.935182+0000","last_became_active":"2024-04-26T20:51:44.940109+0000","last_became_peered":"2024-04-26T20:51:44.940109+0000","last_unstale":"2024-04-26T20:51:46.935182+0000","last_undegraded":"2024-04-26T20:51:46.935182+0000","last_fullsized":"2024-04-26T20:51:46.935182+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:38.835409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:47.393215+0000","last_change":"2024-04-26T20:51:45.939548+0000","last_active":"2024-04-26T20:51:47.393215+0000","last_peered":"2024-04-26T20:51:47.393215+0000","last_clean":"2024-04-26T20:51:47.393215+0000","last_became_active":"2024-04-26T20:51:45.939188+0000","last_became_peered":"2024-04-26T20:51:45.939188+0000","last_unstale":"2024-04-26T20:51:47.393215+0000","last_undegraded":"2024-04-26T20:51:47.393215+0000","last_fullsized":"2024-04-26T20:51:47.393215+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:19:15.464982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:47.226188+0000","last_change":"2024-04-26T20:51:44.934857+0000","last_active":"2024-04-26T20:51:47.226188+0000","last_peered":"2024-04-26T20:51:47.226188+0000","last_clean":"2024-04-26T20:51:47.226188+0000","last_became_active":"2024-04-26T20:51:44.934556+0000","last_became_peered":"2024-04-26T20:51:44.934556+0000","last_unstale":"2024-04-26T20:51:47.226188+0000","last_undegraded":"2024-04-26T20:51:47.226188+0000","last_fullsized":"2024-04-26T20:51:47.226188+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:59:50.250119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:47.393624+0000","last_change":"2024-04-26T20:51:45.937329+0000","last_active":"2024-04-26T20:51:47.393624+0000","last_peered":"2024-04-26T20:51:47.393624+0000","last_clean":"2024-04-26T20:51:47.393624+0000","last_became_active":"2024-04-26T20:51:45.937071+0000","last_became_peered":"2024-04-26T20:51:45.937071+0000","last_unstale":"2024-04-26T20:51:47.393624+0000","last_undegraded":"2024-04-26T20:51:47.393624+0000","last_fullsized":"2024-04-26T20:51:47.393624+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:39:25.419450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":22,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:47.393093+0000","last_change":"2024-04-26T20:51:44.943632+0000","last_active":"2024-04-26T20:51:47.393093+0000","last_peered":"2024-04-26T20:51:47.393093+0000","last_clean":"2024-04-26T20:51:47.393093+0000","last_became_active":"2024-04-26T20:51:44.943278+0000","last_became_peered":"2024-04-26T20:51:44.943278+0000","last_unstale":"2024-04-26T20:51:47.393093+0000","last_undegraded":"2024-04-26T20:51:47.393093+0000","last_fullsized":"2024-04-26T20:51:47.393093+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:14:24.515988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:47.393267+0000","last_change":"2024-04-26T20:51:45.944434+0000","last_active":"2024-04-26T20:51:47.393267+0000","last_peered":"2024-04-26T20:51:47.393267+0000","last_clean":"2024-04-26T20:51:47.393267+0000","last_became_active":"2024-04-26T20:51:45.944165+0000","last_became_peered":"2024-04-26T20:51:45.944165+0000","last_unstale":"2024-04-26T20:51:47.393267+0000","last_undegraded":"2024-04-26T20:51:47.393267+0000","last_fullsized":"2024-04-26T20:51:47.393267+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:13:56.735997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:46.945815+0000","last_change":"2024-04-26T20:51:44.935150+0000","last_active":"2024-04-26T20:51:46.945815+0000","last_peered":"2024-04-26T20:51:46.945815+0000","last_clean":"2024-04-26T20:51:46.945815+0000","last_became_active":"2024-04-26T20:51:44.934078+0000","last_became_peered":"2024-04-26T20:51:44.934078+0000","last_unstale":"2024-04-26T20:51:46.945815+0000","last_undegraded":"2024-04-26T20:51:46.945815+0000","last_fullsized":"2024-04-26T20:51:46.945815+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:26:12.699134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:46.945925+0000","last_change":"2024-04-26T20:51:45.937563+0000","last_active":"2024-04-26T20:51:46.945925+0000","last_peered":"2024-04-26T20:51:46.945925+0000","last_clean":"2024-04-26T20:51:46.945925+0000","last_became_active":"2024-04-26T20:51:45.936889+0000","last_became_peered":"2024-04-26T20:51:45.936889+0000","last_unstale":"2024-04-26T20:51:46.945925+0000","last_undegraded":"2024-04-26T20:51:46.945925+0000","last_fullsized":"2024-04-26T20:51:46.945925+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:20:45.610758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:46.938906+0000","last_change":"2024-04-26T20:51:44.932407+0000","last_active":"2024-04-26T20:51:46.938906+0000","last_peered":"2024-04-26T20:51:46.938906+0000","last_clean":"2024-04-26T20:51:46.938906+0000","last_became_active":"2024-04-26T20:51:44.931987+0000","last_became_peered":"2024-04-26T20:51:44.931987+0000","last_unstale":"2024-04-26T20:51:46.938906+0000","last_undegraded":"2024-04-26T20:51:46.938906+0000","last_fullsized":"2024-04-26T20:51:46.938906+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:54:40.093910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:46.938791+0000","last_change":"2024-04-26T20:51:45.933921+0000","last_active":"2024-04-26T20:51:46.938791+0000","last_peered":"2024-04-26T20:51:46.938791+0000","last_clean":"2024-04-26T20:51:46.938791+0000","last_became_active":"2024-04-26T20:51:45.932745+0000","last_became_peered":"2024-04-26T20:51:45.932745+0000","last_unstale":"2024-04-26T20:51:46.938791+0000","last_undegraded":"2024-04-26T20:51:46.938791+0000","last_fullsized":"2024-04-26T20:51:46.938791+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:00:12.557858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:46.943719+0000","last_change":"2024-04-26T20:51:44.941836+0000","last_active":"2024-04-26T20:51:46.943719+0000","last_peered":"2024-04-26T20:51:46.943719+0000","last_clean":"2024-04-26T20:51:46.943719+0000","last_became_active":"2024-04-26T20:51:44.941581+0000","last_became_peered":"2024-04-26T20:51:44.941581+0000","last_unstale":"2024-04-26T20:51:46.943719+0000","last_undegraded":"2024-04-26T20:51:46.943719+0000","last_fullsized":"2024-04-26T20:51:46.943719+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:21:56.429751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:46.943525+0000","last_change":"2024-04-26T20:51:45.936046+0000","last_active":"2024-04-26T20:51:46.943525+0000","last_peered":"2024-04-26T20:51:46.943525+0000","last_clean":"2024-04-26T20:51:46.943525+0000","last_became_active":"2024-04-26T20:51:45.935772+0000","last_became_peered":"2024-04-26T20:51:45.935772+0000","last_unstale":"2024-04-26T20:51:46.943525+0000","last_undegraded":"2024-04-26T20:51:46.943525+0000","last_fullsized":"2024-04-26T20:51:46.943525+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:55:15.477737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:46.935335+0000","last_change":"2024-04-26T20:51:44.938146+0000","last_active":"2024-04-26T20:51:46.935335+0000","last_peered":"2024-04-26T20:51:46.935335+0000","last_clean":"2024-04-26T20:51:46.935335+0000","last_became_active":"2024-04-26T20:51:44.937861+0000","last_became_peered":"2024-04-26T20:51:44.937861+0000","last_unstale":"2024-04-26T20:51:46.935335+0000","last_undegraded":"2024-04-26T20:51:46.935335+0000","last_fullsized":"2024-04-26T20:51:46.935335+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:47:43.120436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:47.393407+0000","last_change":"2024-04-26T20:51:45.946973+0000","last_active":"2024-04-26T20:51:47.393407+0000","last_peered":"2024-04-26T20:51:47.393407+0000","last_clean":"2024-04-26T20:51:47.393407+0000","last_became_active":"2024-04-26T20:51:45.946725+0000","last_became_peered":"2024-04-26T20:51:45.946725+0000","last_unstale":"2024-04-26T20:51:47.393407+0000","last_undegraded":"2024-04-26T20:51:47.393407+0000","last_fullsized":"2024-04-26T20:51:47.393407+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:31:37.196512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:46.943409+0000","last_change":"2024-04-26T20:51:44.940796+0000","last_active":"2024-04-26T20:51:46.943409+0000","last_peered":"2024-04-26T20:51:46.943409+0000","last_clean":"2024-04-26T20:51:46.943409+0000","last_became_active":"2024-04-26T20:51:44.940524+0000","last_became_peered":"2024-04-26T20:51:44.940524+0000","last_unstale":"2024-04-26T20:51:46.943409+0000","last_undegraded":"2024-04-26T20:51:46.943409+0000","last_fullsized":"2024-04-26T20:51:46.943409+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:43:22.295228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:47.393512+0000","last_change":"2024-04-26T20:51:45.947961+0000","last_active":"2024-04-26T20:51:47.393512+0000","last_peered":"2024-04-26T20:51:47.393512+0000","last_clean":"2024-04-26T20:51:47.393512+0000","last_became_active":"2024-04-26T20:51:45.947677+0000","last_became_peered":"2024-04-26T20:51:45.947677+0000","last_unstale":"2024-04-26T20:51:47.393512+0000","last_undegraded":"2024-04-26T20:51:47.393512+0000","last_fullsized":"2024-04-26T20:51:47.393512+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:19:18.478019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:46.943439+0000","last_change":"2024-04-26T20:51:44.934985+0000","last_active":"2024-04-26T20:51:46.943439+0000","last_peered":"2024-04-26T20:51:46.943439+0000","last_clean":"2024-04-26T20:51:46.943439+0000","last_became_active":"2024-04-26T20:51:44.934690+0000","last_became_peered":"2024-04-26T20:51:44.934690+0000","last_unstale":"2024-04-26T20:51:46.943439+0000","last_undegraded":"2024-04-26T20:51:46.943439+0000","last_fullsized":"2024-04-26T20:51:46.943439+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:43:56.477046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:46.946001+0000","last_change":"2024-04-26T20:51:45.940982+0000","last_active":"2024-04-26T20:51:46.946001+0000","last_peered":"2024-04-26T20:51:46.946001+0000","last_clean":"2024-04-26T20:51:46.946001+0000","last_became_active":"2024-04-26T20:51:45.940690+0000","last_became_peered":"2024-04-26T20:51:45.940690+0000","last_unstale":"2024-04-26T20:51:46.946001+0000","last_undegraded":"2024-04-26T20:51:46.946001+0000","last_fullsized":"2024-04-26T20:51:46.946001+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:43:26.212890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:47.393409+0000","last_change":"2024-04-26T20:51:44.944446+0000","last_active":"2024-04-26T20:51:47.393409+0000","last_peered":"2024-04-26T20:51:47.393409+0000","last_clean":"2024-04-26T20:51:47.393409+0000","last_became_active":"2024-04-26T20:51:44.944147+0000","last_became_peered":"2024-04-26T20:51:44.944147+0000","last_unstale":"2024-04-26T20:51:47.393409+0000","last_undegraded":"2024-04-26T20:51:47.393409+0000","last_fullsized":"2024-04-26T20:51:47.393409+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:48:23.398632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:47.393508+0000","last_change":"2024-04-26T20:51:45.947410+0000","last_active":"2024-04-26T20:51:47.393508+0000","last_peered":"2024-04-26T20:51:47.393508+0000","last_clean":"2024-04-26T20:51:47.393508+0000","last_became_active":"2024-04-26T20:51:45.947161+0000","last_became_peered":"2024-04-26T20:51:45.947161+0000","last_unstale":"2024-04-26T20:51:47.393508+0000","last_undegraded":"2024-04-26T20:51:47.393508+0000","last_fullsized":"2024-04-26T20:51:47.393508+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:29:48.911011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:46.943288+0000","last_change":"2024-04-26T20:51:44.932932+0000","last_active":"2024-04-26T20:51:46.943288+0000","last_peered":"2024-04-26T20:51:46.943288+0000","last_clean":"2024-04-26T20:51:46.943288+0000","last_became_active":"2024-04-26T20:51:44.932657+0000","last_became_peered":"2024-04-26T20:51:44.932657+0000","last_unstale":"2024-04-26T20:51:46.943288+0000","last_undegraded":"2024-04-26T20:51:46.943288+0000","last_fullsized":"2024-04-26T20:51:46.943288+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:14:47.433882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:46.935419+0000","last_change":"2024-04-26T20:51:45.938183+0000","last_active":"2024-04-26T20:51:46.935419+0000","last_peered":"2024-04-26T20:51:46.935419+0000","last_clean":"2024-04-26T20:51:46.935419+0000","last_became_active":"2024-04-26T20:51:45.937934+0000","last_became_peered":"2024-04-26T20:51:45.937934+0000","last_unstale":"2024-04-26T20:51:46.935419+0000","last_undegraded":"2024-04-26T20:51:46.935419+0000","last_fullsized":"2024-04-26T20:51:46.935419+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:41.996127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:46.945974+0000","last_change":"2024-04-26T20:51:44.938761+0000","last_active":"2024-04-26T20:51:46.945974+0000","last_peered":"2024-04-26T20:51:46.945974+0000","last_clean":"2024-04-26T20:51:46.945974+0000","last_became_active":"2024-04-26T20:51:44.938483+0000","last_became_peered":"2024-04-26T20:51:44.938483+0000","last_unstale":"2024-04-26T20:51:46.945974+0000","last_undegraded":"2024-04-26T20:51:46.945974+0000","last_fullsized":"2024-04-26T20:51:46.945974+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:53:35.698065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:47.393492+0000","last_change":"2024-04-26T20:51:45.949689+0000","last_active":"2024-04-26T20:51:47.393492+0000","last_peered":"2024-04-26T20:51:47.393492+0000","last_clean":"2024-04-26T20:51:47.393492+0000","last_became_active":"2024-04-26T20:51:45.949392+0000","last_became_peered":"2024-04-26T20:51:45.949392+0000","last_unstale":"2024-04-26T20:51:47.393492+0000","last_undegraded":"2024-04-26T20:51:47.393492+0000","last_fullsized":"2024-04-26T20:51:47.393492+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:19:18.478019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"39'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:46.943471+0000","last_change":"2024-04-26T20:51:44.932513+0000","last_active":"2024-04-26T20:51:46.943471+0000","last_peered":"2024-04-26T20:51:46.943471+0000","last_clean":"2024-04-26T20:51:46.943471+0000","last_became_active":"2024-04-26T20:51:44.932223+0000","last_became_peered":"2024-04-26T20:51:44.932223+0000","last_unstale":"2024-04-26T20:51:46.943471+0000","last_undegraded":"2024-04-26T20:51:46.943471+0000","last_fullsized":"2024-04-26T20:51:46.943471+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:06:49.166440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:47.393463+0000","last_change":"2024-04-26T20:51:45.942544+0000","last_active":"2024-04-26T20:51:47.393463+0000","last_peered":"2024-04-26T20:51:47.393463+0000","last_clean":"2024-04-26T20:51:47.393463+0000","last_became_active":"2024-04-26T20:51:45.941717+0000","last_became_peered":"2024-04-26T20:51:45.941717+0000","last_unstale":"2024-04-26T20:51:47.393463+0000","last_undegraded":"2024-04-26T20:51:47.393463+0000","last_fullsized":"2024-04-26T20:51:47.393463+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:43:26.212890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:47.393279+0000","last_change":"2024-04-26T20:51:44.938966+0000","last_active":"2024-04-26T20:51:47.393279+0000","last_peered":"2024-04-26T20:51:47.393279+0000","last_clean":"2024-04-26T20:51:47.393279+0000","last_became_active":"2024-04-26T20:51:44.938662+0000","last_became_peered":"2024-04-26T20:51:44.938662+0000","last_unstale":"2024-04-26T20:51:47.393279+0000","last_undegraded":"2024-04-26T20:51:47.393279+0000","last_fullsized":"2024-04-26T20:51:47.393279+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:10:56.604114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:47.393208+0000","last_change":"2024-04-26T20:51:45.937767+0000","last_active":"2024-04-26T20:51:47.393208+0000","last_peered":"2024-04-26T20:51:47.393208+0000","last_clean":"2024-04-26T20:51:47.393208+0000","last_became_active":"2024-04-26T20:51:45.937489+0000","last_became_peered":"2024-04-26T20:51:45.937489+0000","last_unstale":"2024-04-26T20:51:47.393208+0000","last_undegraded":"2024-04-26T20:51:47.393208+0000","last_fullsized":"2024-04-26T20:51:47.393208+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:41.996127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:47.393506+0000","last_change":"2024-04-26T20:51:44.935952+0000","last_active":"2024-04-26T20:51:47.393506+0000","last_peered":"2024-04-26T20:51:47.393506+0000","last_clean":"2024-04-26T20:51:47.393506+0000","last_became_active":"2024-04-26T20:51:44.935682+0000","last_became_peered":"2024-04-26T20:51:44.935682+0000","last_unstale":"2024-04-26T20:51:47.393506+0000","last_undegraded":"2024-04-26T20:51:47.393506+0000","last_fullsized":"2024-04-26T20:51:47.393506+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:17.661527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:46.935173+0000","last_change":"2024-04-26T20:51:45.939639+0000","last_active":"2024-04-26T20:51:46.935173+0000","last_peered":"2024-04-26T20:51:46.935173+0000","last_clean":"2024-04-26T20:51:46.935173+0000","last_became_active":"2024-04-26T20:51:45.939382+0000","last_became_peered":"2024-04-26T20:51:45.939382+0000","last_unstale":"2024-04-26T20:51:46.935173+0000","last_undegraded":"2024-04-26T20:51:46.935173+0000","last_fullsized":"2024-04-26T20:51:46.935173+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:06:47.440852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:47.226205+0000","last_change":"2024-04-26T20:51:44.932688+0000","last_active":"2024-04-26T20:51:47.226205+0000","last_peered":"2024-04-26T20:51:47.226205+0000","last_clean":"2024-04-26T20:51:47.226205+0000","last_became_active":"2024-04-26T20:51:44.932417+0000","last_became_peered":"2024-04-26T20:51:44.932417+0000","last_unstale":"2024-04-26T20:51:47.226205+0000","last_undegraded":"2024-04-26T20:51:47.226205+0000","last_fullsized":"2024-04-26T20:51:47.226205+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:54:19.773392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:46.938409+0000","last_change":"2024-04-26T20:51:45.937227+0000","last_active":"2024-04-26T20:51:46.938409+0000","last_peered":"2024-04-26T20:51:46.938409+0000","last_clean":"2024-04-26T20:51:46.938409+0000","last_became_active":"2024-04-26T20:51:45.936929+0000","last_became_peered":"2024-04-26T20:51:45.936929+0000","last_unstale":"2024-04-26T20:51:46.938409+0000","last_undegraded":"2024-04-26T20:51:46.938409+0000","last_fullsized":"2024-04-26T20:51:46.938409+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:31:37.196512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:46.938199+0000","last_change":"2024-04-26T20:51:44.934242+0000","last_active":"2024-04-26T20:51:46.938199+0000","last_peered":"2024-04-26T20:51:46.938199+0000","last_clean":"2024-04-26T20:51:46.938199+0000","last_became_active":"2024-04-26T20:51:44.933449+0000","last_became_peered":"2024-04-26T20:51:44.933449+0000","last_unstale":"2024-04-26T20:51:46.938199+0000","last_undegraded":"2024-04-26T20:51:46.938199+0000","last_fullsized":"2024-04-26T20:51:46.938199+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:47:43.120436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:47.393578+0000","last_change":"2024-04-26T20:51:45.946767+0000","last_active":"2024-04-26T20:51:47.393578+0000","last_peered":"2024-04-26T20:51:47.393578+0000","last_clean":"2024-04-26T20:51:47.393578+0000","last_became_active":"2024-04-26T20:51:45.945619+0000","last_became_peered":"2024-04-26T20:51:45.945619+0000","last_unstale":"2024-04-26T20:51:47.393578+0000","last_undegraded":"2024-04-26T20:51:47.393578+0000","last_fullsized":"2024-04-26T20:51:47.393578+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:06:29.255667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:47.393066+0000","last_change":"2024-04-26T20:51:44.934065+0000","last_active":"2024-04-26T20:51:47.393066+0000","last_peered":"2024-04-26T20:51:47.393066+0000","last_clean":"2024-04-26T20:51:47.393066+0000","last_became_active":"2024-04-26T20:51:44.933763+0000","last_became_peered":"2024-04-26T20:51:44.933763+0000","last_unstale":"2024-04-26T20:51:47.393066+0000","last_undegraded":"2024-04-26T20:51:47.393066+0000","last_fullsized":"2024-04-26T20:51:47.393066+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:06:46.438682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:46.945693+0000","last_change":"2024-04-26T20:51:45.953526+0000","last_active":"2024-04-26T20:51:46.945693+0000","last_peered":"2024-04-26T20:51:46.945693+0000","last_clean":"2024-04-26T20:51:46.945693+0000","last_became_active":"2024-04-26T20:51:45.953259+0000","last_became_peered":"2024-04-26T20:51:45.953259+0000","last_unstale":"2024-04-26T20:51:46.945693+0000","last_undegraded":"2024-04-26T20:51:46.945693+0000","last_fullsized":"2024-04-26T20:51:46.945693+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:44.916026+0000","last_clean_scrub_stamp":"2024-04-26T20:51:44.916026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:09:05.115224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:47.393223+0000","last_change":"2024-04-26T20:51:44.944770+0000","last_active":"2024-04-26T20:51:47.393223+0000","last_peered":"2024-04-26T20:51:47.393223+0000","last_clean":"2024-04-26T20:51:47.393223+0000","last_became_active":"2024-04-26T20:51:44.944478+0000","last_became_peered":"2024-04-26T20:51:44.944478+0000","last_unstale":"2024-04-26T20:51:47.393223+0000","last_undegraded":"2024-04-26T20:51:47.393223+0000","last_fullsized":"2024-04-26T20:51:47.393223+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:51:43.913856+0000","last_clean_scrub_stamp":"2024-04-26T20:51:43.913856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:38:35.270443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":203,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T20:51:47.393583+0000","last_change":"2024-04-26T20:50:21.150132+0000","last_active":"2024-04-26T20:51:47.393583+0000","last_peered":"2024-04-26T20:51:47.393583+0000","last_clean":"2024-04-26T20:51:47.393583+0000","last_became_active":"2024-04-26T20:50:21.147439+0000","last_became_peered":"2024-04-26T20:50:21.147439+0000","last_unstale":"2024-04-26T20:51:47.393583+0000","last_undegraded":"2024-04-26T20:51:47.393583+0000","last_fullsized":"2024-04-26T20:51:47.393583+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:20.084499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:20.084499+0000","last_clean_scrub_stamp":"2024-04-26T20:50:20.084499+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:19:29.395477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343784,"statfs":{"total":96636764160,"available":96608034816,"internally_reserved":0,"allocated":1060864,"data_stored":789207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":197739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":458752,"data_stored":197269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":458752,"data_stored":197761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":466944,"data_stored":198803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":197227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":197247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343796,"statfs":{"total":96636764160,"available":96608047104,"internally_reserved":0,"allocated":1048576,"data_stored":787559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:51:53.593 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T20:51:53.593 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-26T20:51:53.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.730+0000 7fb3ced0a640 1 Processor -- start 2024-04-26T20:51:53.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.730+0000 7fb3ced0a640 1 -- start start 2024-04-26T20:51:53.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.730+0000 7fb3ced0a640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fb3c80b0b70 0x7fb3c80b0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:53.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.730+0000 7fb3ced0a640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb3c80b4d40 0x7fb3c80b6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:53.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.730+0000 7fb3ced0a640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb3c80b6c50 0x7fb3c815f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:53.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.730+0000 7fb3ced0a640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7fb3c8073bc0 con 0x7fb3c80b6c50 2024-04-26T20:51:53.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.730+0000 7fb3ced0a640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fb3c80b1490 con 0x7fb3c80b0b70 2024-04-26T20:51:53.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.730+0000 7fb3ced0a640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fb3c8072fb0 con 0x7fb3c80b4d40 2024-04-26T20:51:53.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.731+0000 7fb3cd2a7640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb3c80b6c50 0x7fb3c815f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:53.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.731+0000 7fb3cd2a7640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb3c80b6c50 0x7fb3c815f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41048/0 (socket says 172.21.15.73:41048) 2024-04-26T20:51:53.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.731+0000 7fb3cd2a7640 1 -- 172.21.15.73:0/303353328 learned_addr learned my addr 172.21.15.73:0/303353328 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:53.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.731+0000 7fb3cd2a7640 1 -- 172.21.15.73:0/303353328 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb3c80b4d40 msgr2=0x7fb3c80b6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:53.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.731+0000 7fb3ccaa6640 1 --2- 172.21.15.73:0/303353328 >> v2:172.21.15.120:3300/0 conn(0x7fb3c80b0b70 0x7fb3c80b0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:53.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.731+0000 7fb3cd2a7640 1 --2- 172.21.15.73:0/303353328 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb3c80b4d40 0x7fb3c80b6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:53.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.731+0000 7fb3cd2a7640 1 -- 172.21.15.73:0/303353328 >> v2:172.21.15.120:3300/0 conn(0x7fb3c80b0b70 msgr2=0x7fb3c80b0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:53.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.731+0000 7fb3cd2a7640 1 --2- 172.21.15.73:0/303353328 >> v2:172.21.15.120:3300/0 conn(0x7fb3c80b0b70 0x7fb3c80b0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:53.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.731+0000 7fb3cd2a7640 1 -- 172.21.15.73:0/303353328 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3c8073290 con 0x7fb3c80b6c50 2024-04-26T20:51:53.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.731+0000 7fb3ccaa6640 1 --2- 172.21.15.73:0/303353328 >> v2:172.21.15.120:3300/0 conn(0x7fb3c80b0b70 0x7fb3c80b0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:53.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.731+0000 7fb3cd2a7640 1 --2- 172.21.15.73:0/303353328 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3c80b6c50 0x7fb3c815f350 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb3c806edf0 tx=0x7fb3c002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd4b05183b4e2401 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:53.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.732+0000 7fb3c77fe640 1 -- 172.21.15.73:0/303353328 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c0036070 con 0x7fb3c80b6c50 2024-04-26T20:51:53.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.732+0000 7fb3c77fe640 1 -- 172.21.15.73:0/303353328 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3c0031600 con 0x7fb3c80b6c50 2024-04-26T20:51:53.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.732+0000 7fb3c77fe640 1 -- 172.21.15.73:0/303353328 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c00319d0 con 0x7fb3c80b6c50 2024-04-26T20:51:53.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.732+0000 7fb3ced0a640 1 -- 172.21.15.73:0/303353328 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3c80b6c50 msgr2=0x7fb3c815f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:53.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.732+0000 7fb3ced0a640 1 --2- 172.21.15.73:0/303353328 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3c80b6c50 0x7fb3c815f350 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb3c806edf0 tx=0x7fb3c002a6a0 comp rx=0 tx=0).stop 2024-04-26T20:51:53.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.732+0000 7fb3ced0a640 1 -- 172.21.15.73:0/303353328 shutdown_connections 2024-04-26T20:51:53.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.732+0000 7fb3ced0a640 1 --2- 172.21.15.73:0/303353328 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3c80b6c50 0x7fb3c815f350 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:53.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.732+0000 7fb3ced0a640 1 --2- 172.21.15.73:0/303353328 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb3c80b4d40 0x7fb3c80b6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:53.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.732+0000 7fb3ced0a640 1 --2- 172.21.15.73:0/303353328 >> v2:172.21.15.120:3300/0 conn(0x7fb3c80b0b70 0x7fb3c80b0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:53.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.732+0000 7fb3ced0a640 1 -- 172.21.15.73:0/303353328 >> 172.21.15.73:0/303353328 conn(0x7fb3c80b39b0 msgr2=0x7fb3c80b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:53.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.732+0000 7fb3ced0a640 1 -- 172.21.15.73:0/303353328 shutdown_connections 2024-04-26T20:51:53.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.732+0000 7fb3ced0a640 1 -- 172.21.15.73:0/303353328 wait complete. 2024-04-26T20:51:53.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.733+0000 7fb3ced0a640 1 Processor -- start 2024-04-26T20:51:53.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.733+0000 7fb3ced0a640 1 -- start start 2024-04-26T20:51:53.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.734+0000 7fb3ced0a640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb3c80b0b70 0x7fb3c814e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:53.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.734+0000 7fb3ced0a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb3c80b4d40 0x7fb3c81e9250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:53.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.734+0000 7fb3ced0a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3c80b6c50 0x7fb3c81e7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:53.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.734+0000 7fb3ced0a640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c815f890 con 0x7fb3c80b6c50 2024-04-26T20:51:53.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.734+0000 7fb3ced0a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c8073850 con 0x7fb3c80b4d40 2024-04-26T20:51:53.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.734+0000 7fb3ced0a640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fb3c815fa10 con 0x7fb3c80b0b70 2024-04-26T20:51:53.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.734+0000 7fb3cd2a7640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3c80b6c50 0x7fb3c81e7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:53.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.734+0000 7fb3cd2a7640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3c80b6c50 0x7fb3c81e7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41056/0 (socket says 172.21.15.73:41056) 2024-04-26T20:51:53.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.734+0000 7fb3cd2a7640 1 -- 172.21.15.73:0/3502900072 learned_addr learned my addr 172.21.15.73:0/3502900072 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:53.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.735+0000 7fb3c7fff640 1 --2- 172.21.15.73:0/3502900072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb3c80b4d40 0x7fb3c81e9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:53.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.735+0000 7fb3ccaa6640 1 --2- 172.21.15.73:0/3502900072 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb3c80b0b70 0x7fb3c814e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:53.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.735+0000 7fb3cd2a7640 1 -- 172.21.15.73:0/3502900072 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb3c80b0b70 msgr2=0x7fb3c814e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:53.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.735+0000 7fb3cd2a7640 1 --2- 172.21.15.73:0/3502900072 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb3c80b0b70 0x7fb3c814e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:53.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.735+0000 7fb3cd2a7640 1 -- 172.21.15.73:0/3502900072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb3c80b4d40 msgr2=0x7fb3c81e9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:53.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.735+0000 7fb3cd2a7640 1 --2- 172.21.15.73:0/3502900072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb3c80b4d40 0x7fb3c81e9250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:53.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.735+0000 7fb3cd2a7640 1 -- 172.21.15.73:0/3502900072 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3c81e81b0 con 0x7fb3c80b6c50 2024-04-26T20:51:53.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.735+0000 7fb3c7fff640 1 --2- 172.21.15.73:0/3502900072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb3c80b4d40 0x7fb3c81e9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:53.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.735+0000 7fb3ccaa6640 1 --2- 172.21.15.73:0/3502900072 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb3c80b0b70 0x7fb3c814e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:53.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.735+0000 7fb3cd2a7640 1 --2- 172.21.15.73:0/3502900072 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3c80b6c50 0x7fb3c81e7760 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb3c002ac90 tx=0x7fb3c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:53.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.735+0000 7fb3c57fa640 1 -- 172.21.15.73:0/3502900072 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c0041020 con 0x7fb3c80b6c50 2024-04-26T20:51:53.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.735+0000 7fb3c57fa640 1 -- 172.21.15.73:0/3502900072 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3c0007780 con 0x7fb3c80b6c50 2024-04-26T20:51:53.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.735+0000 7fb3ced0a640 1 -- 172.21.15.73:0/3502900072 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3c81e8490 con 0x7fb3c80b6c50 2024-04-26T20:51:53.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.735+0000 7fb3c57fa640 1 -- 172.21.15.73:0/3502900072 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c0036040 con 0x7fb3c80b6c50 2024-04-26T20:51:53.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.735+0000 7fb3ced0a640 1 -- 172.21.15.73:0/3502900072 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3c81e8a50 con 0x7fb3c80b6c50 2024-04-26T20:51:53.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.736+0000 7fb3ced0a640 1 -- 172.21.15.73:0/3502900072 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3c80b5690 con 0x7fb3c80b6c50 2024-04-26T20:51:53.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.736+0000 7fb3c57fa640 1 -- 172.21.15.73:0/3502900072 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb3c0044020 con 0x7fb3c80b6c50 2024-04-26T20:51:53.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.737+0000 7fb3c57fa640 1 --2- 172.21.15.73:0/3502900072 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb3ac06c920 0x7fb3ac06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:53.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.737+0000 7fb3c57fa640 1 -- 172.21.15.73:0/3502900072 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fb3c00aca60 con 0x7fb3c80b6c50 2024-04-26T20:51:53.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.737+0000 7fb3ccaa6640 1 --2- 172.21.15.73:0/3502900072 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb3ac06c920 0x7fb3ac06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:53.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.737+0000 7fb3ccaa6640 1 --2- 172.21.15.73:0/3502900072 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb3ac06c920 0x7fb3ac06ede0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc006fd0 tx=0x7fb3bc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:53.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.741+0000 7fb3c57fa640 1 -- 172.21.15.73:0/3502900072 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb3c80b5690 con 0x7fb3c80b6c50 2024-04-26T20:51:53.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.987+0000 7fb3ced0a640 1 -- 172.21.15.73:0/3502900072 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fb3c80694f0 con 0x7fb3c80b6c50 2024-04-26T20:51:53.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.989+0000 7fb3c57fa640 1 -- 172.21.15.73:0/3502900072 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) ==== 190+0+147 (secure 0 0 0) 0x7fb3c004b030 con 0x7fb3c80b6c50 2024-04-26T20:51:53.990 INFO:teuthology.orchestra.run.smithi073.stdout:[client.0] 2024-04-26T20:51:53.991 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQBUEyxmch+BMBAAap3kvW3+/FlCuAkhRtbWxQ== 2024-04-26T20:51:53.991 INFO:teuthology.orchestra.run.smithi073.stdout: caps mds = "allow" 2024-04-26T20:51:53.991 INFO:teuthology.orchestra.run.smithi073.stdout: caps mon = "allow r" 2024-04-26T20:51:53.991 INFO:teuthology.orchestra.run.smithi073.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-26T20:51:53.991 INFO:teuthology.orchestra.run.smithi073.stderr:updated caps for client.0 2024-04-26T20:51:53.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.991+0000 7fb3ced0a640 1 -- 172.21.15.73:0/3502900072 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb3ac06c920 msgr2=0x7fb3ac06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:53.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.991+0000 7fb3ced0a640 1 --2- 172.21.15.73:0/3502900072 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb3ac06c920 0x7fb3ac06ede0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc006fd0 tx=0x7fb3bc008040 comp rx=0 tx=0).stop 2024-04-26T20:51:53.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.991+0000 7fb3ced0a640 1 -- 172.21.15.73:0/3502900072 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3c80b6c50 msgr2=0x7fb3c81e7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:53.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.991+0000 7fb3ced0a640 1 --2- 172.21.15.73:0/3502900072 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3c80b6c50 0x7fb3c81e7760 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb3c002ac90 tx=0x7fb3c0004a40 comp rx=0 tx=0).stop 2024-04-26T20:51:53.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.991+0000 7fb3ced0a640 1 -- 172.21.15.73:0/3502900072 shutdown_connections 2024-04-26T20:51:53.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.991+0000 7fb3ced0a640 1 --2- 172.21.15.73:0/3502900072 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3c80b6c50 0x7fb3c81e7760 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:53.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.991+0000 7fb3ced0a640 1 --2- 172.21.15.73:0/3502900072 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb3ac06c920 0x7fb3ac06ede0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:53.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.991+0000 7fb3ced0a640 1 --2- 172.21.15.73:0/3502900072 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb3c80b4d40 0x7fb3c81e9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:53.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.991+0000 7fb3ced0a640 1 --2- 172.21.15.73:0/3502900072 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb3c80b0b70 0x7fb3c814e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:53.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.991+0000 7fb3ced0a640 1 -- 172.21.15.73:0/3502900072 >> 172.21.15.73:0/3502900072 conn(0x7fb3c80b39b0 msgr2=0x7fb3c80b1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:53.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.991+0000 7fb3ced0a640 1 -- 172.21.15.73:0/3502900072 shutdown_connections 2024-04-26T20:51:53.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:53.991+0000 7fb3ced0a640 1 -- 172.21.15.73:0/3502900072 wait complete. 2024-04-26T20:51:54.003 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:51:54.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.099+0000 7fdc44e8c640 1 Processor -- start 2024-04-26T20:51:54.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.099+0000 7fdc44e8c640 1 -- start start 2024-04-26T20:51:54.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.100+0000 7fdc44e8c640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdc4015d100 0x7fdc40160ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:54.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.100+0000 7fdc44e8c640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fdc40070930 0x7fdc400aed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:54.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.100+0000 7fdc44e8c640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc400b0990 con 0x7fdc4006c240 2024-04-26T20:51:54.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.100+0000 7fdc44e8c640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fdc400af2d0 con 0x7fdc40070930 2024-04-26T20:51:54.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.100+0000 7fdc44e8c640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fdc400b0c70 con 0x7fdc4015d100 2024-04-26T20:51:54.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.101+0000 7fdc3dd9b640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fdc40070930 0x7fdc400aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:54.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.101+0000 7fdc3ed9d640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7fdc4006c240 0x7fdc40073160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.73:51620/0 (socket says 172.21.15.73:51620) 2024-04-26T20:51:54.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.101+0000 7fdc3e59c640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdc4015d100 0x7fdc40160ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:54.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.101+0000 7fdc3ed9d640 1 -- 172.21.15.73:0/2475748446 learned_addr learned my addr 172.21.15.73:0/2475748446 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:54.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.101+0000 7fdc3d59a640 1 -- 172.21.15.73:0/2475748446 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1035744743 0 0) 0x7fdc400b0990 con 0x7fdc4006c240 2024-04-26T20:51:54.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.101+0000 7fdc3e59c640 1 -- 172.21.15.73:0/2475748446 >> v2:172.21.15.120:3300/0 conn(0x7fdc40070930 msgr2=0x7fdc400aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:54.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.101+0000 7fdc3e59c640 1 --2- 172.21.15.73:0/2475748446 >> v2:172.21.15.120:3300/0 conn(0x7fdc40070930 0x7fdc400aed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:54.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.101+0000 7fdc3e59c640 1 -- 172.21.15.73:0/2475748446 >> v1:172.21.15.120:6789/0 conn(0x7fdc4006c240 legacy=0x7fdc40073160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:54.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.101+0000 7fdc3e59c640 1 -- 172.21.15.73:0/2475748446 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc400b0f50 con 0x7fdc4015d100 2024-04-26T20:51:54.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.102+0000 7fdc3e59c640 1 --2- 172.21.15.73:0/2475748446 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdc4015d100 0x7fdc40160ba0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdc30009d90 tx=0x7fdc3002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60297f3508188f34 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:54.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.102+0000 7fdc3d59a640 1 -- 172.21.15.73:0/2475748446 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc30007740 con 0x7fdc4015d100 2024-04-26T20:51:54.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.102+0000 7fdc3d59a640 1 -- 172.21.15.73:0/2475748446 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc300078e0 con 0x7fdc4015d100 2024-04-26T20:51:54.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.102+0000 7fdc3d59a640 1 -- 172.21.15.73:0/2475748446 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc30007cb0 con 0x7fdc4015d100 2024-04-26T20:51:54.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.103+0000 7fdc44e8c640 1 -- 172.21.15.73:0/2475748446 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdc4015d100 msgr2=0x7fdc40160ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:54.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.103+0000 7fdc44e8c640 1 --2- 172.21.15.73:0/2475748446 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdc4015d100 0x7fdc40160ba0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdc30009d90 tx=0x7fdc3002a660 comp rx=0 tx=0).stop 2024-04-26T20:51:54.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.103+0000 7fdc44e8c640 1 -- 172.21.15.73:0/2475748446 shutdown_connections 2024-04-26T20:51:54.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.103+0000 7fdc44e8c640 1 --2- 172.21.15.73:0/2475748446 >> v2:172.21.15.120:3300/0 conn(0x7fdc40070930 0x7fdc400aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:54.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.103+0000 7fdc44e8c640 1 --2- 172.21.15.73:0/2475748446 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdc4015d100 0x7fdc40160ba0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:54.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.103+0000 7fdc44e8c640 1 -- 172.21.15.73:0/2475748446 >> 172.21.15.73:0/2475748446 conn(0x7fdc400b39b0 msgr2=0x7fdc400b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:54.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.103+0000 7fdc44e8c640 1 -- 172.21.15.73:0/2475748446 shutdown_connections 2024-04-26T20:51:54.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.103+0000 7fdc44e8c640 1 -- 172.21.15.73:0/2475748446 wait complete. 2024-04-26T20:51:54.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.104+0000 7fdc44e8c640 1 Processor -- start 2024-04-26T20:51:54.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.104+0000 7fdc44e8c640 1 -- start start 2024-04-26T20:51:54.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.105+0000 7fdc44e8c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc4006c240 0x7fdc401477b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:54.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.105+0000 7fdc44e8c640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdc40070930 0x7fdc401f2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:54.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.105+0000 7fdc44e8c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc4015d100 0x7fdc4015bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:54.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.105+0000 7fdc44e8c640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4006c6d0 con 0x7fdc4006c240 2024-04-26T20:51:54.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.105+0000 7fdc44e8c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdc400b1510 con 0x7fdc4015d100 2024-04-26T20:51:54.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.105+0000 7fdc44e8c640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fdc4006c850 con 0x7fdc40070930 2024-04-26T20:51:54.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.105+0000 7fdc3e59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc4006c240 0x7fdc401477b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:54.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.105+0000 7fdc3e59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc4006c240 0x7fdc401477b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41072/0 (socket says 172.21.15.73:41072) 2024-04-26T20:51:54.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.105+0000 7fdc3e59c640 1 -- 172.21.15.73:0/1888933036 learned_addr learned my addr 172.21.15.73:0/1888933036 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:54.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.105+0000 7fdc3dd9b640 1 --2- 172.21.15.73:0/1888933036 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdc40070930 0x7fdc401f2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:54.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.105+0000 7fdc3ed9d640 1 --2- 172.21.15.73:0/1888933036 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc4015d100 0x7fdc4015bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:54.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.105+0000 7fdc3e59c640 1 -- 172.21.15.73:0/1888933036 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdc40070930 msgr2=0x7fdc401f2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:54.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.105+0000 7fdc3e59c640 1 --2- 172.21.15.73:0/1888933036 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdc40070930 0x7fdc401f2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:54.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.105+0000 7fdc3e59c640 1 -- 172.21.15.73:0/1888933036 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc4015d100 msgr2=0x7fdc4015bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:54.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.105+0000 7fdc3e59c640 1 --2- 172.21.15.73:0/1888933036 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc4015d100 0x7fdc4015bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:54.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.105+0000 7fdc3e59c640 1 -- 172.21.15.73:0/1888933036 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc40147cf0 con 0x7fdc4006c240 2024-04-26T20:51:54.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.105+0000 7fdc3e59c640 1 --2- 172.21.15.73:0/1888933036 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc4006c240 0x7fdc401477b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fdc30009d90 tx=0x7fdc30004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:54.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.105+0000 7fdc2effd640 1 -- 172.21.15.73:0/1888933036 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc3003f070 con 0x7fdc4006c240 2024-04-26T20:51:54.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.105+0000 7fdc2effd640 1 -- 172.21.15.73:0/1888933036 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc30031c40 con 0x7fdc4006c240 2024-04-26T20:51:54.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.105+0000 7fdc44e8c640 1 -- 172.21.15.73:0/1888933036 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc40159150 con 0x7fdc4006c240 2024-04-26T20:51:54.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.105+0000 7fdc2effd640 1 -- 172.21.15.73:0/1888933036 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc30009450 con 0x7fdc4006c240 2024-04-26T20:51:54.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.105+0000 7fdc44e8c640 1 -- 172.21.15.73:0/1888933036 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc401542f0 con 0x7fdc4006c240 2024-04-26T20:51:54.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.106+0000 7fdc44e8c640 1 -- 172.21.15.73:0/1888933036 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc400694f0 con 0x7fdc4006c240 2024-04-26T20:51:54.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.107+0000 7fdc2effd640 1 -- 172.21.15.73:0/1888933036 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdc30003990 con 0x7fdc4006c240 2024-04-26T20:51:54.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.107+0000 7fdc2effd640 1 --2- 172.21.15.73:0/1888933036 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fdc2006c8f0 0x7fdc2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:54.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.107+0000 7fdc2effd640 1 -- 172.21.15.73:0/1888933036 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fdc300aceb0 con 0x7fdc4006c240 2024-04-26T20:51:54.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.107+0000 7fdc3dd9b640 1 --2- 172.21.15.73:0/1888933036 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fdc2006c8f0 0x7fdc2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:54.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.108+0000 7fdc3dd9b640 1 --2- 172.21.15.73:0/1888933036 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fdc2006c8f0 0x7fdc2006edb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdc40160470 tx=0x7fdc34008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:54.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.111+0000 7fdc2effd640 1 -- 172.21.15.73:0/1888933036 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdc3007b0c0 con 0x7fdc4006c240 2024-04-26T20:51:54.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.323+0000 7fdc44e8c640 1 -- 172.21.15.73:0/1888933036 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdc4006cd70 con 0x7fdc4006c240 2024-04-26T20:51:54.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.323+0000 7fdc2effd640 1 -- 172.21.15.73:0/1888933036 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+5181 (secure 0 0 0) 0x7fdc3007a9c0 con 0x7fdc4006c240 2024-04-26T20:51:54.324 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:51:54.324 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4802,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/272662913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":272662913},{"type":"v1","addr":"172.21.15.73:6836","nonce":272662913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5047,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/1438858294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":1438858294},{"type":"v1","addr":"172.21.15.73:6837","nonce":1438858294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5050,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6834/3883540031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3883540031},{"type":"v1","addr":"172.21.15.120:6834","nonce":3883540031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:51:45.273953+0000","modified":"2024-04-26T20:51:47.270309+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5053":{"gid":5053,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.120:6835/761435580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6833","nonce":761435580},{"type":"v1","addr":"172.21.15.120:6835","nonce":761435580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5053,"qdb_cluster":[5053]},"id":2}]} 2024-04-26T20:51:54.325 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 15 2024-04-26T20:51:54.326 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.325+0000 7fdc44e8c640 1 -- 172.21.15.73:0/1888933036 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fdc2006c8f0 msgr2=0x7fdc2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:54.326 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.325+0000 7fdc44e8c640 1 --2- 172.21.15.73:0/1888933036 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fdc2006c8f0 0x7fdc2006edb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdc40160470 tx=0x7fdc34008040 comp rx=0 tx=0).stop 2024-04-26T20:51:54.326 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.325+0000 7fdc44e8c640 1 -- 172.21.15.73:0/1888933036 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc4006c240 msgr2=0x7fdc401477b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:54.326 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.325+0000 7fdc44e8c640 1 --2- 172.21.15.73:0/1888933036 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc4006c240 0x7fdc401477b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fdc30009d90 tx=0x7fdc30004a40 comp rx=0 tx=0).stop 2024-04-26T20:51:54.326 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.325+0000 7fdc44e8c640 1 -- 172.21.15.73:0/1888933036 shutdown_connections 2024-04-26T20:51:54.326 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.325+0000 7fdc44e8c640 1 --2- 172.21.15.73:0/1888933036 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fdc2006c8f0 0x7fdc2006edb0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:54.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.325+0000 7fdc44e8c640 1 --2- 172.21.15.73:0/1888933036 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdc4015d100 0x7fdc4015bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:54.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.325+0000 7fdc44e8c640 1 --2- 172.21.15.73:0/1888933036 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdc40070930 0x7fdc401f2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:54.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.325+0000 7fdc44e8c640 1 --2- 172.21.15.73:0/1888933036 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc4006c240 0x7fdc401477b0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:54.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.325+0000 7fdc44e8c640 1 -- 172.21.15.73:0/1888933036 >> 172.21.15.73:0/1888933036 conn(0x7fdc400b39b0 msgr2=0x7fdc40073c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:54.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.325+0000 7fdc44e8c640 1 -- 172.21.15.73:0/1888933036 shutdown_connections 2024-04-26T20:51:54.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.325+0000 7fdc44e8c640 1 -- 172.21.15.73:0/1888933036 wait complete. 2024-04-26T20:51:54.740 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 15, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T20:51:45.273953+0000', 'modified': '2024-04-26T20:51:47.270309+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5053}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5053': {'gid': 5053, 'name': 'd', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.120:6835/761435580', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6833', 'nonce': 761435580}, {'type': 'v1', 'addr': '172.21.15.120:6835', 'nonce': 761435580}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5053, 'qdb_cluster': [5053]} 2024-04-26T20:51:54.740 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T20:51:54.740 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-26T20:51:54.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.838+0000 7faba7052640 1 Processor -- start 2024-04-26T20:51:54.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.838+0000 7faba7052640 1 -- start start 2024-04-26T20:51:54.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.839+0000 7faba7052640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7faba00cb7e0 0x7faba0162e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:54.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.839+0000 7faba7052640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7faba01633c0 0x7faba00726c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:54.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.839+0000 7faba7052640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faba006f5a0 con 0x7faba015f290 2024-04-26T20:51:54.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.839+0000 7faba7052640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7faba006f350 con 0x7faba01633c0 2024-04-26T20:51:54.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.839+0000 7faba7052640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7faba00cbc20 con 0x7faba00cb7e0 2024-04-26T20:51:54.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.839+0000 7faba55ef640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7faba01633c0 0x7faba00726c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:54.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.839+0000 7fab9ffff640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7faba00cb7e0 0x7faba0162e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:54.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.839+0000 7faba4dee640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7faba015f290 0x7faba00cd060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.73:51632/0 (socket says 172.21.15.73:51632) 2024-04-26T20:51:54.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.839+0000 7faba55ef640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7faba01633c0 0x7faba00726c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.73:36456/0 (socket says 172.21.15.73:36456) 2024-04-26T20:51:54.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.839+0000 7fab9ffff640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7faba00cb7e0 0x7faba0162e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.73:37364/0 (socket says 172.21.15.73:37364) 2024-04-26T20:51:54.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.839+0000 7faba4dee640 1 -- 172.21.15.73:0/2840076068 learned_addr learned my addr 172.21.15.73:0/2840076068 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:54.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.839+0000 7fab9ffff640 1 -- 172.21.15.73:0/2840076068 >> v2:172.21.15.120:3300/0 conn(0x7faba01633c0 msgr2=0x7faba00726c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:54.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.839+0000 7fab9ffff640 1 --2- 172.21.15.73:0/2840076068 >> v2:172.21.15.120:3300/0 conn(0x7faba01633c0 0x7faba00726c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:54.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.839+0000 7fab9ffff640 1 -- 172.21.15.73:0/2840076068 >> v1:172.21.15.120:6789/0 conn(0x7faba015f290 legacy=0x7faba00cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:54.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.839+0000 7fab9ffff640 1 -- 172.21.15.73:0/2840076068 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faba006f880 con 0x7faba00cb7e0 2024-04-26T20:51:54.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.839+0000 7faba55ef640 1 --2- 172.21.15.73:0/2840076068 >> v2:172.21.15.120:3300/0 conn(0x7faba01633c0 0x7faba00726c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:51:54.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.840+0000 7fab9ffff640 1 --2- 172.21.15.73:0/2840076068 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7faba00cb7e0 0x7faba0162e80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fab8c0097a0 tx=0x7fab8c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f59f8b17a450935 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:54.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.840+0000 7fab9f7fe640 1 -- 172.21.15.73:0/2840076068 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8c037070 con 0x7faba00cb7e0 2024-04-26T20:51:54.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.840+0000 7fab9f7fe640 1 -- 172.21.15.73:0/2840076068 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab8c032440 con 0x7faba00cb7e0 2024-04-26T20:51:54.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.840+0000 7fab9f7fe640 1 -- 172.21.15.73:0/2840076068 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8c032810 con 0x7faba00cb7e0 2024-04-26T20:51:54.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.841+0000 7faba7052640 1 -- 172.21.15.73:0/2840076068 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7faba00cb7e0 msgr2=0x7faba0162e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:54.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.841+0000 7faba7052640 1 --2- 172.21.15.73:0/2840076068 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7faba00cb7e0 0x7faba0162e80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fab8c0097a0 tx=0x7fab8c02a210 comp rx=0 tx=0).stop 2024-04-26T20:51:54.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.841+0000 7faba7052640 1 -- 172.21.15.73:0/2840076068 shutdown_connections 2024-04-26T20:51:54.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.841+0000 7faba7052640 1 --2- 172.21.15.73:0/2840076068 >> v2:172.21.15.120:3300/0 conn(0x7faba01633c0 0x7faba00726c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:54.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.841+0000 7faba7052640 1 --2- 172.21.15.73:0/2840076068 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7faba00cb7e0 0x7faba0162e80 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:54.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.841+0000 7faba7052640 1 -- 172.21.15.73:0/2840076068 >> 172.21.15.73:0/2840076068 conn(0x7faba00d0fd0 msgr2=0x7faba00d13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:54.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.841+0000 7faba7052640 1 -- 172.21.15.73:0/2840076068 shutdown_connections 2024-04-26T20:51:54.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.841+0000 7faba7052640 1 -- 172.21.15.73:0/2840076068 wait complete. 2024-04-26T20:51:54.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.842+0000 7faba7052640 1 Processor -- start 2024-04-26T20:51:54.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.842+0000 7faba7052640 1 -- start start 2024-04-26T20:51:54.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.843+0000 7faba7052640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7faba00cb7e0 0x7faba01523f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:54.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.843+0000 7faba7052640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faba015f290 0x7faba0160600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:54.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.843+0000 7faba7052640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faba01633c0 0x7faba01613c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:54.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.843+0000 7faba7052640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7faba015f670 con 0x7faba01633c0 2024-04-26T20:51:54.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.843+0000 7faba7052640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7faba0072c00 con 0x7faba015f290 2024-04-26T20:51:54.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.843+0000 7faba7052640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7faba0160170 con 0x7faba00cb7e0 2024-04-26T20:51:54.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.843+0000 7faba55ef640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faba01633c0 0x7faba01613c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:54.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.843+0000 7faba55ef640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faba01633c0 0x7faba01613c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41076/0 (socket says 172.21.15.73:41076) 2024-04-26T20:51:54.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.843+0000 7faba55ef640 1 -- 172.21.15.73:0/708744203 learned_addr learned my addr 172.21.15.73:0/708744203 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:54.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.843+0000 7fab9ffff640 1 --2- 172.21.15.73:0/708744203 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faba015f290 0x7faba0160600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:54.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.843+0000 7faba4dee640 1 --2- 172.21.15.73:0/708744203 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7faba00cb7e0 0x7faba01523f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:54.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.843+0000 7faba55ef640 1 -- 172.21.15.73:0/708744203 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7faba00cb7e0 msgr2=0x7faba01523f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:54.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.843+0000 7faba55ef640 1 --2- 172.21.15.73:0/708744203 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7faba00cb7e0 0x7faba01523f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:54.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.843+0000 7faba55ef640 1 -- 172.21.15.73:0/708744203 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faba015f290 msgr2=0x7faba0160600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:54.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.843+0000 7faba55ef640 1 --2- 172.21.15.73:0/708744203 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faba015f290 0x7faba0160600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:54.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.843+0000 7faba55ef640 1 -- 172.21.15.73:0/708744203 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faba0161f30 con 0x7faba01633c0 2024-04-26T20:51:54.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.844+0000 7faba55ef640 1 --2- 172.21.15.73:0/708744203 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faba01633c0 0x7faba01613c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fab9800efc0 tx=0x7fab9800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:54.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.844+0000 7fab9d7fa640 1 -- 172.21.15.73:0/708744203 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab98018030 con 0x7faba01633c0 2024-04-26T20:51:54.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.844+0000 7fab9d7fa640 1 -- 172.21.15.73:0/708744203 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab9800cdf0 con 0x7faba01633c0 2024-04-26T20:51:54.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.844+0000 7faba7052640 1 -- 172.21.15.73:0/708744203 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faba015fb20 con 0x7faba01633c0 2024-04-26T20:51:54.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.844+0000 7fab9d7fa640 1 -- 172.21.15.73:0/708744203 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab98003bb0 con 0x7faba01633c0 2024-04-26T20:51:54.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.844+0000 7faba7052640 1 -- 172.21.15.73:0/708744203 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faba015cca0 con 0x7faba01633c0 2024-04-26T20:51:54.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.845+0000 7fab9d7fa640 1 -- 172.21.15.73:0/708744203 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fab9801b070 con 0x7faba01633c0 2024-04-26T20:51:54.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.846+0000 7fab9d7fa640 1 --2- 172.21.15.73:0/708744203 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fab8406c8f0 0x7fab8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:54.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.846+0000 7faba4dee640 1 --2- 172.21.15.73:0/708744203 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fab8406c8f0 0x7fab8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:54.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.846+0000 7fab9d7fa640 1 -- 172.21.15.73:0/708744203 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fab9808c730 con 0x7faba01633c0 2024-04-26T20:51:54.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.846+0000 7faba4dee640 1 --2- 172.21.15.73:0/708744203 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fab8406c8f0 0x7fab8406edb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fab94000f60 tx=0x7fab94009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:54.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.847+0000 7faba9d6f740 1 Processor -- start 2024-04-26T20:51:54.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.847+0000 7faba9d6f740 1 -- start start 2024-04-26T20:51:54.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.847+0000 7faba9d6f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ba39e43440 0x55ba39e43820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:54.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.847+0000 7faba9d6f740 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55ba39e60ce0 0x55ba39e61230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:54.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.847+0000 7faba9d6f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba39e19120 0x55ba39e5f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:54.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.847+0000 7faba9d6f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ba39d7d740 con 0x55ba39e43440 2024-04-26T20:51:54.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.847+0000 7faba9d6f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ba39d193a0 con 0x55ba39e19120 2024-04-26T20:51:54.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.847+0000 7faba9d6f740 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x55ba39d79700 con 0x55ba39e60ce0 2024-04-26T20:51:54.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.848+0000 7faba4dee640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ba39e43440 0x55ba39e43820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:54.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.848+0000 7faba4dee640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ba39e43440 0x55ba39e43820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41090/0 (socket says 172.21.15.73:41090) 2024-04-26T20:51:54.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.848+0000 7faba4dee640 1 -- 172.21.15.73:0/3150124535 learned_addr learned my addr 172.21.15.73:0/3150124535 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:54.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.848+0000 7faba4dee640 1 -- 172.21.15.73:0/3150124535 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55ba39e60ce0 msgr2=0x55ba39e61230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:54.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.848+0000 7faba4dee640 1 --2- 172.21.15.73:0/3150124535 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55ba39e60ce0 0x55ba39e61230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:54.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.848+0000 7faba4dee640 1 -- 172.21.15.73:0/3150124535 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba39e19120 msgr2=0x55ba39e5f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:54.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.848+0000 7faba4dee640 1 --2- 172.21.15.73:0/3150124535 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba39e19120 0x55ba39e5f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:54.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.848+0000 7faba4dee640 1 -- 172.21.15.73:0/3150124535 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba39d35750 con 0x55ba39e43440 2024-04-26T20:51:54.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.848+0000 7faba4dee640 1 --2- 172.21.15.73:0/3150124535 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ba39e43440 0x55ba39e43820 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fab94005a90 tx=0x7fab94011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e72300f5b9a85ec server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:54.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.848+0000 7fab8a7fc640 1 -- 172.21.15.73:0/3150124535 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab94007710 con 0x55ba39e43440 2024-04-26T20:51:54.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.848+0000 7fab8a7fc640 1 -- 172.21.15.73:0/3150124535 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab9400e740 con 0x55ba39e43440 2024-04-26T20:51:54.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.848+0000 7fab8a7fc640 1 -- 172.21.15.73:0/3150124535 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9400eaf0 con 0x55ba39e43440 2024-04-26T20:51:54.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.849+0000 7faba9d6f740 1 -- 172.21.15.73:0/3150124535 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ba39e43440 msgr2=0x55ba39e43820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:54.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.849+0000 7faba9d6f740 1 --2- 172.21.15.73:0/3150124535 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ba39e43440 0x55ba39e43820 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fab94005a90 tx=0x7fab94011000 comp rx=0 tx=0).stop 2024-04-26T20:51:54.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.849+0000 7faba9d6f740 1 -- 172.21.15.73:0/3150124535 shutdown_connections 2024-04-26T20:51:54.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.849+0000 7faba9d6f740 1 --2- 172.21.15.73:0/3150124535 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba39e19120 0x55ba39e5f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:54.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.849+0000 7faba9d6f740 1 --2- 172.21.15.73:0/3150124535 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55ba39e60ce0 0x55ba39e61230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:54.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.849+0000 7faba9d6f740 1 --2- 172.21.15.73:0/3150124535 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ba39e43440 0x55ba39e43820 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:54.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.849+0000 7faba9d6f740 1 -- 172.21.15.73:0/3150124535 >> 172.21.15.73:0/3150124535 conn(0x55ba39e15a50 msgr2=0x55ba39e5e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:54.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.849+0000 7faba9d6f740 1 -- 172.21.15.73:0/3150124535 shutdown_connections 2024-04-26T20:51:54.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.849+0000 7faba9d6f740 1 -- 172.21.15.73:0/3150124535 wait complete. 2024-04-26T20:51:54.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.849+0000 7faba9d6f740 1 -- start start 2024-04-26T20:51:54.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.849+0000 7faba9d6f740 1 Processor -- start 2024-04-26T20:51:54.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.850+0000 7faba9d6f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ba39e43440 0x55ba39e34c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:54.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.850+0000 7faba9d6f740 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55ba39e60ce0 0x55ba39e46750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:54.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.850+0000 7faba9d6f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba39e172c0 0x55ba39e17740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:54.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.850+0000 7faba9d6f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ba39e19ec0 con 0x55ba39e43440 2024-04-26T20:51:54.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.850+0000 7faba9d6f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ba39d04fe0 con 0x55ba39e172c0 2024-04-26T20:51:54.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.850+0000 7faba9d6f740 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x55ba39e13980 con 0x55ba39e60ce0 2024-04-26T20:51:54.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.850+0000 7faba9d6f740 10 client.0 _ll_register_callbacks cb 0x55ba39d1b150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:51:54.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.850+0000 7faba4dee640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ba39e43440 0x55ba39e34c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:54.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.850+0000 7faba4dee640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ba39e43440 0x55ba39e34c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41106/0 (socket says 172.21.15.73:41106) 2024-04-26T20:51:54.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.850+0000 7faba4dee640 1 -- 172.21.15.73:0/286622658 learned_addr learned my addr 172.21.15.73:0/286622658 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:54.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.850+0000 7faba55ef640 1 --2- 172.21.15.73:0/286622658 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba39e172c0 0x55ba39e17740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:54.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.850+0000 7fab9ffff640 1 --2- 172.21.15.73:0/286622658 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55ba39e60ce0 0x55ba39e46750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:54.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.851+0000 7faba4dee640 1 -- 172.21.15.73:0/286622658 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55ba39e60ce0 msgr2=0x55ba39e46750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:54.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.851+0000 7faba4dee640 1 --2- 172.21.15.73:0/286622658 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55ba39e60ce0 0x55ba39e46750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:54.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.851+0000 7faba4dee640 1 -- 172.21.15.73:0/286622658 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba39e172c0 msgr2=0x55ba39e17740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:54.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.851+0000 7faba4dee640 1 --2- 172.21.15.73:0/286622658 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba39e172c0 0x55ba39e17740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:54.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.851+0000 7faba4dee640 1 -- 172.21.15.73:0/286622658 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba39a8f2d0 con 0x55ba39e43440 2024-04-26T20:51:54.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.851+0000 7faba55ef640 1 --2- 172.21.15.73:0/286622658 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba39e172c0 0x55ba39e17740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:54.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.851+0000 7faba4dee640 1 --2- 172.21.15.73:0/286622658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ba39e43440 0x55ba39e34c10 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fab94010bc0 tx=0x7fab94000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beb2111f3e3ee267 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:54.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.851+0000 7fab7f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-04-26T20:51:54.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.851+0000 7fab7f7fe640 1 -- 172.21.15.73:0/286622658 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9400eaf0 con 0x55ba39e43440 2024-04-26T20:51:54.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.851+0000 7fab7f7fe640 1 -- 172.21.15.73:0/286622658 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fab9400ec90 con 0x55ba39e43440 2024-04-26T20:51:54.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.851+0000 7faba9d6f740 1 -- 172.21.15.73:0/286622658 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab9400e740 con 0x55ba39e43440 2024-04-26T20:51:54.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.851+0000 7fab7f7fe640 1 -- 172.21.15.73:0/286622658 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) ==== 24+0+0 (secure 0 0 0) 0x7fab94031050 con 0x55ba39e43440 2024-04-26T20:51:54.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.851+0000 7faba9d6f740 10 client.5550 fetch_fsmap learned FSMap version 15 2024-04-26T20:51:54.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.851+0000 7faba9d6f740 1 -- 172.21.15.73:0/286622658 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) -- 0x55ba39b596b0 con 0x55ba39e43440 2024-04-26T20:51:54.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.851+0000 7fab7f7fe640 1 -- 172.21.15.73:0/286622658 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab9400e740 con 0x55ba39e43440 2024-04-26T20:51:54.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.851+0000 7fab7f7fe640 1 -- 172.21.15.73:0/286622658 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 15) ==== 3494+0+0 (secure 0 0 0) 0x7fab94034020 con 0x55ba39e43440 2024-04-26T20:51:54.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.851+0000 7faba9d6f740 10 client.5550 fetch_fsmap finished waiting for FSMap version 15 2024-04-26T20:51:54.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.851+0000 7faba9d6f740 10 client.5550 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-26T20:51:54.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.851+0000 7faba9d6f740 20 client.5550 populate_metadata read hostname 'smithi073' 2024-04-26T20:51:54.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.851+0000 7faba9d6f740 1 --2- 172.21.15.73:0/286622658 >> [v2:172.21.15.120:6833/761435580,v1:172.21.15.120:6835/761435580] conn(0x55ba39e6eae0 0x55ba39e6eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:54.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.851+0000 7faba9d6f740 4 client.5550 mds_command: new command op to 5053 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:51:54.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.851+0000 7faba9d6f740 1 -- 172.21.15.73:0/286622658 --> [v2:172.21.15.120:6833/761435580,v1:172.21.15.120:6835/761435580] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba39d8b350 con 0x55ba39e6eae0 2024-04-26T20:51:54.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.852+0000 7fab9ffff640 1 --2- 172.21.15.73:0/286622658 >> [v2:172.21.15.120:6833/761435580,v1:172.21.15.120:6835/761435580] conn(0x55ba39e6eae0 0x55ba39e6eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:54.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.852+0000 7fab7f7fe640 10 client.5550 ms_handle_connect on v2:172.21.15.120:6833/761435580 2024-04-26T20:51:54.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.852+0000 7fab9ffff640 1 --2- 172.21.15.73:0/286622658 >> [v2:172.21.15.120:6833/761435580,v1:172.21.15.120:6835/761435580] conn(0x55ba39e6eae0 0x55ba39e6eec0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8932e8bfb5d39e21 server_cookie=e5b66c77403ffd29 in_seq=0 out_seq=0 2024-04-26T20:51:54.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.853+0000 7fab7f7fe640 1 -- 172.21.15.73:0/286622658 <== mds.0 v2:172.21.15.120:6833/761435580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ba39d8b350 con 0x55ba39e6eae0 2024-04-26T20:51:54.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.853+0000 7fab7f7fe640 10 client.5550 handle_command_reply: tid=0 2024-04-26T20:51:54.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.853+0000 7faba9d6f740 1 client.5550 shutdown 2024-04-26T20:51:54.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.853+0000 7faba9d6f740 1 -- 172.21.15.73:0/286622658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ba39e43440 msgr2=0x55ba39e34c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:54.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.853+0000 7faba9d6f740 1 --2- 172.21.15.73:0/286622658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ba39e43440 0x55ba39e34c10 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fab94010bc0 tx=0x7fab94000bc0 comp rx=0 tx=0).stop 2024-04-26T20:51:54.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.853+0000 7faba9d6f740 1 -- 172.21.15.73:0/286622658 shutdown_connections 2024-04-26T20:51:54.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.853+0000 7faba9d6f740 1 --2- 172.21.15.73:0/286622658 >> [v2:172.21.15.120:6833/761435580,v1:172.21.15.120:6835/761435580] conn(0x55ba39e6eae0 0x55ba39e6eec0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:54.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.853+0000 7faba9d6f740 1 --2- 172.21.15.73:0/286622658 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba39e172c0 0x55ba39e17740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:54.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.853+0000 7fab7f7fe640 0 client.5550 ms_handle_reset on v2:172.21.15.120:6833/761435580 2024-04-26T20:51:54.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.853+0000 7faba9d6f740 1 --2- 172.21.15.73:0/286622658 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55ba39e60ce0 0x55ba39e46750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:54.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.853+0000 7faba9d6f740 1 --2- 172.21.15.73:0/286622658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ba39e43440 0x55ba39e34c10 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:54.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.853+0000 7faba9d6f740 1 -- 172.21.15.73:0/286622658 >> 172.21.15.73:0/286622658 conn(0x55ba39e19120 msgr2=0x55ba39e617d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:54.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.853+0000 7faba9d6f740 1 -- 172.21.15.73:0/286622658 shutdown_connections 2024-04-26T20:51:54.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.853+0000 7faba9d6f740 1 -- 172.21.15.73:0/286622658 wait complete. 2024-04-26T20:51:54.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.853+0000 7faba9d6f740 20 client.5550 trim_cache size 0 max 16384 2024-04-26T20:51:54.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.871+0000 7faba9d6f740 1 Processor -- start 2024-04-26T20:51:54.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.871+0000 7faba9d6f740 1 -- start start 2024-04-26T20:51:54.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.872+0000 7faba9d6f740 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55ba39e172c0 0x55ba39e824d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:54.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.872+0000 7faba9d6f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ba39e43440 0x55ba39e0ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:54.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.872+0000 7faba9d6f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba39e60ce0 0x55ba39e19790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:54.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.872+0000 7faba9d6f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7faba0200ee0 con 0x55ba39e43440 2024-04-26T20:51:54.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.872+0000 7faba9d6f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7faba009d4d0 con 0x55ba39e60ce0 2024-04-26T20:51:54.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.872+0000 7faba9d6f740 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7faba0200890 con 0x55ba39e172c0 2024-04-26T20:51:54.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.872+0000 7fab9ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ba39e43440 0x55ba39e0ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:54.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.872+0000 7fab9ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ba39e43440 0x55ba39e0ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41114/0 (socket says 172.21.15.73:41114) 2024-04-26T20:51:54.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.872+0000 7fab9ffff640 1 -- 172.21.15.73:0/3247869274 learned_addr learned my addr 172.21.15.73:0/3247869274 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:54.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.872+0000 7faba55ef640 1 --2- 172.21.15.73:0/3247869274 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba39e60ce0 0x55ba39e19790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:54.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.872+0000 7fab9ffff640 1 -- 172.21.15.73:0/3247869274 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55ba39e172c0 msgr2=0x55ba39e824d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:54.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.872+0000 7faba4dee640 1 --2- 172.21.15.73:0/3247869274 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55ba39e172c0 0x55ba39e824d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:54.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.872+0000 7fab9ffff640 1 --2- 172.21.15.73:0/3247869274 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55ba39e172c0 0x55ba39e824d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:54.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.872+0000 7fab9ffff640 1 -- 172.21.15.73:0/3247869274 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba39e60ce0 msgr2=0x55ba39e19790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:54.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.872+0000 7fab9ffff640 1 --2- 172.21.15.73:0/3247869274 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba39e60ce0 0x55ba39e19790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:54.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.872+0000 7fab9ffff640 1 -- 172.21.15.73:0/3247869274 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba39e1ade0 con 0x55ba39e43440 2024-04-26T20:51:54.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.872+0000 7fab9ffff640 1 --2- 172.21.15.73:0/3247869274 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ba39e43440 0x55ba39e0ef90 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fab8c0098c0 tx=0x7fab8c02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f098621f86464488 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:54.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.872+0000 7fab89ffb640 1 -- 172.21.15.73:0/3247869274 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8c048070 con 0x55ba39e43440 2024-04-26T20:51:54.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.872+0000 7fab89ffb640 1 -- 172.21.15.73:0/3247869274 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab8c004380 con 0x55ba39e43440 2024-04-26T20:51:54.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.872+0000 7fab89ffb640 1 -- 172.21.15.73:0/3247869274 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8c043e60 con 0x55ba39e43440 2024-04-26T20:51:54.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.873+0000 7faba9d6f740 1 -- 172.21.15.73:0/3247869274 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ba39e43440 msgr2=0x55ba39e0ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:54.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.873+0000 7faba9d6f740 1 --2- 172.21.15.73:0/3247869274 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ba39e43440 0x55ba39e0ef90 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fab8c0098c0 tx=0x7fab8c02c000 comp rx=0 tx=0).stop 2024-04-26T20:51:54.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.873+0000 7faba9d6f740 1 -- 172.21.15.73:0/3247869274 shutdown_connections 2024-04-26T20:51:54.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.873+0000 7faba9d6f740 1 --2- 172.21.15.73:0/3247869274 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ba39e43440 0x55ba39e0ef90 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:54.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.873+0000 7faba9d6f740 1 --2- 172.21.15.73:0/3247869274 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba39e60ce0 0x55ba39e19790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:54.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.873+0000 7faba9d6f740 1 --2- 172.21.15.73:0/3247869274 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55ba39e172c0 0x55ba39e824d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:54.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.873+0000 7faba9d6f740 1 -- 172.21.15.73:0/3247869274 >> 172.21.15.73:0/3247869274 conn(0x55ba39e19120 msgr2=0x55ba39e6eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:54.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.873+0000 7faba9d6f740 1 -- 172.21.15.73:0/3247869274 shutdown_connections 2024-04-26T20:51:54.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.873+0000 7faba9d6f740 1 -- 172.21.15.73:0/3247869274 wait complete. 2024-04-26T20:51:54.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.873+0000 7faba9d6f740 1 -- start start 2024-04-26T20:51:54.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.874+0000 7faba9d6f740 1 Processor -- start 2024-04-26T20:51:54.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.874+0000 7faba9d6f740 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55ba39e172c0 0x55ba39e53be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:54.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.874+0000 7faba9d6f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ba39e43440 0x55ba39e54120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:54.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.874+0000 7faba9d6f740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba39e60ce0 0x55ba39e44470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:54.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.874+0000 7faba9d6f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ba39e6fa20 con 0x55ba39e43440 2024-04-26T20:51:54.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.874+0000 7faba9d6f740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55ba39e55db0 con 0x55ba39e60ce0 2024-04-26T20:51:54.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.874+0000 7faba9d6f740 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x55ba39e42060 con 0x55ba39e172c0 2024-04-26T20:51:54.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.874+0000 7faba9d6f740 10 client.0 _ll_register_callbacks cb 0x7fab94005990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:51:54.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.874+0000 7fab9ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ba39e43440 0x55ba39e54120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:54.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.874+0000 7fab9ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ba39e43440 0x55ba39e54120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41116/0 (socket says 172.21.15.73:41116) 2024-04-26T20:51:54.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.874+0000 7fab9ffff640 1 -- 172.21.15.73:0/2981231856 learned_addr learned my addr 172.21.15.73:0/2981231856 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:54.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.874+0000 7faba55ef640 1 --2- 172.21.15.73:0/2981231856 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba39e60ce0 0x55ba39e44470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:54.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.875+0000 7faba4dee640 1 --2- 172.21.15.73:0/2981231856 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55ba39e172c0 0x55ba39e53be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:54.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.875+0000 7fab9ffff640 1 -- 172.21.15.73:0/2981231856 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55ba39e172c0 msgr2=0x55ba39e53be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:54.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.875+0000 7fab9ffff640 1 --2- 172.21.15.73:0/2981231856 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55ba39e172c0 0x55ba39e53be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:54.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.875+0000 7fab9ffff640 1 -- 172.21.15.73:0/2981231856 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba39e60ce0 msgr2=0x55ba39e44470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:54.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.875+0000 7fab9ffff640 1 --2- 172.21.15.73:0/2981231856 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba39e60ce0 0x55ba39e44470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:54.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.875+0000 7fab9ffff640 1 -- 172.21.15.73:0/2981231856 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba39e56ed0 con 0x55ba39e43440 2024-04-26T20:51:54.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.875+0000 7faba4dee640 1 --2- 172.21.15.73:0/2981231856 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55ba39e172c0 0x55ba39e53be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:54.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.875+0000 7faba55ef640 1 --2- 172.21.15.73:0/2981231856 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba39e60ce0 0x55ba39e44470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:54.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.875+0000 7fab9ffff640 1 --2- 172.21.15.73:0/2981231856 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ba39e43440 0x55ba39e54120 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fab8c006f80 tx=0x7fab8c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af12930a9c9fa14e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:54.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.875+0000 7fab7f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-04-26T20:51:54.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.875+0000 7fab7f7fe640 1 -- 172.21.15.73:0/2981231856 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8c043e60 con 0x55ba39e43440 2024-04-26T20:51:54.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.875+0000 7fab7f7fe640 1 -- 172.21.15.73:0/2981231856 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fab8c004180 con 0x55ba39e43440 2024-04-26T20:51:54.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.875+0000 7faba9d6f740 1 -- 172.21.15.73:0/2981231856 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab8c004380 con 0x55ba39e43440 2024-04-26T20:51:54.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.875+0000 7fab7f7fe640 1 -- 172.21.15.73:0/2981231856 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) ==== 24+0+0 (secure 0 0 0) 0x7fab8c048030 con 0x55ba39e43440 2024-04-26T20:51:54.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.875+0000 7faba9d6f740 10 client.5556 fetch_fsmap learned FSMap version 15 2024-04-26T20:51:54.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.875+0000 7faba9d6f740 1 -- 172.21.15.73:0/2981231856 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) -- 0x55ba39e818c0 con 0x55ba39e43440 2024-04-26T20:51:54.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.875+0000 7fab7f7fe640 1 -- 172.21.15.73:0/2981231856 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab8c004380 con 0x55ba39e43440 2024-04-26T20:51:54.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.875+0000 7fab7f7fe640 1 -- 172.21.15.73:0/2981231856 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 15) ==== 3494+0+0 (secure 0 0 0) 0x7fab8c005990 con 0x55ba39e43440 2024-04-26T20:51:54.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.875+0000 7faba9d6f740 10 client.5556 fetch_fsmap finished waiting for FSMap version 15 2024-04-26T20:51:54.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.875+0000 7faba9d6f740 10 client.5556 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-26T20:51:54.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.875+0000 7faba9d6f740 20 client.5556 populate_metadata read hostname 'smithi073' 2024-04-26T20:51:54.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.875+0000 7faba9d6f740 1 --2- 172.21.15.73:0/2981231856 >> [v2:172.21.15.120:6833/761435580,v1:172.21.15.120:6835/761435580] conn(0x55ba39e44a70 0x55ba39e61460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:54.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.875+0000 7faba9d6f740 4 client.5556 mds_command: new command op to 5053 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T20:51:54.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.875+0000 7faba9d6f740 1 -- 172.21.15.73:0/2981231856 --> [v2:172.21.15.120:6833/761435580,v1:172.21.15.120:6835/761435580] -- command(tid 0: {"prefix": "status"}) -- 0x55ba39d851c0 con 0x55ba39e44a70 2024-04-26T20:51:54.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.876+0000 7faba4dee640 1 --2- 172.21.15.73:0/2981231856 >> [v2:172.21.15.120:6833/761435580,v1:172.21.15.120:6835/761435580] conn(0x55ba39e44a70 0x55ba39e61460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:54.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.876+0000 7faba4dee640 1 --2- 172.21.15.73:0/2981231856 >> [v2:172.21.15.120:6833/761435580,v1:172.21.15.120:6835/761435580] conn(0x55ba39e44a70 0x55ba39e61460 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28ec120921787ee0 server_cookie=f3178de1526d332c in_seq=0 out_seq=0 2024-04-26T20:51:54.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.876+0000 7fab7f7fe640 10 client.5556 ms_handle_connect on v2:172.21.15.120:6833/761435580 2024-04-26T20:51:54.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.876+0000 7fab7f7fe640 1 -- 172.21.15.73:0/2981231856 <== mds.0 v2:172.21.15.120:6833/761435580 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55ba39d851c0 con 0x55ba39e44a70 2024-04-26T20:51:54.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.876+0000 7fab7f7fe640 10 client.5556 handle_command_reply: tid=0 2024-04-26T20:51:54.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.876+0000 7faba9d6f740 1 client.5556 shutdown 2024-04-26T20:51:54.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.876+0000 7faba9d6f740 1 -- 172.21.15.73:0/2981231856 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ba39e43440 msgr2=0x55ba39e54120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:54.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.876+0000 7faba9d6f740 1 --2- 172.21.15.73:0/2981231856 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ba39e43440 0x55ba39e54120 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fab8c006f80 tx=0x7fab8c004730 comp rx=0 tx=0).stop 2024-04-26T20:51:54.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.876+0000 7faba9d6f740 1 -- 172.21.15.73:0/2981231856 shutdown_connections 2024-04-26T20:51:54.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.876+0000 7faba9d6f740 1 --2- 172.21.15.73:0/2981231856 >> [v2:172.21.15.120:6833/761435580,v1:172.21.15.120:6835/761435580] conn(0x55ba39e44a70 0x55ba39e61460 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:54.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.876+0000 7faba9d6f740 1 --2- 172.21.15.73:0/2981231856 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ba39e43440 0x55ba39e54120 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:54.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.876+0000 7faba9d6f740 1 --2- 172.21.15.73:0/2981231856 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55ba39e60ce0 0x55ba39e44470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:54.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.876+0000 7fab7f7fe640 0 client.5556 ms_handle_reset on v2:172.21.15.120:6833/761435580 2024-04-26T20:51:54.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.876+0000 7faba9d6f740 1 --2- 172.21.15.73:0/2981231856 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55ba39e172c0 0x55ba39e53be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:54.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.876+0000 7faba9d6f740 1 -- 172.21.15.73:0/2981231856 >> 172.21.15.73:0/2981231856 conn(0x55ba39e19120 msgr2=0x55ba39e5c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:54.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.876+0000 7faba9d6f740 1 -- 172.21.15.73:0/2981231856 shutdown_connections 2024-04-26T20:51:54.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.877+0000 7faba9d6f740 1 -- 172.21.15.73:0/2981231856 wait complete. 2024-04-26T20:51:54.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.877+0000 7faba9d6f740 20 client.5556 trim_cache size 0 max 16384 2024-04-26T20:51:54.885 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-26T20:51:54.885 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_fsid": "fbd3f548-b547-4713-856b-09f98f5d1764", 2024-04-26T20:51:54.885 INFO:teuthology.orchestra.run.smithi073.stdout: "whoami": 0, 2024-04-26T20:51:54.885 INFO:teuthology.orchestra.run.smithi073.stdout: "id": 5053, 2024-04-26T20:51:54.885 INFO:teuthology.orchestra.run.smithi073.stdout: "want_state": "up:active", 2024-04-26T20:51:54.885 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "up:active", 2024-04-26T20:51:54.885 INFO:teuthology.orchestra.run.smithi073.stdout: "fs_name": "cephfs", 2024-04-26T20:51:54.885 INFO:teuthology.orchestra.run.smithi073.stdout: "rank_uptime": 8.9446340000000006, 2024-04-26T20:51:54.885 INFO:teuthology.orchestra.run.smithi073.stdout: "mdsmap_epoch": 15, 2024-04-26T20:51:54.885 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch": 39, 2024-04-26T20:51:54.885 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch_barrier": 39, 2024-04-26T20:51:54.885 INFO:teuthology.orchestra.run.smithi073.stdout: "uptime": 16.428038625999999 2024-04-26T20:51:54.885 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-26T20:51:54.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.877+0000 7faba7052640 1 -- 172.21.15.73:0/708744203 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fab8406c8f0 msgr2=0x7fab8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:54.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.877+0000 7faba7052640 1 --2- 172.21.15.73:0/708744203 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fab8406c8f0 0x7fab8406edb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fab94000f60 tx=0x7fab94009040 comp rx=0 tx=0).stop 2024-04-26T20:51:54.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.877+0000 7faba7052640 1 -- 172.21.15.73:0/708744203 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faba01633c0 msgr2=0x7faba01613c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:54.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.877+0000 7faba7052640 1 --2- 172.21.15.73:0/708744203 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faba01633c0 0x7faba01613c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fab9800efc0 tx=0x7fab9800c490 comp rx=0 tx=0).stop 2024-04-26T20:51:54.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.877+0000 7faba7052640 1 -- 172.21.15.73:0/708744203 shutdown_connections 2024-04-26T20:51:54.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.877+0000 7faba7052640 1 --2- 172.21.15.73:0/708744203 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faba01633c0 0x7faba01613c0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:54.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.877+0000 7faba7052640 1 --2- 172.21.15.73:0/708744203 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fab8406c8f0 0x7fab8406edb0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:54.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.877+0000 7faba7052640 1 --2- 172.21.15.73:0/708744203 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7faba015f290 0x7faba0160600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:54.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.877+0000 7faba7052640 1 --2- 172.21.15.73:0/708744203 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7faba00cb7e0 0x7faba01523f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:54.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.877+0000 7faba7052640 1 -- 172.21.15.73:0/708744203 >> 172.21.15.73:0/708744203 conn(0x7faba00d0fd0 msgr2=0x7faba00b1490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:54.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.877+0000 7faba7052640 1 -- 172.21.15.73:0/708744203 shutdown_connections 2024-04-26T20:51:54.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.877+0000 7faba7052640 1 -- 172.21.15.73:0/708744203 wait complete. 2024-04-26T20:51:54.889 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T20:51:54.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.984+0000 7f6111cc4640 1 Processor -- start 2024-04-26T20:51:54.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.985+0000 7f6111cc4640 1 -- start start 2024-04-26T20:51:54.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.985+0000 7f6111cc4640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f610c0b3860 0x7f610c16a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:54.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.985+0000 7f6111cc4640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f610c0d0ba0 con 0x7f610c15f3a0 2024-04-26T20:51:54.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.985+0000 7f6111cc4640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f610c0d2590 con 0x7f610c0d06f0 2024-04-26T20:51:54.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.985+0000 7f6111cc4640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f610c0d0e20 con 0x7f610c0b3860 2024-04-26T20:51:54.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.985+0000 7f610bfff640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f610c0b3860 0x7f610c16a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:54.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.985+0000 7f610b7fe640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f610c15f3a0 0x7f610c0d1770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40562/0 (socket says 172.21.15.73:40562) 2024-04-26T20:51:54.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.985+0000 7f610b7fe640 1 -- 172.21.15.73:0/1648902169 learned_addr learned my addr 172.21.15.73:0/1648902169 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:54.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.986+0000 7f610bfff640 1 -- 172.21.15.73:0/1648902169 >> v1:172.21.15.120:6789/0 conn(0x7f610c0d06f0 legacy=0x7f610c0d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:54.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.986+0000 7f610bfff640 1 -- 172.21.15.73:0/1648902169 >> v1:172.21.15.73:6789/0 conn(0x7f610c15f3a0 legacy=0x7f610c0d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:54.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.986+0000 7f610affd640 1 -- 172.21.15.73:0/1648902169 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 76975183 0 0) 0x7f60fc001110 con 0x7f610c15f3a0 2024-04-26T20:51:54.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.986+0000 7f610bfff640 1 -- 172.21.15.73:0/1648902169 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f610c0b1d70 con 0x7f610c0b3860 2024-04-26T20:51:54.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.986+0000 7f610bfff640 1 --2- 172.21.15.73:0/1648902169 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f610c0b3860 0x7f610c16a000 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f6104009d90 tx=0x7f610402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b7b67d297cd59d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:54.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.987+0000 7f610affd640 1 -- 172.21.15.73:0/1648902169 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6104031600 con 0x7f610c0b3860 2024-04-26T20:51:54.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.987+0000 7f610affd640 1 -- 172.21.15.73:0/1648902169 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61040317a0 con 0x7f610c0b3860 2024-04-26T20:51:54.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.987+0000 7f610affd640 1 -- 172.21.15.73:0/1648902169 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6104031b70 con 0x7f610c0b3860 2024-04-26T20:51:54.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.987+0000 7f6111cc4640 1 -- 172.21.15.73:0/1648902169 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f610c0b3860 msgr2=0x7f610c16a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:54.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.987+0000 7f6111cc4640 1 --2- 172.21.15.73:0/1648902169 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f610c0b3860 0x7f610c16a000 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f6104009d90 tx=0x7f610402a610 comp rx=0 tx=0).stop 2024-04-26T20:51:54.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.987+0000 7f6111cc4640 1 -- 172.21.15.73:0/1648902169 shutdown_connections 2024-04-26T20:51:54.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.987+0000 7f6111cc4640 1 --2- 172.21.15.73:0/1648902169 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f610c0b3860 0x7f610c16a000 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:54.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.987+0000 7f6111cc4640 1 -- 172.21.15.73:0/1648902169 >> 172.21.15.73:0/1648902169 conn(0x7f610c0d4f90 msgr2=0x7f610c0d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:54.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.987+0000 7f6111cc4640 1 -- 172.21.15.73:0/1648902169 shutdown_connections 2024-04-26T20:51:54.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.987+0000 7f6111cc4640 1 -- 172.21.15.73:0/1648902169 wait complete. 2024-04-26T20:51:54.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.988+0000 7f6111cc4640 1 Processor -- start 2024-04-26T20:51:54.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.988+0000 7f6111cc4640 1 -- start start 2024-04-26T20:51:54.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.989+0000 7f6111cc4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f610c0b3860 0x7f610c151340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:54.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.989+0000 7f6111cc4640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f610c0d06f0 0x7f610c0ce370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:54.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.989+0000 7f6111cc4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f610c15f3a0 0x7f610c0ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:54.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.989+0000 7f6111cc4640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f610c15f780 con 0x7f610c0b3860 2024-04-26T20:51:54.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.989+0000 7f6111cc4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f610c0b2330 con 0x7f610c15f3a0 2024-04-26T20:51:54.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.989+0000 7f6111cc4640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f610c15f900 con 0x7f610c0d06f0 2024-04-26T20:51:54.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.989+0000 7f610b7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f610c0b3860 0x7f610c151340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:54.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.989+0000 7f610b7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f610c0b3860 0x7f610c151340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41136/0 (socket says 172.21.15.73:41136) 2024-04-26T20:51:54.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.989+0000 7f610b7fe640 1 -- 172.21.15.73:0/1830654535 learned_addr learned my addr 172.21.15.73:0/1830654535 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:54.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.989+0000 7f610bfff640 1 --2- 172.21.15.73:0/1830654535 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f610c15f3a0 0x7f610c0ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:54.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.989+0000 7f6103fff640 1 --2- 172.21.15.73:0/1830654535 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f610c0d06f0 0x7f610c0ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:54.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.989+0000 7f610b7fe640 1 -- 172.21.15.73:0/1830654535 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f610c0d06f0 msgr2=0x7f610c0ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:54.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.989+0000 7f610b7fe640 1 --2- 172.21.15.73:0/1830654535 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f610c0d06f0 0x7f610c0ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:54.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.989+0000 7f610b7fe640 1 -- 172.21.15.73:0/1830654535 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f610c15f3a0 msgr2=0x7f610c0ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:54.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.989+0000 7f610b7fe640 1 --2- 172.21.15.73:0/1830654535 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f610c15f3a0 0x7f610c0ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:54.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.990+0000 7f610b7fe640 1 -- 172.21.15.73:0/1830654535 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f610c0cf0f0 con 0x7f610c0b3860 2024-04-26T20:51:54.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.990+0000 7f610b7fe640 1 --2- 172.21.15.73:0/1830654535 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f610c0b3860 0x7f610c151340 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f60fc00c430 tx=0x7f60fc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:54.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.990+0000 7f6108ff9640 1 -- 172.21.15.73:0/1830654535 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60fc0074e0 con 0x7f610c0b3860 2024-04-26T20:51:54.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.990+0000 7f6108ff9640 1 -- 172.21.15.73:0/1830654535 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60fc0047e0 con 0x7f610c0b3860 2024-04-26T20:51:54.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.990+0000 7f6111cc4640 1 -- 172.21.15.73:0/1830654535 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f610c0cf3d0 con 0x7f610c0b3860 2024-04-26T20:51:54.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.990+0000 7f6108ff9640 1 -- 172.21.15.73:0/1830654535 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60fc004bd0 con 0x7f610c0b3860 2024-04-26T20:51:54.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.990+0000 7f6111cc4640 1 -- 172.21.15.73:0/1830654535 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f610c1e8ec0 con 0x7f610c0b3860 2024-04-26T20:51:54.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.991+0000 7f6111cc4640 1 -- 172.21.15.73:0/1830654535 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f610c0694f0 con 0x7f610c0b3860 2024-04-26T20:51:54.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.993+0000 7f6108ff9640 1 -- 172.21.15.73:0/1830654535 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f60fc011070 con 0x7f610c0b3860 2024-04-26T20:51:54.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.993+0000 7f6108ff9640 1 --2- 172.21.15.73:0/1830654535 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f60ec06c8f0 0x7f60ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:54.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.993+0000 7f6103fff640 1 --2- 172.21.15.73:0/1830654535 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f60ec06c8f0 0x7f60ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:54.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.993+0000 7f6108ff9640 1 -- 172.21.15.73:0/1830654535 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f60fc08bef0 con 0x7f610c0b3860 2024-04-26T20:51:54.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.994+0000 7f6103fff640 1 --2- 172.21.15.73:0/1830654535 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f60ec06c8f0 0x7f60ec06edb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f610c15e350 tx=0x7f60f4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:54.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:54.996+0000 7f6108ff9640 1 -- 172.21.15.73:0/1830654535 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f610c0d0ba0 con 0x7f610c0b3860 2024-04-26T20:51:55.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:55.212+0000 7f6111cc4640 1 -- 172.21.15.73:0/1830654535 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f610c0b6860 con 0x7f610c0b3860 2024-04-26T20:51:55.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:55.213+0000 7f6108ff9640 1 -- 172.21.15.73:0/1830654535 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) ==== 53+0+73 (secure 0 0 0) 0x7f60fc00e280 con 0x7f610c0b3860 2024-04-26T20:51:55.214 INFO:teuthology.orchestra.run.smithi073.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T20:51:55.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:55.215+0000 7f6111cc4640 1 -- 172.21.15.73:0/1830654535 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f60ec06c8f0 msgr2=0x7f60ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:55.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:55.215+0000 7f6111cc4640 1 --2- 172.21.15.73:0/1830654535 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f60ec06c8f0 0x7f60ec06edb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f610c15e350 tx=0x7f60f4009040 comp rx=0 tx=0).stop 2024-04-26T20:51:55.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:55.215+0000 7f6111cc4640 1 -- 172.21.15.73:0/1830654535 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f610c0b3860 msgr2=0x7f610c151340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:55.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:55.215+0000 7f6111cc4640 1 --2- 172.21.15.73:0/1830654535 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f610c0b3860 0x7f610c151340 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f60fc00c430 tx=0x7f60fc00c8f0 comp rx=0 tx=0).stop 2024-04-26T20:51:55.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:55.215+0000 7f6111cc4640 1 -- 172.21.15.73:0/1830654535 shutdown_connections 2024-04-26T20:51:55.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:55.215+0000 7f6111cc4640 1 --2- 172.21.15.73:0/1830654535 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f60ec06c8f0 0x7f60ec06edb0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:55.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:55.215+0000 7f6111cc4640 1 --2- 172.21.15.73:0/1830654535 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f610c15f3a0 0x7f610c0ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:55.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:55.215+0000 7f6111cc4640 1 --2- 172.21.15.73:0/1830654535 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f610c0d06f0 0x7f610c0ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:55.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:55.215+0000 7f6111cc4640 1 --2- 172.21.15.73:0/1830654535 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f610c0b3860 0x7f610c151340 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:55.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:55.215+0000 7f6111cc4640 1 -- 172.21.15.73:0/1830654535 >> 172.21.15.73:0/1830654535 conn(0x7f610c0d4f90 msgr2=0x7f610c0d14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:55.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:55.215+0000 7f6111cc4640 1 -- 172.21.15.73:0/1830654535 shutdown_connections 2024-04-26T20:51:55.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:55.215+0000 7f6111cc4640 1 -- 172.21.15.73:0/1830654535 wait complete. 2024-04-26T20:51:55.227 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T20:51:55.227 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T20:51:55.227 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi073.front.sepia.ceph.com 2024-04-26T20:51:55.227 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T20:51:55.227 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T20:51:55.227 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T20:51:55.227 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T20:51:55.227 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T20:51:55.227 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T20:51:55.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:55.228 DEBUG:teuthology.orchestra.run.smithi073:> ip addr 2024-04-26T20:51:55.243 INFO:teuthology.orchestra.run.smithi073.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:51:55.243 INFO:teuthology.orchestra.run.smithi073.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:51:55.243 INFO:teuthology.orchestra.run.smithi073.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:51:55.243 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:51:55.243 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 ::1/128 scope host 2024-04-26T20:51:55.243 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:51:55.244 INFO:teuthology.orchestra.run.smithi073.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:51:55.244 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:ea brd ff:ff:ff:ff:ff:ff 2024-04-26T20:51:55.244 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f0 2024-04-26T20:51:55.244 INFO:teuthology.orchestra.run.smithi073.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:51:55.244 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:eb brd ff:ff:ff:ff:ff:ff 2024-04-26T20:51:55.244 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f1 2024-04-26T20:51:55.244 INFO:teuthology.orchestra.run.smithi073.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:51:55.244 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7c brd ff:ff:ff:ff:ff:ff 2024-04-26T20:51:55.244 INFO:teuthology.orchestra.run.smithi073.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T20:51:55.244 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7d brd ff:ff:ff:ff:ff:ff 2024-04-26T20:51:55.244 INFO:teuthology.orchestra.run.smithi073.stdout: inet 172.21.15.73/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-26T20:51:55.244 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft 42672sec preferred_lft 42672sec 2024-04-26T20:51:55.244 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::b99b:8e91:ffc2:71f/64 scope link noprefixroute 2024-04-26T20:51:55.244 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:51:55.244 INFO:teuthology.orchestra.run.smithi073.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T20:51:55.245 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 3e:22:aa:11:2d:0e brd ff:ff:ff:ff:ff:ff 2024-04-26T20:51:55.245 INFO:teuthology.orchestra.run.smithi073.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T20:51:55.245 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:51:55.245 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::1cd0:bfff:fec2:3f98/64 scope link 2024-04-26T20:51:55.245 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:51:55.245 INFO:teuthology.orchestra.run.smithi073.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T20:51:55.245 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 66:7f:fa:7c:30:18 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:51:55.245 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::647f:faff:fe7c:3018/64 scope link 2024-04-26T20:51:55.245 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:51:55.245 INFO:teuthology.orchestra.run.smithi073.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T20:51:55.245 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 3e:22:aa:11:2d:0e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:51:55.245 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::3c22:aaff:fe11:2d0e/64 scope link 2024-04-26T20:51:55.245 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:51:55.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:55.246 DEBUG:teuthology.orchestra.run.smithi073:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T20:51:55.305 INFO:teuthology.orchestra.run.smithi073.stdout:1 2024-04-26T20:51:55.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:55.307 DEBUG:teuthology.orchestra.run.smithi073:> route 2024-04-26T20:51:55.363 INFO:teuthology.orchestra.run.smithi073.stdout:Kernel IP routing table 2024-04-26T20:51:55.364 INFO:teuthology.orchestra.run.smithi073.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T20:51:55.364 INFO:teuthology.orchestra.run.smithi073.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T20:51:55.364 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T20:51:55.364 INFO:teuthology.orchestra.run.smithi073.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T20:51:55.364 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:51:55.364 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2024-04-26T20:51:55.364 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T20:51:55.364 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T20:51:55.364 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T20:51:55.365 DEBUG:teuthology.orchestra.run.smithi073:> ') 2024-04-26T20:51:55.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:51:55.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:51:55.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:55.487 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list 2024-04-26T20:51:55.501 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T20:51:55.501 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T20:51:55.502 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T20:51:55.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:55.502 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link set brx.0 up 2024-04-26T20:51:55.566 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:51:55.566 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T20:51:55.620 INFO:teuthology.orchestra.run.smithi073.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T20:51:55.621 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:51:55.621 DEBUG:teuthology.orchestra.run.smithi073:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T20:51:55.675 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T20:51:55.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:55.675 DEBUG:teuthology.orchestra.run.smithi073:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T20:51:55.756 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T20:51:55.757 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T20:51:55.757 INFO:teuthology.orchestra.run.smithi073.stdout:invalid new device string format 2024-04-26T20:51:55.765 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T20:51:55.765 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T20:51:55.765 INFO:teuthology.orchestra.run.smithi073.stdout:invalid new device string format 2024-04-26T20:51:55.765 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: resolved to: "172.21.15.73:3300,172.21.15.120:3300,172.21.15.120:3301" 2024-04-26T20:51:55.765 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T20:51:55.770 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T20:51:55.771 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T20:51:55.771 INFO:teuthology.orchestra.run.smithi073.stdout:invalid new device string format 2024-04-26T20:51:55.771 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: resolved to: "172.21.15.73:3300,172.21.15.120:3300,172.21.15.120:3301" 2024-04-26T20:51:55.771 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T20:51:55.771 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: resolved to: "172.21.15.73:6789,172.21.15.120:6789,172.21.15.120:6790" 2024-04-26T20:51:55.771 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.73:6789,172.21.15.120:6789,172.21.15.120:6790:/ 2024-04-26T20:51:55.771 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=fbd3f548-b547-4713-856b-09f98f5d1764" will pass to kernel 2024-04-26T20:51:55.771 INFO:teuthology.orchestra.run.smithi073.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T20:51:55.771 INFO:teuthology.orchestra.run.smithi073.stdout: You just mounted a file system that supports labels which does not 2024-04-26T20:51:55.772 INFO:teuthology.orchestra.run.smithi073.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T20:51:55.772 INFO:teuthology.orchestra.run.smithi073.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T20:51:55.772 INFO:teuthology.orchestra.run.smithi073.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T20:51:55.773 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T20:51:55.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:55.773 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T20:51:55.838 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T20:51:55.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T20:51:55.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:51:55.939 INFO:teuthology.orchestra.run.smithi073.stdout:client5568 2024-04-26T20:51:55.940 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-26T20:51:55.940 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:51:55.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:55.955 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir testdir1) 2024-04-26T20:51:56.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:56.016 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch testdir1/testfile) 2024-04-26T20:51:56.073 DEBUG:teuthology.orchestra.run.smithi073:> stdin-killer -- python3 -c ' 2024-04-26T20:51:56.074 DEBUG:teuthology.orchestra.run.smithi073:> import os 2024-04-26T20:51:56.074 DEBUG:teuthology.orchestra.run.smithi073:> import stat 2024-04-26T20:51:56.074 DEBUG:teuthology.orchestra.run.smithi073:> 2024-04-26T20:51:56.074 DEBUG:teuthology.orchestra.run.smithi073:> print(os.stat("/home/ubuntu/cephtest/mnt.0/testdir1").st_ino) 2024-04-26T20:51:56.074 DEBUG:teuthology.orchestra.run.smithi073:> ' 2024-04-26T20:51:56.178 INFO:teuthology.orchestra.run.smithi073.stdout:1099511627776 2024-04-26T20:51:56.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:51:56.187 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-26T20:51:56.187 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:51:56.244 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-26T20:51:56.245 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:51:56.300 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:51:56.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.435+0000 7fd15527e640 1 Processor -- start 2024-04-26T20:51:56.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.435+0000 7fd15527e640 1 -- start start 2024-04-26T20:51:56.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.436+0000 7fd15527e640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fd15015f3a0 0x7fd1500d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:56.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.436+0000 7fd15527e640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd1500d2330 0x7fd1500b2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:56.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.436+0000 7fd15527e640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1500d0e20 con 0x7fd1500d1770 2024-04-26T20:51:56.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.436+0000 7fd15527e640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fd1500b1d70 con 0x7fd15015f3a0 2024-04-26T20:51:56.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.436+0000 7fd15527e640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fd1500b2050 con 0x7fd1500d2330 2024-04-26T20:51:56.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.436+0000 7fd14effd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fd15015f3a0 0x7fd1500d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:56.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.436+0000 7fd14f7fe640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd1500d2330 0x7fd1500b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:56.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.436+0000 7fd14effd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fd15015f3a0 0x7fd1500d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.73:36492/0 (socket says 172.21.15.73:36492) 2024-04-26T20:51:56.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.436+0000 7fd14effd640 1 -- 172.21.15.73:0/4229860930 learned_addr learned my addr 172.21.15.73:0/4229860930 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:56.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.436+0000 7fd14e7fc640 1 --1- 172.21.15.73:0/4229860930 >> v1:172.21.15.120:6789/0 conn(0x7fd1500d1770 0x7fd1500d1c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.73:51644/0 (socket says 172.21.15.73:51644) 2024-04-26T20:51:56.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.437+0000 7fd14f7fe640 1 -- 172.21.15.73:0/4229860930 >> v2:172.21.15.120:3300/0 conn(0x7fd15015f3a0 msgr2=0x7fd1500d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:56.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.437+0000 7fd14f7fe640 1 --2- 172.21.15.73:0/4229860930 >> v2:172.21.15.120:3300/0 conn(0x7fd15015f3a0 0x7fd1500d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:56.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.437+0000 7fd14f7fe640 1 -- 172.21.15.73:0/4229860930 >> v1:172.21.15.120:6789/0 conn(0x7fd1500d1770 legacy=0x7fd1500d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:56.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.437+0000 7fd14f7fe640 1 -- 172.21.15.73:0/4229860930 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1500b2330 con 0x7fd1500d2330 2024-04-26T20:51:56.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.437+0000 7fd14effd640 1 --2- 172.21.15.73:0/4229860930 >> v2:172.21.15.120:3300/0 conn(0x7fd15015f3a0 0x7fd1500d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:51:56.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.437+0000 7fd14f7fe640 1 --2- 172.21.15.73:0/4229860930 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd1500d2330 0x7fd1500b2850 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd1480097a0 tx=0x7fd14802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bf29ab2994c6a31 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:56.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.437+0000 7fd14dffb640 1 -- 172.21.15.73:0/4229860930 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd148037070 con 0x7fd1500d2330 2024-04-26T20:51:56.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.438+0000 7fd14dffb640 1 -- 172.21.15.73:0/4229860930 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd148032440 con 0x7fd1500d2330 2024-04-26T20:51:56.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.438+0000 7fd14dffb640 1 -- 172.21.15.73:0/4229860930 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd148032810 con 0x7fd1500d2330 2024-04-26T20:51:56.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.438+0000 7fd15527e640 1 -- 172.21.15.73:0/4229860930 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd1500d2330 msgr2=0x7fd1500b2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:56.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.438+0000 7fd15527e640 1 --2- 172.21.15.73:0/4229860930 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd1500d2330 0x7fd1500b2850 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd1480097a0 tx=0x7fd14802a180 comp rx=0 tx=0).stop 2024-04-26T20:51:56.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.438+0000 7fd15527e640 1 -- 172.21.15.73:0/4229860930 shutdown_connections 2024-04-26T20:51:56.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.438+0000 7fd15527e640 1 --2- 172.21.15.73:0/4229860930 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd1500d2330 0x7fd1500b2850 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:56.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.438+0000 7fd15527e640 1 --2- 172.21.15.73:0/4229860930 >> v2:172.21.15.120:3300/0 conn(0x7fd15015f3a0 0x7fd1500d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:56.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.438+0000 7fd15527e640 1 -- 172.21.15.73:0/4229860930 >> 172.21.15.73:0/4229860930 conn(0x7fd1500d4f90 msgr2=0x7fd1500d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:56.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.438+0000 7fd15527e640 1 -- 172.21.15.73:0/4229860930 shutdown_connections 2024-04-26T20:51:56.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.438+0000 7fd15527e640 1 -- 172.21.15.73:0/4229860930 wait complete. 2024-04-26T20:51:56.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.439+0000 7fd15527e640 1 Processor -- start 2024-04-26T20:51:56.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.439+0000 7fd15527e640 1 -- start start 2024-04-26T20:51:56.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.440+0000 7fd15527e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1500d1770 0x7fd1500baa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:56.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.440+0000 7fd15527e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd1500d2330 0x7fd1500a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:56.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.440+0000 7fd15527e640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd15015f3a0 0x7fd1500a1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:56.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.440+0000 7fd15527e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd1500b2f10 con 0x7fd1500d2330 2024-04-26T20:51:56.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.440+0000 7fd15527e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd1500b2d90 con 0x7fd1500d1770 2024-04-26T20:51:56.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.440+0000 7fd15527e640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fd1500b3090 con 0x7fd15015f3a0 2024-04-26T20:51:56.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.440+0000 7fd14e7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd1500d2330 0x7fd1500a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:56.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.440+0000 7fd14e7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd1500d2330 0x7fd1500a1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41152/0 (socket says 172.21.15.73:41152) 2024-04-26T20:51:56.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.440+0000 7fd14e7fc640 1 -- 172.21.15.73:0/1686996634 learned_addr learned my addr 172.21.15.73:0/1686996634 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:56.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.440+0000 7fd14effd640 1 --2- 172.21.15.73:0/1686996634 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1500d1770 0x7fd1500baa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:56.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.440+0000 7fd14f7fe640 1 --2- 172.21.15.73:0/1686996634 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd15015f3a0 0x7fd1500a1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:56.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.440+0000 7fd14e7fc640 1 -- 172.21.15.73:0/1686996634 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd15015f3a0 msgr2=0x7fd1500a1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:56.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.440+0000 7fd14e7fc640 1 --2- 172.21.15.73:0/1686996634 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd15015f3a0 0x7fd1500a1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:56.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.440+0000 7fd14e7fc640 1 -- 172.21.15.73:0/1686996634 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1500d1770 msgr2=0x7fd1500baa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:56.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.440+0000 7fd14e7fc640 1 --2- 172.21.15.73:0/1686996634 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1500d1770 0x7fd1500baa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:56.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.440+0000 7fd14e7fc640 1 -- 172.21.15.73:0/1686996634 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1500a22e0 con 0x7fd1500d2330 2024-04-26T20:51:56.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.441+0000 7fd14e7fc640 1 --2- 172.21.15.73:0/1686996634 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd1500d2330 0x7fd1500a1580 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fd14400c400 tx=0x7fd14400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:56.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.441+0000 7fd1377fe640 1 -- 172.21.15.73:0/1686996634 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1440074e0 con 0x7fd1500d2330 2024-04-26T20:51:56.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.441+0000 7fd1377fe640 1 -- 172.21.15.73:0/1686996634 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd144007680 con 0x7fd1500d2330 2024-04-26T20:51:56.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.441+0000 7fd15527e640 1 -- 172.21.15.73:0/1686996634 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1500a25c0 con 0x7fd1500d2330 2024-04-26T20:51:56.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.441+0000 7fd1377fe640 1 -- 172.21.15.73:0/1686996634 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd144004a00 con 0x7fd1500d2330 2024-04-26T20:51:56.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.441+0000 7fd15527e640 1 -- 172.21.15.73:0/1686996634 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1500a2000 con 0x7fd1500d2330 2024-04-26T20:51:56.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.442+0000 7fd15527e640 1 -- 172.21.15.73:0/1686996634 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1500c8540 con 0x7fd1500d2330 2024-04-26T20:51:56.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.442+0000 7fd1377fe640 1 -- 172.21.15.73:0/1686996634 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd14401a020 con 0x7fd1500d2330 2024-04-26T20:51:56.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.443+0000 7fd1377fe640 1 --2- 172.21.15.73:0/1686996634 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd13c06c8f0 0x7fd13c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:56.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.443+0000 7fd14effd640 1 --2- 172.21.15.73:0/1686996634 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd13c06c8f0 0x7fd13c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:56.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.443+0000 7fd1377fe640 1 -- 172.21.15.73:0/1686996634 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fd14408b320 con 0x7fd1500d2330 2024-04-26T20:51:56.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.443+0000 7fd14effd640 1 --2- 172.21.15.73:0/1686996634 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd13c06c8f0 0x7fd13c06edb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd1400072c0 tx=0x7fd14000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:56.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.447+0000 7fd1377fe640 1 -- 172.21.15.73:0/1686996634 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd144059000 con 0x7fd1500d2330 2024-04-26T20:51:56.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.623+0000 7fd15527e640 1 -- 172.21.15.73:0/1686996634 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd1500694f0 con 0x7fd1500d2330 2024-04-26T20:51:56.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.624+0000 7fd1377fe640 1 -- 172.21.15.73:0/1686996634 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+16942 (secure 0 0 0) 0x7fd14405d300 con 0x7fd1500d2330 2024-04-26T20:51:56.625 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:51:56.625 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":39,"fsid":"fbd3f548-b547-4713-856b-09f98f5d1764","created":"2024-04-26T20:50:07.876786+0000","modified":"2024-04-26T20:51:46.928774+0000","last_up_change":"2024-04-26T20:50:19.082741+0000","last_in_change":"2024-04-26T20:50:10.018400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:50:19.993731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:51:43.110774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-26T20:51:44.235846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d6f4f3-214a-4392-a02f-c33cbcab3c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6809","nonce":3461705124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6811","nonce":3461705124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6815","nonce":3461705124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6813","nonce":3461705124}]},"public_addr":"172.21.15.73:6809/3461705124","cluster_addr":"172.21.15.73:6811/3461705124","heartbeat_back_addr":"172.21.15.73:6815/3461705124","heartbeat_front_addr":"172.21.15.73:6813/3461705124","state":["exists","up"]},{"osd":1,"uuid":"786bf8ca-6ddc-4b3c-be6c-9152398bd5fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6817","nonce":2827619403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6819","nonce":2827619403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6825","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6827","nonce":2827619403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6821","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6823","nonce":2827619403}]},"public_addr":"172.21.15.73:6817/2827619403","cluster_addr":"172.21.15.73:6819/2827619403","heartbeat_back_addr":"172.21.15.73:6827/2827619403","heartbeat_front_addr":"172.21.15.73:6823/2827619403","state":["exists","up"]},{"osd":2,"uuid":"90963b1a-5437-4ef8-ad8b-6b74c534eeb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6801","nonce":2542681064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6803","nonce":2542681064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6808","nonce":2542681064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6805","nonce":2542681064}]},"public_addr":"172.21.15.73:6801/2542681064","cluster_addr":"172.21.15.73:6803/2542681064","heartbeat_back_addr":"172.21.15.73:6808/2542681064","heartbeat_front_addr":"172.21.15.73:6805/2542681064","state":["exists","up"]},{"osd":3,"uuid":"bd3d2148-6e38-41b4-a453-fe6e3b8b4ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6822","nonce":1604042718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6826","nonce":1604042718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6831","nonce":1604042718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6829","nonce":1604042718}]},"public_addr":"172.21.15.73:6822/1604042718","cluster_addr":"172.21.15.73:6826/1604042718","heartbeat_back_addr":"172.21.15.73:6831/1604042718","heartbeat_front_addr":"172.21.15.73:6829/1604042718","state":["exists","up"]},{"osd":4,"uuid":"e8668dc8-a23f-46d2-8311-cb882da08e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6817","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6819","nonce":3591055561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6824","nonce":3591055561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6831","nonce":3591055561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6828","nonce":3591055561}]},"public_addr":"172.21.15.120:6819/3591055561","cluster_addr":"172.21.15.120:6824/3591055561","heartbeat_back_addr":"172.21.15.120:6831/3591055561","heartbeat_front_addr":"172.21.15.120:6828/3591055561","state":["exists","up"]},{"osd":5,"uuid":"cbac0bdb-96bd-4852-bcce-e564495c3fcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6809","nonce":819450433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6811","nonce":819450433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6815","nonce":819450433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6813","nonce":819450433}]},"public_addr":"172.21.15.120:6809/819450433","cluster_addr":"172.21.15.120:6811/819450433","heartbeat_back_addr":"172.21.15.120:6815/819450433","heartbeat_front_addr":"172.21.15.120:6813/819450433","state":["exists","up"]},{"osd":6,"uuid":"1884b2a8-569f-4272-96a9-f25a09364dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6801","nonce":3946210727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6803","nonce":3946210727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6807","nonce":3946210727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6805","nonce":3946210727}]},"public_addr":"172.21.15.120:6801/3946210727","cluster_addr":"172.21.15.120:6803/3946210727","heartbeat_back_addr":"172.21.15.120:6807/3946210727","heartbeat_front_addr":"172.21.15.120:6805/3946210727","state":["exists","up"]},{"osd":7,"uuid":"c0c3bf6d-a4c5-4f1e-aa2d-330951830666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6818","nonce":2915574674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6821","nonce":2915574674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6827","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6829","nonce":2915574674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6823","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6825","nonce":2915574674}]},"public_addr":"172.21.15.120:6818/2915574674","cluster_addr":"172.21.15.120:6821/2915574674","heartbeat_back_addr":"172.21.15.120:6829/2915574674","heartbeat_front_addr":"172.21.15.120:6825/2915574674","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.218216+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.208710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.112705+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.182789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.164324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.174212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.225396+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.261559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:51:56.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.626+0000 7fd15527e640 1 -- 172.21.15.73:0/1686996634 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd13c06c8f0 msgr2=0x7fd13c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:56.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.626+0000 7fd15527e640 1 --2- 172.21.15.73:0/1686996634 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd13c06c8f0 0x7fd13c06edb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd1400072c0 tx=0x7fd14000b040 comp rx=0 tx=0).stop 2024-04-26T20:51:56.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.626+0000 7fd15527e640 1 -- 172.21.15.73:0/1686996634 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd1500d2330 msgr2=0x7fd1500a1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:56.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.626+0000 7fd15527e640 1 --2- 172.21.15.73:0/1686996634 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd1500d2330 0x7fd1500a1580 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fd14400c400 tx=0x7fd14400c8c0 comp rx=0 tx=0).stop 2024-04-26T20:51:56.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.626+0000 7fd15527e640 1 -- 172.21.15.73:0/1686996634 shutdown_connections 2024-04-26T20:51:56.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.626+0000 7fd15527e640 1 --2- 172.21.15.73:0/1686996634 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd1500d2330 0x7fd1500a1580 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:56.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.626+0000 7fd15527e640 1 --2- 172.21.15.73:0/1686996634 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd13c06c8f0 0x7fd13c06edb0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:56.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.626+0000 7fd15527e640 1 --2- 172.21.15.73:0/1686996634 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd15015f3a0 0x7fd1500a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:56.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.626+0000 7fd15527e640 1 --2- 172.21.15.73:0/1686996634 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd1500d1770 0x7fd1500baa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:56.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.626+0000 7fd15527e640 1 -- 172.21.15.73:0/1686996634 >> 172.21.15.73:0/1686996634 conn(0x7fd1500d4f90 msgr2=0x7fd1500ce070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:56.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.626+0000 7fd15527e640 1 -- 172.21.15.73:0/1686996634 shutdown_connections 2024-04-26T20:51:56.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.626+0000 7fd15527e640 1 -- 172.21.15.73:0/1686996634 wait complete. 2024-04-26T20:51:56.638 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:50:19.993731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:51:43.110774+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:51:44.235846+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:51:56.638 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:51:56.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.737+0000 7f1e1e4a2640 1 Processor -- start 2024-04-26T20:51:56.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.737+0000 7f1e1e4a2640 1 -- start start 2024-04-26T20:51:56.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.737+0000 7f1e1e4a2640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f1e18071260 0x7f1e1815e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:56.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.738+0000 7f1e1e4a2640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e180d2390 con 0x7f1e180cfde0 2024-04-26T20:51:56.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.738+0000 7f1e1e4a2640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e1806d3f0 con 0x7f1e18070350 2024-04-26T20:51:56.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.738+0000 7f1e1e4a2640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f1e180afab0 con 0x7f1e18071260 2024-04-26T20:51:56.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.738+0000 7f1e17fff640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f1e180cfde0 0x7f1e180d1b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40570/0 (socket says 172.21.15.73:40570) 2024-04-26T20:51:56.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.738+0000 7f1e17fff640 1 -- 172.21.15.73:0/433315426 learned_addr learned my addr 172.21.15.73:0/433315426 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:56.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.738+0000 7f1e16ffd640 1 -- 172.21.15.73:0/433315426 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 755821549 0 0) 0x7f1e180d2390 con 0x7f1e180cfde0 2024-04-26T20:51:56.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.738+0000 7f1e16ffd640 1 -- 172.21.15.73:0/433315426 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e00003520 con 0x7f1e180cfde0 2024-04-26T20:51:56.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.738+0000 7f1e1ca3f640 1 --2- 172.21.15.73:0/433315426 >> v2:172.21.15.120:3300/0 conn(0x7f1e18071260 0x7f1e1815e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:56.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.738+0000 7f1e16ffd640 1 -- 172.21.15.73:0/433315426 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2739770726 0 0) 0x7f1e00003520 con 0x7f1e180cfde0 2024-04-26T20:51:56.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.738+0000 7f1e16ffd640 1 -- 172.21.15.73:0/433315426 >> v2:172.21.15.120:3300/0 conn(0x7f1e18071260 msgr2=0x7f1e1815e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:56.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.738+0000 7f1e16ffd640 1 --2- 172.21.15.73:0/433315426 >> v2:172.21.15.120:3300/0 conn(0x7f1e18071260 0x7f1e1815e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:56.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.738+0000 7f1e16ffd640 1 -- 172.21.15.73:0/433315426 >> v1:172.21.15.120:6789/0 conn(0x7f1e18070350 legacy=0x7f1e18070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:56.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.738+0000 7f1e16ffd640 1 -- 172.21.15.73:0/433315426 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e1806d6d0 con 0x7f1e180cfde0 2024-04-26T20:51:56.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.738+0000 7f1e16ffd640 1 -- 172.21.15.73:0/433315426 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7f1e08001670 con 0x7f1e180cfde0 2024-04-26T20:51:56.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.738+0000 7f1e16ffd640 1 -- 172.21.15.73:0/433315426 >> v1:172.21.15.73:6789/0 conn(0x7f1e180cfde0 legacy=0x7f1e180d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:56.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.738+0000 7f1e16ffd640 1 --2- 172.21.15.73:0/433315426 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1e000039b0 0x7f1e00003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:56.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.738+0000 7f1e16ffd640 1 --2- 172.21.15.73:0/433315426 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e00004800 0x7f1e00004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:56.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.738+0000 7f1e16ffd640 1 --2- 172.21.15.73:0/433315426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e000051c0 0x7f1e0000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:56.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.738+0000 7f1e16ffd640 1 -- 172.21.15.73:0/433315426 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0000c130 con 0x7f1e000051c0 2024-04-26T20:51:56.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.738+0000 7f1e16ffd640 1 -- 172.21.15.73:0/433315426 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0000c2e0 con 0x7f1e00004800 2024-04-26T20:51:56.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.738+0000 7f1e16ffd640 1 -- 172.21.15.73:0/433315426 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f1e0000c4e0 con 0x7f1e000039b0 2024-04-26T20:51:56.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.738+0000 7f1e1ca3f640 1 --2- 172.21.15.73:0/433315426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e000051c0 0x7f1e0000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:56.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.738+0000 7f1e177fe640 1 --2- 172.21.15.73:0/433315426 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e00004800 0x7f1e00004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:56.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.738+0000 7f1e17fff640 1 --2- 172.21.15.73:0/433315426 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1e000039b0 0x7f1e00003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:56.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.739+0000 7f1e1ca3f640 1 -- 172.21.15.73:0/433315426 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1e000039b0 msgr2=0x7f1e00003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:56.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.739+0000 7f1e1ca3f640 1 --2- 172.21.15.73:0/433315426 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1e000039b0 0x7f1e00003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:56.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.739+0000 7f1e1ca3f640 1 -- 172.21.15.73:0/433315426 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e00004800 msgr2=0x7f1e00004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:56.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.739+0000 7f1e1ca3f640 1 --2- 172.21.15.73:0/433315426 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e00004800 0x7f1e00004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:56.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.739+0000 7f1e1ca3f640 1 -- 172.21.15.73:0/433315426 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e00003520 con 0x7f1e000051c0 2024-04-26T20:51:56.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.739+0000 7f1e1ca3f640 1 -- 172.21.15.73:0/433315426 reap_dead start 2024-04-26T20:51:56.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.739+0000 7f1e1ca3f640 1 --2- 172.21.15.73:0/433315426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e000051c0 0x7f1e0000b9c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f1e10009f90 tx=0x7f1e00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c78c5bce81a34ac server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:56.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.740+0000 7f1e16ffd640 1 -- 172.21.15.73:0/433315426 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e10009c10 con 0x7f1e000051c0 2024-04-26T20:51:56.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.740+0000 7f1e16ffd640 1 -- 172.21.15.73:0/433315426 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e10031440 con 0x7f1e000051c0 2024-04-26T20:51:56.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.740+0000 7f1e16ffd640 1 -- 172.21.15.73:0/433315426 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e10031810 con 0x7f1e000051c0 2024-04-26T20:51:56.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.740+0000 7f1e1e4a2640 1 -- 172.21.15.73:0/433315426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e000051c0 msgr2=0x7f1e0000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:56.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.740+0000 7f1e1e4a2640 1 --2- 172.21.15.73:0/433315426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e000051c0 0x7f1e0000b9c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f1e10009f90 tx=0x7f1e00005830 comp rx=0 tx=0).stop 2024-04-26T20:51:56.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.740+0000 7f1e1e4a2640 1 -- 172.21.15.73:0/433315426 shutdown_connections 2024-04-26T20:51:56.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.740+0000 7f1e1e4a2640 1 --2- 172.21.15.73:0/433315426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e000051c0 0x7f1e0000b9c0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:56.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.740+0000 7f1e1e4a2640 1 -- 172.21.15.73:0/433315426 >> 172.21.15.73:0/433315426 conn(0x7f1e180d2d80 msgr2=0x7f1e180d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:56.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.740+0000 7f1e1e4a2640 1 -- 172.21.15.73:0/433315426 shutdown_connections 2024-04-26T20:51:56.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.740+0000 7f1e1e4a2640 1 -- 172.21.15.73:0/433315426 wait complete. 2024-04-26T20:51:56.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.741+0000 7f1e1e4a2640 1 Processor -- start 2024-04-26T20:51:56.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.742+0000 7f1e1e4a2640 1 -- start start 2024-04-26T20:51:56.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.742+0000 7f1e1e4a2640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1e000051c0 0x7f1e180b7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:56.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.742+0000 7f1e1e4a2640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e180a0300 0x7f1e180b0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:56.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.742+0000 7f1e1e4a2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e180b5fd0 0x7f1e180b6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:56.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.742+0000 7f1e1e4a2640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1e18070b50 con 0x7f1e180a0300 2024-04-26T20:51:56.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.742+0000 7f1e1e4a2640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1806dd20 con 0x7f1e180b5fd0 2024-04-26T20:51:56.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.742+0000 7f1e1e4a2640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f1e18070cd0 con 0x7f1e000051c0 2024-04-26T20:51:56.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.742+0000 7f1e177fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e180a0300 0x7f1e180b0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:56.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.742+0000 7f1e177fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e180a0300 0x7f1e180b0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41162/0 (socket says 172.21.15.73:41162) 2024-04-26T20:51:56.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.742+0000 7f1e177fe640 1 -- 172.21.15.73:0/3786334802 learned_addr learned my addr 172.21.15.73:0/3786334802 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:56.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.743+0000 7f1e177fe640 1 -- 172.21.15.73:0/3786334802 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1e000051c0 msgr2=0x7f1e180b7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:56.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.743+0000 7f1e177fe640 1 --2- 172.21.15.73:0/3786334802 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1e000051c0 0x7f1e180b7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:56.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.743+0000 7f1e177fe640 1 -- 172.21.15.73:0/3786334802 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e180b5fd0 msgr2=0x7f1e180b6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:56.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.743+0000 7f1e177fe640 1 --2- 172.21.15.73:0/3786334802 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e180b5fd0 0x7f1e180b6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:56.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.743+0000 7f1e177fe640 1 -- 172.21.15.73:0/3786334802 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e181f7f00 con 0x7f1e180a0300 2024-04-26T20:51:56.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.743+0000 7f1e177fe640 1 --2- 172.21.15.73:0/3786334802 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e180a0300 0x7f1e180b0d80 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c00f5f0 tx=0x7f1e0c00fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:56.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.743+0000 7f1e14ff9640 1 -- 172.21.15.73:0/3786334802 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c0052e0 con 0x7f1e180a0300 2024-04-26T20:51:56.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.743+0000 7f1e14ff9640 1 -- 172.21.15.73:0/3786334802 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e0c018070 con 0x7f1e180a0300 2024-04-26T20:51:56.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.743+0000 7f1e14ff9640 1 -- 172.21.15.73:0/3786334802 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c003810 con 0x7f1e180a0300 2024-04-26T20:51:56.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.743+0000 7f1e1e4a2640 1 -- 172.21.15.73:0/3786334802 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e181f81a0 con 0x7f1e180a0300 2024-04-26T20:51:56.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.743+0000 7f1e1e4a2640 1 -- 172.21.15.73:0/3786334802 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e181f8760 con 0x7f1e180a0300 2024-04-26T20:51:56.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.744+0000 7f1e1e4a2640 1 -- 172.21.15.73:0/3786334802 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e180d01c0 con 0x7f1e180a0300 2024-04-26T20:51:56.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.744+0000 7f1e14ff9640 1 -- 172.21.15.73:0/3786334802 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1e0c0039b0 con 0x7f1e180a0300 2024-04-26T20:51:56.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.745+0000 7f1e14ff9640 1 --2- 172.21.15.73:0/3786334802 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1e0406c920 0x7f1e0406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:56.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.745+0000 7f1e17fff640 1 --2- 172.21.15.73:0/3786334802 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1e0406c920 0x7f1e0406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:56.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.745+0000 7f1e14ff9640 1 -- 172.21.15.73:0/3786334802 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f1e0c08ba40 con 0x7f1e180a0300 2024-04-26T20:51:56.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.745+0000 7f1e17fff640 1 --2- 172.21.15.73:0/3786334802 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1e0406c920 0x7f1e0406ede0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1e080011e0 tx=0x7f1e08001110 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:56.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.749+0000 7f1e14ff9640 1 -- 172.21.15.73:0/3786334802 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1e180d01c0 con 0x7f1e180a0300 2024-04-26T20:51:56.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.926+0000 7f1e1e4a2640 1 -- 172.21.15.73:0/3786334802 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1e180694f0 con 0x7f1e180a0300 2024-04-26T20:51:56.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.927+0000 7f1e14ff9640 1 -- 172.21.15.73:0/3786334802 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v39) ==== 93+0+10 (secure 0 0 0) 0x7f1e0c059d70 con 0x7f1e180a0300 2024-04-26T20:51:56.928 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-26T20:51:56.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.929+0000 7f1e1e4a2640 1 -- 172.21.15.73:0/3786334802 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1e0406c920 msgr2=0x7f1e0406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:56.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.929+0000 7f1e1e4a2640 1 --2- 172.21.15.73:0/3786334802 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1e0406c920 0x7f1e0406ede0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1e080011e0 tx=0x7f1e08001110 comp rx=0 tx=0).stop 2024-04-26T20:51:56.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.929+0000 7f1e1e4a2640 1 -- 172.21.15.73:0/3786334802 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e180a0300 msgr2=0x7f1e180b0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:56.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.929+0000 7f1e1e4a2640 1 --2- 172.21.15.73:0/3786334802 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e180a0300 0x7f1e180b0d80 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c00f5f0 tx=0x7f1e0c00fab0 comp rx=0 tx=0).stop 2024-04-26T20:51:56.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.929+0000 7f1e1e4a2640 1 -- 172.21.15.73:0/3786334802 shutdown_connections 2024-04-26T20:51:56.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.929+0000 7f1e1e4a2640 1 --2- 172.21.15.73:0/3786334802 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e180a0300 0x7f1e180b0d80 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:56.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.929+0000 7f1e1e4a2640 1 --2- 172.21.15.73:0/3786334802 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1e0406c920 0x7f1e0406ede0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:56.930 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.929+0000 7f1e1e4a2640 1 --2- 172.21.15.73:0/3786334802 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1e180b5fd0 0x7f1e180b6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:56.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.929+0000 7f1e1e4a2640 1 --2- 172.21.15.73:0/3786334802 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1e000051c0 0x7f1e180b7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:56.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.929+0000 7f1e1e4a2640 1 -- 172.21.15.73:0/3786334802 >> 172.21.15.73:0/3786334802 conn(0x7f1e180d2d80 msgr2=0x7f1e180b1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:56.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.929+0000 7f1e1e4a2640 1 -- 172.21.15.73:0/3786334802 shutdown_connections 2024-04-26T20:51:56.931 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:56.929+0000 7f1e1e4a2640 1 -- 172.21.15.73:0/3786334802 wait complete. 2024-04-26T20:51:56.940 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:51:57.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.040+0000 7fdd998c4640 1 Processor -- start 2024-04-26T20:51:57.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.040+0000 7fdd998c4640 1 -- start start 2024-04-26T20:51:57.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.041+0000 7fdd998c4640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdd9415b600 0x7fdd94071f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:57.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.041+0000 7fdd998c4640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdd940cf0e0 0x7fdd9406f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:57.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.041+0000 7fdd998c4640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd94072450 con 0x7fdd940cf5e0 2024-04-26T20:51:57.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.041+0000 7fdd998c4640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7fdd940cb9b0 con 0x7fdd9415b600 2024-04-26T20:51:57.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.041+0000 7fdd998c4640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fdd9415b9e0 con 0x7fdd940cf0e0 2024-04-26T20:51:57.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.041+0000 7fdd92ffd640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdd9415b600 0x7fdd94071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:57.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.041+0000 7fdd92ffd640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdd9415b600 0x7fdd94071f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41174/0 (socket says 172.21.15.73:41174) 2024-04-26T20:51:57.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.041+0000 7fdd92ffd640 1 -- 172.21.15.73:0/270867482 learned_addr learned my addr 172.21.15.73:0/270867482 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:57.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.041+0000 7fdd937fe640 1 --2- 172.21.15.73:0/270867482 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdd940cf0e0 0x7fdd9406f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:57.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.041+0000 7fdd92ffd640 1 -- 172.21.15.73:0/270867482 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdd940cf0e0 msgr2=0x7fdd9406f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:57.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.041+0000 7fdd92ffd640 1 --2- 172.21.15.73:0/270867482 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdd940cf0e0 0x7fdd9406f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:57.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.041+0000 7fdd92ffd640 1 -- 172.21.15.73:0/270867482 >> v1:172.21.15.120:6789/0 conn(0x7fdd940cf5e0 legacy=0x7fdd940ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:57.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.041+0000 7fdd92ffd640 1 -- 172.21.15.73:0/270867482 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd940d0740 con 0x7fdd9415b600 2024-04-26T20:51:57.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.042+0000 7fdd92ffd640 1 --2- 172.21.15.73:0/270867482 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd9415b600 0x7fdd94071f10 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fdd8400bb10 tx=0x7fdd8402c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d474e1fd39951b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:57.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.042+0000 7fdd927fc640 1 -- 172.21.15.73:0/270867482 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd84036020 con 0x7fdd9415b600 2024-04-26T20:51:57.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.042+0000 7fdd927fc640 1 -- 172.21.15.73:0/270867482 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd84032820 con 0x7fdd9415b600 2024-04-26T20:51:57.044 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.042+0000 7fdd927fc640 1 -- 172.21.15.73:0/270867482 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd84032bf0 con 0x7fdd9415b600 2024-04-26T20:51:57.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.042+0000 7fdd998c4640 1 -- 172.21.15.73:0/270867482 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd9415b600 msgr2=0x7fdd94071f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:57.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.042+0000 7fdd998c4640 1 --2- 172.21.15.73:0/270867482 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd9415b600 0x7fdd94071f10 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fdd8400bb10 tx=0x7fdd8402c920 comp rx=0 tx=0).stop 2024-04-26T20:51:57.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.042+0000 7fdd998c4640 1 -- 172.21.15.73:0/270867482 shutdown_connections 2024-04-26T20:51:57.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.042+0000 7fdd998c4640 1 --2- 172.21.15.73:0/270867482 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdd940cf0e0 0x7fdd9406f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:57.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.042+0000 7fdd998c4640 1 --2- 172.21.15.73:0/270867482 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd9415b600 0x7fdd94071f10 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:57.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.042+0000 7fdd998c4640 1 -- 172.21.15.73:0/270867482 >> 172.21.15.73:0/270867482 conn(0x7fdd940d0be0 msgr2=0x7fdd940d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:57.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.043+0000 7fdd998c4640 1 -- 172.21.15.73:0/270867482 shutdown_connections 2024-04-26T20:51:57.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.043+0000 7fdd998c4640 1 -- 172.21.15.73:0/270867482 wait complete. 2024-04-26T20:51:57.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.044+0000 7fdd998c4640 1 Processor -- start 2024-04-26T20:51:57.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.044+0000 7fdd998c4640 1 -- start start 2024-04-26T20:51:57.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.044+0000 7fdd998c4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdd940cf0e0 0x7fdd9414ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:57.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.044+0000 7fdd998c4640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdd940cf5e0 0x7fdd940ca310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:57.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.044+0000 7fdd998c4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd9415b600 0x7fdd940b5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:57.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.044+0000 7fdd998c4640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdd940d02f0 con 0x7fdd9415b600 2024-04-26T20:51:57.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.044+0000 7fdd998c4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdd940cb640 con 0x7fdd940cf0e0 2024-04-26T20:51:57.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.044+0000 7fdd998c4640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fdd940d0470 con 0x7fdd940cf5e0 2024-04-26T20:51:57.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.044+0000 7fdd937fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd9415b600 0x7fdd940b5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:57.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.045+0000 7fdd937fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd9415b600 0x7fdd940b5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41182/0 (socket says 172.21.15.73:41182) 2024-04-26T20:51:57.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.045+0000 7fdd937fe640 1 -- 172.21.15.73:0/3215978567 learned_addr learned my addr 172.21.15.73:0/3215978567 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:57.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.045+0000 7fdd92ffd640 1 --2- 172.21.15.73:0/3215978567 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdd940cf0e0 0x7fdd9414ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:57.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.045+0000 7fdd937fe640 1 -- 172.21.15.73:0/3215978567 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdd940cf5e0 msgr2=0x7fdd940ca310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:57.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.045+0000 7fdd8bfff640 1 --2- 172.21.15.73:0/3215978567 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdd940cf5e0 0x7fdd940ca310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:57.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.045+0000 7fdd937fe640 1 --2- 172.21.15.73:0/3215978567 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdd940cf5e0 0x7fdd940ca310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:57.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.045+0000 7fdd937fe640 1 -- 172.21.15.73:0/3215978567 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdd940cf0e0 msgr2=0x7fdd9414ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:57.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.045+0000 7fdd937fe640 1 --2- 172.21.15.73:0/3215978567 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdd940cf0e0 0x7fdd9414ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:57.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.045+0000 7fdd937fe640 1 -- 172.21.15.73:0/3215978567 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd9414f150 con 0x7fdd9415b600 2024-04-26T20:51:57.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.045+0000 7fdd92ffd640 1 --2- 172.21.15.73:0/3215978567 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdd940cf0e0 0x7fdd9414ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:57.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.045+0000 7fdd937fe640 1 --2- 172.21.15.73:0/3215978567 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd9415b600 0x7fdd940b5a50 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c007e10 tx=0x7fdd8c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:57.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.045+0000 7fdd90ff9640 1 -- 172.21.15.73:0/3215978567 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c014620 con 0x7fdd9415b600 2024-04-26T20:51:57.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.045+0000 7fdd90ff9640 1 -- 172.21.15.73:0/3215978567 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd8c019070 con 0x7fdd9415b600 2024-04-26T20:51:57.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.045+0000 7fdd998c4640 1 -- 172.21.15.73:0/3215978567 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd940c8070 con 0x7fdd9415b600 2024-04-26T20:51:57.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.045+0000 7fdd90ff9640 1 -- 172.21.15.73:0/3215978567 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c014a20 con 0x7fdd9415b600 2024-04-26T20:51:57.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.045+0000 7fdd998c4640 1 -- 172.21.15.73:0/3215978567 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd940c8630 con 0x7fdd9415b600 2024-04-26T20:51:57.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.046+0000 7fdd998c4640 1 -- 172.21.15.73:0/3215978567 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd940694f0 con 0x7fdd9415b600 2024-04-26T20:51:57.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.047+0000 7fdd90ff9640 1 -- 172.21.15.73:0/3215978567 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdd8c014bc0 con 0x7fdd9415b600 2024-04-26T20:51:57.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.047+0000 7fdd90ff9640 1 --2- 172.21.15.73:0/3215978567 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fdd7406c920 0x7fdd7406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:57.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.047+0000 7fdd90ff9640 1 -- 172.21.15.73:0/3215978567 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fdd8c08c560 con 0x7fdd9415b600 2024-04-26T20:51:57.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.047+0000 7fdd92ffd640 1 --2- 172.21.15.73:0/3215978567 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fdd7406c920 0x7fdd7406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:57.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.047+0000 7fdd92ffd640 1 --2- 172.21.15.73:0/3215978567 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fdd7406c920 0x7fdd7406ede0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fdd84037000 tx=0x7fdd84008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:57.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.051+0000 7fdd90ff9640 1 -- 172.21.15.73:0/3215978567 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdd8c05a800 con 0x7fdd9415b600 2024-04-26T20:51:57.227 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.226+0000 7fdd998c4640 1 -- 172.21.15.73:0/3215978567 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdd940ce7b0 con 0x7fdd9415b600 2024-04-26T20:51:57.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.227+0000 7fdd90ff9640 1 -- 172.21.15.73:0/3215978567 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v39) ==== 104+0+11 (secure 0 0 0) 0x7fdd8c05a100 con 0x7fdd9415b600 2024-04-26T20:51:57.228 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-26T20:51:57.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.229+0000 7fdd998c4640 1 -- 172.21.15.73:0/3215978567 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fdd7406c920 msgr2=0x7fdd7406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:57.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.229+0000 7fdd998c4640 1 --2- 172.21.15.73:0/3215978567 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fdd7406c920 0x7fdd7406ede0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fdd84037000 tx=0x7fdd84008040 comp rx=0 tx=0).stop 2024-04-26T20:51:57.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.229+0000 7fdd998c4640 1 -- 172.21.15.73:0/3215978567 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd9415b600 msgr2=0x7fdd940b5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:57.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.229+0000 7fdd998c4640 1 --2- 172.21.15.73:0/3215978567 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd9415b600 0x7fdd940b5a50 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c007e10 tx=0x7fdd8c00e4b0 comp rx=0 tx=0).stop 2024-04-26T20:51:57.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.229+0000 7fdd998c4640 1 -- 172.21.15.73:0/3215978567 shutdown_connections 2024-04-26T20:51:57.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.229+0000 7fdd998c4640 1 --2- 172.21.15.73:0/3215978567 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd9415b600 0x7fdd940b5a50 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:57.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.229+0000 7fdd998c4640 1 --2- 172.21.15.73:0/3215978567 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fdd7406c920 0x7fdd7406ede0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:57.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.229+0000 7fdd998c4640 1 --2- 172.21.15.73:0/3215978567 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdd940cf5e0 0x7fdd940ca310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:57.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.229+0000 7fdd998c4640 1 --2- 172.21.15.73:0/3215978567 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdd940cf0e0 0x7fdd9414ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:57.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.229+0000 7fdd998c4640 1 -- 172.21.15.73:0/3215978567 >> 172.21.15.73:0/3215978567 conn(0x7fdd940d0be0 msgr2=0x7fdd940b14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:57.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.229+0000 7fdd998c4640 1 -- 172.21.15.73:0/3215978567 shutdown_connections 2024-04-26T20:51:57.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.229+0000 7fdd998c4640 1 -- 172.21.15.73:0/3215978567 wait complete. 2024-04-26T20:51:57.240 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:51:57.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.339+0000 7f16ebb9d640 1 Processor -- start 2024-04-26T20:51:57.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.339+0000 7f16ebb9d640 1 -- start start 2024-04-26T20:51:57.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.340+0000 7f16ebb9d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f16e415f1d0 0x7f16e406f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:57.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.340+0000 7f16ebb9d640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f16e40afd60 0x7f16e406c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:57.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.340+0000 7f16ebb9d640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16e40b4480 con 0x7f16e406c240 2024-04-26T20:51:57.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.340+0000 7f16ebb9d640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f16e406cbe0 con 0x7f16e40afd60 2024-04-26T20:51:57.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.340+0000 7f16ebb9d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f16e40ae8c0 con 0x7f16e415f1d0 2024-04-26T20:51:57.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.340+0000 7f16ea13a640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f16e40afd60 0x7f16e406c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:57.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.340+0000 7f16ea13a640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f16e40afd60 0x7f16e406c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41190/0 (socket says 172.21.15.73:41190) 2024-04-26T20:51:57.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.340+0000 7f16ea13a640 1 -- 172.21.15.73:0/589385991 learned_addr learned my addr 172.21.15.73:0/589385991 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:57.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.340+0000 7f16e9939640 1 --2- 172.21.15.73:0/589385991 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f16e415f1d0 0x7f16e406f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:57.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.340+0000 7f16ea13a640 1 -- 172.21.15.73:0/589385991 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f16e415f1d0 msgr2=0x7f16e406f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:57.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.340+0000 7f16ea13a640 1 --2- 172.21.15.73:0/589385991 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f16e415f1d0 0x7f16e406f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:57.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.340+0000 7f16ea13a640 1 -- 172.21.15.73:0/589385991 >> v1:172.21.15.120:6789/0 conn(0x7f16e406c240 legacy=0x7f16e40726e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:57.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.340+0000 7f16ea13a640 1 -- 172.21.15.73:0/589385991 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16e40b4720 con 0x7f16e40afd60 2024-04-26T20:51:57.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.340+0000 7f16e9939640 1 --2- 172.21.15.73:0/589385991 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f16e415f1d0 0x7f16e406f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:57.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.341+0000 7f16ea13a640 1 --2- 172.21.15.73:0/589385991 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16e40afd60 0x7f16e406c6a0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f16dc009e80 tx=0x7f16dc02a740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5a4c91d54900922 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:57.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.341+0000 7f16e8937640 1 -- 172.21.15.73:0/589385991 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16dc036070 con 0x7f16e40afd60 2024-04-26T20:51:57.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.341+0000 7f16e8937640 1 -- 172.21.15.73:0/589385991 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16dc031600 con 0x7f16e40afd60 2024-04-26T20:51:57.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.341+0000 7f16e8937640 1 -- 172.21.15.73:0/589385991 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16dc0319d0 con 0x7f16e40afd60 2024-04-26T20:51:57.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.341+0000 7f16ebb9d640 1 -- 172.21.15.73:0/589385991 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16e40afd60 msgr2=0x7f16e406c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:57.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.341+0000 7f16ebb9d640 1 --2- 172.21.15.73:0/589385991 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16e40afd60 0x7f16e406c6a0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f16dc009e80 tx=0x7f16dc02a740 comp rx=0 tx=0).stop 2024-04-26T20:51:57.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.342+0000 7f16ebb9d640 1 -- 172.21.15.73:0/589385991 shutdown_connections 2024-04-26T20:51:57.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.342+0000 7f16ebb9d640 1 --2- 172.21.15.73:0/589385991 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16e40afd60 0x7f16e406c6a0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:57.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.342+0000 7f16ebb9d640 1 --2- 172.21.15.73:0/589385991 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f16e415f1d0 0x7f16e406f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:57.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.342+0000 7f16ebb9d640 1 -- 172.21.15.73:0/589385991 >> 172.21.15.73:0/589385991 conn(0x7f16e40b3180 msgr2=0x7f16e40b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:57.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.342+0000 7f16ebb9d640 1 -- 172.21.15.73:0/589385991 shutdown_connections 2024-04-26T20:51:57.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.342+0000 7f16ebb9d640 1 -- 172.21.15.73:0/589385991 wait complete. 2024-04-26T20:51:57.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.343+0000 7f16ebb9d640 1 Processor -- start 2024-04-26T20:51:57.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.343+0000 7f16ebb9d640 1 -- start start 2024-04-26T20:51:57.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.343+0000 7f16ebb9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16e406c240 0x7f16e4159af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:57.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.343+0000 7f16ebb9d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f16e40afd60 0x7f16e4143610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:57.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.343+0000 7f16ebb9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16e415f1d0 0x7f16e4143f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:57.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.343+0000 7f16ebb9d640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f16e40b4fc0 con 0x7f16e415f1d0 2024-04-26T20:51:57.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.343+0000 7f16ebb9d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f16e40b4ce0 con 0x7f16e406c240 2024-04-26T20:51:57.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.343+0000 7f16ebb9d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f16e406f6b0 con 0x7f16e40afd60 2024-04-26T20:51:57.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.344+0000 7f16ea13a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16e415f1d0 0x7f16e4143f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:57.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.344+0000 7f16ea13a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16e415f1d0 0x7f16e4143f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41194/0 (socket says 172.21.15.73:41194) 2024-04-26T20:51:57.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.344+0000 7f16ea13a640 1 -- 172.21.15.73:0/3922048296 learned_addr learned my addr 172.21.15.73:0/3922048296 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:57.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.344+0000 7f16ea13a640 1 -- 172.21.15.73:0/3922048296 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f16e40afd60 msgr2=0x7f16e4143610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:57.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.344+0000 7f16ea13a640 1 --2- 172.21.15.73:0/3922048296 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f16e40afd60 0x7f16e4143610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:57.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.344+0000 7f16ea13a640 1 -- 172.21.15.73:0/3922048296 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16e406c240 msgr2=0x7f16e4159af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:57.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.344+0000 7f16e9939640 1 --2- 172.21.15.73:0/3922048296 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16e406c240 0x7f16e4159af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:57.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.344+0000 7f16ea13a640 1 --2- 172.21.15.73:0/3922048296 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16e406c240 0x7f16e4159af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:57.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.344+0000 7f16ea13a640 1 -- 172.21.15.73:0/3922048296 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16e4153bb0 con 0x7f16e415f1d0 2024-04-26T20:51:57.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.344+0000 7f16e9939640 1 --2- 172.21.15.73:0/3922048296 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16e406c240 0x7f16e4159af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:57.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.344+0000 7f16ea13a640 1 --2- 172.21.15.73:0/3922048296 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16e415f1d0 0x7f16e4143f20 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f16dc034040 tx=0x7f16dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:57.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.344+0000 7f16da7fc640 1 -- 172.21.15.73:0/3922048296 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16dc036050 con 0x7f16e415f1d0 2024-04-26T20:51:57.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.344+0000 7f16da7fc640 1 -- 172.21.15.73:0/3922048296 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16dc042070 con 0x7f16e415f1d0 2024-04-26T20:51:57.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.344+0000 7f16ebb9d640 1 -- 172.21.15.73:0/3922048296 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16e415fee0 con 0x7f16e415f1d0 2024-04-26T20:51:57.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.344+0000 7f16da7fc640 1 -- 172.21.15.73:0/3922048296 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16dc009510 con 0x7f16e415f1d0 2024-04-26T20:51:57.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.344+0000 7f16ebb9d640 1 -- 172.21.15.73:0/3922048296 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16e41604a0 con 0x7f16e415f1d0 2024-04-26T20:51:57.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.345+0000 7f16ebb9d640 1 -- 172.21.15.73:0/3922048296 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16e40694f0 con 0x7f16e415f1d0 2024-04-26T20:51:57.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.346+0000 7f16da7fc640 1 -- 172.21.15.73:0/3922048296 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f16dc007bd0 con 0x7f16e415f1d0 2024-04-26T20:51:57.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.346+0000 7f16da7fc640 1 --2- 172.21.15.73:0/3922048296 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f16c806c8f0 0x7f16c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:57.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.346+0000 7f16da7fc640 1 -- 172.21.15.73:0/3922048296 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f16dc0acba0 con 0x7f16e415f1d0 2024-04-26T20:51:57.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.346+0000 7f16e9939640 1 --2- 172.21.15.73:0/3922048296 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f16c806c8f0 0x7f16c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:57.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.346+0000 7f16e9939640 1 --2- 172.21.15.73:0/3922048296 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f16c806c8f0 0x7f16c806edb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f16e0006fd0 tx=0x7f16e0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:57.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.350+0000 7f16da7fc640 1 -- 172.21.15.73:0/3922048296 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f16dc07ae40 con 0x7f16e415f1d0 2024-04-26T20:51:57.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.525+0000 7f16ebb9d640 1 -- 172.21.15.73:0/3922048296 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f16e40b5c90 con 0x7f16e415f1d0 2024-04-26T20:51:57.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.526+0000 7f16da7fc640 1 -- 172.21.15.73:0/3922048296 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v39) ==== 100+0+11 (secure 0 0 0) 0x7f16dc07a740 con 0x7f16e415f1d0 2024-04-26T20:51:57.527 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-26T20:51:57.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.528+0000 7f16ebb9d640 1 -- 172.21.15.73:0/3922048296 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f16c806c8f0 msgr2=0x7f16c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:57.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.528+0000 7f16ebb9d640 1 --2- 172.21.15.73:0/3922048296 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f16c806c8f0 0x7f16c806edb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f16e0006fd0 tx=0x7f16e0008040 comp rx=0 tx=0).stop 2024-04-26T20:51:57.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.528+0000 7f16ebb9d640 1 -- 172.21.15.73:0/3922048296 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16e415f1d0 msgr2=0x7f16e4143f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:57.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.528+0000 7f16ebb9d640 1 --2- 172.21.15.73:0/3922048296 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16e415f1d0 0x7f16e4143f20 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f16dc034040 tx=0x7f16dc004a40 comp rx=0 tx=0).stop 2024-04-26T20:51:57.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.528+0000 7f16ebb9d640 1 -- 172.21.15.73:0/3922048296 shutdown_connections 2024-04-26T20:51:57.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.528+0000 7f16ebb9d640 1 --2- 172.21.15.73:0/3922048296 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16e415f1d0 0x7f16e4143f20 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:57.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.528+0000 7f16ebb9d640 1 --2- 172.21.15.73:0/3922048296 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f16c806c8f0 0x7f16c806edb0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:57.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.528+0000 7f16ebb9d640 1 --2- 172.21.15.73:0/3922048296 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f16e40afd60 0x7f16e4143610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:57.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.528+0000 7f16ebb9d640 1 --2- 172.21.15.73:0/3922048296 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f16e406c240 0x7f16e4159af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:57.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.528+0000 7f16ebb9d640 1 -- 172.21.15.73:0/3922048296 >> 172.21.15.73:0/3922048296 conn(0x7f16e40b3180 msgr2=0x7f16e40aed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:57.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.528+0000 7f16ebb9d640 1 -- 172.21.15.73:0/3922048296 shutdown_connections 2024-04-26T20:51:57.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.528+0000 7f16ebb9d640 1 -- 172.21.15.73:0/3922048296 wait complete. 2024-04-26T20:51:57.539 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T20:51:57.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.635+0000 7f324f714640 1 Processor -- start 2024-04-26T20:51:57.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.636+0000 7f324f714640 1 -- start start 2024-04-26T20:51:57.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.636+0000 7f324f714640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f324815f3a0 0x7f32480d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:57.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.636+0000 7f324f714640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f32480d2330 0x7f32480b2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:57.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.636+0000 7f324f714640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32480d0e20 con 0x7f32480d1770 2024-04-26T20:51:57.638 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.636+0000 7f324f714640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f32480b1d70 con 0x7f32480d2330 2024-04-26T20:51:57.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.636+0000 7f324f714640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f32480b2050 con 0x7f324815f3a0 2024-04-26T20:51:57.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.636+0000 7f324dcb1640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f32480d2330 0x7f32480b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:57.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.637+0000 7f324dcb1640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f32480d2330 0x7f32480b2850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41202/0 (socket says 172.21.15.73:41202) 2024-04-26T20:51:57.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.637+0000 7f324dcb1640 1 -- 172.21.15.73:0/2337274289 learned_addr learned my addr 172.21.15.73:0/2337274289 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:57.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.637+0000 7f324d4b0640 1 --2- 172.21.15.73:0/2337274289 >> v2:172.21.15.120:3300/0 conn(0x7f324815f3a0 0x7f32480d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:57.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.637+0000 7f324dcb1640 1 -- 172.21.15.73:0/2337274289 >> v2:172.21.15.120:3300/0 conn(0x7f324815f3a0 msgr2=0x7f32480d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:57.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.637+0000 7f324dcb1640 1 --2- 172.21.15.73:0/2337274289 >> v2:172.21.15.120:3300/0 conn(0x7f324815f3a0 0x7f32480d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:57.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.637+0000 7f324dcb1640 1 -- 172.21.15.73:0/2337274289 >> v1:172.21.15.120:6789/0 conn(0x7f32480d1770 legacy=0x7f32480d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:57.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.637+0000 7f324dcb1640 1 -- 172.21.15.73:0/2337274289 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32480b2330 con 0x7f32480d2330 2024-04-26T20:51:57.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.637+0000 7f324d4b0640 1 --2- 172.21.15.73:0/2337274289 >> v2:172.21.15.120:3300/0 conn(0x7f324815f3a0 0x7f32480d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:57.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.637+0000 7f324dcb1640 1 --2- 172.21.15.73:0/2337274289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32480d2330 0x7f32480b2850 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f32480c89f0 tx=0x7f324002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1be84f1a71a1cb70 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:57.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.638+0000 7f323ffff640 1 -- 172.21.15.73:0/2337274289 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3240036070 con 0x7f32480d2330 2024-04-26T20:51:57.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.638+0000 7f323ffff640 1 -- 172.21.15.73:0/2337274289 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3240031600 con 0x7f32480d2330 2024-04-26T20:51:57.639 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.638+0000 7f323ffff640 1 -- 172.21.15.73:0/2337274289 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32400319d0 con 0x7f32480d2330 2024-04-26T20:51:57.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.638+0000 7f324f714640 1 -- 172.21.15.73:0/2337274289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32480d2330 msgr2=0x7f32480b2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:57.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.638+0000 7f324f714640 1 --2- 172.21.15.73:0/2337274289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32480d2330 0x7f32480b2850 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f32480c89f0 tx=0x7f324002a6c0 comp rx=0 tx=0).stop 2024-04-26T20:51:57.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.638+0000 7f324f714640 1 -- 172.21.15.73:0/2337274289 shutdown_connections 2024-04-26T20:51:57.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.638+0000 7f324f714640 1 --2- 172.21.15.73:0/2337274289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32480d2330 0x7f32480b2850 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:57.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.638+0000 7f324f714640 1 --2- 172.21.15.73:0/2337274289 >> v2:172.21.15.120:3300/0 conn(0x7f324815f3a0 0x7f32480d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:57.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.638+0000 7f324f714640 1 -- 172.21.15.73:0/2337274289 >> 172.21.15.73:0/2337274289 conn(0x7f32480d4f90 msgr2=0x7f32480d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:57.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.638+0000 7f324f714640 1 -- 172.21.15.73:0/2337274289 shutdown_connections 2024-04-26T20:51:57.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.638+0000 7f324f714640 1 -- 172.21.15.73:0/2337274289 wait complete. 2024-04-26T20:51:57.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.639+0000 7f324f714640 1 Processor -- start 2024-04-26T20:51:57.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.639+0000 7f324f714640 1 -- start start 2024-04-26T20:51:57.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.640+0000 7f324f714640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f32480d1770 0x7f324814d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:57.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.640+0000 7f324f714640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32480d2330 0x7f32481502d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:57.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.640+0000 7f324f714640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f324815f3a0 0x7f32480ba320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:57.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.640+0000 7f324f714640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f32480b2f10 con 0x7f324815f3a0 2024-04-26T20:51:57.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.640+0000 7f324f714640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f32480b2d90 con 0x7f32480d2330 2024-04-26T20:51:57.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.640+0000 7f324f714640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f32480b3090 con 0x7f32480d1770 2024-04-26T20:51:57.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.640+0000 7f324dcb1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f324815f3a0 0x7f32480ba320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:57.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.640+0000 7f324dcb1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f324815f3a0 0x7f32480ba320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41204/0 (socket says 172.21.15.73:41204) 2024-04-26T20:51:57.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.640+0000 7f324dcb1640 1 -- 172.21.15.73:0/328929280 learned_addr learned my addr 172.21.15.73:0/328929280 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:57.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.640+0000 7f324ccaf640 1 --2- 172.21.15.73:0/328929280 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32480d2330 0x7f32481502d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:57.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.640+0000 7f324d4b0640 1 --2- 172.21.15.73:0/328929280 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f32480d1770 0x7f324814d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:57.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.640+0000 7f324dcb1640 1 -- 172.21.15.73:0/328929280 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f32480d1770 msgr2=0x7f324814d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:57.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.640+0000 7f324dcb1640 1 --2- 172.21.15.73:0/328929280 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f32480d1770 0x7f324814d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:57.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.640+0000 7f324dcb1640 1 -- 172.21.15.73:0/328929280 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32480d2330 msgr2=0x7f32481502d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:57.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.640+0000 7f324dcb1640 1 --2- 172.21.15.73:0/328929280 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32480d2330 0x7f32481502d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:57.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.640+0000 7f324dcb1640 1 -- 172.21.15.73:0/328929280 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32480baa50 con 0x7f324815f3a0 2024-04-26T20:51:57.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.641+0000 7f324ccaf640 1 --2- 172.21.15.73:0/328929280 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32480d2330 0x7f32481502d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:57.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.641+0000 7f324d4b0640 1 --2- 172.21.15.73:0/328929280 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f32480d1770 0x7f324814d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:51:57.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.641+0000 7f324dcb1640 1 --2- 172.21.15.73:0/328929280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f324815f3a0 0x7f32480ba320 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f3240034040 tx=0x7f3240004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:57.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.641+0000 7f323dffb640 1 -- 172.21.15.73:0/328929280 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3240036050 con 0x7f324815f3a0 2024-04-26T20:51:57.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.641+0000 7f323dffb640 1 -- 172.21.15.73:0/328929280 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3240042070 con 0x7f324815f3a0 2024-04-26T20:51:57.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.641+0000 7f324f714640 1 -- 172.21.15.73:0/328929280 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32480bad30 con 0x7f324815f3a0 2024-04-26T20:51:57.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.641+0000 7f323dffb640 1 -- 172.21.15.73:0/328929280 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3240009510 con 0x7f324815f3a0 2024-04-26T20:51:57.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.641+0000 7f324f714640 1 -- 172.21.15.73:0/328929280 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f324814da60 con 0x7f324815f3a0 2024-04-26T20:51:57.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.642+0000 7f324f714640 1 -- 172.21.15.73:0/328929280 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32480694f0 con 0x7f324815f3a0 2024-04-26T20:51:57.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.642+0000 7f323dffb640 1 -- 172.21.15.73:0/328929280 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3240007bf0 con 0x7f324815f3a0 2024-04-26T20:51:57.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.643+0000 7f323dffb640 1 --2- 172.21.15.73:0/328929280 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f322c06c8f0 0x7f322c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:57.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.643+0000 7f323dffb640 1 -- 172.21.15.73:0/328929280 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f32400accc0 con 0x7f324815f3a0 2024-04-26T20:51:57.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.643+0000 7f324d4b0640 1 --2- 172.21.15.73:0/328929280 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f322c06c8f0 0x7f322c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:57.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.643+0000 7f324d4b0640 1 --2- 172.21.15.73:0/328929280 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f322c06c8f0 0x7f322c06edb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3244006fd0 tx=0x7f3244008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:57.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.647+0000 7f323dffb640 1 -- 172.21.15.73:0/328929280 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f324007af60 con 0x7f324815f3a0 2024-04-26T20:51:57.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.834+0000 7f324f714640 1 -- 172.21.15.73:0/328929280 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f32480b6860 con 0x7f324815f3a0 2024-04-26T20:51:57.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.835+0000 7f323dffb640 1 -- 172.21.15.73:0/328929280 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v39) ==== 80+0+0 (secure 0 0 0) 0x7f324007a860 con 0x7f324815f3a0 2024-04-26T20:51:57.836 INFO:teuthology.orchestra.run.smithi073.stderr:listed 0 entries 2024-04-26T20:51:57.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.837+0000 7f324f714640 1 -- 172.21.15.73:0/328929280 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f322c06c8f0 msgr2=0x7f322c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:57.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.837+0000 7f324f714640 1 --2- 172.21.15.73:0/328929280 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f322c06c8f0 0x7f322c06edb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3244006fd0 tx=0x7f3244008040 comp rx=0 tx=0).stop 2024-04-26T20:51:57.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.837+0000 7f324f714640 1 -- 172.21.15.73:0/328929280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f324815f3a0 msgr2=0x7f32480ba320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:57.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.837+0000 7f324f714640 1 --2- 172.21.15.73:0/328929280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f324815f3a0 0x7f32480ba320 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f3240034040 tx=0x7f3240004a40 comp rx=0 tx=0).stop 2024-04-26T20:51:57.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.837+0000 7f324f714640 1 -- 172.21.15.73:0/328929280 shutdown_connections 2024-04-26T20:51:57.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.837+0000 7f324f714640 1 --2- 172.21.15.73:0/328929280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f324815f3a0 0x7f32480ba320 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:57.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.837+0000 7f324f714640 1 --2- 172.21.15.73:0/328929280 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f322c06c8f0 0x7f322c06edb0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:57.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.837+0000 7f324f714640 1 --2- 172.21.15.73:0/328929280 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32480d2330 0x7f32481502d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:57.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.837+0000 7f324f714640 1 --2- 172.21.15.73:0/328929280 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f32480d1770 0x7f324814d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:57.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.837+0000 7f324f714640 1 -- 172.21.15.73:0/328929280 >> 172.21.15.73:0/328929280 conn(0x7f32480d4f90 msgr2=0x7f32480d1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:57.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.837+0000 7f324f714640 1 -- 172.21.15.73:0/328929280 shutdown_connections 2024-04-26T20:51:57.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:57.837+0000 7f324f714640 1 -- 172.21.15.73:0/328929280 wait complete. 2024-04-26T20:51:57.848 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T20:51:57.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:57.849 DEBUG:teuthology.orchestra.run.smithi073:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T20:51:57.901 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi073.front.sepia.ceph.com 2024-04-26T20:51:57.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:51:57.901 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T20:51:57.919 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:51:58.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.056+0000 7f1dd8025640 1 Processor -- start 2024-04-26T20:51:58.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.056+0000 7f1dd8025640 1 -- start start 2024-04-26T20:51:58.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.057+0000 7f1dd8025640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f1dd015f290 0x7f1dd00cb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:58.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.057+0000 7f1dd8025640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1dd00cd060 0x7f1dd0162e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:58.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.057+0000 7f1dd8025640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1dd00cd4a0 0x7f1dd01633c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:58.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.057+0000 7f1dd8025640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f1dd006f5a0 con 0x7f1dd00cd4a0 2024-04-26T20:51:58.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.057+0000 7f1dd8025640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f1dd006f880 con 0x7f1dd015f290 2024-04-26T20:51:58.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.057+0000 7f1dd8025640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f1dd006fb60 con 0x7f1dd00cd060 2024-04-26T20:51:58.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.057+0000 7f1dd65c2640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1dd00cd4a0 0x7f1dd01633c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:58.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.057+0000 7f1dd65c2640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1dd00cd4a0 0x7f1dd01633c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41206/0 (socket says 172.21.15.73:41206) 2024-04-26T20:51:58.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.057+0000 7f1dd65c2640 1 -- 172.21.15.73:0/2162018832 learned_addr learned my addr 172.21.15.73:0/2162018832 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:58.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.057+0000 7f1dd5dc1640 1 --2- 172.21.15.73:0/2162018832 >> v2:172.21.15.120:3300/0 conn(0x7f1dd015f290 0x7f1dd00cb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:58.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.057+0000 7f1dd55c0640 1 --2- 172.21.15.73:0/2162018832 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1dd00cd060 0x7f1dd0162e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:58.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.057+0000 7f1dd65c2640 1 -- 172.21.15.73:0/2162018832 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1dd00cd060 msgr2=0x7f1dd0162e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:58.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.057+0000 7f1dd65c2640 1 --2- 172.21.15.73:0/2162018832 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1dd00cd060 0x7f1dd0162e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:58.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.057+0000 7f1dd65c2640 1 -- 172.21.15.73:0/2162018832 >> v2:172.21.15.120:3300/0 conn(0x7f1dd015f290 msgr2=0x7f1dd00cb7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:58.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.057+0000 7f1dd65c2640 1 --2- 172.21.15.73:0/2162018832 >> v2:172.21.15.120:3300/0 conn(0x7f1dd015f290 0x7f1dd00cb7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:58.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.057+0000 7f1dd65c2640 1 -- 172.21.15.73:0/2162018832 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dd015f670 con 0x7f1dd00cd4a0 2024-04-26T20:51:58.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.057+0000 7f1dd5dc1640 1 --2- 172.21.15.73:0/2162018832 >> v2:172.21.15.120:3300/0 conn(0x7f1dd015f290 0x7f1dd00cb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:58.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.058+0000 7f1dd65c2640 1 --2- 172.21.15.73:0/2162018832 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1dd00cd4a0 0x7f1dd01633c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0075b00 tx=0x7f1dc802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c6e639ed4e3d9d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:58.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.058+0000 7f1dd4dbf640 1 -- 172.21.15.73:0/2162018832 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc8036070 con 0x7f1dd00cd4a0 2024-04-26T20:51:58.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.058+0000 7f1dd4dbf640 1 -- 172.21.15.73:0/2162018832 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dc8031600 con 0x7f1dd00cd4a0 2024-04-26T20:51:58.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.058+0000 7f1dd4dbf640 1 -- 172.21.15.73:0/2162018832 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc80319d0 con 0x7f1dd00cd4a0 2024-04-26T20:51:58.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.058+0000 7f1dd8025640 1 -- 172.21.15.73:0/2162018832 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1dd00cd4a0 msgr2=0x7f1dd01633c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:58.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.058+0000 7f1dd8025640 1 --2- 172.21.15.73:0/2162018832 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1dd00cd4a0 0x7f1dd01633c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0075b00 tx=0x7f1dc802a730 comp rx=0 tx=0).stop 2024-04-26T20:51:58.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.058+0000 7f1dd8025640 1 -- 172.21.15.73:0/2162018832 shutdown_connections 2024-04-26T20:51:58.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.059+0000 7f1dd8025640 1 --2- 172.21.15.73:0/2162018832 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1dd00cd4a0 0x7f1dd01633c0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:58.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.059+0000 7f1dd8025640 1 --2- 172.21.15.73:0/2162018832 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1dd00cd060 0x7f1dd0162e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:58.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.059+0000 7f1dd8025640 1 --2- 172.21.15.73:0/2162018832 >> v2:172.21.15.120:3300/0 conn(0x7f1dd015f290 0x7f1dd00cb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:58.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.059+0000 7f1dd8025640 1 -- 172.21.15.73:0/2162018832 >> 172.21.15.73:0/2162018832 conn(0x7f1dd00d0fd0 msgr2=0x7f1dd00d13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:58.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.059+0000 7f1dd8025640 1 -- 172.21.15.73:0/2162018832 shutdown_connections 2024-04-26T20:51:58.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.059+0000 7f1dd8025640 1 -- 172.21.15.73:0/2162018832 wait complete. 2024-04-26T20:51:58.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.060+0000 7f1dd8025640 1 Processor -- start 2024-04-26T20:51:58.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.060+0000 7f1dd8025640 1 -- start start 2024-04-26T20:51:58.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.061+0000 7f1dd8025640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1dd00cd060 0x7f1dd014ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:58.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.061+0000 7f1dd8025640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1dd00cd4a0 0x7f1dd009caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:58.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.061+0000 7f1dd8025640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1dd015f290 0x7f1dd01f88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:58.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.061+0000 7f1dd8025640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd006e6d0 con 0x7f1dd00cd060 2024-04-26T20:51:58.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.061+0000 7f1dd8025640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd006e550 con 0x7f1dd00cd4a0 2024-04-26T20:51:58.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.061+0000 7f1dd8025640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f1dd006e850 con 0x7f1dd015f290 2024-04-26T20:51:58.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.061+0000 7f1dd5dc1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1dd00cd060 0x7f1dd014ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:58.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.061+0000 7f1dd5dc1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1dd00cd060 0x7f1dd014ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41208/0 (socket says 172.21.15.73:41208) 2024-04-26T20:51:58.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.061+0000 7f1dd5dc1640 1 -- 172.21.15.73:0/1886077869 learned_addr learned my addr 172.21.15.73:0/1886077869 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:58.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.061+0000 7f1dd55c0640 1 --2- 172.21.15.73:0/1886077869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1dd00cd4a0 0x7f1dd009caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:58.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.061+0000 7f1dd65c2640 1 --2- 172.21.15.73:0/1886077869 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1dd015f290 0x7f1dd01f88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:58.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.061+0000 7f1dd5dc1640 1 -- 172.21.15.73:0/1886077869 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1dd015f290 msgr2=0x7f1dd01f88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:58.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.061+0000 7f1dd5dc1640 1 --2- 172.21.15.73:0/1886077869 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1dd015f290 0x7f1dd01f88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:58.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.061+0000 7f1dd5dc1640 1 -- 172.21.15.73:0/1886077869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1dd00cd4a0 msgr2=0x7f1dd009caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:58.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.061+0000 7f1dd5dc1640 1 --2- 172.21.15.73:0/1886077869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1dd00cd4a0 0x7f1dd009caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:58.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.061+0000 7f1dd5dc1640 1 -- 172.21.15.73:0/1886077869 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dd01f9210 con 0x7f1dd00cd060 2024-04-26T20:51:58.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.061+0000 7f1dd65c2640 1 --2- 172.21.15.73:0/1886077869 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1dd015f290 0x7f1dd01f88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:51:58.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.061+0000 7f1dd5dc1640 1 --2- 172.21.15.73:0/1886077869 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1dd00cd060 0x7f1dd014ef10 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc00d7d0 tx=0x7f1dcc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:58.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.061+0000 7f1dc67fc640 1 -- 172.21.15.73:0/1886077869 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc004030 con 0x7f1dd00cd060 2024-04-26T20:51:58.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.061+0000 7f1dc67fc640 1 -- 172.21.15.73:0/1886077869 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dcc0041d0 con 0x7f1dd00cd060 2024-04-26T20:51:58.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.061+0000 7f1dd8025640 1 -- 172.21.15.73:0/1886077869 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dd01f94f0 con 0x7f1dd00cd060 2024-04-26T20:51:58.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.061+0000 7f1dc67fc640 1 -- 172.21.15.73:0/1886077869 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dcc0096a0 con 0x7f1dd00cd060 2024-04-26T20:51:58.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.062+0000 7f1dd8025640 1 -- 172.21.15.73:0/1886077869 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dd00ca5a0 con 0x7f1dd00cd060 2024-04-26T20:51:58.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.062+0000 7f1dd8025640 1 -- 172.21.15.73:0/1886077869 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dd00694f0 con 0x7f1dd00cd060 2024-04-26T20:51:58.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.063+0000 7f1dc67fc640 1 -- 172.21.15.73:0/1886077869 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1dcc009840 con 0x7f1dd00cd060 2024-04-26T20:51:58.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.063+0000 7f1dc67fc640 1 --2- 172.21.15.73:0/1886077869 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1db406c8f0 0x7f1db406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:58.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.063+0000 7f1dc67fc640 1 -- 172.21.15.73:0/1886077869 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f1dcc08beb0 con 0x7f1dd00cd060 2024-04-26T20:51:58.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.063+0000 7f1dd55c0640 1 --2- 172.21.15.73:0/1886077869 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1db406c8f0 0x7f1db406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:58.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.064+0000 7f1dd55c0640 1 --2- 172.21.15.73:0/1886077869 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1db406c8f0 0x7f1db406edb0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1dd015d910 tx=0x7f1dc0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:58.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.069+0000 7f1dc67fc640 1 -- 172.21.15.73:0/1886077869 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1dcc05a1e0 con 0x7f1dd00cd060 2024-04-26T20:51:58.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.280+0000 7f1dd8025640 1 -- 172.21.15.73:0/1886077869 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1dd006ff80 con 0x7f1dd00cd060 2024-04-26T20:51:58.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.280+0000 7f1dc67fc640 1 -- 172.21.15.73:0/1886077869 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+5181 (secure 0 0 0) 0x7f1dcc059ae0 con 0x7f1dd00cd060 2024-04-26T20:51:58.281 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:51:58.281 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4802,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/272662913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":272662913},{"type":"v1","addr":"172.21.15.73:6836","nonce":272662913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5047,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/1438858294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":1438858294},{"type":"v1","addr":"172.21.15.73:6837","nonce":1438858294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5050,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6834/3883540031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3883540031},{"type":"v1","addr":"172.21.15.120:6834","nonce":3883540031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:51:45.273953+0000","modified":"2024-04-26T20:51:47.270309+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5053},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5053":{"gid":5053,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.120:6835/761435580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6833","nonce":761435580},{"type":"v1","addr":"172.21.15.120:6835","nonce":761435580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5053,"qdb_cluster":[5053]},"id":2}]} 2024-04-26T20:51:58.282 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 15 2024-04-26T20:51:58.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.282+0000 7f1dd8025640 1 -- 172.21.15.73:0/1886077869 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1db406c8f0 msgr2=0x7f1db406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:58.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.282+0000 7f1dd8025640 1 --2- 172.21.15.73:0/1886077869 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1db406c8f0 0x7f1db406edb0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1dd015d910 tx=0x7f1dc0008040 comp rx=0 tx=0).stop 2024-04-26T20:51:58.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.282+0000 7f1dd8025640 1 -- 172.21.15.73:0/1886077869 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1dd00cd060 msgr2=0x7f1dd014ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:58.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.282+0000 7f1dd8025640 1 --2- 172.21.15.73:0/1886077869 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1dd00cd060 0x7f1dd014ef10 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc00d7d0 tx=0x7f1dcc00dc90 comp rx=0 tx=0).stop 2024-04-26T20:51:58.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.282+0000 7f1dd8025640 1 -- 172.21.15.73:0/1886077869 shutdown_connections 2024-04-26T20:51:58.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.282+0000 7f1dd8025640 1 --2- 172.21.15.73:0/1886077869 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1db406c8f0 0x7f1db406edb0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:58.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.282+0000 7f1dd8025640 1 --2- 172.21.15.73:0/1886077869 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1dd015f290 0x7f1dd01f88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:58.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.282+0000 7f1dd8025640 1 --2- 172.21.15.73:0/1886077869 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1dd00cd4a0 0x7f1dd009caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:58.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.282+0000 7f1dd8025640 1 --2- 172.21.15.73:0/1886077869 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1dd00cd060 0x7f1dd014ef10 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:58.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.282+0000 7f1dd8025640 1 -- 172.21.15.73:0/1886077869 >> 172.21.15.73:0/1886077869 conn(0x7f1dd00d0fd0 msgr2=0x7f1dd00b0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:58.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.282+0000 7f1dd8025640 1 -- 172.21.15.73:0/1886077869 shutdown_connections 2024-04-26T20:51:58.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.282+0000 7f1dd8025640 1 -- 172.21.15.73:0/1886077869 wait complete. 2024-04-26T20:51:58.294 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2024-04-26T20:51:58.429 INFO:teuthology.orchestra.run.smithi120.stdout:{"message":"","return_code":0} 2024-04-26T20:51:58.429 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-26T20:51:58.430 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 2:0 2024-04-26T20:51:58.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.527+0000 7fa0519af640 1 Processor -- start 2024-04-26T20:51:58.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.527+0000 7fa0519af640 1 -- start start 2024-04-26T20:51:58.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.528+0000 7fa0519af640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fa04c15f3a0 0x7fa04c0d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:58.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.528+0000 7fa0519af640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa04c0d1770 0x7fa04c0d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:58.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.528+0000 7fa0519af640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa04c0d25a0 con 0x7fa04c0d2130 2024-04-26T20:51:58.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.528+0000 7fa0519af640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fa04c0d0e20 con 0x7fa04c15f3a0 2024-04-26T20:51:58.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.528+0000 7fa0519af640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fa04c0b1d70 con 0x7fa04c0d1770 2024-04-26T20:51:58.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.528+0000 7fa04a7fc640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa04c0d1770 0x7fa04c0d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:58.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.528+0000 7fa04affd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fa04c15f3a0 0x7fa04c0d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:58.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.528+0000 7fa04b7fe640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7fa04c0d2130 0x7fa04c0d2e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.73:51694/0 (socket says 172.21.15.73:51694) 2024-04-26T20:51:58.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.528+0000 7fa04b7fe640 1 -- 172.21.15.73:0/4260868341 learned_addr learned my addr 172.21.15.73:0/4260868341 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:58.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.529+0000 7fa04affd640 1 -- 172.21.15.73:0/4260868341 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa04c0d1770 msgr2=0x7fa04c0d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:58.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.529+0000 7fa04affd640 1 --2- 172.21.15.73:0/4260868341 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa04c0d1770 0x7fa04c0d1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:58.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.529+0000 7fa04affd640 1 -- 172.21.15.73:0/4260868341 >> v1:172.21.15.120:6789/0 conn(0x7fa04c0d2130 legacy=0x7fa04c0d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:58.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.529+0000 7fa04affd640 1 -- 172.21.15.73:0/4260868341 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa04c0b2050 con 0x7fa04c15f3a0 2024-04-26T20:51:58.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.529+0000 7fa049ffb640 1 -- 172.21.15.73:0/4260868341 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3855426517 0 0) 0x7fa044001490 con 0x7fa04c0d2130 2024-04-26T20:51:58.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.529+0000 7fa04a7fc640 1 --2- 172.21.15.73:0/4260868341 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa04c0d1770 0x7fa04c0d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:51:58.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.529+0000 7fa04affd640 1 --2- 172.21.15.73:0/4260868341 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa04c15f3a0 0x7fa04c0d06f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa03c0097a0 tx=0x7fa03c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47b7a1bfb2db1004 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:58.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.529+0000 7fa049ffb640 1 -- 172.21.15.73:0/4260868341 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03c037070 con 0x7fa04c15f3a0 2024-04-26T20:51:58.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.529+0000 7fa049ffb640 1 -- 172.21.15.73:0/4260868341 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa03c032440 con 0x7fa04c15f3a0 2024-04-26T20:51:58.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.529+0000 7fa049ffb640 1 -- 172.21.15.73:0/4260868341 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03c032810 con 0x7fa04c15f3a0 2024-04-26T20:51:58.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.530+0000 7fa0519af640 1 -- 172.21.15.73:0/4260868341 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa04c15f3a0 msgr2=0x7fa04c0d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:58.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.530+0000 7fa0519af640 1 --2- 172.21.15.73:0/4260868341 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa04c15f3a0 0x7fa04c0d06f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa03c0097a0 tx=0x7fa03c02a180 comp rx=0 tx=0).stop 2024-04-26T20:51:58.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.530+0000 7fa0519af640 1 -- 172.21.15.73:0/4260868341 shutdown_connections 2024-04-26T20:51:58.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.530+0000 7fa0519af640 1 --2- 172.21.15.73:0/4260868341 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa04c0d1770 0x7fa04c0d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:58.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.530+0000 7fa0519af640 1 --2- 172.21.15.73:0/4260868341 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa04c15f3a0 0x7fa04c0d06f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:58.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.530+0000 7fa0519af640 1 -- 172.21.15.73:0/4260868341 >> 172.21.15.73:0/4260868341 conn(0x7fa04c0d4f90 msgr2=0x7fa04c0d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:58.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.530+0000 7fa0519af640 1 -- 172.21.15.73:0/4260868341 shutdown_connections 2024-04-26T20:51:58.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.530+0000 7fa0519af640 1 -- 172.21.15.73:0/4260868341 wait complete. 2024-04-26T20:51:58.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.531+0000 7fa0519af640 1 Processor -- start 2024-04-26T20:51:58.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.531+0000 7fa0519af640 1 -- start start 2024-04-26T20:51:58.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.532+0000 7fa0519af640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa04c0d1770 0x7fa04c159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:58.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.532+0000 7fa0519af640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa04c0d2130 0x7fa04c0ce9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:58.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.532+0000 7fa0519af640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa04c15f3a0 0x7fa04c160bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:58.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.532+0000 7fa0519af640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa04c15f900 con 0x7fa04c0d2130 2024-04-26T20:51:58.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.532+0000 7fa0519af640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa04c15f780 con 0x7fa04c15f3a0 2024-04-26T20:51:58.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.532+0000 7fa0519af640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fa04c0b3860 con 0x7fa04c0d1770 2024-04-26T20:51:58.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.532+0000 7fa04a7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa04c0d2130 0x7fa04c0ce9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:58.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.532+0000 7fa04a7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa04c0d2130 0x7fa04c0ce9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41218/0 (socket says 172.21.15.73:41218) 2024-04-26T20:51:58.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.532+0000 7fa04a7fc640 1 -- 172.21.15.73:0/735909258 learned_addr learned my addr 172.21.15.73:0/735909258 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:58.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.532+0000 7fa04b7fe640 1 --2- 172.21.15.73:0/735909258 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa04c15f3a0 0x7fa04c160bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:58.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.532+0000 7fa04affd640 1 --2- 172.21.15.73:0/735909258 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa04c0d1770 0x7fa04c159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:58.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.532+0000 7fa04a7fc640 1 -- 172.21.15.73:0/735909258 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa04c0d1770 msgr2=0x7fa04c159890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:58.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.532+0000 7fa04a7fc640 1 --2- 172.21.15.73:0/735909258 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa04c0d1770 0x7fa04c159890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:58.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.532+0000 7fa04a7fc640 1 -- 172.21.15.73:0/735909258 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa04c15f3a0 msgr2=0x7fa04c160bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:58.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.532+0000 7fa04a7fc640 1 --2- 172.21.15.73:0/735909258 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa04c15f3a0 0x7fa04c160bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:58.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.532+0000 7fa04a7fc640 1 -- 172.21.15.73:0/735909258 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa04c0cd900 con 0x7fa04c0d2130 2024-04-26T20:51:58.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.533+0000 7fa04a7fc640 1 --2- 172.21.15.73:0/735909258 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa04c0d2130 0x7fa04c0ce9a0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa04000ebe0 tx=0x7fa04000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:58.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.533+0000 7fa033fff640 1 -- 172.21.15.73:0/735909258 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa040017070 con 0x7fa04c0d2130 2024-04-26T20:51:58.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.533+0000 7fa033fff640 1 -- 172.21.15.73:0/735909258 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa04000cc30 con 0x7fa04c0d2130 2024-04-26T20:51:58.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.533+0000 7fa0519af640 1 -- 172.21.15.73:0/735909258 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa04c0cdbe0 con 0x7fa04c0d2130 2024-04-26T20:51:58.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.533+0000 7fa033fff640 1 -- 172.21.15.73:0/735909258 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa040013470 con 0x7fa04c0d2130 2024-04-26T20:51:58.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.533+0000 7fa0519af640 1 -- 172.21.15.73:0/735909258 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa04c0ce1a0 con 0x7fa04c0d2130 2024-04-26T20:51:58.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.534+0000 7fa0519af640 1 -- 172.21.15.73:0/735909258 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa04c0694f0 con 0x7fa04c0d2130 2024-04-26T20:51:58.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.534+0000 7fa033fff640 1 -- 172.21.15.73:0/735909258 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa04001a070 con 0x7fa04c0d2130 2024-04-26T20:51:58.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.535+0000 7fa033fff640 1 --2- 172.21.15.73:0/735909258 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa03806c920 0x7fa03806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:58.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.535+0000 7fa04affd640 1 --2- 172.21.15.73:0/735909258 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa03806c920 0x7fa03806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:58.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.535+0000 7fa033fff640 1 -- 172.21.15.73:0/735909258 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fa04008be50 con 0x7fa04c0d2130 2024-04-26T20:51:58.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.535+0000 7fa04affd640 1 --2- 172.21.15.73:0/735909258 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa03806c920 0x7fa03806ede0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa03c034000 tx=0x7fa03c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:58.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.539+0000 7fa033fff640 1 -- 172.21.15.73:0/735909258 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa040059b30 con 0x7fa04c0d2130 2024-04-26T20:51:58.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.781+0000 7fa0519af640 1 -- 172.21.15.73:0/735909258 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "2:0"} v 0) -- 0x7fa04c15e9c0 con 0x7fa04c0d2130 2024-04-26T20:51:58.783 INFO:tasks.ceph.mon.a.smithi073.stderr:2024-04-26T20:51:58.782+0000 7f712aaa2640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T20:51:58.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.783+0000 7fa033fff640 1 -- 172.21.15.73:0/735909258 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "2:0"}]=0 failed mds gid 5053 v16) ==== 97+0+0 (secure 0 0 0) 0x7fa04005de30 con 0x7fa04c0d2130 2024-04-26T20:51:58.784 INFO:teuthology.orchestra.run.smithi073.stderr:failed mds gid 5053 2024-04-26T20:51:58.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.787+0000 7fa0519af640 1 -- 172.21.15.73:0/735909258 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa03806c920 msgr2=0x7fa03806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:58.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.787+0000 7fa0519af640 1 --2- 172.21.15.73:0/735909258 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa03806c920 0x7fa03806ede0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa03c034000 tx=0x7fa03c02e000 comp rx=0 tx=0).stop 2024-04-26T20:51:58.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.787+0000 7fa0519af640 1 -- 172.21.15.73:0/735909258 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa04c0d2130 msgr2=0x7fa04c0ce9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:58.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.787+0000 7fa0519af640 1 --2- 172.21.15.73:0/735909258 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa04c0d2130 0x7fa04c0ce9a0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa04000ebe0 tx=0x7fa04000c520 comp rx=0 tx=0).stop 2024-04-26T20:51:58.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.787+0000 7fa0519af640 1 -- 172.21.15.73:0/735909258 shutdown_connections 2024-04-26T20:51:58.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.787+0000 7fa0519af640 1 --2- 172.21.15.73:0/735909258 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa04c0d2130 0x7fa04c0ce9a0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:58.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.787+0000 7fa0519af640 1 --2- 172.21.15.73:0/735909258 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa03806c920 0x7fa03806ede0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:58.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.787+0000 7fa0519af640 1 --2- 172.21.15.73:0/735909258 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa04c15f3a0 0x7fa04c160bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:58.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.787+0000 7fa0519af640 1 --2- 172.21.15.73:0/735909258 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa04c0d1770 0x7fa04c159890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:58.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.787+0000 7fa0519af640 1 -- 172.21.15.73:0/735909258 >> 172.21.15.73:0/735909258 conn(0x7fa04c0d4f90 msgr2=0x7fa04c0d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:58.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.787+0000 7fa0519af640 1 -- 172.21.15.73:0/735909258 shutdown_connections 2024-04-26T20:51:58.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.787+0000 7fa0519af640 1 -- 172.21.15.73:0/735909258 wait complete. 2024-04-26T20:51:58.789 INFO:tasks.ceph.mds.d.smithi120.stderr: -4143> 2024-04-26T20:51:50.310+0000 7f01c8c1b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T20:51:58.800 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:51:58.822 INFO:tasks.ceph.mds.d.smithi120.stdout:starting mds.d at 2024-04-26T20:51:58.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.895+0000 7fbbb68f9640 1 Processor -- start 2024-04-26T20:51:58.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.895+0000 7fbbb68f9640 1 -- start start 2024-04-26T20:51:58.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.896+0000 7fbbb68f9640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbbb0070750 0x7fbbb00b56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:58.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.896+0000 7fbbb68f9640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbbb00af6e0 0x7fbbb00afb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:58.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.896+0000 7fbbb68f9640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbb00b5e30 con 0x7fbbb0072fb0 2024-04-26T20:51:58.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.896+0000 7fbbb68f9640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7fbbb00b5c00 con 0x7fbbb00af6e0 2024-04-26T20:51:58.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.896+0000 7fbbb68f9640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fbbb006f2b0 con 0x7fbbb0070750 2024-04-26T20:51:58.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.896+0000 7fbbaffff640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fbbb0072fb0 0x7fbbb0073390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40578/0 (socket says 172.21.15.73:40578) 2024-04-26T20:51:58.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.896+0000 7fbbaffff640 1 -- 172.21.15.73:0/1166709720 learned_addr learned my addr 172.21.15.73:0/1166709720 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:51:58.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.896+0000 7fbbb4e96640 1 --2- 172.21.15.73:0/1166709720 >> v2:172.21.15.73:3300/0 conn(0x7fbbb00af6e0 0x7fbbb00afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:58.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.897+0000 7fbbaf7fe640 1 --2- 172.21.15.73:0/1166709720 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbbb0070750 0x7fbbb00b56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:58.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.897+0000 7fbbb4e96640 1 -- 172.21.15.73:0/1166709720 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbbb0070750 msgr2=0x7fbbb00b56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:51:58.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.897+0000 7fbbb4e96640 1 --2- 172.21.15.73:0/1166709720 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbbb0070750 0x7fbbb00b56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:58.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.897+0000 7fbbb4e96640 1 -- 172.21.15.73:0/1166709720 >> v1:172.21.15.73:6789/0 conn(0x7fbbb0072fb0 legacy=0x7fbbb0073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:58.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.897+0000 7fbbb4e96640 1 -- 172.21.15.73:0/1166709720 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbb00b6110 con 0x7fbbb00af6e0 2024-04-26T20:51:58.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.897+0000 7fbbb4e96640 1 --2- 172.21.15.73:0/1166709720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbbb00af6e0 0x7fbbb00afb70 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fbba8009d90 tx=0x7fbba802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=403996bb88c8cd2d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:58.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.897+0000 7fbbaeffd640 1 -- 172.21.15.73:0/1166709720 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba8007740 con 0x7fbbb00af6e0 2024-04-26T20:51:58.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.898+0000 7fbbaeffd640 1 -- 172.21.15.73:0/1166709720 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbba80078e0 con 0x7fbbb00af6e0 2024-04-26T20:51:58.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.898+0000 7fbbaeffd640 1 -- 172.21.15.73:0/1166709720 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba8007cb0 con 0x7fbbb00af6e0 2024-04-26T20:51:58.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.898+0000 7fbbb68f9640 1 -- 172.21.15.73:0/1166709720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbbb00af6e0 msgr2=0x7fbbb00afb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:58.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.898+0000 7fbbb68f9640 1 --2- 172.21.15.73:0/1166709720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbbb00af6e0 0x7fbbb00afb70 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fbba8009d90 tx=0x7fbba802a660 comp rx=0 tx=0).stop 2024-04-26T20:51:58.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.898+0000 7fbbb68f9640 1 -- 172.21.15.73:0/1166709720 shutdown_connections 2024-04-26T20:51:58.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.898+0000 7fbbb68f9640 1 --2- 172.21.15.73:0/1166709720 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbbb0070750 0x7fbbb00b56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:58.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.898+0000 7fbbb68f9640 1 --2- 172.21.15.73:0/1166709720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbbb00af6e0 0x7fbbb00afb70 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:58.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.898+0000 7fbbb68f9640 1 -- 172.21.15.73:0/1166709720 >> 172.21.15.73:0/1166709720 conn(0x7fbbb00b39b0 msgr2=0x7fbbb00b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:58.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.898+0000 7fbbb68f9640 1 -- 172.21.15.73:0/1166709720 shutdown_connections 2024-04-26T20:51:58.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.898+0000 7fbbb68f9640 1 -- 172.21.15.73:0/1166709720 wait complete. 2024-04-26T20:51:58.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.899+0000 7fbbb68f9640 1 Processor -- start 2024-04-26T20:51:58.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.899+0000 7fbbb68f9640 1 -- start start 2024-04-26T20:51:58.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.900+0000 7fbbb68f9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbbb0070750 0x7fbbb01f1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:58.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.900+0000 7fbbb68f9640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbbb0072fb0 0x7fbbb0152530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:58.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.900+0000 7fbbb68f9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbb00af6e0 0x7fbbb00d1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:58.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.900+0000 7fbbb68f9640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb015c2e0 con 0x7fbbb0070750 2024-04-26T20:51:58.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.900+0000 7fbbb68f9640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb015c160 con 0x7fbbb00af6e0 2024-04-26T20:51:58.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.900+0000 7fbbb68f9640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fbbb00b00b0 con 0x7fbbb0072fb0 2024-04-26T20:51:58.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.900+0000 7fbbaffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbbb0070750 0x7fbbb01f1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:58.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.900+0000 7fbbaffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbbb0070750 0x7fbbb01f1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41228/0 (socket says 172.21.15.73:41228) 2024-04-26T20:51:58.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.900+0000 7fbbaffff640 1 -- 172.21.15.73:0/3699017587 learned_addr learned my addr 172.21.15.73:0/3699017587 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:51:58.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.900+0000 7fbbaf7fe640 1 --2- 172.21.15.73:0/3699017587 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbbb0072fb0 0x7fbbb0152530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:58.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.900+0000 7fbbaffff640 1 -- 172.21.15.73:0/3699017587 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbbb0072fb0 msgr2=0x7fbbb0152530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:58.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.900+0000 7fbbaffff640 1 --2- 172.21.15.73:0/3699017587 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbbb0072fb0 0x7fbbb0152530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:58.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.900+0000 7fbbb4e96640 1 --2- 172.21.15.73:0/3699017587 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbb00af6e0 0x7fbbb00d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:58.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.900+0000 7fbbaffff640 1 -- 172.21.15.73:0/3699017587 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbb00af6e0 msgr2=0x7fbbb00d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:58.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.900+0000 7fbbaffff640 1 --2- 172.21.15.73:0/3699017587 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbb00af6e0 0x7fbbb00d1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:58.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.900+0000 7fbbaffff640 1 -- 172.21.15.73:0/3699017587 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbb0152a70 con 0x7fbbb0070750 2024-04-26T20:51:58.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.901+0000 7fbbaffff640 1 --2- 172.21.15.73:0/3699017587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbbb0070750 0x7fbbb01f1b10 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fbba000c400 tx=0x7fbba000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:58.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.901+0000 7fbbacff9640 1 -- 172.21.15.73:0/3699017587 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba0017070 con 0x7fbbb0070750 2024-04-26T20:51:58.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.901+0000 7fbbacff9640 1 -- 172.21.15.73:0/3699017587 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbba0005ce0 con 0x7fbbb0070750 2024-04-26T20:51:58.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.901+0000 7fbbb68f9640 1 -- 172.21.15.73:0/3699017587 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbb00d21f0 con 0x7fbbb0070750 2024-04-26T20:51:58.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.901+0000 7fbbacff9640 1 -- 172.21.15.73:0/3699017587 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba00072f0 con 0x7fbbb0070750 2024-04-26T20:51:58.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.901+0000 7fbbb68f9640 1 -- 172.21.15.73:0/3699017587 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbb00cec90 con 0x7fbbb0070750 2024-04-26T20:51:58.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.902+0000 7fbbb68f9640 1 -- 172.21.15.73:0/3699017587 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbb00694f0 con 0x7fbbb0070750 2024-04-26T20:51:58.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.902+0000 7fbbacff9640 1 -- 172.21.15.73:0/3699017587 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbba001a070 con 0x7fbbb0070750 2024-04-26T20:51:58.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.903+0000 7fbbacff9640 1 --2- 172.21.15.73:0/3699017587 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fbb9c06c920 0x7fbb9c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:51:58.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.903+0000 7fbbacff9640 1 -- 172.21.15.73:0/3699017587 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6187+0+0 (secure 0 0 0) 0x7fbba008b1c0 con 0x7fbbb0070750 2024-04-26T20:51:58.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.903+0000 7fbbaf7fe640 1 --2- 172.21.15.73:0/3699017587 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fbb9c06c920 0x7fbb9c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:51:58.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.903+0000 7fbbaf7fe640 1 --2- 172.21.15.73:0/3699017587 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fbb9c06c920 0x7fbb9c06ede0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbbb00d2630 tx=0x7fbba4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:51:58.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:58.907+0000 7fbbacff9640 1 -- 172.21.15.73:0/3699017587 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbba0059380 con 0x7fbbb0070750 2024-04-26T20:51:59.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:59.119+0000 7fbbb68f9640 1 -- 172.21.15.73:0/3699017587 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbbb00b0780 con 0x7fbbb0070750 2024-04-26T20:51:59.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:59.119+0000 7fbbacff9640 1 -- 172.21.15.73:0/3699017587 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+4350 (secure 0 0 0) 0x7fbba0058c80 con 0x7fbbb0070750 2024-04-26T20:51:59.120 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:51:59.120 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4802,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/272662913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":272662913},{"type":"v1","addr":"172.21.15.73:6836","nonce":272662913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5047,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/1438858294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":1438858294},{"type":"v1","addr":"172.21.15.73:6837","nonce":1438858294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:51:45.273953+0000","modified":"2024-04-26T20:51:58.787929+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5050},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5050":{"gid":5050,"name":"b","rank":0,"incarnation":17,"state":"up:replay","state_seq":1,"addr":"172.21.15.120:6834/3883540031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3883540031},{"type":"v1","addr":"172.21.15.120:6834","nonce":3883540031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-04-26T20:51:59.120 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 17 2024-04-26T20:51:59.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:59.121+0000 7fbbb68f9640 1 -- 172.21.15.73:0/3699017587 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fbb9c06c920 msgr2=0x7fbb9c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:59.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:59.121+0000 7fbbb68f9640 1 --2- 172.21.15.73:0/3699017587 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fbb9c06c920 0x7fbb9c06ede0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbbb00d2630 tx=0x7fbba4008040 comp rx=0 tx=0).stop 2024-04-26T20:51:59.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:59.121+0000 7fbbb68f9640 1 -- 172.21.15.73:0/3699017587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbbb0070750 msgr2=0x7fbbb01f1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:51:59.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:59.121+0000 7fbbb68f9640 1 --2- 172.21.15.73:0/3699017587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbbb0070750 0x7fbbb01f1b10 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fbba000c400 tx=0x7fbba000c8c0 comp rx=0 tx=0).stop 2024-04-26T20:51:59.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:59.121+0000 7fbbb68f9640 1 -- 172.21.15.73:0/3699017587 shutdown_connections 2024-04-26T20:51:59.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:59.121+0000 7fbbb68f9640 1 --2- 172.21.15.73:0/3699017587 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fbb9c06c920 0x7fbb9c06ede0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:59.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:59.121+0000 7fbbb68f9640 1 --2- 172.21.15.73:0/3699017587 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbb00af6e0 0x7fbbb00d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:59.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:59.121+0000 7fbbb68f9640 1 --2- 172.21.15.73:0/3699017587 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbbb0072fb0 0x7fbbb0152530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:59.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:59.121+0000 7fbbb68f9640 1 --2- 172.21.15.73:0/3699017587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbbb0070750 0x7fbbb01f1b10 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:51:59.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:59.121+0000 7fbbb68f9640 1 -- 172.21.15.73:0/3699017587 >> 172.21.15.73:0/3699017587 conn(0x7fbbb00b39b0 msgr2=0x7fbbb0073390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:51:59.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:59.121+0000 7fbbb68f9640 1 -- 172.21.15.73:0/3699017587 shutdown_connections 2024-04-26T20:51:59.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:51:59.121+0000 7fbbb68f9640 1 -- 172.21.15.73:0/3699017587 wait complete. 2024-04-26T20:51:59.535 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T20:51:45.273953+0000', 'modified': '2024-04-26T20:51:58.787929+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 40, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5050}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5050': {'gid': 5050, 'name': 'b', 'rank': 0, 'incarnation': 17, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.120:6834/3883540031', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6832', 'nonce': 3883540031}, {'type': 'v1', 'addr': '172.21.15.120:6834', 'nonce': 3883540031}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-26T20:51:59.536 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5050:up:replay 2024-04-26T20:52:00.537 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:52:00.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.645+0000 7fea34647640 1 Processor -- start 2024-04-26T20:52:00.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.645+0000 7fea34647640 1 -- start start 2024-04-26T20:52:00.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.646+0000 7fea34647640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fea2c15f2d0 0x7fea2c1604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:00.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.646+0000 7fea34647640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fea2c072700 0x7fea2c06f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:00.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.646+0000 7fea34647640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea2c072f70 con 0x7fea2c06fcd0 2024-04-26T20:52:00.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.646+0000 7fea34647640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7fea2c074010 con 0x7fea2c15f2d0 2024-04-26T20:52:00.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.646+0000 7fea34647640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fea2c070100 con 0x7fea2c072700 2024-04-26T20:52:00.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.646+0000 7fea323e3640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fea2c15f2d0 0x7fea2c1604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:00.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.646+0000 7fea32be4640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fea2c06fcd0 0x7fea2c0cbb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:39532/0 (socket says 172.21.15.73:39532) 2024-04-26T20:52:00.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.646+0000 7fea323e3640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fea2c15f2d0 0x7fea2c1604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39690/0 (socket says 172.21.15.73:39690) 2024-04-26T20:52:00.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.646+0000 7fea32be4640 1 -- 172.21.15.73:0/2226726112 learned_addr learned my addr 172.21.15.73:0/2226726112 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:52:00.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.646+0000 7fea31be2640 1 --2- 172.21.15.73:0/2226726112 >> v2:172.21.15.120:3300/0 conn(0x7fea2c072700 0x7fea2c06f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:00.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.646+0000 7fea323e3640 1 -- 172.21.15.73:0/2226726112 >> v2:172.21.15.120:3300/0 conn(0x7fea2c072700 msgr2=0x7fea2c06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:00.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.646+0000 7fea323e3640 1 --2- 172.21.15.73:0/2226726112 >> v2:172.21.15.120:3300/0 conn(0x7fea2c072700 0x7fea2c06f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:00.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.646+0000 7fea323e3640 1 -- 172.21.15.73:0/2226726112 >> v1:172.21.15.73:6789/0 conn(0x7fea2c06fcd0 legacy=0x7fea2c0cbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:00.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.646+0000 7fea313e1640 1 -- 172.21.15.73:0/2226726112 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2803001283 0 0) 0x7fea2c072f70 con 0x7fea2c06fcd0 2024-04-26T20:52:00.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.646+0000 7fea323e3640 1 -- 172.21.15.73:0/2226726112 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea2c073250 con 0x7fea2c15f2d0 2024-04-26T20:52:00.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.647+0000 7fea323e3640 1 --2- 172.21.15.73:0/2226726112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea2c15f2d0 0x7fea2c1604e0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fea2c0717a0 tx=0x7fea2802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed0de67da2fa9e12 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:00.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.647+0000 7fea313e1640 1 -- 172.21.15.73:0/2226726112 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea28037070 con 0x7fea2c15f2d0 2024-04-26T20:52:00.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.647+0000 7fea313e1640 1 -- 172.21.15.73:0/2226726112 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea28032440 con 0x7fea2c15f2d0 2024-04-26T20:52:00.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.647+0000 7fea313e1640 1 -- 172.21.15.73:0/2226726112 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea28032810 con 0x7fea2c15f2d0 2024-04-26T20:52:00.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.648+0000 7fea34647640 1 -- 172.21.15.73:0/2226726112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea2c15f2d0 msgr2=0x7fea2c1604e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:00.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.648+0000 7fea34647640 1 --2- 172.21.15.73:0/2226726112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea2c15f2d0 0x7fea2c1604e0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fea2c0717a0 tx=0x7fea2802a1e0 comp rx=0 tx=0).stop 2024-04-26T20:52:00.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.648+0000 7fea34647640 1 -- 172.21.15.73:0/2226726112 shutdown_connections 2024-04-26T20:52:00.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.648+0000 7fea34647640 1 --2- 172.21.15.73:0/2226726112 >> v2:172.21.15.120:3300/0 conn(0x7fea2c072700 0x7fea2c06f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:00.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.648+0000 7fea34647640 1 --2- 172.21.15.73:0/2226726112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea2c15f2d0 0x7fea2c1604e0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:00.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.648+0000 7fea34647640 1 -- 172.21.15.73:0/2226726112 >> 172.21.15.73:0/2226726112 conn(0x7fea2c0d0be0 msgr2=0x7fea2c0d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:00.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.648+0000 7fea34647640 1 -- 172.21.15.73:0/2226726112 shutdown_connections 2024-04-26T20:52:00.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.648+0000 7fea34647640 1 -- 172.21.15.73:0/2226726112 wait complete. 2024-04-26T20:52:00.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.649+0000 7fea34647640 1 Processor -- start 2024-04-26T20:52:00.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.649+0000 7fea34647640 1 -- start start 2024-04-26T20:52:00.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.650+0000 7fea34647640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea2c06fcd0 0x7fea2c1e4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:00.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.650+0000 7fea34647640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea2c072700 0x7fea2c0c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:00.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.650+0000 7fea34647640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fea2c15f2d0 0x7fea2c14f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:00.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.650+0000 7fea34647640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c073af0 con 0x7fea2c06fcd0 2024-04-26T20:52:00.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.650+0000 7fea34647640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c073810 con 0x7fea2c072700 2024-04-26T20:52:00.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.650+0000 7fea34647640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fea2c15f6b0 con 0x7fea2c15f2d0 2024-04-26T20:52:00.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.650+0000 7fea323e3640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea2c06fcd0 0x7fea2c1e4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:00.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.650+0000 7fea323e3640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea2c06fcd0 0x7fea2c1e4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39700/0 (socket says 172.21.15.73:39700) 2024-04-26T20:52:00.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.650+0000 7fea323e3640 1 -- 172.21.15.73:0/2069688512 learned_addr learned my addr 172.21.15.73:0/2069688512 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:00.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.650+0000 7fea323e3640 1 -- 172.21.15.73:0/2069688512 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fea2c15f2d0 msgr2=0x7fea2c14f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:00.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.650+0000 7fea31be2640 1 --2- 172.21.15.73:0/2069688512 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea2c072700 0x7fea2c0c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:00.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.650+0000 7fea323e3640 1 --2- 172.21.15.73:0/2069688512 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fea2c15f2d0 0x7fea2c14f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:00.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.650+0000 7fea323e3640 1 -- 172.21.15.73:0/2069688512 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea2c072700 msgr2=0x7fea2c0c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:00.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.650+0000 7fea323e3640 1 --2- 172.21.15.73:0/2069688512 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea2c072700 0x7fea2c0c7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:00.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.650+0000 7fea323e3640 1 -- 172.21.15.73:0/2069688512 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea2c150110 con 0x7fea2c06fcd0 2024-04-26T20:52:00.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.650+0000 7fea31be2640 1 --2- 172.21.15.73:0/2069688512 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea2c072700 0x7fea2c0c7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:52:00.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.650+0000 7fea323e3640 1 --2- 172.21.15.73:0/2069688512 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea2c06fcd0 0x7fea2c1e4660 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fea28035040 tx=0x7fea280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:00.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.650+0000 7fea22ffd640 1 -- 172.21.15.73:0/2069688512 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea28037040 con 0x7fea2c06fcd0 2024-04-26T20:52:00.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.650+0000 7fea22ffd640 1 -- 172.21.15.73:0/2069688512 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea28030500 con 0x7fea2c06fcd0 2024-04-26T20:52:00.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.650+0000 7fea34647640 1 -- 172.21.15.73:0/2069688512 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea2c1e4c40 con 0x7fea2c06fcd0 2024-04-26T20:52:00.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.651+0000 7fea22ffd640 1 -- 172.21.15.73:0/2069688512 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea28030910 con 0x7fea2c06fcd0 2024-04-26T20:52:00.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.651+0000 7fea34647640 1 -- 172.21.15.73:0/2069688512 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea2c1e5200 con 0x7fea2c06fcd0 2024-04-26T20:52:00.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.651+0000 7fea34647640 1 -- 172.21.15.73:0/2069688512 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea2c0694f0 con 0x7fea2c06fcd0 2024-04-26T20:52:00.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.652+0000 7fea22ffd640 1 -- 172.21.15.73:0/2069688512 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fea28044020 con 0x7fea2c06fcd0 2024-04-26T20:52:00.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.652+0000 7fea22ffd640 1 --2- 172.21.15.73:0/2069688512 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fea1006c8f0 0x7fea1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:00.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.653+0000 7fea22ffd640 1 -- 172.21.15.73:0/2069688512 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6187+0+0 (secure 0 0 0) 0x7fea280abf40 con 0x7fea2c06fcd0 2024-04-26T20:52:00.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.653+0000 7fea31be2640 1 --2- 172.21.15.73:0/2069688512 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fea1006c8f0 0x7fea1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:00.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.653+0000 7fea31be2640 1 --2- 172.21.15.73:0/2069688512 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fea1006c8f0 0x7fea1006edb0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fea2c0c84b0 tx=0x7fea1c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:00.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.657+0000 7fea22ffd640 1 -- 172.21.15.73:0/2069688512 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fea2807a220 con 0x7fea2c06fcd0 2024-04-26T20:52:00.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.869+0000 7fea34647640 1 -- 172.21.15.73:0/2069688512 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea2c0c3b50 con 0x7fea2c06fcd0 2024-04-26T20:52:00.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.869+0000 7fea22ffd640 1 -- 172.21.15.73:0/2069688512 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+5178 (secure 0 0 0) 0x7fea28079b20 con 0x7fea2c06fcd0 2024-04-26T20:52:00.870 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:52:00.870 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4802,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/272662913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":272662913},{"type":"v1","addr":"172.21.15.73:6836","nonce":272662913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4943,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2624899096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6833","nonce":2624899096},{"type":"v1","addr":"172.21.15.120:6835","nonce":2624899096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5047,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/1438858294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":1438858294},{"type":"v1","addr":"172.21.15.73:6837","nonce":1438858294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:51:45.273953+0000","modified":"2024-04-26T20:51:59.793451+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5050},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5050":{"gid":5050,"name":"b","rank":0,"incarnation":17,"state":"up:rejoin","state_seq":8,"addr":"172.21.15.120:6834/3883540031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3883540031},{"type":"v1","addr":"172.21.15.120:6834","nonce":3883540031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-04-26T20:52:00.871 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 19 2024-04-26T20:52:00.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.871+0000 7fea34647640 1 -- 172.21.15.73:0/2069688512 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fea1006c8f0 msgr2=0x7fea1006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:00.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.871+0000 7fea34647640 1 --2- 172.21.15.73:0/2069688512 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fea1006c8f0 0x7fea1006edb0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fea2c0c84b0 tx=0x7fea1c008040 comp rx=0 tx=0).stop 2024-04-26T20:52:00.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.871+0000 7fea34647640 1 -- 172.21.15.73:0/2069688512 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea2c06fcd0 msgr2=0x7fea2c1e4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:00.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.871+0000 7fea34647640 1 --2- 172.21.15.73:0/2069688512 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea2c06fcd0 0x7fea2c1e4660 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fea28035040 tx=0x7fea280046c0 comp rx=0 tx=0).stop 2024-04-26T20:52:00.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.871+0000 7fea34647640 1 -- 172.21.15.73:0/2069688512 shutdown_connections 2024-04-26T20:52:00.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.871+0000 7fea34647640 1 --2- 172.21.15.73:0/2069688512 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fea1006c8f0 0x7fea1006edb0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:00.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.871+0000 7fea34647640 1 --2- 172.21.15.73:0/2069688512 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fea2c15f2d0 0x7fea2c14f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:00.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.871+0000 7fea34647640 1 --2- 172.21.15.73:0/2069688512 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea2c072700 0x7fea2c0c7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:00.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.871+0000 7fea34647640 1 --2- 172.21.15.73:0/2069688512 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea2c06fcd0 0x7fea2c1e4660 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:00.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.871+0000 7fea34647640 1 -- 172.21.15.73:0/2069688512 >> 172.21.15.73:0/2069688512 conn(0x7fea2c0d0be0 msgr2=0x7fea2c0b61d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:00.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.871+0000 7fea34647640 1 -- 172.21.15.73:0/2069688512 shutdown_connections 2024-04-26T20:52:00.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:00.871+0000 7fea34647640 1 -- 172.21.15.73:0/2069688512 wait complete. 2024-04-26T20:52:01.285 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 19, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T20:51:45.273953+0000', 'modified': '2024-04-26T20:51:59.793451+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 40, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5050}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5050': {'gid': 5050, 'name': 'b', 'rank': 0, 'incarnation': 17, 'state': 'up:rejoin', 'state_seq': 8, 'addr': '172.21.15.120:6834/3883540031', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6832', 'nonce': 3883540031}, {'type': 'v1', 'addr': '172.21.15.120:6834', 'nonce': 3883540031}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-26T20:52:01.286 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5050:up:rejoin 2024-04-26T20:52:02.287 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:52:02.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.396+0000 7f894e351640 1 Processor -- start 2024-04-26T20:52:02.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.396+0000 7f894e351640 1 -- start start 2024-04-26T20:52:02.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.397+0000 7f894e351640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f894815b600 0x7f8948071f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:02.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.397+0000 7f894e351640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f89480cf5e0 0x7f89480ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:02.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.397+0000 7f894e351640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8948072450 con 0x7f89480cbbd0 2024-04-26T20:52:02.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.397+0000 7f894e351640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f89480cb9b0 con 0x7f89480cf5e0 2024-04-26T20:52:02.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.397+0000 7f894e351640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f894815b9e0 con 0x7f894815b600 2024-04-26T20:52:02.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.397+0000 7f894c8ee640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f89480cbbd0 0x7f894806f370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:39542/0 (socket says 172.21.15.73:39542) 2024-04-26T20:52:02.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.397+0000 7f894c8ee640 1 -- 172.21.15.73:0/1749226167 learned_addr learned my addr 172.21.15.73:0/1749226167 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:52:02.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.398+0000 7f8946ffd640 1 -- 172.21.15.73:0/1749226167 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1972501781 0 0) 0x7f8940001490 con 0x7f89480cbbd0 2024-04-26T20:52:02.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.398+0000 7f89477fe640 1 --2- 172.21.15.73:0/1749226167 >> v2:172.21.15.120:3300/0 conn(0x7f89480cf5e0 0x7f89480ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:02.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.398+0000 7f8947fff640 1 --2- 172.21.15.73:0/1749226167 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f894815b600 0x7f8948071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:02.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.398+0000 7f8946ffd640 1 -- 172.21.15.73:0/1749226167 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8930005730 con 0x7f89480cbbd0 2024-04-26T20:52:02.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.398+0000 7f8946ffd640 1 -- 172.21.15.73:0/1749226167 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1422227451 0 0) 0x7f8930005730 con 0x7f89480cbbd0 2024-04-26T20:52:02.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.398+0000 7f8946ffd640 1 -- 172.21.15.73:0/1749226167 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f894815b600 msgr2=0x7f8948071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:02.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.398+0000 7f8946ffd640 1 --2- 172.21.15.73:0/1749226167 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f894815b600 0x7f8948071f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:02.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.398+0000 7f8946ffd640 1 -- 172.21.15.73:0/1749226167 >> v2:172.21.15.120:3300/0 conn(0x7f89480cf5e0 msgr2=0x7f89480ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:02.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.398+0000 7f8946ffd640 1 --2- 172.21.15.73:0/1749226167 >> v2:172.21.15.120:3300/0 conn(0x7f89480cf5e0 0x7f89480ce9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:02.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.398+0000 7f8946ffd640 1 -- 172.21.15.73:0/1749226167 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89480d0740 con 0x7f89480cbbd0 2024-04-26T20:52:02.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.398+0000 7f8947fff640 1 --2- 172.21.15.73:0/1749226167 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f894815b600 0x7f8948071f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:52:02.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.398+0000 7f8946ffd640 1 -- 172.21.15.73:0/1749226167 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7f8940001bc0 con 0x7f89480cbbd0 2024-04-26T20:52:02.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.398+0000 7f89477fe640 1 --2- 172.21.15.73:0/1749226167 >> v2:172.21.15.120:3300/0 conn(0x7f89480cf5e0 0x7f89480ce9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:52:02.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.398+0000 7f8946ffd640 1 -- 172.21.15.73:0/1749226167 >> v1:172.21.15.73:6789/0 conn(0x7f89480cbbd0 legacy=0x7f894806f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:02.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.398+0000 7f8946ffd640 1 --2- 172.21.15.73:0/1749226167 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f894815b600 0x7f8930005bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:02.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.398+0000 7f8946ffd640 1 --2- 172.21.15.73:0/1749226167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89300061b0 0x7f8930006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:02.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.398+0000 7f8946ffd640 1 --2- 172.21.15.73:0/1749226167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8930006f50 0x7f89300073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:02.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.398+0000 7f8946ffd640 1 -- 172.21.15.73:0/1749226167 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f89480cb9b0 con 0x7f8930006f50 2024-04-26T20:52:02.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.398+0000 7f8946ffd640 1 -- 172.21.15.73:0/1749226167 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f894815b9e0 con 0x7f89300061b0 2024-04-26T20:52:02.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.398+0000 7f8946ffd640 1 -- 172.21.15.73:0/1749226167 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f893000dcb0 con 0x7f894815b600 2024-04-26T20:52:02.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.398+0000 7f894c8ee640 1 --2- 172.21.15.73:0/1749226167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8930006f50 0x7f89300073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:02.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.398+0000 7f8947fff640 1 --2- 172.21.15.73:0/1749226167 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f894815b600 0x7f8930005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:02.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.398+0000 7f89477fe640 1 --2- 172.21.15.73:0/1749226167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89300061b0 0x7f8930006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:02.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.398+0000 7f894c8ee640 1 -- 172.21.15.73:0/1749226167 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f894815b600 msgr2=0x7f8930005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:02.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.398+0000 7f894c8ee640 1 --2- 172.21.15.73:0/1749226167 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f894815b600 0x7f8930005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:02.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.398+0000 7f894c8ee640 1 -- 172.21.15.73:0/1749226167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89300061b0 msgr2=0x7f8930006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:02.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.398+0000 7f894c8ee640 1 --2- 172.21.15.73:0/1749226167 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89300061b0 0x7f8930006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:02.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.398+0000 7f894c8ee640 1 -- 172.21.15.73:0/1749226167 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8930005730 con 0x7f8930006f50 2024-04-26T20:52:02.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.399+0000 7f894c8ee640 1 --2- 172.21.15.73:0/1749226167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8930006f50 0x7f89300073d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f894000aa40 tx=0x7f8930007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=641f60fdffb5b25e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:02.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.399+0000 7f8946ffd640 1 -- 172.21.15.73:0/1749226167 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894000a690 con 0x7f8930006f50 2024-04-26T20:52:02.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.399+0000 7f8946ffd640 1 -- 172.21.15.73:0/1749226167 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f894002f070 con 0x7f8930006f50 2024-04-26T20:52:02.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.399+0000 7f8946ffd640 1 -- 172.21.15.73:0/1749226167 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8940034470 con 0x7f8930006f50 2024-04-26T20:52:02.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.400+0000 7f894e351640 1 -- 172.21.15.73:0/1749226167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8930006f50 msgr2=0x7f89300073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:02.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.400+0000 7f894e351640 1 --2- 172.21.15.73:0/1749226167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8930006f50 0x7f89300073d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f894000aa40 tx=0x7f8930007a40 comp rx=0 tx=0).stop 2024-04-26T20:52:02.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.400+0000 7f894c8ee640 1 -- 172.21.15.73:0/1749226167 reap_dead start 2024-04-26T20:52:02.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.400+0000 7f894e351640 1 -- 172.21.15.73:0/1749226167 shutdown_connections 2024-04-26T20:52:02.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.400+0000 7f894e351640 1 -- 172.21.15.73:0/1749226167 >> 172.21.15.73:0/1749226167 conn(0x7f89480d0be0 msgr2=0x7f89480d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:02.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.400+0000 7f894e351640 1 -- 172.21.15.73:0/1749226167 shutdown_connections 2024-04-26T20:52:02.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.400+0000 7f894e351640 1 -- 172.21.15.73:0/1749226167 wait complete. 2024-04-26T20:52:02.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.401+0000 7f894e351640 1 Processor -- start 2024-04-26T20:52:02.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.401+0000 7f894e351640 1 -- start start 2024-04-26T20:52:02.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.402+0000 7f894e351640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89481e1380 0x7f89481e2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:02.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.402+0000 7f894e351640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89480b62d0 0x7f8948158010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:02.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.402+0000 7f894e351640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f89481e1760 0x7f8948158550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:02.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.402+0000 7f894e351640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f89480d02f0 con 0x7f89481e1380 2024-04-26T20:52:02.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.402+0000 7f894e351640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f89480cb640 con 0x7f89480b62d0 2024-04-26T20:52:02.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.402+0000 7f894e351640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f89480d0470 con 0x7f89481e1760 2024-04-26T20:52:02.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.402+0000 7f8947fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89481e1380 0x7f89481e2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:02.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.402+0000 7f8947fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89481e1380 0x7f89481e2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39718/0 (socket says 172.21.15.73:39718) 2024-04-26T20:52:02.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.402+0000 7f8947fff640 1 -- 172.21.15.73:0/2140627495 learned_addr learned my addr 172.21.15.73:0/2140627495 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:02.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.402+0000 7f8947fff640 1 -- 172.21.15.73:0/2140627495 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f89481e1760 msgr2=0x7f8948158550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:02.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.402+0000 7f8947fff640 1 --2- 172.21.15.73:0/2140627495 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f89481e1760 0x7f8948158550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:02.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.402+0000 7f8947fff640 1 -- 172.21.15.73:0/2140627495 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89480b62d0 msgr2=0x7f8948158010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:02.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.402+0000 7f89477fe640 1 --2- 172.21.15.73:0/2140627495 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89480b62d0 0x7f8948158010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:02.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.402+0000 7f8947fff640 1 --2- 172.21.15.73:0/2140627495 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89480b62d0 0x7f8948158010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:02.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.402+0000 7f8947fff640 1 -- 172.21.15.73:0/2140627495 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89481e2760 con 0x7f89481e1380 2024-04-26T20:52:02.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.403+0000 7f8947fff640 1 --2- 172.21.15.73:0/2140627495 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89481e1380 0x7f89481e2220 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f89380115a0 tx=0x7f8938011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:02.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.403+0000 7f8944ff9640 1 -- 172.21.15.73:0/2140627495 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8938019070 con 0x7f89481e1380 2024-04-26T20:52:02.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.403+0000 7f8944ff9640 1 -- 172.21.15.73:0/2140627495 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8938004220 con 0x7f89481e1380 2024-04-26T20:52:02.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.403+0000 7f894e351640 1 -- 172.21.15.73:0/2140627495 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89481f9750 con 0x7f89481e1380 2024-04-26T20:52:02.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.403+0000 7f8944ff9640 1 -- 172.21.15.73:0/2140627495 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f893800f040 con 0x7f89481e1380 2024-04-26T20:52:02.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.403+0000 7f894e351640 1 -- 172.21.15.73:0/2140627495 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89481f8c70 con 0x7f89481e1380 2024-04-26T20:52:02.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.404+0000 7f894e351640 1 -- 172.21.15.73:0/2140627495 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89480694f0 con 0x7f89481e1380 2024-04-26T20:52:02.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.404+0000 7f8944ff9640 1 -- 172.21.15.73:0/2140627495 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8938015670 con 0x7f89481e1380 2024-04-26T20:52:02.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.405+0000 7f8944ff9640 1 --2- 172.21.15.73:0/2140627495 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f893406c920 0x7f893406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:02.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.405+0000 7f8944ff9640 1 -- 172.21.15.73:0/2140627495 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6187+0+0 (secure 0 0 0) 0x7f893808bf60 con 0x7f89481e1380 2024-04-26T20:52:02.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.405+0000 7f89477fe640 1 --2- 172.21.15.73:0/2140627495 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f893406c920 0x7f893406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:02.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.405+0000 7f89477fe640 1 --2- 172.21.15.73:0/2140627495 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f893406c920 0x7f893406ede0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f89480af970 tx=0x7f893c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:02.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.409+0000 7f8944ff9640 1 -- 172.21.15.73:0/2140627495 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f89481e2760 con 0x7f89481e1380 2024-04-26T20:52:02.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.619+0000 7f894e351640 1 -- 172.21.15.73:0/2140627495 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f89480cef70 con 0x7f89481e1380 2024-04-26T20:52:02.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.620+0000 7f8944ff9640 1 -- 172.21.15.73:0/2140627495 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5185 (secure 0 0 0) 0x7f893805a1b0 con 0x7f89481e1380 2024-04-26T20:52:02.621 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:52:02.621 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4802,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/272662913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":272662913},{"type":"v1","addr":"172.21.15.73:6836","nonce":272662913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4943,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2624899096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6833","nonce":2624899096},{"type":"v1","addr":"172.21.15.120:6835","nonce":2624899096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5047,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/1438858294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":1438858294},{"type":"v1","addr":"172.21.15.73:6837","nonce":1438858294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:51:45.273953+0000","modified":"2024-04-26T20:52:01.795359+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5050},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5050":{"gid":5050,"name":"b","rank":0,"incarnation":17,"state":"up:active","state_seq":9,"addr":"172.21.15.120:6834/3883540031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3883540031},{"type":"v1","addr":"172.21.15.120:6834","nonce":3883540031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5050,"qdb_cluster":[5050]},"id":2}]} 2024-04-26T20:52:02.621 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 20 2024-04-26T20:52:02.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.622+0000 7f894e351640 1 -- 172.21.15.73:0/2140627495 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f893406c920 msgr2=0x7f893406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:02.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.622+0000 7f894e351640 1 --2- 172.21.15.73:0/2140627495 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f893406c920 0x7f893406ede0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f89480af970 tx=0x7f893c008040 comp rx=0 tx=0).stop 2024-04-26T20:52:02.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.622+0000 7f894e351640 1 -- 172.21.15.73:0/2140627495 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89481e1380 msgr2=0x7f89481e2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:02.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.622+0000 7f894e351640 1 --2- 172.21.15.73:0/2140627495 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89481e1380 0x7f89481e2220 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f89380115a0 tx=0x7f8938011a60 comp rx=0 tx=0).stop 2024-04-26T20:52:02.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.622+0000 7f894e351640 1 -- 172.21.15.73:0/2140627495 shutdown_connections 2024-04-26T20:52:02.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.622+0000 7f894e351640 1 --2- 172.21.15.73:0/2140627495 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f893406c920 0x7f893406ede0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:02.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.622+0000 7f894e351640 1 --2- 172.21.15.73:0/2140627495 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f89481e1760 0x7f8948158550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:02.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.622+0000 7f894e351640 1 --2- 172.21.15.73:0/2140627495 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f89480b62d0 0x7f8948158010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:02.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.622+0000 7f894e351640 1 --2- 172.21.15.73:0/2140627495 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89481e1380 0x7f89481e2220 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:02.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.622+0000 7f894e351640 1 -- 172.21.15.73:0/2140627495 >> 172.21.15.73:0/2140627495 conn(0x7f89480d0be0 msgr2=0x7f894815a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:02.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.622+0000 7f894e351640 1 -- 172.21.15.73:0/2140627495 shutdown_connections 2024-04-26T20:52:02.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:02.622+0000 7f894e351640 1 -- 172.21.15.73:0/2140627495 wait complete. 2024-04-26T20:52:03.036 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T20:51:45.273953+0000', 'modified': '2024-04-26T20:52:01.795359+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 40, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5050}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5050': {'gid': 5050, 'name': 'b', 'rank': 0, 'incarnation': 17, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.120:6834/3883540031', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.120:6832', 'nonce': 3883540031}, {'type': 'v1', 'addr': '172.21.15.120:6834', 'nonce': 3883540031}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5050, 'qdb_cluster': [5050]} 2024-04-26T20:52:03.037 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T20:52:03.037 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-26T20:52:03.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.136+0000 7fd26ff51640 1 Processor -- start 2024-04-26T20:52:03.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.136+0000 7fd26ff51640 1 -- start start 2024-04-26T20:52:03.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.137+0000 7fd26ff51640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd2681604e0 0x7fd26806f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:03.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.137+0000 7fd26ff51640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd26806fcd0 0x7fd2680ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:03.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.137+0000 7fd26ff51640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd268160970 con 0x7fd26815f2d0 2024-04-26T20:52:03.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.137+0000 7fd26ff51640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7fd268074010 con 0x7fd2681604e0 2024-04-26T20:52:03.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.137+0000 7fd26ff51640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fd268070100 con 0x7fd26806fcd0 2024-04-26T20:52:03.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.137+0000 7fd26d4ec640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd2681604e0 0x7fd26806f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:03.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.137+0000 7fd26d4ec640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd2681604e0 0x7fd26806f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39724/0 (socket says 172.21.15.73:39724) 2024-04-26T20:52:03.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.137+0000 7fd26d4ec640 1 -- 172.21.15.73:0/1225153321 learned_addr learned my addr 172.21.15.73:0/1225153321 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:03.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.137+0000 7fd26d4ec640 1 -- 172.21.15.73:0/1225153321 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd26806fcd0 msgr2=0x7fd2680ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:03.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.137+0000 7fd26e4ee640 1 --2- 172.21.15.73:0/1225153321 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd26806fcd0 0x7fd2680ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:03.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.137+0000 7fd26d4ec640 1 --2- 172.21.15.73:0/1225153321 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd26806fcd0 0x7fd2680ce7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:03.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.137+0000 7fd26d4ec640 1 -- 172.21.15.73:0/1225153321 >> v1:172.21.15.120:6789/0 conn(0x7fd26815f2d0 legacy=0x7fd268072700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:03.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.137+0000 7fd26d4ec640 1 -- 172.21.15.73:0/1225153321 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd268072f70 con 0x7fd2681604e0 2024-04-26T20:52:03.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.138+0000 7fd26d4ec640 1 --2- 172.21.15.73:0/1225153321 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd2681604e0 0x7fd26806f790 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fd2580098a0 tx=0x7fd25802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93842d5ec76f1d52 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:03.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.138+0000 7fd26cceb640 1 -- 172.21.15.73:0/1225153321 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd258037070 con 0x7fd2681604e0 2024-04-26T20:52:03.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.138+0000 7fd26cceb640 1 -- 172.21.15.73:0/1225153321 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd258032440 con 0x7fd2681604e0 2024-04-26T20:52:03.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.138+0000 7fd26cceb640 1 -- 172.21.15.73:0/1225153321 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd258032810 con 0x7fd2681604e0 2024-04-26T20:52:03.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.138+0000 7fd26ff51640 1 -- 172.21.15.73:0/1225153321 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd2681604e0 msgr2=0x7fd26806f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:03.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.138+0000 7fd26ff51640 1 --2- 172.21.15.73:0/1225153321 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd2681604e0 0x7fd26806f790 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fd2580098a0 tx=0x7fd25802a250 comp rx=0 tx=0).stop 2024-04-26T20:52:03.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.138+0000 7fd26ff51640 1 -- 172.21.15.73:0/1225153321 shutdown_connections 2024-04-26T20:52:03.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.138+0000 7fd26ff51640 1 --2- 172.21.15.73:0/1225153321 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd26806fcd0 0x7fd2680ce7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:03.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.138+0000 7fd26ff51640 1 --2- 172.21.15.73:0/1225153321 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd2681604e0 0x7fd26806f790 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:03.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.138+0000 7fd26ff51640 1 -- 172.21.15.73:0/1225153321 >> 172.21.15.73:0/1225153321 conn(0x7fd2680d0be0 msgr2=0x7fd2680d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:03.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.138+0000 7fd26ff51640 1 -- 172.21.15.73:0/1225153321 shutdown_connections 2024-04-26T20:52:03.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.139+0000 7fd26ff51640 1 -- 172.21.15.73:0/1225153321 wait complete. 2024-04-26T20:52:03.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.140+0000 7fd26ff51640 1 Processor -- start 2024-04-26T20:52:03.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.140+0000 7fd26ff51640 1 -- start start 2024-04-26T20:52:03.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.140+0000 7fd26ff51640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd26806fcd0 0x7fd26806ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:03.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.140+0000 7fd26ff51640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd26815f2d0 0x7fd268153890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:03.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.140+0000 7fd26ff51640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd2681604e0 0x7fd268153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:03.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.140+0000 7fd26ff51640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd268073810 con 0x7fd26815f2d0 2024-04-26T20:52:03.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.140+0000 7fd26ff51640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd268073530 con 0x7fd2681604e0 2024-04-26T20:52:03.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.140+0000 7fd26ff51640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fd268073af0 con 0x7fd26806fcd0 2024-04-26T20:52:03.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.140+0000 7fd26d4ec640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd26815f2d0 0x7fd268153890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:03.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.140+0000 7fd26d4ec640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd26815f2d0 0x7fd268153890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39730/0 (socket says 172.21.15.73:39730) 2024-04-26T20:52:03.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.140+0000 7fd26d4ec640 1 -- 172.21.15.73:0/2784141152 learned_addr learned my addr 172.21.15.73:0/2784141152 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:03.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.141+0000 7fd26e4ee640 1 --2- 172.21.15.73:0/2784141152 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd2681604e0 0x7fd268153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:03.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.141+0000 7fd26dced640 1 --2- 172.21.15.73:0/2784141152 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd26806fcd0 0x7fd26806ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:03.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.141+0000 7fd26d4ec640 1 -- 172.21.15.73:0/2784141152 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd26806fcd0 msgr2=0x7fd26806ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:03.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.141+0000 7fd26d4ec640 1 --2- 172.21.15.73:0/2784141152 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd26806fcd0 0x7fd26806ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:03.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.141+0000 7fd26d4ec640 1 -- 172.21.15.73:0/2784141152 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd2681604e0 msgr2=0x7fd268153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:03.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.141+0000 7fd26d4ec640 1 --2- 172.21.15.73:0/2784141152 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd2681604e0 0x7fd268153dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:03.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.141+0000 7fd26d4ec640 1 -- 172.21.15.73:0/2784141152 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2680c7f00 con 0x7fd26815f2d0 2024-04-26T20:52:03.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.141+0000 7fd26d4ec640 1 --2- 172.21.15.73:0/2784141152 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd26815f2d0 0x7fd268153890 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fd258000c00 tx=0x7fd2580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:03.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.141+0000 7fd25e7fc640 1 -- 172.21.15.73:0/2784141152 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd258037040 con 0x7fd26815f2d0 2024-04-26T20:52:03.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.141+0000 7fd25e7fc640 1 -- 172.21.15.73:0/2784141152 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd258004230 con 0x7fd26815f2d0 2024-04-26T20:52:03.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.141+0000 7fd25e7fc640 1 -- 172.21.15.73:0/2784141152 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd258030620 con 0x7fd26815f2d0 2024-04-26T20:52:03.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.141+0000 7fd26ff51640 1 -- 172.21.15.73:0/2784141152 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2680c81e0 con 0x7fd26815f2d0 2024-04-26T20:52:03.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.141+0000 7fd26ff51640 1 -- 172.21.15.73:0/2784141152 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2680c87a0 con 0x7fd26815f2d0 2024-04-26T20:52:03.152 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.143+0000 7fd25e7fc640 1 -- 172.21.15.73:0/2784141152 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd2580307c0 con 0x7fd26815f2d0 2024-04-26T20:52:03.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.143+0000 7fd25e7fc640 1 --2- 172.21.15.73:0/2784141152 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd24c06c8f0 0x7fd24c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:03.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.143+0000 7fd25e7fc640 1 -- 172.21.15.73:0/2784141152 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6187+0+0 (secure 0 0 0) 0x7fd2580acc10 con 0x7fd26815f2d0 2024-04-26T20:52:03.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.143+0000 7fd26dced640 1 --2- 172.21.15.73:0/2784141152 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd24c06c8f0 0x7fd24c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:03.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.144+0000 7fd26dced640 1 --2- 172.21.15.73:0/2784141152 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd24c06c8f0 0x7fd24c06edb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd264004ea0 tx=0x7fd264009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:03.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.144+0000 7fd272c6b740 1 Processor -- start 2024-04-26T20:52:03.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.144+0000 7fd272c6b740 1 -- start start 2024-04-26T20:52:03.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.145+0000 7fd272c6b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4ea9489f0 0x55b4ea921050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:03.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.145+0000 7fd272c6b740 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55b4ea948dd0 0x55b4ea94b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:03.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.145+0000 7fd272c6b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4ea94b790 0x55b4ea91eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:03.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.145+0000 7fd272c6b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55b4ea885740 con 0x55b4ea9489f0 2024-04-26T20:52:03.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.145+0000 7fd272c6b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b4ea8213a0 con 0x55b4ea94b790 2024-04-26T20:52:03.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.145+0000 7fd272c6b740 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x55b4ea881700 con 0x55b4ea948dd0 2024-04-26T20:52:03.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.145+0000 7fd26dced640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4ea9489f0 0x55b4ea921050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:03.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.145+0000 7fd26dced640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4ea9489f0 0x55b4ea921050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39746/0 (socket says 172.21.15.73:39746) 2024-04-26T20:52:03.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.145+0000 7fd26dced640 1 -- 172.21.15.73:0/1904682454 learned_addr learned my addr 172.21.15.73:0/1904682454 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:03.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.145+0000 7fd26d4ec640 1 --2- 172.21.15.73:0/1904682454 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55b4ea948dd0 0x55b4ea94b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:03.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.145+0000 7fd26e4ee640 1 --2- 172.21.15.73:0/1904682454 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4ea94b790 0x55b4ea91eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:03.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.145+0000 7fd26dced640 1 -- 172.21.15.73:0/1904682454 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55b4ea948dd0 msgr2=0x55b4ea94b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:03.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.145+0000 7fd26dced640 1 --2- 172.21.15.73:0/1904682454 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55b4ea948dd0 0x55b4ea94b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:03.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.145+0000 7fd26dced640 1 -- 172.21.15.73:0/1904682454 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4ea94b790 msgr2=0x55b4ea91eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:03.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.145+0000 7fd26dced640 1 --2- 172.21.15.73:0/1904682454 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4ea94b790 0x55b4ea91eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:03.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.145+0000 7fd26dced640 1 -- 172.21.15.73:0/1904682454 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4ea83d750 con 0x55b4ea9489f0 2024-04-26T20:52:03.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.145+0000 7fd26e4ee640 1 --2- 172.21.15.73:0/1904682454 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4ea94b790 0x55b4ea91eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:52:03.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.145+0000 7fd26d4ec640 1 --2- 172.21.15.73:0/1904682454 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55b4ea948dd0 0x55b4ea94b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:52:03.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.145+0000 7fd26dced640 1 --2- 172.21.15.73:0/1904682454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4ea9489f0 0x55b4ea921050 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd26400e810 tx=0x7fd264011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f80de0e0ce40ad9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:03.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.145+0000 7fd24b7fe640 1 -- 172.21.15.73:0/1904682454 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd264025070 con 0x55b4ea9489f0 2024-04-26T20:52:03.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.146+0000 7fd24b7fe640 1 -- 172.21.15.73:0/1904682454 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2640159b0 con 0x55b4ea9489f0 2024-04-26T20:52:03.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.146+0000 7fd24b7fe640 1 -- 172.21.15.73:0/1904682454 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd264015d60 con 0x55b4ea9489f0 2024-04-26T20:52:03.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.146+0000 7fd272c6b740 1 -- 172.21.15.73:0/1904682454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4ea9489f0 msgr2=0x55b4ea921050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:03.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.146+0000 7fd272c6b740 1 --2- 172.21.15.73:0/1904682454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4ea9489f0 0x55b4ea921050 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd26400e810 tx=0x7fd264011000 comp rx=0 tx=0).stop 2024-04-26T20:52:03.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.146+0000 7fd272c6b740 1 -- 172.21.15.73:0/1904682454 shutdown_connections 2024-04-26T20:52:03.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.146+0000 7fd272c6b740 1 --2- 172.21.15.73:0/1904682454 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4ea94b790 0x55b4ea91eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:03.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.146+0000 7fd272c6b740 1 --2- 172.21.15.73:0/1904682454 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55b4ea948dd0 0x55b4ea94b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:03.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.146+0000 7fd272c6b740 1 --2- 172.21.15.73:0/1904682454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4ea9489f0 0x55b4ea921050 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:03.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.146+0000 7fd272c6b740 1 -- 172.21.15.73:0/1904682454 >> 172.21.15.73:0/1904682454 conn(0x55b4ea91da50 msgr2=0x55b4ea966250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:03.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.146+0000 7fd272c6b740 1 -- 172.21.15.73:0/1904682454 shutdown_connections 2024-04-26T20:52:03.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.146+0000 7fd272c6b740 1 -- 172.21.15.73:0/1904682454 wait complete. 2024-04-26T20:52:03.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.147+0000 7fd272c6b740 1 -- start start 2024-04-26T20:52:03.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.147+0000 7fd272c6b740 1 Processor -- start 2024-04-26T20:52:03.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.148+0000 7fd272c6b740 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55b4ea948dd0 0x55b4ea932b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:03.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.148+0000 7fd272c6b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4ea94b790 0x55b4ea95d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:03.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.148+0000 7fd272c6b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4ea94ccd0 0x55b4ea95d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:03.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.148+0000 7fd272c6b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55b4ea80cfe0 con 0x55b4ea94ccd0 2024-04-26T20:52:03.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.148+0000 7fd272c6b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b4ea963f50 con 0x55b4ea94b790 2024-04-26T20:52:03.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.148+0000 7fd272c6b740 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x55b4ea94d160 con 0x55b4ea948dd0 2024-04-26T20:52:03.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.148+0000 7fd272c6b740 10 client.0 _ll_register_callbacks cb 0x55b4ea823150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:52:03.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.148+0000 7fd26e4ee640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4ea94ccd0 0x55b4ea95d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:03.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.148+0000 7fd26e4ee640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4ea94ccd0 0x55b4ea95d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39758/0 (socket says 172.21.15.73:39758) 2024-04-26T20:52:03.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.148+0000 7fd26e4ee640 1 -- 172.21.15.73:0/3817889494 learned_addr learned my addr 172.21.15.73:0/3817889494 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:03.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.148+0000 7fd26e4ee640 1 -- 172.21.15.73:0/3817889494 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55b4ea948dd0 msgr2=0x55b4ea932b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:03.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.148+0000 7fd26dced640 1 --2- 172.21.15.73:0/3817889494 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55b4ea948dd0 0x55b4ea932b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:03.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.148+0000 7fd26d4ec640 1 --2- 172.21.15.73:0/3817889494 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4ea94b790 0x55b4ea95d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:03.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.148+0000 7fd26e4ee640 1 --2- 172.21.15.73:0/3817889494 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55b4ea948dd0 0x55b4ea932b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:03.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.148+0000 7fd26e4ee640 1 -- 172.21.15.73:0/3817889494 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4ea94b790 msgr2=0x55b4ea95d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:03.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.148+0000 7fd26e4ee640 1 --2- 172.21.15.73:0/3817889494 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4ea94b790 0x55b4ea95d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:03.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.148+0000 7fd26e4ee640 1 -- 172.21.15.73:0/3817889494 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4ea5972d0 con 0x55b4ea94ccd0 2024-04-26T20:52:03.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.148+0000 7fd26d4ec640 1 --2- 172.21.15.73:0/3817889494 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4ea94b790 0x55b4ea95d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:52:03.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.148+0000 7fd26e4ee640 1 --2- 172.21.15.73:0/3817889494 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4ea94ccd0 0x55b4ea95d800 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fd260010bd0 tx=0x7fd26000e760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=261edd24d0164473 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:03.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.148+0000 7fd248ff9640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-04-26T20:52:03.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.148+0000 7fd248ff9640 1 -- 172.21.15.73:0/3817889494 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd26001d060 con 0x55b4ea94ccd0 2024-04-26T20:52:03.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.148+0000 7fd248ff9640 1 -- 172.21.15.73:0/3817889494 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(40..40 src has 1..40) ==== 6187+0+0 (secure 0 0 0) 0x7fd26000a790 con 0x55b4ea94ccd0 2024-04-26T20:52:03.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.148+0000 7fd272c6b740 1 -- 172.21.15.73:0/3817889494 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2640159b0 con 0x55b4ea94ccd0 2024-04-26T20:52:03.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.149+0000 7fd248ff9640 1 -- 172.21.15.73:0/3817889494 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fd260023050 con 0x55b4ea94ccd0 2024-04-26T20:52:03.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.149+0000 7fd272c6b740 10 client.5649 fetch_fsmap learned FSMap version 20 2024-04-26T20:52:03.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.149+0000 7fd272c6b740 1 -- 172.21.15.73:0/3817889494 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x55b4ea6616b0 con 0x55b4ea94ccd0 2024-04-26T20:52:03.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.149+0000 7fd248ff9640 1 -- 172.21.15.73:0/3817889494 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2640159b0 con 0x55b4ea94ccd0 2024-04-26T20:52:03.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.149+0000 7fd248ff9640 1 -- 172.21.15.73:0/3817889494 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7fd260026020 con 0x55b4ea94ccd0 2024-04-26T20:52:03.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.149+0000 7fd272c6b740 10 client.5649 fetch_fsmap finished waiting for FSMap version 20 2024-04-26T20:52:03.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.149+0000 7fd272c6b740 10 client.5649 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T20:52:03.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.149+0000 7fd272c6b740 20 client.5649 populate_metadata read hostname 'smithi073' 2024-04-26T20:52:03.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.149+0000 7fd272c6b740 1 --2- 172.21.15.73:0/3817889494 >> [v2:172.21.15.120:6832/3883540031,v1:172.21.15.120:6834/3883540031] conn(0x55b4ea964670 0x55b4ea94e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:03.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.149+0000 7fd272c6b740 4 client.5649 mds_command: new command op to 5050 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:52:03.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.149+0000 7fd272c6b740 1 -- 172.21.15.73:0/3817889494 --> [v2:172.21.15.120:6832/3883540031,v1:172.21.15.120:6834/3883540031] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4ea893350 con 0x55b4ea964670 2024-04-26T20:52:03.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.149+0000 7fd26dced640 1 --2- 172.21.15.73:0/3817889494 >> [v2:172.21.15.120:6832/3883540031,v1:172.21.15.120:6834/3883540031] conn(0x55b4ea964670 0x55b4ea94e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:03.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.149+0000 7fd26dced640 1 --2- 172.21.15.73:0/3817889494 >> [v2:172.21.15.120:6832/3883540031,v1:172.21.15.120:6834/3883540031] conn(0x55b4ea964670 0x55b4ea94e750 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0d3dd6b278f1836 server_cookie=363204ccc2eb0526 in_seq=0 out_seq=0 2024-04-26T20:52:03.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.149+0000 7fd248ff9640 10 client.5649 ms_handle_connect on v2:172.21.15.120:6832/3883540031 2024-04-26T20:52:03.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.150+0000 7fd248ff9640 1 -- 172.21.15.73:0/3817889494 <== mds.0 v2:172.21.15.120:6832/3883540031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b4ea893350 con 0x55b4ea964670 2024-04-26T20:52:03.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.150+0000 7fd248ff9640 10 client.5649 handle_command_reply: tid=0 2024-04-26T20:52:03.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.150+0000 7fd272c6b740 1 client.5649 shutdown 2024-04-26T20:52:03.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.150+0000 7fd272c6b740 1 -- 172.21.15.73:0/3817889494 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4ea94ccd0 msgr2=0x55b4ea95d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:03.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.150+0000 7fd272c6b740 1 --2- 172.21.15.73:0/3817889494 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4ea94ccd0 0x55b4ea95d800 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fd260010bd0 tx=0x7fd26000e760 comp rx=0 tx=0).stop 2024-04-26T20:52:03.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.150+0000 7fd272c6b740 1 -- 172.21.15.73:0/3817889494 shutdown_connections 2024-04-26T20:52:03.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.150+0000 7fd272c6b740 1 --2- 172.21.15.73:0/3817889494 >> [v2:172.21.15.120:6832/3883540031,v1:172.21.15.120:6834/3883540031] conn(0x55b4ea964670 0x55b4ea94e750 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:03.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.150+0000 7fd272c6b740 1 --2- 172.21.15.73:0/3817889494 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4ea94ccd0 0x55b4ea95d800 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:03.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.150+0000 7fd248ff9640 0 client.5649 ms_handle_reset on v2:172.21.15.120:6832/3883540031 2024-04-26T20:52:03.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.150+0000 7fd272c6b740 1 --2- 172.21.15.73:0/3817889494 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4ea94b790 0x55b4ea95d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:03.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.150+0000 7fd272c6b740 1 --2- 172.21.15.73:0/3817889494 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55b4ea948dd0 0x55b4ea932b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:03.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.150+0000 7fd272c6b740 1 -- 172.21.15.73:0/3817889494 >> 172.21.15.73:0/3817889494 conn(0x55b4ea9489f0 msgr2=0x55b4ea91eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:03.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.151+0000 7fd272c6b740 1 -- 172.21.15.73:0/3817889494 shutdown_connections 2024-04-26T20:52:03.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.151+0000 7fd272c6b740 1 -- 172.21.15.73:0/3817889494 wait complete. 2024-04-26T20:52:03.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.151+0000 7fd272c6b740 20 client.5649 trim_cache size 0 max 16384 2024-04-26T20:52:03.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.169+0000 7fd272c6b740 1 Processor -- start 2024-04-26T20:52:03.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.169+0000 7fd272c6b740 1 -- start start 2024-04-26T20:52:03.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.169+0000 7fd272c6b740 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55b4ea948dd0 0x55b4ea9dbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:03.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.169+0000 7fd272c6b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4ea94b790 0x55b4ea95d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:03.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.169+0000 7fd272c6b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4ea94ccd0 0x55b4ea95d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:03.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.169+0000 7fd272c6b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd26815df30 con 0x55b4ea94ccd0 2024-04-26T20:52:03.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.169+0000 7fd272c6b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd2680b5ee0 con 0x55b4ea94b790 2024-04-26T20:52:03.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.169+0000 7fd272c6b740 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fd2680b6530 con 0x55b4ea948dd0 2024-04-26T20:52:03.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.169+0000 7fd26e4ee640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4ea94ccd0 0x55b4ea95d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:03.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.169+0000 7fd26e4ee640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4ea94ccd0 0x55b4ea95d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39766/0 (socket says 172.21.15.73:39766) 2024-04-26T20:52:03.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.169+0000 7fd26e4ee640 1 -- 172.21.15.73:0/2473124871 learned_addr learned my addr 172.21.15.73:0/2473124871 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:03.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.170+0000 7fd26dced640 1 --2- 172.21.15.73:0/2473124871 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55b4ea948dd0 0x55b4ea9dbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:03.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.170+0000 7fd26e4ee640 1 -- 172.21.15.73:0/2473124871 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55b4ea948dd0 msgr2=0x55b4ea9dbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:03.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.170+0000 7fd26d4ec640 1 --2- 172.21.15.73:0/2473124871 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4ea94b790 0x55b4ea95d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:03.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.170+0000 7fd26e4ee640 1 --2- 172.21.15.73:0/2473124871 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55b4ea948dd0 0x55b4ea9dbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:03.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.170+0000 7fd26e4ee640 1 -- 172.21.15.73:0/2473124871 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4ea94b790 msgr2=0x55b4ea95d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:03.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.170+0000 7fd26e4ee640 1 --2- 172.21.15.73:0/2473124871 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4ea94b790 0x55b4ea95d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:03.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.170+0000 7fd26e4ee640 1 -- 172.21.15.73:0/2473124871 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4ea9485c0 con 0x55b4ea94ccd0 2024-04-26T20:52:03.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.170+0000 7fd26dced640 1 --2- 172.21.15.73:0/2473124871 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55b4ea948dd0 0x55b4ea9dbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:52:03.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.170+0000 7fd26d4ec640 1 --2- 172.21.15.73:0/2473124871 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4ea94b790 0x55b4ea95d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:52:03.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.170+0000 7fd26e4ee640 1 --2- 172.21.15.73:0/2473124871 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4ea94ccd0 0x55b4ea95d6d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fd260002230 tx=0x7fd260013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21a87f45f3d60069 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:03.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.170+0000 7fd24affd640 1 -- 172.21.15.73:0/2473124871 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd26000ec60 con 0x55b4ea94ccd0 2024-04-26T20:52:03.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.170+0000 7fd24affd640 1 -- 172.21.15.73:0/2473124871 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd26000a590 con 0x55b4ea94ccd0 2024-04-26T20:52:03.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.170+0000 7fd24affd640 1 -- 172.21.15.73:0/2473124871 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd260010030 con 0x55b4ea94ccd0 2024-04-26T20:52:03.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.170+0000 7fd272c6b740 1 -- 172.21.15.73:0/2473124871 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4ea94ccd0 msgr2=0x55b4ea95d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:03.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.170+0000 7fd272c6b740 1 --2- 172.21.15.73:0/2473124871 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4ea94ccd0 0x55b4ea95d6d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fd260002230 tx=0x7fd260013000 comp rx=0 tx=0).stop 2024-04-26T20:52:03.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.171+0000 7fd272c6b740 1 -- 172.21.15.73:0/2473124871 shutdown_connections 2024-04-26T20:52:03.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.171+0000 7fd272c6b740 1 --2- 172.21.15.73:0/2473124871 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4ea94ccd0 0x55b4ea95d6d0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:03.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.171+0000 7fd272c6b740 1 --2- 172.21.15.73:0/2473124871 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4ea94b790 0x55b4ea95d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:03.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.171+0000 7fd272c6b740 1 --2- 172.21.15.73:0/2473124871 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55b4ea948dd0 0x55b4ea9dbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:03.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.171+0000 7fd272c6b740 1 -- 172.21.15.73:0/2473124871 >> 172.21.15.73:0/2473124871 conn(0x55b4ea9489f0 msgr2=0x55b4ea932970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:03.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.171+0000 7fd272c6b740 1 -- 172.21.15.73:0/2473124871 shutdown_connections 2024-04-26T20:52:03.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.171+0000 7fd272c6b740 1 -- 172.21.15.73:0/2473124871 wait complete. 2024-04-26T20:52:03.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.171+0000 7fd272c6b740 1 -- start start 2024-04-26T20:52:03.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.171+0000 7fd272c6b740 1 Processor -- start 2024-04-26T20:52:03.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.172+0000 7fd272c6b740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4ea948dd0 0x55b4ea9d8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:03.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.172+0000 7fd272c6b740 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55b4ea94b790 0x55b4ea9dc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:03.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.172+0000 7fd272c6b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4ea94ccd0 0x55b4ea920c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:03.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.172+0000 7fd272c6b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55b4ea9644f0 con 0x55b4ea94ccd0 2024-04-26T20:52:03.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.172+0000 7fd272c6b740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55b4ea94df90 con 0x55b4ea948dd0 2024-04-26T20:52:03.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.172+0000 7fd272c6b740 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x55b4ea964b40 con 0x55b4ea94b790 2024-04-26T20:52:03.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.172+0000 7fd272c6b740 10 client.0 _ll_register_callbacks cb 0x7fd260007770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:52:03.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.172+0000 7fd26e4ee640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4ea94ccd0 0x55b4ea920c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:03.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.172+0000 7fd26e4ee640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4ea94ccd0 0x55b4ea920c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39780/0 (socket says 172.21.15.73:39780) 2024-04-26T20:52:03.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.172+0000 7fd26e4ee640 1 -- 172.21.15.73:0/1007801928 learned_addr learned my addr 172.21.15.73:0/1007801928 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:03.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.172+0000 7fd26d4ec640 1 --2- 172.21.15.73:0/1007801928 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55b4ea94b790 0x55b4ea9dc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:03.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.172+0000 7fd26dced640 1 --2- 172.21.15.73:0/1007801928 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4ea948dd0 0x55b4ea9d8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:03.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.172+0000 7fd26e4ee640 1 -- 172.21.15.73:0/1007801928 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55b4ea94b790 msgr2=0x55b4ea9dc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:03.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.172+0000 7fd26e4ee640 1 --2- 172.21.15.73:0/1007801928 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55b4ea94b790 0x55b4ea9dc790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:03.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.172+0000 7fd26e4ee640 1 -- 172.21.15.73:0/1007801928 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4ea948dd0 msgr2=0x55b4ea9d8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:03.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.172+0000 7fd26e4ee640 1 --2- 172.21.15.73:0/1007801928 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4ea948dd0 0x55b4ea9d8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:03.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.172+0000 7fd26e4ee640 1 -- 172.21.15.73:0/1007801928 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4ea91b870 con 0x55b4ea94ccd0 2024-04-26T20:52:03.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.172+0000 7fd26d4ec640 1 --2- 172.21.15.73:0/1007801928 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55b4ea94b790 0x55b4ea9dc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:52:03.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.172+0000 7fd26dced640 1 --2- 172.21.15.73:0/1007801928 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4ea948dd0 0x55b4ea9d8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:52:03.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.173+0000 7fd26e4ee640 1 --2- 172.21.15.73:0/1007801928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4ea94ccd0 0x55b4ea920c60 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fd26000aa80 tx=0x7fd260004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c10b21527f482fda server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:03.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.173+0000 7fd248ff9640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-04-26T20:52:03.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.173+0000 7fd248ff9640 1 -- 172.21.15.73:0/1007801928 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd260010030 con 0x55b4ea94ccd0 2024-04-26T20:52:03.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.173+0000 7fd248ff9640 1 -- 172.21.15.73:0/1007801928 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(40..40 src has 1..40) ==== 6187+0+0 (secure 0 0 0) 0x7fd2600101d0 con 0x55b4ea94ccd0 2024-04-26T20:52:03.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.173+0000 7fd272c6b740 1 -- 172.21.15.73:0/1007801928 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd26000a590 con 0x55b4ea94ccd0 2024-04-26T20:52:03.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.173+0000 7fd248ff9640 1 -- 172.21.15.73:0/1007801928 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fd260041050 con 0x55b4ea94ccd0 2024-04-26T20:52:03.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.173+0000 7fd272c6b740 10 client.5655 fetch_fsmap learned FSMap version 20 2024-04-26T20:52:03.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.173+0000 7fd272c6b740 1 -- 172.21.15.73:0/1007801928 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x55b4ea920460 con 0x55b4ea94ccd0 2024-04-26T20:52:03.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.173+0000 7fd248ff9640 1 -- 172.21.15.73:0/1007801928 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd26000a590 con 0x55b4ea94ccd0 2024-04-26T20:52:03.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.173+0000 7fd248ff9640 1 -- 172.21.15.73:0/1007801928 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7fd260044020 con 0x55b4ea94ccd0 2024-04-26T20:52:03.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.173+0000 7fd272c6b740 10 client.5655 fetch_fsmap finished waiting for FSMap version 20 2024-04-26T20:52:03.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.173+0000 7fd272c6b740 10 client.5655 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T20:52:03.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.173+0000 7fd272c6b740 20 client.5655 populate_metadata read hostname 'smithi073' 2024-04-26T20:52:03.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.173+0000 7fd272c6b740 1 --2- 172.21.15.73:0/1007801928 >> [v2:172.21.15.120:6832/3883540031,v1:172.21.15.120:6834/3883540031] conn(0x55b4ea95cb30 0x55b4ea95cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:03.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.173+0000 7fd272c6b740 4 client.5655 mds_command: new command op to 5050 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T20:52:03.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.173+0000 7fd272c6b740 1 -- 172.21.15.73:0/1007801928 --> [v2:172.21.15.120:6832/3883540031,v1:172.21.15.120:6834/3883540031] -- command(tid 0: {"prefix": "status"}) -- 0x55b4ea88d1c0 con 0x55b4ea95cb30 2024-04-26T20:52:03.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.173+0000 7fd26dced640 1 --2- 172.21.15.73:0/1007801928 >> [v2:172.21.15.120:6832/3883540031,v1:172.21.15.120:6834/3883540031] conn(0x55b4ea95cb30 0x55b4ea95cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:03.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.174+0000 7fd248ff9640 10 client.5655 ms_handle_connect on v2:172.21.15.120:6832/3883540031 2024-04-26T20:52:03.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.174+0000 7fd26dced640 1 --2- 172.21.15.73:0/1007801928 >> [v2:172.21.15.120:6832/3883540031,v1:172.21.15.120:6834/3883540031] conn(0x55b4ea95cb30 0x55b4ea95cf10 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a80cf60cb5aa3eae server_cookie=23d5c2e9be887c0d in_seq=0 out_seq=0 2024-04-26T20:52:03.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.174+0000 7fd248ff9640 1 -- 172.21.15.73:0/1007801928 <== mds.0 v2:172.21.15.120:6832/3883540031 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55b4ea88d1c0 con 0x55b4ea95cb30 2024-04-26T20:52:03.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.174+0000 7fd248ff9640 10 client.5655 handle_command_reply: tid=0 2024-04-26T20:52:03.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.174+0000 7fd272c6b740 1 client.5655 shutdown 2024-04-26T20:52:03.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.174+0000 7fd272c6b740 1 -- 172.21.15.73:0/1007801928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4ea94ccd0 msgr2=0x55b4ea920c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:03.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.174+0000 7fd272c6b740 1 --2- 172.21.15.73:0/1007801928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4ea94ccd0 0x55b4ea920c60 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fd26000aa80 tx=0x7fd260004a40 comp rx=0 tx=0).stop 2024-04-26T20:52:03.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.174+0000 7fd272c6b740 1 -- 172.21.15.73:0/1007801928 shutdown_connections 2024-04-26T20:52:03.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.174+0000 7fd272c6b740 1 --2- 172.21.15.73:0/1007801928 >> [v2:172.21.15.120:6832/3883540031,v1:172.21.15.120:6834/3883540031] conn(0x55b4ea95cb30 0x55b4ea95cf10 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:03.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.174+0000 7fd272c6b740 1 --2- 172.21.15.73:0/1007801928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b4ea94ccd0 0x55b4ea920c60 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:03.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.174+0000 7fd248ff9640 0 client.5655 ms_handle_reset on v2:172.21.15.120:6832/3883540031 2024-04-26T20:52:03.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.174+0000 7fd272c6b740 1 --2- 172.21.15.73:0/1007801928 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55b4ea94b790 0x55b4ea9dc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:03.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.174+0000 7fd272c6b740 1 --2- 172.21.15.73:0/1007801928 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55b4ea948dd0 0x55b4ea9d8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:03.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.174+0000 7fd272c6b740 1 -- 172.21.15.73:0/1007801928 >> 172.21.15.73:0/1007801928 conn(0x55b4ea9489f0 msgr2=0x55b4ea965780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:03.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.174+0000 7fd272c6b740 1 -- 172.21.15.73:0/1007801928 shutdown_connections 2024-04-26T20:52:03.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.174+0000 7fd272c6b740 1 -- 172.21.15.73:0/1007801928 wait complete. 2024-04-26T20:52:03.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.174+0000 7fd272c6b740 20 client.5655 trim_cache size 0 max 16384 2024-04-26T20:52:03.183 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-26T20:52:03.183 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_fsid": "fbd3f548-b547-4713-856b-09f98f5d1764", 2024-04-26T20:52:03.183 INFO:teuthology.orchestra.run.smithi073.stdout: "whoami": 0, 2024-04-26T20:52:03.183 INFO:teuthology.orchestra.run.smithi073.stdout: "id": 5050, 2024-04-26T20:52:03.183 INFO:teuthology.orchestra.run.smithi073.stdout: "want_state": "up:active", 2024-04-26T20:52:03.183 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "up:active", 2024-04-26T20:52:03.183 INFO:teuthology.orchestra.run.smithi073.stdout: "fs_name": "cephfs", 2024-04-26T20:52:03.184 INFO:teuthology.orchestra.run.smithi073.stdout: "rank_uptime": 4.3852288760000002, 2024-04-26T20:52:03.184 INFO:teuthology.orchestra.run.smithi073.stdout: "mdsmap_epoch": 20, 2024-04-26T20:52:03.184 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch": 40, 2024-04-26T20:52:03.184 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch_barrier": 40, 2024-04-26T20:52:03.184 INFO:teuthology.orchestra.run.smithi073.stdout: "uptime": 24.725785129999998 2024-04-26T20:52:03.184 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-26T20:52:03.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.175+0000 7fd26ff51640 1 -- 172.21.15.73:0/2784141152 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd24c06c8f0 msgr2=0x7fd24c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:03.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.175+0000 7fd26ff51640 1 --2- 172.21.15.73:0/2784141152 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd24c06c8f0 0x7fd24c06edb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd264004ea0 tx=0x7fd264009040 comp rx=0 tx=0).stop 2024-04-26T20:52:03.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.175+0000 7fd26ff51640 1 -- 172.21.15.73:0/2784141152 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd26815f2d0 msgr2=0x7fd268153890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:03.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.175+0000 7fd26ff51640 1 --2- 172.21.15.73:0/2784141152 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd26815f2d0 0x7fd268153890 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fd258000c00 tx=0x7fd2580046c0 comp rx=0 tx=0).stop 2024-04-26T20:52:03.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.175+0000 7fd26ff51640 1 -- 172.21.15.73:0/2784141152 shutdown_connections 2024-04-26T20:52:03.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.175+0000 7fd26ff51640 1 --2- 172.21.15.73:0/2784141152 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd26815f2d0 0x7fd268153890 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:03.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.175+0000 7fd26ff51640 1 --2- 172.21.15.73:0/2784141152 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd24c06c8f0 0x7fd24c06edb0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:03.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.175+0000 7fd26ff51640 1 --2- 172.21.15.73:0/2784141152 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd2681604e0 0x7fd268153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:03.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.175+0000 7fd26ff51640 1 --2- 172.21.15.73:0/2784141152 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd26806fcd0 0x7fd26806ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:03.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.175+0000 7fd26ff51640 1 -- 172.21.15.73:0/2784141152 >> 172.21.15.73:0/2784141152 conn(0x7fd2680d0be0 msgr2=0x7fd26815bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:03.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.175+0000 7fd26ff51640 1 -- 172.21.15.73:0/2784141152 shutdown_connections 2024-04-26T20:52:03.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.175+0000 7fd26ff51640 1 -- 172.21.15.73:0/2784141152 wait complete. 2024-04-26T20:52:03.187 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000000.00000000 parent 2024-04-26T20:52:03.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.227+0000 7f9294c43f40 1 Processor -- start 2024-04-26T20:52:03.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.227+0000 7f9294c43f40 1 -- start start 2024-04-26T20:52:03.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.228+0000 7f9294c43f40 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ceb215a80 0x562ceb22fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:03.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.228+0000 7f9294c43f40 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562ceb215e60 0x562ceb233c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:03.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.228+0000 7f9294c43f40 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x562ceb218f30 0x562ceb22f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:03.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.228+0000 7f9294c43f40 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x562ceb1a22d0 con 0x562ceb215a80 2024-04-26T20:52:03.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.228+0000 7f9294c43f40 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562ceb101e40 con 0x562ceb215e60 2024-04-26T20:52:03.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.228+0000 7f9294c43f40 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x562ceb1a2570 con 0x562ceb218f30 2024-04-26T20:52:03.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.228+0000 7f929273d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ceb215a80 0x562ceb22fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:03.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.228+0000 7f929273d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ceb215a80 0x562ceb22fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39794/0 (socket says 172.21.15.73:39794) 2024-04-26T20:52:03.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.228+0000 7f929273d640 1 -- 172.21.15.73:0/396385656 learned_addr learned my addr 172.21.15.73:0/396385656 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:03.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.228+0000 7f9291f3c640 1 --2- 172.21.15.73:0/396385656 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562ceb215e60 0x562ceb233c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:03.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.228+0000 7f9292f3e640 1 --2- 172.21.15.73:0/396385656 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x562ceb218f30 0x562ceb22f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:03.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.228+0000 7f929273d640 1 -- 172.21.15.73:0/396385656 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x562ceb218f30 msgr2=0x562ceb22f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:03.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.228+0000 7f929273d640 1 --2- 172.21.15.73:0/396385656 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x562ceb218f30 0x562ceb22f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:03.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.228+0000 7f929273d640 1 -- 172.21.15.73:0/396385656 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562ceb215e60 msgr2=0x562ceb233c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:03.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.228+0000 7f929273d640 1 --2- 172.21.15.73:0/396385656 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562ceb215e60 0x562ceb233c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:03.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.228+0000 7f929273d640 1 -- 172.21.15.73:0/396385656 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ceb22f5d0 con 0x562ceb215a80 2024-04-26T20:52:03.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.228+0000 7f9291f3c640 1 --2- 172.21.15.73:0/396385656 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562ceb215e60 0x562ceb233c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:52:03.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.228+0000 7f929273d640 1 --2- 172.21.15.73:0/396385656 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ceb215a80 0x562ceb22fe10 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f9284001d80 tx=0x7f9284001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e29b1778de02278 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:03.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.228+0000 7f9290f3a640 1 -- 172.21.15.73:0/396385656 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928400bd70 con 0x562ceb215a80 2024-04-26T20:52:03.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.228+0000 7f9290f3a640 1 -- 172.21.15.73:0/396385656 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f928400f400 con 0x562ceb215a80 2024-04-26T20:52:03.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.228+0000 7f9290f3a640 1 -- 172.21.15.73:0/396385656 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928400f7f0 con 0x562ceb215a80 2024-04-26T20:52:03.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.229+0000 7f9294c43f40 1 -- 172.21.15.73:0/396385656 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ceb215a80 msgr2=0x562ceb22fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:03.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.229+0000 7f9294c43f40 1 --2- 172.21.15.73:0/396385656 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ceb215a80 0x562ceb22fe10 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f9284001d80 tx=0x7f9284001e70 comp rx=0 tx=0).stop 2024-04-26T20:52:03.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.229+0000 7f9294c43f40 1 -- 172.21.15.73:0/396385656 shutdown_connections 2024-04-26T20:52:03.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.229+0000 7f9294c43f40 1 --2- 172.21.15.73:0/396385656 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x562ceb218f30 0x562ceb22f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:03.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.229+0000 7f9294c43f40 1 --2- 172.21.15.73:0/396385656 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562ceb215e60 0x562ceb233c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:03.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.229+0000 7f9294c43f40 1 --2- 172.21.15.73:0/396385656 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ceb215a80 0x562ceb22fe10 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:03.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.229+0000 7f9294c43f40 1 -- 172.21.15.73:0/396385656 >> 172.21.15.73:0/396385656 conn(0x562ceb158ab0 msgr2=0x562ceb159720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:03.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.229+0000 7f9294c43f40 1 -- 172.21.15.73:0/396385656 shutdown_connections 2024-04-26T20:52:03.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.229+0000 7f9294c43f40 1 -- 172.21.15.73:0/396385656 wait complete. 2024-04-26T20:52:03.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.229+0000 7f9294c43f40 1 Processor -- start 2024-04-26T20:52:03.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.229+0000 7f9294c43f40 1 -- start start 2024-04-26T20:52:03.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.230+0000 7f9294c43f40 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ceb215a80 0x562ceb22d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:03.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.230+0000 7f9294c43f40 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562ceb215e60 0x562ceb210b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:03.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.230+0000 7f9294c43f40 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x562ceb218f30 0x562ceb233960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:03.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.230+0000 7f9294c43f40 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x562ceb185a70 con 0x562ceb215a80 2024-04-26T20:52:03.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.230+0000 7f9294c43f40 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x562ceb234170 con 0x562ceb215e60 2024-04-26T20:52:03.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.230+0000 7f9294c43f40 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x562ceb214b70 con 0x562ceb218f30 2024-04-26T20:52:03.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.230+0000 7f929273d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ceb215a80 0x562ceb22d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:03.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.230+0000 7f929273d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ceb215a80 0x562ceb22d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39806/0 (socket says 172.21.15.73:39806) 2024-04-26T20:52:03.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.230+0000 7f929273d640 1 -- 172.21.15.73:0/3088487613 learned_addr learned my addr 172.21.15.73:0/3088487613 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:03.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.230+0000 7f9291f3c640 1 --2- 172.21.15.73:0/3088487613 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562ceb215e60 0x562ceb210b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:03.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.230+0000 7f9292f3e640 1 --2- 172.21.15.73:0/3088487613 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x562ceb218f30 0x562ceb233960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:03.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.230+0000 7f929273d640 1 -- 172.21.15.73:0/3088487613 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x562ceb218f30 msgr2=0x562ceb233960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:03.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.230+0000 7f929273d640 1 --2- 172.21.15.73:0/3088487613 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x562ceb218f30 0x562ceb233960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:03.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.230+0000 7f929273d640 1 -- 172.21.15.73:0/3088487613 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562ceb215e60 msgr2=0x562ceb210b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:03.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.230+0000 7f929273d640 1 --2- 172.21.15.73:0/3088487613 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562ceb215e60 0x562ceb210b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:03.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.230+0000 7f929273d640 1 -- 172.21.15.73:0/3088487613 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ceb2bf530 con 0x562ceb215a80 2024-04-26T20:52:03.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.230+0000 7f9291f3c640 1 --2- 172.21.15.73:0/3088487613 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x562ceb215e60 0x562ceb210b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:52:03.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.231+0000 7f929273d640 1 --2- 172.21.15.73:0/3088487613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ceb215a80 0x562ceb22d920 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f9284001b50 tx=0x7f9284000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:03.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.231+0000 7f92837fe640 1 -- 172.21.15.73:0/3088487613 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9284020070 con 0x562ceb215a80 2024-04-26T20:52:03.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.231+0000 7f92837fe640 1 -- 172.21.15.73:0/3088487613 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f928401c440 con 0x562ceb215a80 2024-04-26T20:52:03.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.231+0000 7f9294c43f40 1 -- 172.21.15.73:0/3088487613 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562ceb2bec90 con 0x562ceb215a80 2024-04-26T20:52:03.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.231+0000 7f92837fe640 1 -- 172.21.15.73:0/3088487613 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928401c810 con 0x562ceb215a80 2024-04-26T20:52:03.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.231+0000 7f9294c43f40 1 -- 172.21.15.73:0/3088487613 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ceb2bf250 con 0x562ceb215a80 2024-04-26T20:52:03.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.232+0000 7f92837fe640 1 -- 172.21.15.73:0/3088487613 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9284023070 con 0x562ceb215a80 2024-04-26T20:52:03.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.232+0000 7f92837fe640 1 --2- 172.21.15.73:0/3088487613 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f927406b300 0x7f927406d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:03.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.233+0000 7f92837fe640 1 -- 172.21.15.73:0/3088487613 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6187+0+0 (secure 0 0 0) 0x7f928408c1f0 con 0x562ceb215a80 2024-04-26T20:52:03.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.233+0000 7f9291f3c640 1 --2- 172.21.15.73:0/3088487613 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f927406b300 0x7f927406d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:03.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.233+0000 7f9294c43f40 1 --2- 172.21.15.73:0/3088487613 >> [v2:172.21.15.120:6816/2915574674,v1:172.21.15.120:6818/2915574674] conn(0x562ceb184fe0 0x562ceb2c05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:03.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.233+0000 7f9294c43f40 1 -- 172.21.15.73:0/3088487613 --> [v2:172.21.15.120:6816/2915574674,v1:172.21.15.120:6818/2915574674] -- osd_op(unknown.0.0:1 5.10 5:0cf6ad0f:::10000000000.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40) -- 0x562ceb2c0b20 con 0x562ceb184fe0 2024-04-26T20:52:03.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.233+0000 7f9292f3e640 1 --2- 172.21.15.73:0/3088487613 >> [v2:172.21.15.120:6816/2915574674,v1:172.21.15.120:6818/2915574674] conn(0x562ceb184fe0 0x562ceb2c05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:03.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.233+0000 7f9291f3c640 1 --2- 172.21.15.73:0/3088487613 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f927406b300 0x7f927406d7c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f92880057f0 tx=0x7f9288005ac0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:03.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.233+0000 7f9292f3e640 1 --2- 172.21.15.73:0/3088487613 >> [v2:172.21.15.120:6816/2915574674,v1:172.21.15.120:6818/2915574674] conn(0x562ceb184fe0 0x562ceb2c05e0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:03.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.235+0000 7f9292f3e640 1 -- 172.21.15.73:0/3088487613 <== osd.7 v2:172.21.15.120:6816/2915574674 1 ==== osd_op_reply(1 10000000000.00000000 [rmxattr] v40'11 uv11 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f928c0518f0 con 0x562ceb184fe0 2024-04-26T20:52:03.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.235+0000 7f9294c43f40 1 -- 172.21.15.73:0/3088487613 >> [v2:172.21.15.120:6816/2915574674,v1:172.21.15.120:6818/2915574674] conn(0x562ceb184fe0 msgr2=0x562ceb2c05e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:03.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.235+0000 7f9294c43f40 1 --2- 172.21.15.73:0/3088487613 >> [v2:172.21.15.120:6816/2915574674,v1:172.21.15.120:6818/2915574674] conn(0x562ceb184fe0 0x562ceb2c05e0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:03.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.235+0000 7f9294c43f40 1 -- 172.21.15.73:0/3088487613 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f927406b300 msgr2=0x7f927406d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:03.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.235+0000 7f9294c43f40 1 --2- 172.21.15.73:0/3088487613 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f927406b300 0x7f927406d7c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f92880057f0 tx=0x7f9288005ac0 comp rx=0 tx=0).stop 2024-04-26T20:52:03.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.235+0000 7f9294c43f40 1 -- 172.21.15.73:0/3088487613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ceb215a80 msgr2=0x562ceb22d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:03.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.235+0000 7f9294c43f40 1 --2- 172.21.15.73:0/3088487613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562ceb215a80 0x562ceb22d920 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f9284001b50 tx=0x7f9284000bc0 comp rx=0 tx=0).stop 2024-04-26T20:52:03.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.235+0000 7f9292f3e640 1 -- 172.21.15.73:0/3088487613 reap_dead start 2024-04-26T20:52:03.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.235+0000 7f9294c43f40 1 -- 172.21.15.73:0/3088487613 shutdown_connections 2024-04-26T20:52:03.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.235+0000 7f9294c43f40 1 -- 172.21.15.73:0/3088487613 >> 172.21.15.73:0/3088487613 conn(0x562ceb158ab0 msgr2=0x562ceb158e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:03.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.235+0000 7f9294c43f40 1 -- 172.21.15.73:0/3088487613 shutdown_connections 2024-04-26T20:52:03.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.235+0000 7f9294c43f40 1 -- 172.21.15.73:0/3088487613 wait complete. 2024-04-26T20:52:03.240 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T20:52:03.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.374+0000 7fce9f837640 1 Processor -- start 2024-04-26T20:52:03.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.374+0000 7fce9f837640 1 -- start start 2024-04-26T20:52:03.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.375+0000 7fce9f837640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fce9815f4b0 0x7fce980d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:03.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.375+0000 7fce9f837640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fce9806d5f0 0x7fce980cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:03.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.375+0000 7fce9f837640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fce980cfd10 0x7fce980d1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:03.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.375+0000 7fce9f837640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7fce980d24b0 con 0x7fce980cfd10 2024-04-26T20:52:03.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.375+0000 7fce9f837640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fce980ce050 con 0x7fce9815f4b0 2024-04-26T20:52:03.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.375+0000 7fce9f837640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fce9815f890 con 0x7fce9806d5f0 2024-04-26T20:52:03.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.375+0000 7fce9ddd4640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fce980cfd10 0x7fce980d1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:03.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.375+0000 7fce9ddd4640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fce980cfd10 0x7fce980d1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39822/0 (socket says 172.21.15.73:39822) 2024-04-26T20:52:03.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.375+0000 7fce9ddd4640 1 -- 172.21.15.73:0/985194346 learned_addr learned my addr 172.21.15.73:0/985194346 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:03.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.376+0000 7fce9cdd2640 1 --2- 172.21.15.73:0/985194346 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fce9806d5f0 0x7fce980cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:03.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.376+0000 7fce9d5d3640 1 --2- 172.21.15.73:0/985194346 >> v2:172.21.15.120:3300/0 conn(0x7fce9815f4b0 0x7fce980d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:03.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.376+0000 7fce9ddd4640 1 -- 172.21.15.73:0/985194346 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fce9806d5f0 msgr2=0x7fce980cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:03.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.376+0000 7fce9ddd4640 1 --2- 172.21.15.73:0/985194346 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fce9806d5f0 0x7fce980cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:03.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.376+0000 7fce9ddd4640 1 -- 172.21.15.73:0/985194346 >> v2:172.21.15.120:3300/0 conn(0x7fce9815f4b0 msgr2=0x7fce980d38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:03.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.376+0000 7fce9ddd4640 1 --2- 172.21.15.73:0/985194346 >> v2:172.21.15.120:3300/0 conn(0x7fce9815f4b0 0x7fce980d38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:03.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.376+0000 7fce9ddd4640 1 -- 172.21.15.73:0/985194346 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce9806dd70 con 0x7fce980cfd10 2024-04-26T20:52:03.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.376+0000 7fce9ddd4640 1 --2- 172.21.15.73:0/985194346 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce980cfd10 0x7fce980d1090 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fce9806c4e0 tx=0x7fce9002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4def0bdcda2cdb6c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:03.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.377+0000 7fce8ffff640 1 -- 172.21.15.73:0/985194346 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce90037070 con 0x7fce980cfd10 2024-04-26T20:52:03.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.377+0000 7fce8ffff640 1 -- 172.21.15.73:0/985194346 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce90032440 con 0x7fce980cfd10 2024-04-26T20:52:03.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.377+0000 7fce8ffff640 1 -- 172.21.15.73:0/985194346 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce90032810 con 0x7fce980cfd10 2024-04-26T20:52:03.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.377+0000 7fce9f837640 1 -- 172.21.15.73:0/985194346 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce980cfd10 msgr2=0x7fce980d1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:03.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.377+0000 7fce9f837640 1 --2- 172.21.15.73:0/985194346 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce980cfd10 0x7fce980d1090 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fce9806c4e0 tx=0x7fce9002a140 comp rx=0 tx=0).stop 2024-04-26T20:52:03.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.377+0000 7fce9f837640 1 -- 172.21.15.73:0/985194346 shutdown_connections 2024-04-26T20:52:03.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.377+0000 7fce9f837640 1 --2- 172.21.15.73:0/985194346 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce980cfd10 0x7fce980d1090 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:03.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.377+0000 7fce9f837640 1 --2- 172.21.15.73:0/985194346 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fce9806d5f0 0x7fce980cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:03.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.377+0000 7fce9f837640 1 --2- 172.21.15.73:0/985194346 >> v2:172.21.15.120:3300/0 conn(0x7fce9815f4b0 0x7fce980d38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:03.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.377+0000 7fce9f837640 1 -- 172.21.15.73:0/985194346 >> 172.21.15.73:0/985194346 conn(0x7fce980d2dc0 msgr2=0x7fce980d31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:03.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.377+0000 7fce9f837640 1 -- 172.21.15.73:0/985194346 shutdown_connections 2024-04-26T20:52:03.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.377+0000 7fce9f837640 1 -- 172.21.15.73:0/985194346 wait complete. 2024-04-26T20:52:03.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.378+0000 7fce9f837640 1 Processor -- start 2024-04-26T20:52:03.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.378+0000 7fce9f837640 1 -- start start 2024-04-26T20:52:03.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.379+0000 7fce9f837640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fce9806d5f0 0x7fce980b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:03.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.379+0000 7fce9f837640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce980cfd10 0x7fce980cb300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:03.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.379+0000 7fce9f837640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce9815f4b0 0x7fce980cb840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:03.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.379+0000 7fce9f837640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fce9806da50 con 0x7fce9815f4b0 2024-04-26T20:52:03.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.379+0000 7fce9f837640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fce9806e330 con 0x7fce980cfd10 2024-04-26T20:52:03.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.379+0000 7fce9f837640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fce980b24e0 con 0x7fce9806d5f0 2024-04-26T20:52:03.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.379+0000 7fce9ddd4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce9815f4b0 0x7fce980cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:03.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.379+0000 7fce9ddd4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce9815f4b0 0x7fce980cb840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39838/0 (socket says 172.21.15.73:39838) 2024-04-26T20:52:03.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.379+0000 7fce9ddd4640 1 -- 172.21.15.73:0/1504147193 learned_addr learned my addr 172.21.15.73:0/1504147193 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:03.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.380+0000 7fce9ddd4640 1 -- 172.21.15.73:0/1504147193 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fce9806d5f0 msgr2=0x7fce980b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:03.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.380+0000 7fce9d5d3640 1 --2- 172.21.15.73:0/1504147193 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fce9806d5f0 0x7fce980b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:03.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.380+0000 7fce9ddd4640 1 --2- 172.21.15.73:0/1504147193 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fce9806d5f0 0x7fce980b3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:03.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.380+0000 7fce9cdd2640 1 --2- 172.21.15.73:0/1504147193 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce980cfd10 0x7fce980cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:03.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.380+0000 7fce9ddd4640 1 -- 172.21.15.73:0/1504147193 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce980cfd10 msgr2=0x7fce980cb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:03.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.380+0000 7fce9ddd4640 1 --2- 172.21.15.73:0/1504147193 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce980cfd10 0x7fce980cb300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:03.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.380+0000 7fce9ddd4640 1 -- 172.21.15.73:0/1504147193 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce98157c90 con 0x7fce9815f4b0 2024-04-26T20:52:03.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.380+0000 7fce9ddd4640 1 --2- 172.21.15.73:0/1504147193 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce9815f4b0 0x7fce980cb840 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fce900090b0 tx=0x7fce900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:03.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.380+0000 7fce8dffb640 1 -- 172.21.15.73:0/1504147193 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce90041070 con 0x7fce9815f4b0 2024-04-26T20:52:03.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.380+0000 7fce8dffb640 1 -- 172.21.15.73:0/1504147193 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce9003c870 con 0x7fce9815f4b0 2024-04-26T20:52:03.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.380+0000 7fce8dffb640 1 -- 172.21.15.73:0/1504147193 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce90037040 con 0x7fce9815f4b0 2024-04-26T20:52:03.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.380+0000 7fce9f837640 1 -- 172.21.15.73:0/1504147193 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce98157f70 con 0x7fce9815f4b0 2024-04-26T20:52:03.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.380+0000 7fce9f837640 1 -- 172.21.15.73:0/1504147193 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce98158530 con 0x7fce9815f4b0 2024-04-26T20:52:03.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.381+0000 7fce9f837640 1 -- 172.21.15.73:0/1504147193 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce980694f0 con 0x7fce9815f4b0 2024-04-26T20:52:03.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.381+0000 7fce8dffb640 1 -- 172.21.15.73:0/1504147193 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fce90044070 con 0x7fce9815f4b0 2024-04-26T20:52:03.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.382+0000 7fce8dffb640 1 --2- 172.21.15.73:0/1504147193 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fce7c06c8f0 0x7fce7c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:03.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.382+0000 7fce9d5d3640 1 --2- 172.21.15.73:0/1504147193 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fce7c06c8f0 0x7fce7c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:03.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.382+0000 7fce8dffb640 1 -- 172.21.15.73:0/1504147193 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6187+0+0 (secure 0 0 0) 0x7fce900ac7a0 con 0x7fce9815f4b0 2024-04-26T20:52:03.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.382+0000 7fce9d5d3640 1 --2- 172.21.15.73:0/1504147193 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fce7c06c8f0 0x7fce7c06edb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fce94009f30 tx=0x7fce9400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:03.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.386+0000 7fce8dffb640 1 -- 172.21.15.73:0/1504147193 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fce9007aba0 con 0x7fce9815f4b0 2024-04-26T20:52:03.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.597+0000 7fce9f837640 1 -- 172.21.15.73:0/1504147193 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fce980d1540 con 0x7fce9815f4b0 2024-04-26T20:52:03.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.598+0000 7fce8dffb640 1 -- 172.21.15.73:0/1504147193 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v20) ==== 53+0+73 (secure 0 0 0) 0x7fce9007a4a0 con 0x7fce9815f4b0 2024-04-26T20:52:03.599 INFO:teuthology.orchestra.run.smithi073.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T20:52:03.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.599+0000 7fce9f837640 1 -- 172.21.15.73:0/1504147193 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fce7c06c8f0 msgr2=0x7fce7c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:03.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.599+0000 7fce9f837640 1 --2- 172.21.15.73:0/1504147193 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fce7c06c8f0 0x7fce7c06edb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fce94009f30 tx=0x7fce9400b040 comp rx=0 tx=0).stop 2024-04-26T20:52:03.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.599+0000 7fce9f837640 1 -- 172.21.15.73:0/1504147193 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce9815f4b0 msgr2=0x7fce980cb840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:03.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.599+0000 7fce9f837640 1 --2- 172.21.15.73:0/1504147193 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce9815f4b0 0x7fce980cb840 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fce900090b0 tx=0x7fce900046c0 comp rx=0 tx=0).stop 2024-04-26T20:52:03.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.599+0000 7fce9f837640 1 -- 172.21.15.73:0/1504147193 shutdown_connections 2024-04-26T20:52:03.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.599+0000 7fce9f837640 1 --2- 172.21.15.73:0/1504147193 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce9815f4b0 0x7fce980cb840 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:03.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.599+0000 7fce9f837640 1 --2- 172.21.15.73:0/1504147193 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fce7c06c8f0 0x7fce7c06edb0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:03.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.599+0000 7fce9f837640 1 --2- 172.21.15.73:0/1504147193 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fce980cfd10 0x7fce980cb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:03.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.599+0000 7fce9f837640 1 --2- 172.21.15.73:0/1504147193 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fce9806d5f0 0x7fce980b3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:03.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.599+0000 7fce9f837640 1 -- 172.21.15.73:0/1504147193 >> 172.21.15.73:0/1504147193 conn(0x7fce980d2dc0 msgr2=0x7fce98154a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:03.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.600+0000 7fce9f837640 1 -- 172.21.15.73:0/1504147193 shutdown_connections 2024-04-26T20:52:03.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03.600+0000 7fce9f837640 1 -- 172.21.15.73:0/1504147193 wait complete. 2024-04-26T20:52:03.611 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T20:52:03.611 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T20:52:03.611 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi073.front.sepia.ceph.com 2024-04-26T20:52:03.611 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T20:52:03.611 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T20:52:03.611 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T20:52:03.612 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T20:52:03.612 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T20:52:03.612 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T20:52:03.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:52:03.612 DEBUG:teuthology.orchestra.run.smithi073:> ip addr 2024-04-26T20:52:03.627 INFO:teuthology.orchestra.run.smithi073.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:52:03.627 INFO:teuthology.orchestra.run.smithi073.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:52:03.627 INFO:teuthology.orchestra.run.smithi073.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:52:03.627 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:52:03.627 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 ::1/128 scope host 2024-04-26T20:52:03.627 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:52:03.627 INFO:teuthology.orchestra.run.smithi073.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:52:03.627 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:ea brd ff:ff:ff:ff:ff:ff 2024-04-26T20:52:03.627 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f0 2024-04-26T20:52:03.628 INFO:teuthology.orchestra.run.smithi073.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:52:03.628 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:eb brd ff:ff:ff:ff:ff:ff 2024-04-26T20:52:03.628 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f1 2024-04-26T20:52:03.628 INFO:teuthology.orchestra.run.smithi073.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:52:03.628 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7c brd ff:ff:ff:ff:ff:ff 2024-04-26T20:52:03.628 INFO:teuthology.orchestra.run.smithi073.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T20:52:03.628 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7d brd ff:ff:ff:ff:ff:ff 2024-04-26T20:52:03.628 INFO:teuthology.orchestra.run.smithi073.stdout: inet 172.21.15.73/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-26T20:52:03.628 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft 42663sec preferred_lft 42663sec 2024-04-26T20:52:03.628 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::b99b:8e91:ffc2:71f/64 scope link noprefixroute 2024-04-26T20:52:03.628 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:52:03.628 INFO:teuthology.orchestra.run.smithi073.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T20:52:03.628 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 3e:22:aa:11:2d:0e brd ff:ff:ff:ff:ff:ff 2024-04-26T20:52:03.628 INFO:teuthology.orchestra.run.smithi073.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T20:52:03.628 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:52:03.629 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::1cd0:bfff:fec2:3f98/64 scope link 2024-04-26T20:52:03.629 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:52:03.629 INFO:teuthology.orchestra.run.smithi073.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T20:52:03.629 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 66:7f:fa:7c:30:18 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:52:03.629 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::647f:faff:fe7c:3018/64 scope link 2024-04-26T20:52:03.629 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:52:03.629 INFO:teuthology.orchestra.run.smithi073.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T20:52:03.629 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 3e:22:aa:11:2d:0e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:52:03.629 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::3c22:aaff:fe11:2d0e/64 scope link 2024-04-26T20:52:03.629 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:52:03.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:52:03.630 DEBUG:teuthology.orchestra.run.smithi073:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T20:52:03.689 INFO:teuthology.orchestra.run.smithi073.stdout:1 2024-04-26T20:52:03.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:52:03.691 DEBUG:teuthology.orchestra.run.smithi073:> route 2024-04-26T20:52:03.748 INFO:teuthology.orchestra.run.smithi073.stdout:Kernel IP routing table 2024-04-26T20:52:03.748 INFO:teuthology.orchestra.run.smithi073.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T20:52:03.748 INFO:teuthology.orchestra.run.smithi073.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T20:52:03.748 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T20:52:03.748 INFO:teuthology.orchestra.run.smithi073.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T20:52:03.748 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:52:03.748 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2024-04-26T20:52:03.748 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T20:52:03.748 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T20:52:03.748 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T20:52:03.749 DEBUG:teuthology.orchestra.run.smithi073:> ') 2024-04-26T20:52:03.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:52:03.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:52:03.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:52:03.871 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list 2024-04-26T20:52:03.886 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T20:52:03.886 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T20:52:03.886 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T20:52:03.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:52:03.886 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link set brx.0 up 2024-04-26T20:52:03.951 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:52:03.951 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T20:52:04.005 INFO:teuthology.orchestra.run.smithi073.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T20:52:04.006 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:52:04.006 DEBUG:teuthology.orchestra.run.smithi073:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T20:52:04.060 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T20:52:04.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:52:04.060 DEBUG:teuthology.orchestra.run.smithi073:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T20:52:04.141 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T20:52:04.141 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T20:52:04.142 INFO:teuthology.orchestra.run.smithi073.stdout:invalid new device string format 2024-04-26T20:52:04.150 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T20:52:04.150 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T20:52:04.150 INFO:teuthology.orchestra.run.smithi073.stdout:invalid new device string format 2024-04-26T20:52:04.150 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: resolved to: "172.21.15.73:3300,172.21.15.120:3300,172.21.15.120:3301" 2024-04-26T20:52:04.150 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T20:52:04.155 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T20:52:04.155 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T20:52:04.155 INFO:teuthology.orchestra.run.smithi073.stdout:invalid new device string format 2024-04-26T20:52:04.155 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: resolved to: "172.21.15.73:3300,172.21.15.120:3300,172.21.15.120:3301" 2024-04-26T20:52:04.155 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T20:52:04.155 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: resolved to: "172.21.15.73:6789,172.21.15.120:6789,172.21.15.120:6790" 2024-04-26T20:52:04.156 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.73:6789,172.21.15.120:6789,172.21.15.120:6790:/ 2024-04-26T20:52:04.156 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=fbd3f548-b547-4713-856b-09f98f5d1764" will pass to kernel 2024-04-26T20:52:04.156 INFO:teuthology.orchestra.run.smithi073.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T20:52:04.156 INFO:teuthology.orchestra.run.smithi073.stdout: You just mounted a file system that supports labels which does not 2024-04-26T20:52:04.156 INFO:teuthology.orchestra.run.smithi073.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T20:52:04.156 INFO:teuthology.orchestra.run.smithi073.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T20:52:04.156 INFO:teuthology.orchestra.run.smithi073.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T20:52:04.158 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T20:52:04.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:52:04.158 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T20:52:04.222 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T20:52:04.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T20:52:04.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:52:04.318 INFO:teuthology.orchestra.run.smithi073.stdout:client5673 2024-04-26T20:52:04.319 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-26T20:52:04.319 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:52:04.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:52:04.334 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls testdir1) 2024-04-26T20:52:04.389 INFO:tasks.ceph.mds.b.smithi120.stderr:2024-04-26T20:52:04.389+0000 7fbad0340640 -1 log_channel(cluster) log [ERR] : bad backtrace on directory inode 0x10000000000 2024-04-26T20:52:04.390 INFO:teuthology.orchestra.run.smithi073.stdout:testfile 2024-04-26T20:52:04.391 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:52:04.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.527+0000 7fe2a25f2640 1 Processor -- start 2024-04-26T20:52:04.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.527+0000 7fe2a25f2640 1 -- start start 2024-04-26T20:52:04.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.527+0000 7fe2a25f2640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe29c071e60 0x7fe29c15df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:04.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.527+0000 7fe2a25f2640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe29c072240 0x7fe29c0726a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:04.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.527+0000 7fe2a25f2640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe29c15e4d0 con 0x7fe29c0cb5a0 2024-04-26T20:52:04.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.528+0000 7fe2a25f2640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7fe29c0701f0 con 0x7fe29c072240 2024-04-26T20:52:04.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.528+0000 7fe2a25f2640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fe29c06e240 con 0x7fe29c071e60 2024-04-26T20:52:04.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.528+0000 7fe29b7fe640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe29c072240 0x7fe29c0726a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:04.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.528+0000 7fe29b7fe640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe29c072240 0x7fe29c0726a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39848/0 (socket says 172.21.15.73:39848) 2024-04-26T20:52:04.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.528+0000 7fe29b7fe640 1 -- 172.21.15.73:0/4074506531 learned_addr learned my addr 172.21.15.73:0/4074506531 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:04.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.528+0000 7fe29b7fe640 1 -- 172.21.15.73:0/4074506531 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe29c071e60 msgr2=0x7fe29c15df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:04.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.528+0000 7fe29b7fe640 1 --2- 172.21.15.73:0/4074506531 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe29c071e60 0x7fe29c15df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:04.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.528+0000 7fe29b7fe640 1 -- 172.21.15.73:0/4074506531 >> v1:172.21.15.120:6789/0 conn(0x7fe29c0cb5a0 legacy=0x7fe29c0cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:04.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.528+0000 7fe29b7fe640 1 -- 172.21.15.73:0/4074506531 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe29c06fc30 con 0x7fe29c072240 2024-04-26T20:52:04.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.529+0000 7fe29b7fe640 1 --2- 172.21.15.73:0/4074506531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe29c072240 0x7fe29c0726a0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fe290009870 tx=0x7fe29002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f27638ad9530f57e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:04.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.529+0000 7fe29affd640 1 -- 172.21.15.73:0/4074506531 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe290037070 con 0x7fe29c072240 2024-04-26T20:52:04.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.529+0000 7fe29affd640 1 -- 172.21.15.73:0/4074506531 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe290032440 con 0x7fe29c072240 2024-04-26T20:52:04.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.529+0000 7fe29affd640 1 -- 172.21.15.73:0/4074506531 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe290032810 con 0x7fe29c072240 2024-04-26T20:52:04.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.529+0000 7fe2a25f2640 1 -- 172.21.15.73:0/4074506531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe29c072240 msgr2=0x7fe29c0726a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:04.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.529+0000 7fe2a25f2640 1 --2- 172.21.15.73:0/4074506531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe29c072240 0x7fe29c0726a0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fe290009870 tx=0x7fe29002a180 comp rx=0 tx=0).stop 2024-04-26T20:52:04.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.529+0000 7fe2a25f2640 1 -- 172.21.15.73:0/4074506531 shutdown_connections 2024-04-26T20:52:04.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.529+0000 7fe2a25f2640 1 --2- 172.21.15.73:0/4074506531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe29c072240 0x7fe29c0726a0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:04.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.529+0000 7fe2a25f2640 1 --2- 172.21.15.73:0/4074506531 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe29c071e60 0x7fe29c15df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:04.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.529+0000 7fe2a25f2640 1 -- 172.21.15.73:0/4074506531 >> 172.21.15.73:0/4074506531 conn(0x7fe29c0d0be0 msgr2=0x7fe29c0d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:04.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.529+0000 7fe2a25f2640 1 -- 172.21.15.73:0/4074506531 shutdown_connections 2024-04-26T20:52:04.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.529+0000 7fe2a25f2640 1 -- 172.21.15.73:0/4074506531 wait complete. 2024-04-26T20:52:04.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.530+0000 7fe2a25f2640 1 Processor -- start 2024-04-26T20:52:04.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.531+0000 7fe2a25f2640 1 -- start start 2024-04-26T20:52:04.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.531+0000 7fe2a25f2640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe29c071e60 0x7fe29c145950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:04.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.531+0000 7fe2a25f2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe29c072240 0x7fe29c148ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:04.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.531+0000 7fe2a25f2640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe29c0cb5a0 0x7fe29c147790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:04.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.531+0000 7fe2a25f2640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe29c072be0 con 0x7fe29c071e60 2024-04-26T20:52:04.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.531+0000 7fe2a25f2640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe29c073fb0 con 0x7fe29c072240 2024-04-26T20:52:04.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.531+0000 7fe2a25f2640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fe29c06f8e0 con 0x7fe29c0cb5a0 2024-04-26T20:52:04.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.531+0000 7fe29bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe29c071e60 0x7fe29c145950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:04.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.531+0000 7fe29bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe29c071e60 0x7fe29c145950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39864/0 (socket says 172.21.15.73:39864) 2024-04-26T20:52:04.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.531+0000 7fe29bfff640 1 -- 172.21.15.73:0/256840886 learned_addr learned my addr 172.21.15.73:0/256840886 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:04.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.532+0000 7fe29bfff640 1 -- 172.21.15.73:0/256840886 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe29c0cb5a0 msgr2=0x7fe29c147790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:04.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.532+0000 7fe2a0b8f640 1 --2- 172.21.15.73:0/256840886 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe29c0cb5a0 0x7fe29c147790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:04.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.532+0000 7fe29bfff640 1 --2- 172.21.15.73:0/256840886 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe29c0cb5a0 0x7fe29c147790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:04.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.532+0000 7fe29b7fe640 1 --2- 172.21.15.73:0/256840886 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe29c072240 0x7fe29c148ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:04.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.532+0000 7fe29bfff640 1 -- 172.21.15.73:0/256840886 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe29c072240 msgr2=0x7fe29c148ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:04.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.532+0000 7fe29bfff640 1 --2- 172.21.15.73:0/256840886 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe29c072240 0x7fe29c148ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:04.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.532+0000 7fe29bfff640 1 -- 172.21.15.73:0/256840886 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe29c1466f0 con 0x7fe29c071e60 2024-04-26T20:52:04.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.532+0000 7fe29bfff640 1 --2- 172.21.15.73:0/256840886 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe29c071e60 0x7fe29c145950 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe28c00b410 tx=0x7fe28c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:04.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.532+0000 7fe298ff9640 1 -- 172.21.15.73:0/256840886 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe28c015020 con 0x7fe29c071e60 2024-04-26T20:52:04.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.532+0000 7fe298ff9640 1 -- 172.21.15.73:0/256840886 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe28c011850 con 0x7fe29c071e60 2024-04-26T20:52:04.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.532+0000 7fe298ff9640 1 -- 172.21.15.73:0/256840886 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe28c011c40 con 0x7fe29c071e60 2024-04-26T20:52:04.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.532+0000 7fe2a25f2640 1 -- 172.21.15.73:0/256840886 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe29c1469d0 con 0x7fe29c071e60 2024-04-26T20:52:04.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.532+0000 7fe2a25f2640 1 -- 172.21.15.73:0/256840886 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe29c146f90 con 0x7fe29c071e60 2024-04-26T20:52:04.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.533+0000 7fe2a25f2640 1 -- 172.21.15.73:0/256840886 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe29c0694f0 con 0x7fe29c071e60 2024-04-26T20:52:04.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.534+0000 7fe298ff9640 1 -- 172.21.15.73:0/256840886 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe28c018020 con 0x7fe29c071e60 2024-04-26T20:52:04.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.534+0000 7fe298ff9640 1 --2- 172.21.15.73:0/256840886 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe28806c8f0 0x7fe28806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:04.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.534+0000 7fe29b7fe640 1 --2- 172.21.15.73:0/256840886 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe28806c8f0 0x7fe28806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:04.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.534+0000 7fe298ff9640 1 -- 172.21.15.73:0/256840886 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6187+0+0 (secure 0 0 0) 0x7fe28c08bd30 con 0x7fe29c071e60 2024-04-26T20:52:04.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.534+0000 7fe29b7fe640 1 --2- 172.21.15.73:0/256840886 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe28806c8f0 0x7fe28806edb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe290034000 tx=0x7fe29002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:04.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.538+0000 7fe298ff9640 1 -- 172.21.15.73:0/256840886 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe28c01f020 con 0x7fe29c071e60 2024-04-26T20:52:04.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.746+0000 7fe2a25f2640 1 -- 172.21.15.73:0/256840886 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe29c069b20 con 0x7fe29c071e60 2024-04-26T20:52:04.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.747+0000 7fe298ff9640 1 -- 172.21.15.73:0/256840886 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5185 (secure 0 0 0) 0x7fe28c0172c0 con 0x7fe29c071e60 2024-04-26T20:52:04.748 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:52:04.748 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4802,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/272662913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":272662913},{"type":"v1","addr":"172.21.15.73:6836","nonce":272662913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4943,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2624899096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6833","nonce":2624899096},{"type":"v1","addr":"172.21.15.120:6835","nonce":2624899096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5047,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/1438858294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":1438858294},{"type":"v1","addr":"172.21.15.73:6837","nonce":1438858294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:51:45.273953+0000","modified":"2024-04-26T20:52:01.795359+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5050},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5050":{"gid":5050,"name":"b","rank":0,"incarnation":17,"state":"up:active","state_seq":9,"addr":"172.21.15.120:6834/3883540031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3883540031},{"type":"v1","addr":"172.21.15.120:6834","nonce":3883540031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5050,"qdb_cluster":[5050]},"id":2}]} 2024-04-26T20:52:04.748 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 20 2024-04-26T20:52:04.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.748+0000 7fe2a25f2640 1 -- 172.21.15.73:0/256840886 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe28806c8f0 msgr2=0x7fe28806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:04.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.748+0000 7fe2a25f2640 1 --2- 172.21.15.73:0/256840886 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe28806c8f0 0x7fe28806edb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe290034000 tx=0x7fe29002e000 comp rx=0 tx=0).stop 2024-04-26T20:52:04.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.748+0000 7fe2a25f2640 1 -- 172.21.15.73:0/256840886 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe29c071e60 msgr2=0x7fe29c145950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:04.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.748+0000 7fe2a25f2640 1 --2- 172.21.15.73:0/256840886 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe29c071e60 0x7fe29c145950 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe28c00b410 tx=0x7fe28c00b8d0 comp rx=0 tx=0).stop 2024-04-26T20:52:04.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.749+0000 7fe2a25f2640 1 -- 172.21.15.73:0/256840886 shutdown_connections 2024-04-26T20:52:04.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.749+0000 7fe2a25f2640 1 --2- 172.21.15.73:0/256840886 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe28806c8f0 0x7fe28806edb0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:04.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.749+0000 7fe2a25f2640 1 --2- 172.21.15.73:0/256840886 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe29c0cb5a0 0x7fe29c147790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:04.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.749+0000 7fe2a25f2640 1 --2- 172.21.15.73:0/256840886 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe29c072240 0x7fe29c148ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:04.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.749+0000 7fe2a25f2640 1 --2- 172.21.15.73:0/256840886 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe29c071e60 0x7fe29c145950 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:04.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.749+0000 7fe2a25f2640 1 -- 172.21.15.73:0/256840886 >> 172.21.15.73:0/256840886 conn(0x7fe29c0d0be0 msgr2=0x7fe29c0b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:04.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.749+0000 7fe2a25f2640 1 -- 172.21.15.73:0/256840886 shutdown_connections 2024-04-26T20:52:04.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.749+0000 7fe2a25f2640 1 -- 172.21.15.73:0/256840886 wait complete. 2024-04-26T20:52:04.761 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-04-26T20:52:04.903 INFO:teuthology.orchestra.run.smithi120.stdout:{"message":"","return_code":0} 2024-04-26T20:52:04.903 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-26T20:52:04.903 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000000.00000000 parent 2024-04-26T20:52:04.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.945+0000 7fe1e7823f40 1 Processor -- start 2024-04-26T20:52:04.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.945+0000 7fe1e7823f40 1 -- start start 2024-04-26T20:52:04.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.946+0000 7fe1e7823f40 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe1d4003cb0 0x55f0ab147550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:04.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.946+0000 7fe1e7823f40 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f0ab14b850 0x55f0ab149d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:04.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.946+0000 7fe1e7823f40 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0ab151820 0x55f0ab0817a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:04.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.946+0000 7fe1e7823f40 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f0ab026e40 con 0x55f0ab14b850 2024-04-26T20:52:04.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.946+0000 7fe1e7823f40 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f0ab02ae80 con 0x55f0ab151820 2024-04-26T20:52:04.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.946+0000 7fe1e7823f40 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x55f0ab154c00 con 0x7fe1d4003cb0 2024-04-26T20:52:04.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.946+0000 7fe1dcb1c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f0ab14b850 0x55f0ab149d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:04.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.946+0000 7fe1dcb1c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f0ab14b850 0x55f0ab149d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39890/0 (socket says 172.21.15.73:39890) 2024-04-26T20:52:04.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.946+0000 7fe1dcb1c640 1 -- 172.21.15.73:0/351642980 learned_addr learned my addr 172.21.15.73:0/351642980 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:04.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.946+0000 7fe1dcb1c640 1 -- 172.21.15.73:0/351642980 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe1d4003cb0 msgr2=0x55f0ab147550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:04.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.946+0000 7fe1dcb1c640 1 --2- 172.21.15.73:0/351642980 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe1d4003cb0 0x55f0ab147550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:04.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.946+0000 7fe1dcb1c640 1 -- 172.21.15.73:0/351642980 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0ab151820 msgr2=0x55f0ab0817a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:04.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.946+0000 7fe1dcb1c640 1 --2- 172.21.15.73:0/351642980 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0ab151820 0x55f0ab0817a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:04.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.946+0000 7fe1dcb1c640 1 -- 172.21.15.73:0/351642980 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0ab07e170 con 0x55f0ab14b850 2024-04-26T20:52:04.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.946+0000 7fe1dcb1c640 1 --2- 172.21.15.73:0/351642980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f0ab14b850 0x55f0ab149d00 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fe1d000c600 tx=0x7fe1d000cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad775ecdba074f06 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:04.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.946+0000 7fe1dffff640 1 -- 172.21.15.73:0/351642980 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d001a070 con 0x55f0ab14b850 2024-04-26T20:52:04.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.946+0000 7fe1dffff640 1 -- 172.21.15.73:0/351642980 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1d000a8e0 con 0x55f0ab14b850 2024-04-26T20:52:04.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.946+0000 7fe1dffff640 1 -- 172.21.15.73:0/351642980 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d000acb0 con 0x55f0ab14b850 2024-04-26T20:52:04.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.947+0000 7fe1e7823f40 1 -- 172.21.15.73:0/351642980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f0ab14b850 msgr2=0x55f0ab149d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:04.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.947+0000 7fe1e7823f40 1 --2- 172.21.15.73:0/351642980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f0ab14b850 0x55f0ab149d00 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fe1d000c600 tx=0x7fe1d000cac0 comp rx=0 tx=0).stop 2024-04-26T20:52:04.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.947+0000 7fe1e7823f40 1 -- 172.21.15.73:0/351642980 shutdown_connections 2024-04-26T20:52:04.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.947+0000 7fe1e7823f40 1 --2- 172.21.15.73:0/351642980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f0ab14b850 0x55f0ab149d00 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:04.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.947+0000 7fe1e7823f40 1 --2- 172.21.15.73:0/351642980 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0ab151820 0x55f0ab0817a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:04.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.947+0000 7fe1e7823f40 1 --2- 172.21.15.73:0/351642980 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe1d4003cb0 0x55f0ab147550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:04.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.947+0000 7fe1e7823f40 1 -- 172.21.15.73:0/351642980 >> 172.21.15.73:0/351642980 conn(0x55f0ab0a98a0 msgr2=0x55f0ab0809b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:04.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.947+0000 7fe1e7823f40 1 -- 172.21.15.73:0/351642980 shutdown_connections 2024-04-26T20:52:04.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.947+0000 7fe1e7823f40 1 -- 172.21.15.73:0/351642980 wait complete. 2024-04-26T20:52:04.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.947+0000 7fe1e7823f40 1 Processor -- start 2024-04-26T20:52:04.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.947+0000 7fe1e7823f40 1 -- start start 2024-04-26T20:52:04.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.948+0000 7fe1e7823f40 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55f0ab14b850 0x55f0ab1ee920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:04.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.948+0000 7fe1e7823f40 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0ab151820 0x55f0ab1533b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:04.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.948+0000 7fe1e7823f40 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1d4003cb0 0x55f0ab1538f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:04.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.948+0000 7fe1e7823f40 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f0ab14be40 con 0x7fe1d4003cb0 2024-04-26T20:52:04.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.948+0000 7fe1e7823f40 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55f0ab14b430 con 0x55f0ab151820 2024-04-26T20:52:04.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.948+0000 7fe1e7823f40 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x55f0ab07e3e0 con 0x55f0ab14b850 2024-04-26T20:52:04.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.948+0000 7fe1e5b1e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1d4003cb0 0x55f0ab1538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:04.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.948+0000 7fe1e5b1e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1d4003cb0 0x55f0ab1538f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39902/0 (socket says 172.21.15.73:39902) 2024-04-26T20:52:04.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.948+0000 7fe1e5b1e640 1 -- 172.21.15.73:0/3006635433 learned_addr learned my addr 172.21.15.73:0/3006635433 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:04.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.948+0000 7fe1e531d640 1 --2- 172.21.15.73:0/3006635433 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55f0ab14b850 0x55f0ab1ee920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:04.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.948+0000 7fe1dcb1c640 1 --2- 172.21.15.73:0/3006635433 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0ab151820 0x55f0ab1533b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:04.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.948+0000 7fe1e5b1e640 1 -- 172.21.15.73:0/3006635433 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55f0ab14b850 msgr2=0x55f0ab1ee920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:04.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.948+0000 7fe1e5b1e640 1 --2- 172.21.15.73:0/3006635433 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55f0ab14b850 0x55f0ab1ee920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:04.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.948+0000 7fe1e5b1e640 1 -- 172.21.15.73:0/3006635433 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0ab151820 msgr2=0x55f0ab1533b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:04.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.948+0000 7fe1e5b1e640 1 --2- 172.21.15.73:0/3006635433 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55f0ab151820 0x55f0ab1533b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:04.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.948+0000 7fe1e5b1e640 1 -- 172.21.15.73:0/3006635433 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0ab14c120 con 0x7fe1d4003cb0 2024-04-26T20:52:04.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.948+0000 7fe1e531d640 1 --2- 172.21.15.73:0/3006635433 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55f0ab14b850 0x55f0ab1ee920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:52:04.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.949+0000 7fe1e5b1e640 1 --2- 172.21.15.73:0/3006635433 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1d4003cb0 0x55f0ab1538f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0025890 tx=0x7fe1e0054b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:04.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.949+0000 7fe1deffd640 1 -- 172.21.15.73:0/3006635433 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e0058070 con 0x7fe1d4003cb0 2024-04-26T20:52:04.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.949+0000 7fe1deffd640 1 -- 172.21.15.73:0/3006635433 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1e0059040 con 0x7fe1d4003cb0 2024-04-26T20:52:04.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.949+0000 7fe1e7823f40 1 -- 172.21.15.73:0/3006635433 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f0ab1de8d0 con 0x7fe1d4003cb0 2024-04-26T20:52:04.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.949+0000 7fe1deffd640 1 -- 172.21.15.73:0/3006635433 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e0006d50 con 0x7fe1d4003cb0 2024-04-26T20:52:04.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.949+0000 7fe1e7823f40 1 -- 172.21.15.73:0/3006635433 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0ab1dee90 con 0x7fe1d4003cb0 2024-04-26T20:52:04.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.950+0000 7fe1deffd640 1 -- 172.21.15.73:0/3006635433 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe1e0054030 con 0x7fe1d4003cb0 2024-04-26T20:52:04.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.950+0000 7fe1deffd640 1 --2- 172.21.15.73:0/3006635433 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe1c806b2d0 0x7fe1c806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:04.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.950+0000 7fe1deffd640 1 -- 172.21.15.73:0/3006635433 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6187+0+0 (secure 0 0 0) 0x7fe1e00ce220 con 0x7fe1d4003cb0 2024-04-26T20:52:04.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.950+0000 7fe1e531d640 1 --2- 172.21.15.73:0/3006635433 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe1c806b2d0 0x7fe1c806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:04.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.951+0000 7fe1e7823f40 1 --2- 172.21.15.73:0/3006635433 >> [v2:172.21.15.120:6816/2915574674,v1:172.21.15.120:6818/2915574674] conn(0x55f0ab14a7f0 0x55f0ab1dd460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:04.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.951+0000 7fe1e7823f40 1 -- 172.21.15.73:0/3006635433 --> [v2:172.21.15.120:6816/2915574674,v1:172.21.15.120:6818/2915574674] -- osd_op(unknown.0.0:1 5.10 5:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) -- 0x55f0ab1dd9a0 con 0x55f0ab14a7f0 2024-04-26T20:52:04.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.951+0000 7fe1e531d640 1 --2- 172.21.15.73:0/3006635433 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe1c806b2d0 0x7fe1c806d790 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8000f60 tx=0x7fe1d8000fe0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:04.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.951+0000 7fe1dcb1c640 1 --2- 172.21.15.73:0/3006635433 >> [v2:172.21.15.120:6816/2915574674,v1:172.21.15.120:6818/2915574674] conn(0x55f0ab14a7f0 0x55f0ab1dd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:04.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.951+0000 7fe1dcb1c640 1 --2- 172.21.15.73:0/3006635433 >> [v2:172.21.15.120:6816/2915574674,v1:172.21.15.120:6818/2915574674] conn(0x55f0ab14a7f0 0x55f0ab1dd460 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:04.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.952+0000 7fe1dcb1c640 1 -- 172.21.15.73:0/3006635433 <== osd.7 v2:172.21.15.120:6816/2915574674 1 ==== osd_op_reply(1 10000000000.00000000 [getxattr (64) out=64b] v0'0 uv17 ondisk = 0) ==== 164+0+64 (crc 0 0 0) 0x7fe1d0015410 con 0x55f0ab14a7f0 2024-04-26T20:52:04.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.952+0000 7fe1e7823f40 1 -- 172.21.15.73:0/3006635433 >> [v2:172.21.15.120:6816/2915574674,v1:172.21.15.120:6818/2915574674] conn(0x55f0ab14a7f0 msgr2=0x55f0ab1dd460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:04.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.952+0000 7fe1e7823f40 1 --2- 172.21.15.73:0/3006635433 >> [v2:172.21.15.120:6816/2915574674,v1:172.21.15.120:6818/2915574674] conn(0x55f0ab14a7f0 0x55f0ab1dd460 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:04.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.952+0000 7fe1e7823f40 1 -- 172.21.15.73:0/3006635433 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe1c806b2d0 msgr2=0x7fe1c806d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:04.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.952+0000 7fe1e7823f40 1 --2- 172.21.15.73:0/3006635433 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe1c806b2d0 0x7fe1c806d790 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8000f60 tx=0x7fe1d8000fe0 comp rx=0 tx=0).stop 2024-04-26T20:52:04.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.952+0000 7fe1e7823f40 1 -- 172.21.15.73:0/3006635433 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1d4003cb0 msgr2=0x55f0ab1538f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:04.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.952+0000 7fe1e7823f40 1 --2- 172.21.15.73:0/3006635433 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1d4003cb0 0x55f0ab1538f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0025890 tx=0x7fe1e0054b70 comp rx=0 tx=0).stop 2024-04-26T20:52:04.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.952+0000 7fe1e5b1e640 1 -- 172.21.15.73:0/3006635433 reap_dead start 2024-04-26T20:52:04.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.952+0000 7fe1e7823f40 1 -- 172.21.15.73:0/3006635433 shutdown_connections 2024-04-26T20:52:04.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.952+0000 7fe1e7823f40 1 -- 172.21.15.73:0/3006635433 >> 172.21.15.73:0/3006635433 conn(0x55f0ab0a98a0 msgr2=0x55f0ab147550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:04.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.952+0000 7fe1e7823f40 1 -- 172.21.15.73:0/3006635433 shutdown_connections 2024-04-26T20:52:04.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:04.952+0000 7fe1e7823f40 1 -- 172.21.15.73:0/3006635433 wait complete. 2024-04-26T20:52:04.958 INFO:teuthology.orchestra.run.smithi073.stdout::testdir1 2024-04-26T20:52:04.958 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T20:52:04.959 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T20:52:05.039 DEBUG:teuthology.orchestra.run.smithi120:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T20:52:05.076 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T20:52:05.076 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T20:52:05.076 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-26T20:52:05.077 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:52:05.095 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-26T20:52:05.095 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:52:05.151 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:52:05.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.286+0000 7fea85b5f640 1 Processor -- start 2024-04-26T20:52:05.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.286+0000 7fea85b5f640 1 -- start start 2024-04-26T20:52:05.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.287+0000 7fea85b5f640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fea8006fc40 0x7fea80073360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:05.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.287+0000 7fea85b5f640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea80156840 con 0x7fea8006f790 2024-04-26T20:52:05.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.287+0000 7fea85b5f640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea800cbf20 con 0x7fea80156460 2024-04-26T20:52:05.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.287+0000 7fea85b5f640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7fea800cc200 con 0x7fea8006fc40 2024-04-26T20:52:05.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.287+0000 7fea7ffff640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fea8006fc40 0x7fea80073360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:05.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.287+0000 7fea7effd640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fea8006f790 0x7fea80072c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:39560/0 (socket says 172.21.15.73:39560) 2024-04-26T20:52:05.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.287+0000 7fea7effd640 1 -- 172.21.15.73:0/2001700281 learned_addr learned my addr 172.21.15.73:0/2001700281 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:52:05.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.287+0000 7fea7ffff640 1 -- 172.21.15.73:0/2001700281 >> v1:172.21.15.120:6789/0 conn(0x7fea80156460 legacy=0x7fea800cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:05.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.287+0000 7fea7ffff640 1 -- 172.21.15.73:0/2001700281 >> v1:172.21.15.73:6789/0 conn(0x7fea8006f790 legacy=0x7fea80072c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:05.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.287+0000 7fea7ffff640 1 -- 172.21.15.73:0/2001700281 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea800cc4e0 con 0x7fea8006fc40 2024-04-26T20:52:05.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.288+0000 7fea7ffff640 1 --2- 172.21.15.73:0/2001700281 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea8006fc40 0x7fea80073360 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fea78009d90 tx=0x7fea7802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a362b0533ac425c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:05.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.288+0000 7fea7e7fc640 1 -- 172.21.15.73:0/2001700281 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea78007740 con 0x7fea8006fc40 2024-04-26T20:52:05.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.288+0000 7fea7e7fc640 1 -- 172.21.15.73:0/2001700281 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea780078e0 con 0x7fea8006fc40 2024-04-26T20:52:05.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.288+0000 7fea7e7fc640 1 -- 172.21.15.73:0/2001700281 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea78007cb0 con 0x7fea8006fc40 2024-04-26T20:52:05.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.289+0000 7fea85b5f640 1 -- 172.21.15.73:0/2001700281 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea8006fc40 msgr2=0x7fea80073360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:05.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.289+0000 7fea85b5f640 1 --2- 172.21.15.73:0/2001700281 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea8006fc40 0x7fea80073360 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fea78009d90 tx=0x7fea7802a6a0 comp rx=0 tx=0).stop 2024-04-26T20:52:05.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.289+0000 7fea85b5f640 1 -- 172.21.15.73:0/2001700281 shutdown_connections 2024-04-26T20:52:05.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.289+0000 7fea85b5f640 1 --2- 172.21.15.73:0/2001700281 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea8006fc40 0x7fea80073360 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:05.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.289+0000 7fea85b5f640 1 -- 172.21.15.73:0/2001700281 >> 172.21.15.73:0/2001700281 conn(0x7fea800d0be0 msgr2=0x7fea800d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:05.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.289+0000 7fea85b5f640 1 -- 172.21.15.73:0/2001700281 shutdown_connections 2024-04-26T20:52:05.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.289+0000 7fea85b5f640 1 -- 172.21.15.73:0/2001700281 wait complete. 2024-04-26T20:52:05.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.290+0000 7fea85b5f640 1 Processor -- start 2024-04-26T20:52:05.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.290+0000 7fea85b5f640 1 -- start start 2024-04-26T20:52:05.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.291+0000 7fea85b5f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea8006f790 0x7fea800c86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:05.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.291+0000 7fea85b5f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea8006fc40 0x7fea80151ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:05.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.291+0000 7fea85b5f640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fea80156460 0x7fea801e7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:05.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.291+0000 7fea85b5f640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fea80070050 con 0x7fea8006fc40 2024-04-26T20:52:05.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.291+0000 7fea85b5f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fea80072700 con 0x7fea8006f790 2024-04-26T20:52:05.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.291+0000 7fea85b5f640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fea800cbbe0 con 0x7fea80156460 2024-04-26T20:52:05.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.291+0000 7fea7effd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea8006fc40 0x7fea80151ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:05.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.291+0000 7fea7effd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea8006fc40 0x7fea80151ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39920/0 (socket says 172.21.15.73:39920) 2024-04-26T20:52:05.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.291+0000 7fea7effd640 1 -- 172.21.15.73:0/444186231 learned_addr learned my addr 172.21.15.73:0/444186231 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:05.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.291+0000 7fea7f7fe640 1 --2- 172.21.15.73:0/444186231 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea8006f790 0x7fea800c86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:05.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.291+0000 7fea7ffff640 1 --2- 172.21.15.73:0/444186231 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fea80156460 0x7fea801e7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:05.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.291+0000 7fea7effd640 1 -- 172.21.15.73:0/444186231 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fea80156460 msgr2=0x7fea801e7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:05.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.291+0000 7fea7effd640 1 --2- 172.21.15.73:0/444186231 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fea80156460 0x7fea801e7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:05.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.291+0000 7fea7effd640 1 -- 172.21.15.73:0/444186231 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea8006f790 msgr2=0x7fea800c86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:05.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.291+0000 7fea7effd640 1 --2- 172.21.15.73:0/444186231 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea8006f790 0x7fea800c86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:05.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.291+0000 7fea7effd640 1 -- 172.21.15.73:0/444186231 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea800c49f0 con 0x7fea8006fc40 2024-04-26T20:52:05.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.291+0000 7fea7ffff640 1 --2- 172.21.15.73:0/444186231 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fea80156460 0x7fea801e7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:52:05.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.291+0000 7fea7effd640 1 --2- 172.21.15.73:0/444186231 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea8006fc40 0x7fea80151ef0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fea7400c4d0 tx=0x7fea7400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:05.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.291+0000 7fea67fff640 1 -- 172.21.15.73:0/444186231 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea74017070 con 0x7fea8006fc40 2024-04-26T20:52:05.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.291+0000 7fea67fff640 1 -- 172.21.15.73:0/444186231 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea74005ce0 con 0x7fea8006fc40 2024-04-26T20:52:05.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.292+0000 7fea85b5f640 1 -- 172.21.15.73:0/444186231 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea800c8fc0 con 0x7fea8006fc40 2024-04-26T20:52:05.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.292+0000 7fea67fff640 1 -- 172.21.15.73:0/444186231 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea74007270 con 0x7fea8006fc40 2024-04-26T20:52:05.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.292+0000 7fea85b5f640 1 -- 172.21.15.73:0/444186231 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea800c4c70 con 0x7fea8006fc40 2024-04-26T20:52:05.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.293+0000 7fea85b5f640 1 -- 172.21.15.73:0/444186231 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea800c93b0 con 0x7fea8006fc40 2024-04-26T20:52:05.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.293+0000 7fea67fff640 1 -- 172.21.15.73:0/444186231 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fea7401a070 con 0x7fea8006fc40 2024-04-26T20:52:05.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.293+0000 7fea67fff640 1 --2- 172.21.15.73:0/444186231 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fea6c06c8f0 0x7fea6c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:05.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.293+0000 7fea67fff640 1 -- 172.21.15.73:0/444186231 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6187+0+0 (secure 0 0 0) 0x7fea7408af80 con 0x7fea8006fc40 2024-04-26T20:52:05.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.293+0000 7fea7f7fe640 1 --2- 172.21.15.73:0/444186231 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fea6c06c8f0 0x7fea6c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:05.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.294+0000 7fea7f7fe640 1 --2- 172.21.15.73:0/444186231 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fea6c06c8f0 0x7fea6c06edb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fea70006fd0 tx=0x7fea70009060 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:05.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.298+0000 7fea67fff640 1 -- 172.21.15.73:0/444186231 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fea800c49f0 con 0x7fea8006fc40 2024-04-26T20:52:05.311 INFO:tasks.ceph.mds.b.smithi120.stderr:2024-04-26T20:52:05.311+0000 7fbad4348640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T20:52:05.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.472+0000 7fea85b5f640 1 -- 172.21.15.73:0/444186231 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fea800694f0 con 0x7fea8006fc40 2024-04-26T20:52:05.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.473+0000 7fea67fff640 1 -- 172.21.15.73:0/444186231 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+17071 (secure 0 0 0) 0x7fea74058b80 con 0x7fea8006fc40 2024-04-26T20:52:05.474 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:52:05.474 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":40,"fsid":"fbd3f548-b547-4713-856b-09f98f5d1764","created":"2024-04-26T20:50:07.876786+0000","modified":"2024-04-26T20:51:58.783253+0000","last_up_change":"2024-04-26T20:50:19.082741+0000","last_in_change":"2024-04-26T20:50:10.018400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:50:19.993731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:51:43.110774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-26T20:51:44.235846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d6f4f3-214a-4392-a02f-c33cbcab3c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6809","nonce":3461705124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6811","nonce":3461705124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6815","nonce":3461705124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6813","nonce":3461705124}]},"public_addr":"172.21.15.73:6809/3461705124","cluster_addr":"172.21.15.73:6811/3461705124","heartbeat_back_addr":"172.21.15.73:6815/3461705124","heartbeat_front_addr":"172.21.15.73:6813/3461705124","state":["exists","up"]},{"osd":1,"uuid":"786bf8ca-6ddc-4b3c-be6c-9152398bd5fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6817","nonce":2827619403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6819","nonce":2827619403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6825","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6827","nonce":2827619403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6821","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6823","nonce":2827619403}]},"public_addr":"172.21.15.73:6817/2827619403","cluster_addr":"172.21.15.73:6819/2827619403","heartbeat_back_addr":"172.21.15.73:6827/2827619403","heartbeat_front_addr":"172.21.15.73:6823/2827619403","state":["exists","up"]},{"osd":2,"uuid":"90963b1a-5437-4ef8-ad8b-6b74c534eeb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6801","nonce":2542681064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6803","nonce":2542681064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6808","nonce":2542681064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6805","nonce":2542681064}]},"public_addr":"172.21.15.73:6801/2542681064","cluster_addr":"172.21.15.73:6803/2542681064","heartbeat_back_addr":"172.21.15.73:6808/2542681064","heartbeat_front_addr":"172.21.15.73:6805/2542681064","state":["exists","up"]},{"osd":3,"uuid":"bd3d2148-6e38-41b4-a453-fe6e3b8b4ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6822","nonce":1604042718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6826","nonce":1604042718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6831","nonce":1604042718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6829","nonce":1604042718}]},"public_addr":"172.21.15.73:6822/1604042718","cluster_addr":"172.21.15.73:6826/1604042718","heartbeat_back_addr":"172.21.15.73:6831/1604042718","heartbeat_front_addr":"172.21.15.73:6829/1604042718","state":["exists","up"]},{"osd":4,"uuid":"e8668dc8-a23f-46d2-8311-cb882da08e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6817","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6819","nonce":3591055561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6824","nonce":3591055561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6831","nonce":3591055561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6828","nonce":3591055561}]},"public_addr":"172.21.15.120:6819/3591055561","cluster_addr":"172.21.15.120:6824/3591055561","heartbeat_back_addr":"172.21.15.120:6831/3591055561","heartbeat_front_addr":"172.21.15.120:6828/3591055561","state":["exists","up"]},{"osd":5,"uuid":"cbac0bdb-96bd-4852-bcce-e564495c3fcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6809","nonce":819450433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6811","nonce":819450433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6815","nonce":819450433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6813","nonce":819450433}]},"public_addr":"172.21.15.120:6809/819450433","cluster_addr":"172.21.15.120:6811/819450433","heartbeat_back_addr":"172.21.15.120:6815/819450433","heartbeat_front_addr":"172.21.15.120:6813/819450433","state":["exists","up"]},{"osd":6,"uuid":"1884b2a8-569f-4272-96a9-f25a09364dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6801","nonce":3946210727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6803","nonce":3946210727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6807","nonce":3946210727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6805","nonce":3946210727}]},"public_addr":"172.21.15.120:6801/3946210727","cluster_addr":"172.21.15.120:6803/3946210727","heartbeat_back_addr":"172.21.15.120:6807/3946210727","heartbeat_front_addr":"172.21.15.120:6805/3946210727","state":["exists","up"]},{"osd":7,"uuid":"c0c3bf6d-a4c5-4f1e-aa2d-330951830666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6818","nonce":2915574674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6821","nonce":2915574674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6827","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6829","nonce":2915574674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6823","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6825","nonce":2915574674}]},"public_addr":"172.21.15.120:6818/2915574674","cluster_addr":"172.21.15.120:6821/2915574674","heartbeat_back_addr":"172.21.15.120:6829/2915574674","heartbeat_front_addr":"172.21.15.120:6825/2915574674","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.218216+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.208710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.112705+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.182789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.164324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.174212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.225396+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.261559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.120:6835/761435580":"2024-04-27T20:51:58.783228+0000","172.21.15.120:6833/761435580":"2024-04-27T20:51:58.783228+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:52:05.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.475+0000 7fea85b5f640 1 -- 172.21.15.73:0/444186231 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fea6c06c8f0 msgr2=0x7fea6c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:05.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.475+0000 7fea85b5f640 1 --2- 172.21.15.73:0/444186231 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fea6c06c8f0 0x7fea6c06edb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fea70006fd0 tx=0x7fea70009060 comp rx=0 tx=0).stop 2024-04-26T20:52:05.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.475+0000 7fea85b5f640 1 -- 172.21.15.73:0/444186231 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea8006fc40 msgr2=0x7fea80151ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:05.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.475+0000 7fea85b5f640 1 --2- 172.21.15.73:0/444186231 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea8006fc40 0x7fea80151ef0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fea7400c4d0 tx=0x7fea7400c990 comp rx=0 tx=0).stop 2024-04-26T20:52:05.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.475+0000 7fea85b5f640 1 -- 172.21.15.73:0/444186231 shutdown_connections 2024-04-26T20:52:05.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.475+0000 7fea85b5f640 1 --2- 172.21.15.73:0/444186231 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea8006fc40 0x7fea80151ef0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:05.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.475+0000 7fea85b5f640 1 --2- 172.21.15.73:0/444186231 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fea6c06c8f0 0x7fea6c06edb0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:05.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.475+0000 7fea85b5f640 1 --2- 172.21.15.73:0/444186231 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fea80156460 0x7fea801e7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:05.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.475+0000 7fea85b5f640 1 --2- 172.21.15.73:0/444186231 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea8006f790 0x7fea800c86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:05.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.475+0000 7fea85b5f640 1 -- 172.21.15.73:0/444186231 >> 172.21.15.73:0/444186231 conn(0x7fea800d0be0 msgr2=0x7fea800ce900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:05.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.475+0000 7fea85b5f640 1 -- 172.21.15.73:0/444186231 shutdown_connections 2024-04-26T20:52:05.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.475+0000 7fea85b5f640 1 -- 172.21.15.73:0/444186231 wait complete. 2024-04-26T20:52:05.487 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:50:19.993731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:51:43.110774+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:51:44.235846+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:52:05.487 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:52:05.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.585+0000 7f2680e7b640 1 Processor -- start 2024-04-26T20:52:05.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.585+0000 7f2680e7b640 1 -- start start 2024-04-26T20:52:05.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.586+0000 7f2680e7b640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f267c15f210 0x7f267c0c7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:05.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.586+0000 7f2680e7b640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f267c0736c0 0x7f267c06c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:05.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.586+0000 7f2680e7b640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f267c073ca0 con 0x7f267c06c240 2024-04-26T20:52:05.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.586+0000 7f2680e7b640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f267c073f80 con 0x7f267c0736c0 2024-04-26T20:52:05.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.586+0000 7f2680e7b640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f267c0c7110 con 0x7f267c15f210 2024-04-26T20:52:05.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.586+0000 7f267ad9d640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f267c0736c0 0x7f267c06c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:05.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.587+0000 7f267ad9d640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f267c0736c0 0x7f267c06c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39930/0 (socket says 172.21.15.73:39930) 2024-04-26T20:52:05.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.587+0000 7f267ad9d640 1 -- 172.21.15.73:0/3526020767 learned_addr learned my addr 172.21.15.73:0/3526020767 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:05.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.587+0000 7f267a59c640 1 --2- 172.21.15.73:0/3526020767 >> v2:172.21.15.120:3300/0 conn(0x7f267c15f210 0x7f267c0c7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:05.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.587+0000 7f267ad9d640 1 -- 172.21.15.73:0/3526020767 >> v2:172.21.15.120:3300/0 conn(0x7f267c15f210 msgr2=0x7f267c0c7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:05.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.587+0000 7f267ad9d640 1 --2- 172.21.15.73:0/3526020767 >> v2:172.21.15.120:3300/0 conn(0x7f267c15f210 0x7f267c0c7bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:05.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.587+0000 7f267ad9d640 1 -- 172.21.15.73:0/3526020767 >> v1:172.21.15.120:6789/0 conn(0x7f267c06c240 legacy=0x7f267c072fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:05.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.587+0000 7f267ad9d640 1 -- 172.21.15.73:0/3526020767 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f267c0c73f0 con 0x7f267c0736c0 2024-04-26T20:52:05.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.587+0000 7f267ad9d640 1 --2- 172.21.15.73:0/3526020767 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f267c0736c0 0x7f267c06c6d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f267c0cf810 tx=0x7f267402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4be375dad51076bf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:05.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.588+0000 7f267959a640 1 -- 172.21.15.73:0/3526020767 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2674036070 con 0x7f267c0736c0 2024-04-26T20:52:05.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.588+0000 7f267959a640 1 -- 172.21.15.73:0/3526020767 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2674031600 con 0x7f267c0736c0 2024-04-26T20:52:05.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.588+0000 7f267959a640 1 -- 172.21.15.73:0/3526020767 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26740319d0 con 0x7f267c0736c0 2024-04-26T20:52:05.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.588+0000 7f2680e7b640 1 -- 172.21.15.73:0/3526020767 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f267c0736c0 msgr2=0x7f267c06c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:05.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.588+0000 7f2680e7b640 1 --2- 172.21.15.73:0/3526020767 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f267c0736c0 0x7f267c06c6d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f267c0cf810 tx=0x7f267402a5d0 comp rx=0 tx=0).stop 2024-04-26T20:52:05.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.588+0000 7f2680e7b640 1 -- 172.21.15.73:0/3526020767 shutdown_connections 2024-04-26T20:52:05.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.588+0000 7f2680e7b640 1 --2- 172.21.15.73:0/3526020767 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f267c0736c0 0x7f267c06c6d0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:05.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.588+0000 7f2680e7b640 1 --2- 172.21.15.73:0/3526020767 >> v2:172.21.15.120:3300/0 conn(0x7f267c15f210 0x7f267c0c7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:05.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.588+0000 7f2680e7b640 1 -- 172.21.15.73:0/3526020767 >> 172.21.15.73:0/3526020767 conn(0x7f267c0d1c90 msgr2=0x7f267c0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:05.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.588+0000 7f2680e7b640 1 -- 172.21.15.73:0/3526020767 shutdown_connections 2024-04-26T20:52:05.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.588+0000 7f2680e7b640 1 -- 172.21.15.73:0/3526020767 wait complete. 2024-04-26T20:52:05.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.590+0000 7f2680e7b640 1 Processor -- start 2024-04-26T20:52:05.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.590+0000 7f2680e7b640 1 -- start start 2024-04-26T20:52:05.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.590+0000 7f2680e7b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f267c06c240 0x7f267c15f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:05.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.590+0000 7f2680e7b640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f267c0736c0 0x7f267c0c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:05.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.590+0000 7f2680e7b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f267c15f210 0x7f267c159c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:05.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.590+0000 7f2680e7b640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f267c0cc670 con 0x7f267c06c240 2024-04-26T20:52:05.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.590+0000 7f2680e7b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f267c0cc4f0 con 0x7f267c15f210 2024-04-26T20:52:05.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.590+0000 7f2680e7b640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f267c0cc7f0 con 0x7f267c0736c0 2024-04-26T20:52:05.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.590+0000 7f267a59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f267c06c240 0x7f267c15f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:05.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.590+0000 7f267a59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f267c06c240 0x7f267c15f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39934/0 (socket says 172.21.15.73:39934) 2024-04-26T20:52:05.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.590+0000 7f267a59c640 1 -- 172.21.15.73:0/1440039425 learned_addr learned my addr 172.21.15.73:0/1440039425 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:05.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.591+0000 7f267a59c640 1 -- 172.21.15.73:0/1440039425 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f267c0736c0 msgr2=0x7f267c0c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:05.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.591+0000 7f2679d9b640 1 --2- 172.21.15.73:0/1440039425 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f267c0736c0 0x7f267c0c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:05.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.591+0000 7f267ad9d640 1 --2- 172.21.15.73:0/1440039425 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f267c15f210 0x7f267c159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:05.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.591+0000 7f267a59c640 1 --2- 172.21.15.73:0/1440039425 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f267c0736c0 0x7f267c0c9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:05.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.591+0000 7f267a59c640 1 -- 172.21.15.73:0/1440039425 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f267c15f210 msgr2=0x7f267c159c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:05.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.591+0000 7f267a59c640 1 --2- 172.21.15.73:0/1440039425 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f267c15f210 0x7f267c159c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:05.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.591+0000 7f267a59c640 1 -- 172.21.15.73:0/1440039425 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f267c0ca400 con 0x7f267c06c240 2024-04-26T20:52:05.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.591+0000 7f267ad9d640 1 --2- 172.21.15.73:0/1440039425 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f267c15f210 0x7f267c159c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:52:05.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.591+0000 7f2679d9b640 1 --2- 172.21.15.73:0/1440039425 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f267c0736c0 0x7f267c0c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:52:05.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.591+0000 7f267a59c640 1 --2- 172.21.15.73:0/1440039425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f267c06c240 0x7f267c15f5f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f267000c7f0 tx=0x7f267000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:05.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.591+0000 7f266affd640 1 -- 172.21.15.73:0/1440039425 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2670015070 con 0x7f267c06c240 2024-04-26T20:52:05.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.591+0000 7f266affd640 1 -- 172.21.15.73:0/1440039425 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2670004220 con 0x7f267c06c240 2024-04-26T20:52:05.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.591+0000 7f2680e7b640 1 -- 172.21.15.73:0/1440039425 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f267c0ca6e0 con 0x7f267c06c240 2024-04-26T20:52:05.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.591+0000 7f266affd640 1 -- 172.21.15.73:0/1440039425 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2670007620 con 0x7f267c06c240 2024-04-26T20:52:05.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.591+0000 7f2680e7b640 1 -- 172.21.15.73:0/1440039425 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f267c0caca0 con 0x7f267c06c240 2024-04-26T20:52:05.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.592+0000 7f2680e7b640 1 -- 172.21.15.73:0/1440039425 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f267c0694f0 con 0x7f267c06c240 2024-04-26T20:52:05.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.593+0000 7f266affd640 1 -- 172.21.15.73:0/1440039425 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f26700079d0 con 0x7f267c06c240 2024-04-26T20:52:05.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.593+0000 7f266affd640 1 --2- 172.21.15.73:0/1440039425 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f265c06c920 0x7f265c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:05.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.593+0000 7f266affd640 1 -- 172.21.15.73:0/1440039425 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6187+0+0 (secure 0 0 0) 0x7f267008bd30 con 0x7f267c06c240 2024-04-26T20:52:05.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.593+0000 7f2679d9b640 1 --2- 172.21.15.73:0/1440039425 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f265c06c920 0x7f265c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:05.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.593+0000 7f2679d9b640 1 --2- 172.21.15.73:0/1440039425 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f265c06c920 0x7f265c06ede0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f267c1f8510 tx=0x7f266c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:05.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.597+0000 7f266affd640 1 -- 172.21.15.73:0/1440039425 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2670059f80 con 0x7f267c06c240 2024-04-26T20:52:05.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.771+0000 7f2680e7b640 1 -- 172.21.15.73:0/1440039425 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f267c0d2b30 con 0x7f267c06c240 2024-04-26T20:52:05.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.771+0000 7f266affd640 1 -- 172.21.15.73:0/1440039425 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) ==== 93+0+10 (secure 0 0 0) 0x7f2670059880 con 0x7f267c06c240 2024-04-26T20:52:05.772 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-26T20:52:05.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.773+0000 7f2680e7b640 1 -- 172.21.15.73:0/1440039425 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f265c06c920 msgr2=0x7f265c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:05.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.773+0000 7f2680e7b640 1 --2- 172.21.15.73:0/1440039425 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f265c06c920 0x7f265c06ede0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f267c1f8510 tx=0x7f266c009040 comp rx=0 tx=0).stop 2024-04-26T20:52:05.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.773+0000 7f2680e7b640 1 -- 172.21.15.73:0/1440039425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f267c06c240 msgr2=0x7f267c15f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:05.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.773+0000 7f2680e7b640 1 --2- 172.21.15.73:0/1440039425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f267c06c240 0x7f267c15f5f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f267000c7f0 tx=0x7f267000ccb0 comp rx=0 tx=0).stop 2024-04-26T20:52:05.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.773+0000 7f2680e7b640 1 -- 172.21.15.73:0/1440039425 shutdown_connections 2024-04-26T20:52:05.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.773+0000 7f2680e7b640 1 --2- 172.21.15.73:0/1440039425 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f265c06c920 0x7f265c06ede0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:05.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.773+0000 7f2680e7b640 1 --2- 172.21.15.73:0/1440039425 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f267c15f210 0x7f267c159c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:05.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.773+0000 7f2680e7b640 1 --2- 172.21.15.73:0/1440039425 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f267c0736c0 0x7f267c0c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:05.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.773+0000 7f2680e7b640 1 --2- 172.21.15.73:0/1440039425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f267c06c240 0x7f267c15f5f0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:05.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.773+0000 7f2680e7b640 1 -- 172.21.15.73:0/1440039425 >> 172.21.15.73:0/1440039425 conn(0x7f267c0d1c90 msgr2=0x7f267c0ccbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:05.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.773+0000 7f2680e7b640 1 -- 172.21.15.73:0/1440039425 shutdown_connections 2024-04-26T20:52:05.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.773+0000 7f2680e7b640 1 -- 172.21.15.73:0/1440039425 wait complete. 2024-04-26T20:52:05.786 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:52:05.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.886+0000 7ffa634bd640 1 Processor -- start 2024-04-26T20:52:05.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.886+0000 7ffa634bd640 1 -- start start 2024-04-26T20:52:05.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.886+0000 7ffa634bd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7ffa5c0d1240 0x7ffa5c0d1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:05.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.887+0000 7ffa634bd640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa5c0d0690 con 0x7ffa5c0b3610 2024-04-26T20:52:05.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.887+0000 7ffa634bd640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa5c0d1b60 con 0x7ffa5c0b4360 2024-04-26T20:52:05.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.887+0000 7ffa634bd640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7ffa5c0cf210 con 0x7ffa5c0d1240 2024-04-26T20:52:05.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.887+0000 7ffa61a5a640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7ffa5c0b3610 0x7ffa5c0cff80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:39562/0 (socket says 172.21.15.73:39562) 2024-04-26T20:52:05.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.887+0000 7ffa61a5a640 1 -- 172.21.15.73:0/1517570574 learned_addr learned my addr 172.21.15.73:0/1517570574 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:52:05.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.887+0000 7ffa61259640 1 --2- 172.21.15.73:0/1517570574 >> v2:172.21.15.120:3300/0 conn(0x7ffa5c0d1240 0x7ffa5c0d1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:05.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.887+0000 7ffa53fff640 1 -- 172.21.15.73:0/1517570574 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1621745338 0 0) 0x7ffa5c0d0690 con 0x7ffa5c0b3610 2024-04-26T20:52:05.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.887+0000 7ffa53fff640 1 -- 172.21.15.73:0/1517570574 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa48003520 con 0x7ffa5c0b3610 2024-04-26T20:52:05.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.887+0000 7ffa53fff640 1 -- 172.21.15.73:0/1517570574 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3164755509 0 0) 0x7ffa48003520 con 0x7ffa5c0b3610 2024-04-26T20:52:05.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.887+0000 7ffa53fff640 1 -- 172.21.15.73:0/1517570574 >> v2:172.21.15.120:3300/0 conn(0x7ffa5c0d1240 msgr2=0x7ffa5c0d1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:05.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.887+0000 7ffa53fff640 1 --2- 172.21.15.73:0/1517570574 >> v2:172.21.15.120:3300/0 conn(0x7ffa5c0d1240 0x7ffa5c0d1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:05.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.887+0000 7ffa53fff640 1 -- 172.21.15.73:0/1517570574 >> v1:172.21.15.120:6789/0 conn(0x7ffa5c0b4360 legacy=0x7ffa5c0cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:05.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.887+0000 7ffa53fff640 1 -- 172.21.15.73:0/1517570574 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa5c0ce5e0 con 0x7ffa5c0b3610 2024-04-26T20:52:05.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.887+0000 7ffa53fff640 1 -- 172.21.15.73:0/1517570574 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7ffa540019f0 con 0x7ffa5c0b3610 2024-04-26T20:52:05.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.887+0000 7ffa53fff640 1 -- 172.21.15.73:0/1517570574 >> v1:172.21.15.73:6789/0 conn(0x7ffa5c0b3610 legacy=0x7ffa5c0cff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:05.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.887+0000 7ffa53fff640 1 --2- 172.21.15.73:0/1517570574 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ffa480039b0 0x7ffa48003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:05.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.887+0000 7ffa53fff640 1 --2- 172.21.15.73:0/1517570574 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa48004800 0x7ffa48004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:05.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.887+0000 7ffa53fff640 1 --2- 172.21.15.73:0/1517570574 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa480051c0 0x7ffa4800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:05.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.887+0000 7ffa53fff640 1 -- 172.21.15.73:0/1517570574 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5c0cf210 con 0x7ffa48004800 2024-04-26T20:52:05.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.887+0000 7ffa53fff640 1 -- 172.21.15.73:0/1517570574 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4800c060 con 0x7ffa480051c0 2024-04-26T20:52:05.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.887+0000 7ffa53fff640 1 -- 172.21.15.73:0/1517570574 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7ffa4800c260 con 0x7ffa480039b0 2024-04-26T20:52:05.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.887+0000 7ffa60a58640 1 --2- 172.21.15.73:0/1517570574 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa48004800 0x7ffa48004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:05.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.887+0000 7ffa61259640 1 --2- 172.21.15.73:0/1517570574 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ffa480039b0 0x7ffa48003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:05.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.887+0000 7ffa61a5a640 1 --2- 172.21.15.73:0/1517570574 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa480051c0 0x7ffa4800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:05.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.888+0000 7ffa60a58640 1 -- 172.21.15.73:0/1517570574 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ffa480039b0 msgr2=0x7ffa48003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:05.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.888+0000 7ffa60a58640 1 --2- 172.21.15.73:0/1517570574 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ffa480039b0 0x7ffa48003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:05.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.888+0000 7ffa60a58640 1 -- 172.21.15.73:0/1517570574 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa480051c0 msgr2=0x7ffa4800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:05.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.888+0000 7ffa60a58640 1 --2- 172.21.15.73:0/1517570574 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa480051c0 0x7ffa4800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:05.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.888+0000 7ffa60a58640 1 -- 172.21.15.73:0/1517570574 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa48003520 con 0x7ffa48004800 2024-04-26T20:52:05.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.888+0000 7ffa61a5a640 1 -- 172.21.15.73:0/1517570574 reap_dead start 2024-04-26T20:52:05.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.888+0000 7ffa60a58640 1 --2- 172.21.15.73:0/1517570574 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa48004800 0x7ffa48004c80 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c001750 tx=0x7ffa48005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3fb91e1a2614514 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:05.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.888+0000 7ffa53fff640 1 -- 172.21.15.73:0/1517570574 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c001370 con 0x7ffa48004800 2024-04-26T20:52:05.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.888+0000 7ffa53fff640 1 -- 172.21.15.73:0/1517570574 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa4c0052d0 con 0x7ffa48004800 2024-04-26T20:52:05.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.888+0000 7ffa53fff640 1 -- 172.21.15.73:0/1517570574 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c02abc0 con 0x7ffa48004800 2024-04-26T20:52:05.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.889+0000 7ffa634bd640 1 -- 172.21.15.73:0/1517570574 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa48004800 msgr2=0x7ffa48004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:05.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.889+0000 7ffa634bd640 1 --2- 172.21.15.73:0/1517570574 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa48004800 0x7ffa48004c80 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c001750 tx=0x7ffa48005830 comp rx=0 tx=0).stop 2024-04-26T20:52:05.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.889+0000 7ffa634bd640 1 -- 172.21.15.73:0/1517570574 shutdown_connections 2024-04-26T20:52:05.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.889+0000 7ffa634bd640 1 --2- 172.21.15.73:0/1517570574 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa48004800 0x7ffa48004c80 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:05.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.889+0000 7ffa634bd640 1 -- 172.21.15.73:0/1517570574 >> 172.21.15.73:0/1517570574 conn(0x7ffa5c0d2e60 msgr2=0x7ffa5c0d3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:05.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.889+0000 7ffa634bd640 1 -- 172.21.15.73:0/1517570574 shutdown_connections 2024-04-26T20:52:05.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.889+0000 7ffa634bd640 1 -- 172.21.15.73:0/1517570574 wait complete. 2024-04-26T20:52:05.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.890+0000 7ffa634bd640 1 Processor -- start 2024-04-26T20:52:05.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.890+0000 7ffa634bd640 1 -- start start 2024-04-26T20:52:05.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.891+0000 7ffa634bd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa48004800 0x7ffa5c0c7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:05.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.891+0000 7ffa634bd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa5c1f2af0 0x7ffa5c0cc650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:05.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.891+0000 7ffa634bd640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ffa5c1511e0 0x7ffa5c0c83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:05.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.891+0000 7ffa634bd640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5c0cf460 con 0x7ffa48004800 2024-04-26T20:52:05.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.891+0000 7ffa634bd640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5c160ab0 con 0x7ffa5c1f2af0 2024-04-26T20:52:05.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.891+0000 7ffa634bd640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7ffa5c0c6610 con 0x7ffa5c1511e0 2024-04-26T20:52:05.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.891+0000 7ffa61259640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa48004800 0x7ffa5c0c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:05.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.891+0000 7ffa61259640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa48004800 0x7ffa5c0c7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39956/0 (socket says 172.21.15.73:39956) 2024-04-26T20:52:05.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.891+0000 7ffa61259640 1 -- 172.21.15.73:0/4267779267 learned_addr learned my addr 172.21.15.73:0/4267779267 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:05.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.891+0000 7ffa61a5a640 1 --2- 172.21.15.73:0/4267779267 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ffa5c1511e0 0x7ffa5c0c83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:05.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.891+0000 7ffa60a58640 1 --2- 172.21.15.73:0/4267779267 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa5c1f2af0 0x7ffa5c0cc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:05.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.891+0000 7ffa61259640 1 -- 172.21.15.73:0/4267779267 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ffa5c1511e0 msgr2=0x7ffa5c0c83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:05.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.891+0000 7ffa61259640 1 --2- 172.21.15.73:0/4267779267 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ffa5c1511e0 0x7ffa5c0c83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:05.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.891+0000 7ffa61259640 1 -- 172.21.15.73:0/4267779267 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa5c1f2af0 msgr2=0x7ffa5c0cc650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:05.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.891+0000 7ffa61259640 1 --2- 172.21.15.73:0/4267779267 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa5c1f2af0 0x7ffa5c0cc650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:05.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.891+0000 7ffa61259640 1 -- 172.21.15.73:0/4267779267 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa5c1f2f70 con 0x7ffa48004800 2024-04-26T20:52:05.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.891+0000 7ffa61259640 1 --2- 172.21.15.73:0/4267779267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa48004800 0x7ffa5c0c7600 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ffa58011960 tx=0x7ffa58011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:05.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.892+0000 7ffa51ffb640 1 -- 172.21.15.73:0/4267779267 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5800faf0 con 0x7ffa48004800 2024-04-26T20:52:05.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.892+0000 7ffa51ffb640 1 -- 172.21.15.73:0/4267779267 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa5800fc90 con 0x7ffa48004800 2024-04-26T20:52:05.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.892+0000 7ffa634bd640 1 -- 172.21.15.73:0/4267779267 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa5c0c9f30 con 0x7ffa48004800 2024-04-26T20:52:05.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.892+0000 7ffa51ffb640 1 -- 172.21.15.73:0/4267779267 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa58014440 con 0x7ffa48004800 2024-04-26T20:52:05.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.892+0000 7ffa634bd640 1 -- 172.21.15.73:0/4267779267 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa5c0ca4f0 con 0x7ffa48004800 2024-04-26T20:52:05.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.893+0000 7ffa634bd640 1 -- 172.21.15.73:0/4267779267 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa5c0694f0 con 0x7ffa48004800 2024-04-26T20:52:05.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.893+0000 7ffa51ffb640 1 -- 172.21.15.73:0/4267779267 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffa58012040 con 0x7ffa48004800 2024-04-26T20:52:05.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.893+0000 7ffa51ffb640 1 --2- 172.21.15.73:0/4267779267 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ffa4006c8f0 0x7ffa4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:05.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.894+0000 7ffa51ffb640 1 -- 172.21.15.73:0/4267779267 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6187+0+0 (secure 0 0 0) 0x7ffa5808c270 con 0x7ffa48004800 2024-04-26T20:52:05.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.894+0000 7ffa60a58640 1 --2- 172.21.15.73:0/4267779267 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ffa4006c8f0 0x7ffa4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:05.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.894+0000 7ffa60a58640 1 --2- 172.21.15.73:0/4267779267 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ffa4006c8f0 0x7ffa4006edb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c032000 tx=0x7ffa4c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:05.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:05.898+0000 7ffa51ffb640 1 -- 172.21.15.73:0/4267779267 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ffa58019030 con 0x7ffa48004800 2024-04-26T20:52:06.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.072+0000 7ffa634bd640 1 -- 172.21.15.73:0/4267779267 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ffa5c06cea0 con 0x7ffa48004800 2024-04-26T20:52:06.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.072+0000 7ffa51ffb640 1 -- 172.21.15.73:0/4267779267 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) ==== 104+0+11 (secure 0 0 0) 0x7ffa58059e70 con 0x7ffa48004800 2024-04-26T20:52:06.073 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-26T20:52:06.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.074+0000 7ffa634bd640 1 -- 172.21.15.73:0/4267779267 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ffa4006c8f0 msgr2=0x7ffa4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:06.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.074+0000 7ffa634bd640 1 --2- 172.21.15.73:0/4267779267 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ffa4006c8f0 0x7ffa4006edb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c032000 tx=0x7ffa4c02e000 comp rx=0 tx=0).stop 2024-04-26T20:52:06.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.074+0000 7ffa634bd640 1 -- 172.21.15.73:0/4267779267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa48004800 msgr2=0x7ffa5c0c7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:06.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.074+0000 7ffa634bd640 1 --2- 172.21.15.73:0/4267779267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa48004800 0x7ffa5c0c7600 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ffa58011960 tx=0x7ffa58011e20 comp rx=0 tx=0).stop 2024-04-26T20:52:06.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.074+0000 7ffa634bd640 1 -- 172.21.15.73:0/4267779267 shutdown_connections 2024-04-26T20:52:06.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.074+0000 7ffa634bd640 1 --2- 172.21.15.73:0/4267779267 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ffa4006c8f0 0x7ffa4006edb0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:06.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.074+0000 7ffa634bd640 1 --2- 172.21.15.73:0/4267779267 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ffa5c1511e0 0x7ffa5c0c83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:06.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.074+0000 7ffa634bd640 1 --2- 172.21.15.73:0/4267779267 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa5c1f2af0 0x7ffa5c0cc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:06.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.074+0000 7ffa634bd640 1 --2- 172.21.15.73:0/4267779267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa48004800 0x7ffa5c0c7600 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:06.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.074+0000 7ffa634bd640 1 -- 172.21.15.73:0/4267779267 >> 172.21.15.73:0/4267779267 conn(0x7ffa5c0d2e60 msgr2=0x7ffa5c15e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:06.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.074+0000 7ffa634bd640 1 -- 172.21.15.73:0/4267779267 shutdown_connections 2024-04-26T20:52:06.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.074+0000 7ffa634bd640 1 -- 172.21.15.73:0/4267779267 wait complete. 2024-04-26T20:52:06.086 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:52:06.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.184+0000 7fc20346d640 1 Processor -- start 2024-04-26T20:52:06.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.184+0000 7fc20346d640 1 -- start start 2024-04-26T20:52:06.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.185+0000 7fc20346d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc1fc0d1720 0x7fc1fc0d0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:06.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.185+0000 7fc20346d640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1fc0701e0 con 0x7fc1fc15d250 2024-04-26T20:52:06.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.185+0000 7fc20346d640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1fc0ce370 con 0x7fc1fc0d39f0 2024-04-26T20:52:06.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.185+0000 7fc20346d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fc1fc15d630 con 0x7fc1fc0d1720 2024-04-26T20:52:06.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.185+0000 7fc201209640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fc1fc15d250 0x7fc1fc0d1010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:39568/0 (socket says 172.21.15.73:39568) 2024-04-26T20:52:06.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.185+0000 7fc201209640 1 -- 172.21.15.73:0/1630695061 learned_addr learned my addr 172.21.15.73:0/1630695061 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:52:06.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.185+0000 7fc200a08640 1 --2- 172.21.15.73:0/1630695061 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc1fc0d1720 0x7fc1fc0d0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:06.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.185+0000 7fc1f3fff640 1 -- 172.21.15.73:0/1630695061 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3115821822 0 0) 0x7fc1fc0701e0 con 0x7fc1fc15d250 2024-04-26T20:52:06.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.186+0000 7fc1f3fff640 1 -- 172.21.15.73:0/1630695061 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc1e8003520 con 0x7fc1fc15d250 2024-04-26T20:52:06.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.186+0000 7fc1f3fff640 1 -- 172.21.15.73:0/1630695061 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1472801464 0 0) 0x7fc1f4001110 con 0x7fc1fc0d39f0 2024-04-26T20:52:06.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.186+0000 7fc200a08640 1 -- 172.21.15.73:0/1630695061 >> v1:172.21.15.120:6789/0 conn(0x7fc1fc0d39f0 legacy=0x7fc1fc070970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:06.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.186+0000 7fc200a08640 1 -- 172.21.15.73:0/1630695061 >> v1:172.21.15.73:6789/0 conn(0x7fc1fc15d250 legacy=0x7fc1fc0d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:06.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.186+0000 7fc200a08640 1 -- 172.21.15.73:0/1630695061 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1fc0ce5f0 con 0x7fc1fc0d1720 2024-04-26T20:52:06.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.186+0000 7fc200a08640 1 --2- 172.21.15.73:0/1630695061 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc1fc0d1720 0x7fc1fc0d0950 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec0097d0 tx=0x7fc1ec02a0e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2b727fa6269826b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:06.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.187+0000 7fc1f3fff640 1 -- 172.21.15.73:0/1630695061 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1ec037070 con 0x7fc1fc0d1720 2024-04-26T20:52:06.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.187+0000 7fc1f3fff640 1 -- 172.21.15.73:0/1630695061 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1ec02ab70 con 0x7fc1fc0d1720 2024-04-26T20:52:06.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.187+0000 7fc1f3fff640 1 -- 172.21.15.73:0/1630695061 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f80019f0 con 0x7fc1fc0d1720 2024-04-26T20:52:06.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.187+0000 7fc20346d640 1 -- 172.21.15.73:0/1630695061 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc1fc0d1720 msgr2=0x7fc1fc0d0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:06.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.187+0000 7fc20346d640 1 --2- 172.21.15.73:0/1630695061 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc1fc0d1720 0x7fc1fc0d0950 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec0097d0 tx=0x7fc1ec02a0e0 comp rx=0 tx=0).stop 2024-04-26T20:52:06.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.187+0000 7fc20346d640 1 -- 172.21.15.73:0/1630695061 shutdown_connections 2024-04-26T20:52:06.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.187+0000 7fc20346d640 1 --2- 172.21.15.73:0/1630695061 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc1fc0d1720 0x7fc1fc0d0950 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:06.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.187+0000 7fc20346d640 1 -- 172.21.15.73:0/1630695061 >> 172.21.15.73:0/1630695061 conn(0x7fc1fc0d2d80 msgr2=0x7fc1fc0d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:06.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.187+0000 7fc20346d640 1 -- 172.21.15.73:0/1630695061 shutdown_connections 2024-04-26T20:52:06.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.187+0000 7fc20346d640 1 -- 172.21.15.73:0/1630695061 wait complete. 2024-04-26T20:52:06.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.188+0000 7fc20346d640 1 Processor -- start 2024-04-26T20:52:06.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.188+0000 7fc20346d640 1 -- start start 2024-04-26T20:52:06.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.189+0000 7fc20346d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc1fc0d1720 0x7fc1fc0b6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:06.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.189+0000 7fc20346d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1fc0d39f0 0x7fc1fc1f3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:06.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.189+0000 7fc20346d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1fc15d250 0x7fc1fc0b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:06.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.189+0000 7fc20346d640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc1fc0cebb0 con 0x7fc1fc0d39f0 2024-04-26T20:52:06.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.189+0000 7fc20346d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc1fc0704e0 con 0x7fc1fc15d250 2024-04-26T20:52:06.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.189+0000 7fc20346d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fc1fc071080 con 0x7fc1fc0d1720 2024-04-26T20:52:06.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.189+0000 7fc200a08640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1fc0d39f0 0x7fc1fc1f3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:06.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.189+0000 7fc200a08640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1fc0d39f0 0x7fc1fc1f3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39966/0 (socket says 172.21.15.73:39966) 2024-04-26T20:52:06.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.189+0000 7fc200a08640 1 -- 172.21.15.73:0/3510436826 learned_addr learned my addr 172.21.15.73:0/3510436826 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:06.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.189+0000 7fc201209640 1 --2- 172.21.15.73:0/3510436826 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc1fc0d1720 0x7fc1fc0b6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:06.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.189+0000 7fc201a0a640 1 --2- 172.21.15.73:0/3510436826 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1fc15d250 0x7fc1fc0b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:06.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.190+0000 7fc200a08640 1 -- 172.21.15.73:0/3510436826 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc1fc0d1720 msgr2=0x7fc1fc0b6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:06.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.190+0000 7fc200a08640 1 --2- 172.21.15.73:0/3510436826 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc1fc0d1720 0x7fc1fc0b6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:06.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.190+0000 7fc200a08640 1 -- 172.21.15.73:0/3510436826 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1fc15d250 msgr2=0x7fc1fc0b4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:06.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.190+0000 7fc200a08640 1 --2- 172.21.15.73:0/3510436826 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1fc15d250 0x7fc1fc0b4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:06.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.190+0000 7fc200a08640 1 -- 172.21.15.73:0/3510436826 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1fc1f4410 con 0x7fc1fc0d39f0 2024-04-26T20:52:06.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.190+0000 7fc201a0a640 1 --2- 172.21.15.73:0/3510436826 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1fc15d250 0x7fc1fc0b4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:52:06.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.190+0000 7fc200a08640 1 --2- 172.21.15.73:0/3510436826 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1fc0d39f0 0x7fc1fc1f3ed0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec035040 tx=0x7fc1ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:06.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.190+0000 7fc1f1ffb640 1 -- 172.21.15.73:0/3510436826 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1ec042020 con 0x7fc1fc0d39f0 2024-04-26T20:52:06.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.190+0000 7fc1f1ffb640 1 -- 172.21.15.73:0/3510436826 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1ec003a40 con 0x7fc1fc0d39f0 2024-04-26T20:52:06.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.190+0000 7fc20346d640 1 -- 172.21.15.73:0/3510436826 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1fc0b7270 con 0x7fc1fc0d39f0 2024-04-26T20:52:06.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.190+0000 7fc1f1ffb640 1 -- 172.21.15.73:0/3510436826 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1ec037040 con 0x7fc1fc0d39f0 2024-04-26T20:52:06.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.190+0000 7fc20346d640 1 -- 172.21.15.73:0/3510436826 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1fc0b7830 con 0x7fc1fc0d39f0 2024-04-26T20:52:06.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.191+0000 7fc20346d640 1 -- 172.21.15.73:0/3510436826 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1fc0cd1f0 con 0x7fc1fc0d39f0 2024-04-26T20:52:06.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.191+0000 7fc1f1ffb640 1 -- 172.21.15.73:0/3510436826 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc1ec045020 con 0x7fc1fc0d39f0 2024-04-26T20:52:06.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.192+0000 7fc1f1ffb640 1 --2- 172.21.15.73:0/3510436826 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc1e006c8f0 0x7fc1e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:06.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.192+0000 7fc1f1ffb640 1 -- 172.21.15.73:0/3510436826 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6187+0+0 (secure 0 0 0) 0x7fc1ec0ac430 con 0x7fc1fc0d39f0 2024-04-26T20:52:06.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.192+0000 7fc201209640 1 --2- 172.21.15.73:0/3510436826 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc1e006c8f0 0x7fc1e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:06.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.192+0000 7fc201209640 1 --2- 172.21.15.73:0/3510436826 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc1e006c8f0 0x7fc1e006edb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8005fd0 tx=0x7fc1f8001a90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:06.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.196+0000 7fc1f1ffb640 1 -- 172.21.15.73:0/3510436826 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc1fc0cd1f0 con 0x7fc1fc0d39f0 2024-04-26T20:52:06.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.371+0000 7fc20346d640 1 -- 172.21.15.73:0/3510436826 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc1fc0d0450 con 0x7fc1fc0d39f0 2024-04-26T20:52:06.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.371+0000 7fc1f1ffb640 1 -- 172.21.15.73:0/3510436826 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) ==== 100+0+11 (secure 0 0 0) 0x7fc1ec04c030 con 0x7fc1fc0d39f0 2024-04-26T20:52:06.372 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-26T20:52:06.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.373+0000 7fc20346d640 1 -- 172.21.15.73:0/3510436826 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc1e006c8f0 msgr2=0x7fc1e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:06.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.373+0000 7fc20346d640 1 --2- 172.21.15.73:0/3510436826 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc1e006c8f0 0x7fc1e006edb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8005fd0 tx=0x7fc1f8001a90 comp rx=0 tx=0).stop 2024-04-26T20:52:06.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.373+0000 7fc20346d640 1 -- 172.21.15.73:0/3510436826 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1fc0d39f0 msgr2=0x7fc1fc1f3ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:06.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.373+0000 7fc20346d640 1 --2- 172.21.15.73:0/3510436826 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1fc0d39f0 0x7fc1fc1f3ed0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec035040 tx=0x7fc1ec0046c0 comp rx=0 tx=0).stop 2024-04-26T20:52:06.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.373+0000 7fc20346d640 1 -- 172.21.15.73:0/3510436826 shutdown_connections 2024-04-26T20:52:06.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.373+0000 7fc20346d640 1 --2- 172.21.15.73:0/3510436826 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1fc0d39f0 0x7fc1fc1f3ed0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:06.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.373+0000 7fc20346d640 1 --2- 172.21.15.73:0/3510436826 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc1e006c8f0 0x7fc1e006edb0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:06.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.373+0000 7fc20346d640 1 --2- 172.21.15.73:0/3510436826 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1fc15d250 0x7fc1fc0b4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:06.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.373+0000 7fc20346d640 1 --2- 172.21.15.73:0/3510436826 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc1fc0d1720 0x7fc1fc0b6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:06.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.373+0000 7fc20346d640 1 -- 172.21.15.73:0/3510436826 >> 172.21.15.73:0/3510436826 conn(0x7fc1fc0d2d80 msgr2=0x7fc1fc0ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:06.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.373+0000 7fc20346d640 1 -- 172.21.15.73:0/3510436826 shutdown_connections 2024-04-26T20:52:06.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.373+0000 7fc20346d640 1 -- 172.21.15.73:0/3510436826 wait complete. 2024-04-26T20:52:06.385 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T20:52:06.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.481+0000 7fb1cf0a9640 1 Processor -- start 2024-04-26T20:52:06.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.482+0000 7fb1cf0a9640 1 -- start start 2024-04-26T20:52:06.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.482+0000 7fb1cf0a9640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb1c81608f0 0x7fb1c80d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:06.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.482+0000 7fb1cf0a9640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb1c80cffa0 0x7fb1c80b2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:06.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.482+0000 7fb1cf0a9640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1c80cde20 con 0x7fb1c80d0a90 2024-04-26T20:52:06.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.482+0000 7fb1cf0a9640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7fb1c80cea30 con 0x7fb1c80cffa0 2024-04-26T20:52:06.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.482+0000 7fb1cf0a9640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fb1c80ce100 con 0x7fb1c81608f0 2024-04-26T20:52:06.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.482+0000 7fb1cd646640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb1c80cffa0 0x7fb1c80b2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:06.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.482+0000 7fb1cd646640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb1c80cffa0 0x7fb1c80b2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39974/0 (socket says 172.21.15.73:39974) 2024-04-26T20:52:06.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.483+0000 7fb1c7fff640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fb1c80d0a90 0x7fb1c80cf890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:39570/0 (socket says 172.21.15.73:39570) 2024-04-26T20:52:06.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.483+0000 7fb1cd646640 1 -- 172.21.15.73:0/2301711815 learned_addr learned my addr 172.21.15.73:0/2301711815 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:06.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.483+0000 7fb1cce45640 1 --2- 172.21.15.73:0/2301711815 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb1c81608f0 0x7fb1c80d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:06.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.483+0000 7fb1cd646640 1 -- 172.21.15.73:0/2301711815 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb1c81608f0 msgr2=0x7fb1c80d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:06.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.483+0000 7fb1cd646640 1 --2- 172.21.15.73:0/2301711815 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb1c81608f0 0x7fb1c80d1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:06.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.483+0000 7fb1cd646640 1 -- 172.21.15.73:0/2301711815 >> v1:172.21.15.73:6789/0 conn(0x7fb1c80d0a90 legacy=0x7fb1c80cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:06.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.483+0000 7fb1cd646640 1 -- 172.21.15.73:0/2301711815 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1c80ce3e0 con 0x7fb1c80cffa0 2024-04-26T20:52:06.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.483+0000 7fb1cd646640 1 --2- 172.21.15.73:0/2301711815 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1c80cffa0 0x7fb1c80b2300 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0009e50 tx=0x7fb1c002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6672a7e1c90e9d03 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:06.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.484+0000 7fb1c77fe640 1 -- 172.21.15.73:0/2301711815 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c0036070 con 0x7fb1c80cffa0 2024-04-26T20:52:06.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.484+0000 7fb1c77fe640 1 -- 172.21.15.73:0/2301711815 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1c0031600 con 0x7fb1c80cffa0 2024-04-26T20:52:06.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.484+0000 7fb1c77fe640 1 -- 172.21.15.73:0/2301711815 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c00319d0 con 0x7fb1c80cffa0 2024-04-26T20:52:06.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.484+0000 7fb1cf0a9640 1 -- 172.21.15.73:0/2301711815 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1c80cffa0 msgr2=0x7fb1c80b2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:06.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.484+0000 7fb1cf0a9640 1 --2- 172.21.15.73:0/2301711815 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1c80cffa0 0x7fb1c80b2300 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0009e50 tx=0x7fb1c002a710 comp rx=0 tx=0).stop 2024-04-26T20:52:06.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.484+0000 7fb1cf0a9640 1 -- 172.21.15.73:0/2301711815 shutdown_connections 2024-04-26T20:52:06.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.484+0000 7fb1cf0a9640 1 --2- 172.21.15.73:0/2301711815 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1c80cffa0 0x7fb1c80b2300 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:06.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.484+0000 7fb1cf0a9640 1 --2- 172.21.15.73:0/2301711815 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb1c81608f0 0x7fb1c80d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:06.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.484+0000 7fb1cf0a9640 1 -- 172.21.15.73:0/2301711815 >> 172.21.15.73:0/2301711815 conn(0x7fb1c80d2ec0 msgr2=0x7fb1c80d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:06.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.484+0000 7fb1cf0a9640 1 -- 172.21.15.73:0/2301711815 shutdown_connections 2024-04-26T20:52:06.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.484+0000 7fb1cf0a9640 1 -- 172.21.15.73:0/2301711815 wait complete. 2024-04-26T20:52:06.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.485+0000 7fb1cf0a9640 1 Processor -- start 2024-04-26T20:52:06.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.485+0000 7fb1cf0a9640 1 -- start start 2024-04-26T20:52:06.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.486+0000 7fb1cf0a9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1c80cffa0 0x7fb1c80ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:06.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.486+0000 7fb1cf0a9640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb1c80d0a90 0x7fb1c80d1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:06.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.486+0000 7fb1cf0a9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb1c81608f0 0x7fb1c80c5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:06.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.486+0000 7fb1cf0a9640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c80ceec0 con 0x7fb1c80cffa0 2024-04-26T20:52:06.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.486+0000 7fb1cf0a9640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c80d0ef0 con 0x7fb1c81608f0 2024-04-26T20:52:06.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.486+0000 7fb1cf0a9640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fb1c80cda40 con 0x7fb1c80d0a90 2024-04-26T20:52:06.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.486+0000 7fb1cce45640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1c80cffa0 0x7fb1c80ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:06.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.486+0000 7fb1cce45640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1c80cffa0 0x7fb1c80ccbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39980/0 (socket says 172.21.15.73:39980) 2024-04-26T20:52:06.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.486+0000 7fb1cce45640 1 -- 172.21.15.73:0/36800336 learned_addr learned my addr 172.21.15.73:0/36800336 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:06.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.486+0000 7fb1c7fff640 1 --2- 172.21.15.73:0/36800336 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb1c80d0a90 0x7fb1c80d1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:06.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.486+0000 7fb1cd646640 1 --2- 172.21.15.73:0/36800336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb1c81608f0 0x7fb1c80c5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:06.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.487+0000 7fb1cce45640 1 -- 172.21.15.73:0/36800336 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb1c80d0a90 msgr2=0x7fb1c80d1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:06.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.487+0000 7fb1cce45640 1 --2- 172.21.15.73:0/36800336 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb1c80d0a90 0x7fb1c80d1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:06.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.487+0000 7fb1cce45640 1 -- 172.21.15.73:0/36800336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb1c81608f0 msgr2=0x7fb1c80c5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:06.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.487+0000 7fb1cce45640 1 --2- 172.21.15.73:0/36800336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb1c81608f0 0x7fb1c80c5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:06.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.487+0000 7fb1cce45640 1 -- 172.21.15.73:0/36800336 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1c80cbb10 con 0x7fb1c80cffa0 2024-04-26T20:52:06.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.487+0000 7fb1c7fff640 1 --2- 172.21.15.73:0/36800336 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb1c80d0a90 0x7fb1c80d1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:52:06.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.487+0000 7fb1cd646640 1 --2- 172.21.15.73:0/36800336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb1c81608f0 0x7fb1c80c5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:52:06.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.487+0000 7fb1cce45640 1 --2- 172.21.15.73:0/36800336 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1c80cffa0 0x7fb1c80ccbb0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc00c7f0 tx=0x7fb1bc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:06.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.487+0000 7fb1c57fa640 1 -- 172.21.15.73:0/36800336 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc015070 con 0x7fb1c80cffa0 2024-04-26T20:52:06.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.487+0000 7fb1c57fa640 1 -- 172.21.15.73:0/36800336 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1bc004220 con 0x7fb1c80cffa0 2024-04-26T20:52:06.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.487+0000 7fb1cf0a9640 1 -- 172.21.15.73:0/36800336 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1c80cbdf0 con 0x7fb1c80cffa0 2024-04-26T20:52:06.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.487+0000 7fb1c57fa640 1 -- 172.21.15.73:0/36800336 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc007620 con 0x7fb1c80cffa0 2024-04-26T20:52:06.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.487+0000 7fb1cf0a9640 1 -- 172.21.15.73:0/36800336 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1c80cc690 con 0x7fb1c80cffa0 2024-04-26T20:52:06.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.488+0000 7fb1cf0a9640 1 -- 172.21.15.73:0/36800336 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1c80cc1e0 con 0x7fb1c80cffa0 2024-04-26T20:52:06.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.488+0000 7fb1c57fa640 1 -- 172.21.15.73:0/36800336 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb1bc0077c0 con 0x7fb1c80cffa0 2024-04-26T20:52:06.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.489+0000 7fb1c57fa640 1 --2- 172.21.15.73:0/36800336 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb1ac06c8f0 0x7fb1ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:06.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.489+0000 7fb1c7fff640 1 --2- 172.21.15.73:0/36800336 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb1ac06c8f0 0x7fb1ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:06.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.489+0000 7fb1c57fa640 1 -- 172.21.15.73:0/36800336 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6187+0+0 (secure 0 0 0) 0x7fb1bc08be50 con 0x7fb1c80cffa0 2024-04-26T20:52:06.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.489+0000 7fb1c7fff640 1 --2- 172.21.15.73:0/36800336 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb1ac06c8f0 0x7fb1ac06edb0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb1b4007fd0 tx=0x7fb1b4007d60 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:06.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.493+0000 7fb1c57fa640 1 -- 172.21.15.73:0/36800336 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb1bc05a130 con 0x7fb1c80cffa0 2024-04-26T20:52:06.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.680+0000 7fb1cf0a9640 1 -- 172.21.15.73:0/36800336 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb1c80694f0 con 0x7fb1c80cffa0 2024-04-26T20:52:06.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.680+0000 7fb1c57fa640 1 -- 172.21.15.73:0/36800336 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v40) ==== 80+0+122 (secure 0 0 0) 0x7fb1bc059a30 con 0x7fb1c80cffa0 2024-04-26T20:52:06.681 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.120:6835/761435580 2024-04-27T20:51:58.783228+0000 2024-04-26T20:52:06.681 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.120:6833/761435580 2024-04-27T20:51:58.783228+0000 2024-04-26T20:52:06.681 INFO:teuthology.orchestra.run.smithi073.stderr:listed 2 entries 2024-04-26T20:52:06.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.682+0000 7fb1cf0a9640 1 -- 172.21.15.73:0/36800336 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb1ac06c8f0 msgr2=0x7fb1ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:06.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.682+0000 7fb1cf0a9640 1 --2- 172.21.15.73:0/36800336 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb1ac06c8f0 0x7fb1ac06edb0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb1b4007fd0 tx=0x7fb1b4007d60 comp rx=0 tx=0).stop 2024-04-26T20:52:06.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.682+0000 7fb1cf0a9640 1 -- 172.21.15.73:0/36800336 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1c80cffa0 msgr2=0x7fb1c80ccbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:06.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.682+0000 7fb1cf0a9640 1 --2- 172.21.15.73:0/36800336 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1c80cffa0 0x7fb1c80ccbb0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc00c7f0 tx=0x7fb1bc00ccb0 comp rx=0 tx=0).stop 2024-04-26T20:52:06.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.682+0000 7fb1cf0a9640 1 -- 172.21.15.73:0/36800336 shutdown_connections 2024-04-26T20:52:06.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.682+0000 7fb1cf0a9640 1 --2- 172.21.15.73:0/36800336 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb1ac06c8f0 0x7fb1ac06edb0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:06.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.682+0000 7fb1cf0a9640 1 --2- 172.21.15.73:0/36800336 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb1c81608f0 0x7fb1c80c5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:06.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.682+0000 7fb1cf0a9640 1 --2- 172.21.15.73:0/36800336 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb1c80d0a90 0x7fb1c80d1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:06.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.682+0000 7fb1cf0a9640 1 --2- 172.21.15.73:0/36800336 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1c80cffa0 0x7fb1c80ccbb0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:06.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.682+0000 7fb1cf0a9640 1 -- 172.21.15.73:0/36800336 >> 172.21.15.73:0/36800336 conn(0x7fb1c80d2ec0 msgr2=0x7fb1c815b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:06.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.682+0000 7fb1cf0a9640 1 -- 172.21.15.73:0/36800336 shutdown_connections 2024-04-26T20:52:06.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.682+0000 7fb1cf0a9640 1 -- 172.21.15.73:0/36800336 wait complete. 2024-04-26T20:52:06.694 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T20:52:06.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:52:06.694 DEBUG:teuthology.orchestra.run.smithi073:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T20:52:06.722 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi073.front.sepia.ceph.com 2024-04-26T20:52:06.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:52:06.722 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T20:52:06.776 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T20:52:06.777 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-26T20:52:06.777 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:52:06.796 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056097,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19070 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T20:52:06.796 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T20:52:06.796 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-26T20:52:06.796 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:52:06.833 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19637 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T20:52:06.833 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T20:52:06.833 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-26T20:52:06.833 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:52:06.854 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056097,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19070 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T20:52:06.854 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:52:06.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.969+0000 7f9c8b19d640 1 Processor -- start 2024-04-26T20:52:06.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.969+0000 7f9c8b19d640 1 -- start start 2024-04-26T20:52:06.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.970+0000 7f9c8b19d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f9c84072950 0x7f9c84167ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:06.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.970+0000 7f9c8b19d640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c8415d7c0 con 0x7f9c84071e60 2024-04-26T20:52:06.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.970+0000 7f9c8b19d640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c8415daa0 con 0x7f9c840cc480 2024-04-26T20:52:06.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.970+0000 7f9c8b19d640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f9c8415dd80 con 0x7f9c84072950 2024-04-26T20:52:06.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.970+0000 7f9c88f39640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f9c84071e60 0x7f9c8406f8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:39584/0 (socket says 172.21.15.73:39584) 2024-04-26T20:52:06.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.970+0000 7f9c88f39640 1 -- 172.21.15.73:0/2077409685 learned_addr learned my addr 172.21.15.73:0/2077409685 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:52:06.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.970+0000 7f9c8973a640 1 --2- 172.21.15.73:0/2077409685 >> v2:172.21.15.120:3300/0 conn(0x7f9c84072950 0x7f9c84167ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:06.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.970+0000 7f9c837fe640 1 -- 172.21.15.73:0/2077409685 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1826727930 0 0) 0x7f9c8415d7c0 con 0x7f9c84071e60 2024-04-26T20:52:06.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.970+0000 7f9c837fe640 1 -- 172.21.15.73:0/2077409685 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c6c003520 con 0x7f9c84071e60 2024-04-26T20:52:06.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.971+0000 7f9c837fe640 1 -- 172.21.15.73:0/2077409685 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2532233542 0 0) 0x7f9c8415daa0 con 0x7f9c840cc480 2024-04-26T20:52:06.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.971+0000 7f9c837fe640 1 -- 172.21.15.73:0/2077409685 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c8415d7c0 con 0x7f9c840cc480 2024-04-26T20:52:06.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.971+0000 7f9c837fe640 1 -- 172.21.15.73:0/2077409685 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 641519177 0 0) 0x7f9c6c003520 con 0x7f9c84071e60 2024-04-26T20:52:06.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.971+0000 7f9c837fe640 1 -- 172.21.15.73:0/2077409685 >> v2:172.21.15.120:3300/0 conn(0x7f9c84072950 msgr2=0x7f9c84167ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:06.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.971+0000 7f9c837fe640 1 --2- 172.21.15.73:0/2077409685 >> v2:172.21.15.120:3300/0 conn(0x7f9c84072950 0x7f9c84167ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:06.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.971+0000 7f9c837fe640 1 -- 172.21.15.73:0/2077409685 >> v1:172.21.15.120:6789/0 conn(0x7f9c840cc480 legacy=0x7f9c84072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:06.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.971+0000 7f9c837fe640 1 -- 172.21.15.73:0/2077409685 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c8415e060 con 0x7f9c84071e60 2024-04-26T20:52:06.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.971+0000 7f9c837fe640 1 -- 172.21.15.73:0/2077409685 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7f9c74001670 con 0x7f9c84071e60 2024-04-26T20:52:06.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.971+0000 7f9c837fe640 1 -- 172.21.15.73:0/2077409685 >> v1:172.21.15.73:6789/0 conn(0x7f9c84071e60 legacy=0x7f9c8406f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:06.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.971+0000 7f9c837fe640 1 --2- 172.21.15.73:0/2077409685 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9c6c003b40 0x7f9c6c003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:06.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.971+0000 7f9c837fe640 1 --2- 172.21.15.73:0/2077409685 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c6c004800 0x7f9c6c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:06.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.971+0000 7f9c837fe640 1 --2- 172.21.15.73:0/2077409685 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c6c0051c0 0x7f9c6c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:06.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.971+0000 7f9c837fe640 1 -- 172.21.15.73:0/2077409685 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8415dd80 con 0x7f9c6c0051c0 2024-04-26T20:52:06.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.971+0000 7f9c837fe640 1 -- 172.21.15.73:0/2077409685 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9c6c00c040 con 0x7f9c6c004800 2024-04-26T20:52:06.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.971+0000 7f9c837fe640 1 -- 172.21.15.73:0/2077409685 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f9c6c00c240 con 0x7f9c6c003b40 2024-04-26T20:52:06.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.971+0000 7f9c8973a640 1 --2- 172.21.15.73:0/2077409685 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c6c0051c0 0x7f9c6c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:06.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.971+0000 7f9c83fff640 1 --2- 172.21.15.73:0/2077409685 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c6c004800 0x7f9c6c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:06.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.971+0000 7f9c88f39640 1 --2- 172.21.15.73:0/2077409685 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9c6c003b40 0x7f9c6c003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:06.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.971+0000 7f9c8973a640 1 -- 172.21.15.73:0/2077409685 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9c6c003b40 msgr2=0x7f9c6c003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:06.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.971+0000 7f9c8973a640 1 --2- 172.21.15.73:0/2077409685 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9c6c003b40 0x7f9c6c003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:06.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.971+0000 7f9c8973a640 1 -- 172.21.15.73:0/2077409685 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c6c004800 msgr2=0x7f9c6c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:06.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.971+0000 7f9c8973a640 1 --2- 172.21.15.73:0/2077409685 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c6c004800 0x7f9c6c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:06.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.971+0000 7f9c8973a640 1 -- 172.21.15.73:0/2077409685 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c6c003520 con 0x7f9c6c0051c0 2024-04-26T20:52:06.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.971+0000 7f9c88f39640 1 --2- 172.21.15.73:0/2077409685 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9c6c003b40 0x7f9c6c003f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:52:06.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.972+0000 7f9c8973a640 1 -- 172.21.15.73:0/2077409685 reap_dead start 2024-04-26T20:52:06.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.972+0000 7f9c8973a640 1 --2- 172.21.15.73:0/2077409685 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c6c0051c0 0x7f9c6c00b9c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c009fc0 tx=0x7f9c6c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=488ffdb4a87404b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:06.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.972+0000 7f9c837fe640 1 -- 172.21.15.73:0/2077409685 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c035020 con 0x7f9c6c0051c0 2024-04-26T20:52:06.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.972+0000 7f9c837fe640 1 -- 172.21.15.73:0/2077409685 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c7c003bd0 con 0x7f9c6c0051c0 2024-04-26T20:52:06.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.972+0000 7f9c837fe640 1 -- 172.21.15.73:0/2077409685 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c009c10 con 0x7f9c6c0051c0 2024-04-26T20:52:06.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.972+0000 7f9c8b19d640 1 -- 172.21.15.73:0/2077409685 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c6c0051c0 msgr2=0x7f9c6c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:06.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.972+0000 7f9c8b19d640 1 --2- 172.21.15.73:0/2077409685 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c6c0051c0 0x7f9c6c00b9c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c009fc0 tx=0x7f9c6c005830 comp rx=0 tx=0).stop 2024-04-26T20:52:06.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.973+0000 7f9c8b19d640 1 -- 172.21.15.73:0/2077409685 shutdown_connections 2024-04-26T20:52:06.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.973+0000 7f9c8b19d640 1 --2- 172.21.15.73:0/2077409685 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c6c0051c0 0x7f9c6c00b9c0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:06.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.973+0000 7f9c8b19d640 1 -- 172.21.15.73:0/2077409685 >> 172.21.15.73:0/2077409685 conn(0x7f9c840d0be0 msgr2=0x7f9c840d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:06.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.973+0000 7f9c8b19d640 1 -- 172.21.15.73:0/2077409685 shutdown_connections 2024-04-26T20:52:06.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.973+0000 7f9c8b19d640 1 -- 172.21.15.73:0/2077409685 wait complete. 2024-04-26T20:52:06.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.974+0000 7f9c8b19d640 1 Processor -- start 2024-04-26T20:52:06.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.974+0000 7f9c8b19d640 1 -- start start 2024-04-26T20:52:06.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.975+0000 7f9c8b19d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9c6c0051c0 0x7f9c840b5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:06.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.975+0000 7f9c8b19d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c840c7040 0x7f9c840b4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:06.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.975+0000 7f9c8b19d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c840b4d40 0x7f9c840b51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:06.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.975+0000 7f9c8b19d640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8415d570 con 0x7f9c840b4d40 2024-04-26T20:52:06.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.975+0000 7f9c8b19d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8415d3f0 con 0x7f9c840c7040 2024-04-26T20:52:06.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.975+0000 7f9c8b19d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f9c84072240 con 0x7f9c6c0051c0 2024-04-26T20:52:06.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.975+0000 7f9c8973a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c840b4d40 0x7f9c840b51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:06.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.975+0000 7f9c8973a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c840b4d40 0x7f9c840b51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40000/0 (socket says 172.21.15.73:40000) 2024-04-26T20:52:06.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.975+0000 7f9c8973a640 1 -- 172.21.15.73:0/67490370 learned_addr learned my addr 172.21.15.73:0/67490370 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:06.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.975+0000 7f9c83fff640 1 --2- 172.21.15.73:0/67490370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c840c7040 0x7f9c840b4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:06.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.975+0000 7f9c88f39640 1 --2- 172.21.15.73:0/67490370 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9c6c0051c0 0x7f9c840b5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:06.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.975+0000 7f9c8973a640 1 -- 172.21.15.73:0/67490370 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9c6c0051c0 msgr2=0x7f9c840b5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:06.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.975+0000 7f9c8973a640 1 --2- 172.21.15.73:0/67490370 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9c6c0051c0 0x7f9c840b5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:06.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.975+0000 7f9c8973a640 1 -- 172.21.15.73:0/67490370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c840c7040 msgr2=0x7f9c840b4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:06.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.975+0000 7f9c8973a640 1 --2- 172.21.15.73:0/67490370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c840c7040 0x7f9c840b4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:06.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.975+0000 7f9c8973a640 1 -- 172.21.15.73:0/67490370 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c841507b0 con 0x7f9c840b4d40 2024-04-26T20:52:06.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.975+0000 7f9c8973a640 1 --2- 172.21.15.73:0/67490370 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c840b4d40 0x7f9c840b51c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c02af10 tx=0x7f9c7c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:06.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.975+0000 7f9c817fa640 1 -- 172.21.15.73:0/67490370 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c035030 con 0x7f9c840b4d40 2024-04-26T20:52:06.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.975+0000 7f9c817fa640 1 -- 172.21.15.73:0/67490370 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c7c043070 con 0x7f9c840b4d40 2024-04-26T20:52:06.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.975+0000 7f9c8b19d640 1 -- 172.21.15.73:0/67490370 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c841560d0 con 0x7f9c840b4d40 2024-04-26T20:52:06.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.975+0000 7f9c817fa640 1 -- 172.21.15.73:0/67490370 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c02d070 con 0x7f9c840b4d40 2024-04-26T20:52:06.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.975+0000 7f9c8b19d640 1 -- 172.21.15.73:0/67490370 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c84150a70 con 0x7f9c840b4d40 2024-04-26T20:52:06.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.976+0000 7f9c8b19d640 1 -- 172.21.15.73:0/67490370 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c8406eec0 con 0x7f9c840b4d40 2024-04-26T20:52:06.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.977+0000 7f9c817fa640 1 -- 172.21.15.73:0/67490370 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9c7c005d10 con 0x7f9c840b4d40 2024-04-26T20:52:06.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.977+0000 7f9c817fa640 1 --2- 172.21.15.73:0/67490370 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f9c7006c8f0 0x7f9c7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:06.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.977+0000 7f9c817fa640 1 -- 172.21.15.73:0/67490370 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6187+0+0 (secure 0 0 0) 0x7f9c7c0ad080 con 0x7f9c840b4d40 2024-04-26T20:52:06.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.977+0000 7f9c88f39640 1 --2- 172.21.15.73:0/67490370 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f9c7006c8f0 0x7f9c7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:06.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.978+0000 7f9c88f39640 1 --2- 172.21.15.73:0/67490370 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f9c7006c8f0 0x7f9c7006edb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9c74001030 tx=0x7f9c74009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:06.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:06.982+0000 7f9c817fa640 1 -- 172.21.15.73:0/67490370 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9c8406eec0 con 0x7f9c840b4d40 2024-04-26T20:52:07.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.197+0000 7f9c8b19d640 1 -- 172.21.15.73:0/67490370 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c840694f0 con 0x7f9c840b4d40 2024-04-26T20:52:07.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.198+0000 7f9c817fa640 1 -- 172.21.15.73:0/67490370 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5185 (secure 0 0 0) 0x7f9c7c07b3f0 con 0x7f9c840b4d40 2024-04-26T20:52:07.199 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:52:07.199 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4802,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/272662913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":272662913},{"type":"v1","addr":"172.21.15.73:6836","nonce":272662913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4943,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2624899096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6833","nonce":2624899096},{"type":"v1","addr":"172.21.15.120:6835","nonce":2624899096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5047,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/1438858294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":1438858294},{"type":"v1","addr":"172.21.15.73:6837","nonce":1438858294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:51:45.273953+0000","modified":"2024-04-26T20:52:01.795359+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5050},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5050":{"gid":5050,"name":"b","rank":0,"incarnation":17,"state":"up:active","state_seq":9,"addr":"172.21.15.120:6834/3883540031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3883540031},{"type":"v1","addr":"172.21.15.120:6834","nonce":3883540031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5050,"qdb_cluster":[5050]},"id":2}]} 2024-04-26T20:52:07.199 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 20 2024-04-26T20:52:07.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.200+0000 7f9c8b19d640 1 -- 172.21.15.73:0/67490370 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f9c7006c8f0 msgr2=0x7f9c7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:07.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.200+0000 7f9c8b19d640 1 --2- 172.21.15.73:0/67490370 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f9c7006c8f0 0x7f9c7006edb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9c74001030 tx=0x7f9c74009040 comp rx=0 tx=0).stop 2024-04-26T20:52:07.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.200+0000 7f9c8b19d640 1 -- 172.21.15.73:0/67490370 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c840b4d40 msgr2=0x7f9c840b51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:07.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.200+0000 7f9c8b19d640 1 --2- 172.21.15.73:0/67490370 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c840b4d40 0x7f9c840b51c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c02af10 tx=0x7f9c7c009750 comp rx=0 tx=0).stop 2024-04-26T20:52:07.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.200+0000 7f9c8b19d640 1 -- 172.21.15.73:0/67490370 shutdown_connections 2024-04-26T20:52:07.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.200+0000 7f9c8b19d640 1 --2- 172.21.15.73:0/67490370 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c840b4d40 0x7f9c840b51c0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:07.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.200+0000 7f9c8b19d640 1 --2- 172.21.15.73:0/67490370 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f9c7006c8f0 0x7f9c7006edb0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:07.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.200+0000 7f9c8b19d640 1 --2- 172.21.15.73:0/67490370 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f9c840c7040 0x7f9c840b4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:07.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.200+0000 7f9c8b19d640 1 --2- 172.21.15.73:0/67490370 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f9c6c0051c0 0x7f9c840b5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:07.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.200+0000 7f9c8b19d640 1 -- 172.21.15.73:0/67490370 >> 172.21.15.73:0/67490370 conn(0x7f9c840d0be0 msgr2=0x7f9c840cd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:07.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.200+0000 7f9c8b19d640 1 -- 172.21.15.73:0/67490370 shutdown_connections 2024-04-26T20:52:07.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.200+0000 7f9c8b19d640 1 -- 172.21.15.73:0/67490370 wait complete. 2024-04-26T20:52:07.212 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:52:07.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.308+0000 7fc2c7e59640 1 Processor -- start 2024-04-26T20:52:07.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.308+0000 7fc2c7e59640 1 -- start start 2024-04-26T20:52:07.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.309+0000 7fc2c7e59640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc2c015ab60 0x7fc2c006c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:07.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.309+0000 7fc2c7e59640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fc2c01617f0 0x7fc2c006c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:07.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.309+0000 7fc2c7e59640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc2c015d5f0 0x7fc2c015daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:07.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.309+0000 7fc2c7e59640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7fc2c015af40 con 0x7fc2c015ab60 2024-04-26T20:52:07.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.309+0000 7fc2c7e59640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fc2c015e320 con 0x7fc2c01617f0 2024-04-26T20:52:07.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.309+0000 7fc2c7e59640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fc2c0160d10 con 0x7fc2c015d5f0 2024-04-26T20:52:07.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.309+0000 7fc2c5bf5640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc2c015ab60 0x7fc2c006c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:07.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.309+0000 7fc2c5bf5640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc2c015ab60 0x7fc2c006c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40002/0 (socket says 172.21.15.73:40002) 2024-04-26T20:52:07.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.309+0000 7fc2c5bf5640 1 -- 172.21.15.73:0/1802970380 learned_addr learned my addr 172.21.15.73:0/1802970380 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:07.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.310+0000 7fc2c5bf5640 1 -- 172.21.15.73:0/1802970380 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc2c015d5f0 msgr2=0x7fc2c015daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:07.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.310+0000 7fc2c63f6640 1 --2- 172.21.15.73:0/1802970380 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc2c015d5f0 0x7fc2c015daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:07.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.310+0000 7fc2c53f4640 1 --2- 172.21.15.73:0/1802970380 >> v2:172.21.15.120:3300/0 conn(0x7fc2c01617f0 0x7fc2c006c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:07.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.310+0000 7fc2c5bf5640 1 --2- 172.21.15.73:0/1802970380 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc2c015d5f0 0x7fc2c015daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:07.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.310+0000 7fc2c5bf5640 1 -- 172.21.15.73:0/1802970380 >> v2:172.21.15.120:3300/0 conn(0x7fc2c01617f0 msgr2=0x7fc2c006c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:07.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.310+0000 7fc2c5bf5640 1 --2- 172.21.15.73:0/1802970380 >> v2:172.21.15.120:3300/0 conn(0x7fc2c01617f0 0x7fc2c006c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:07.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.310+0000 7fc2c5bf5640 1 -- 172.21.15.73:0/1802970380 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2c0160ff0 con 0x7fc2c015ab60 2024-04-26T20:52:07.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.310+0000 7fc2c5bf5640 1 --2- 172.21.15.73:0/1802970380 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2c015ab60 0x7fc2c006c240 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fc2c00b48c0 tx=0x7fc2bc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a6d4c8df1c61dfd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:07.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.311+0000 7fc2c4bf3640 1 -- 172.21.15.73:0/1802970380 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc037070 con 0x7fc2c015ab60 2024-04-26T20:52:07.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.311+0000 7fc2c4bf3640 1 -- 172.21.15.73:0/1802970380 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2bc032440 con 0x7fc2c015ab60 2024-04-26T20:52:07.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.311+0000 7fc2c4bf3640 1 -- 172.21.15.73:0/1802970380 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc032810 con 0x7fc2c015ab60 2024-04-26T20:52:07.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.311+0000 7fc2c7e59640 1 -- 172.21.15.73:0/1802970380 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2c015ab60 msgr2=0x7fc2c006c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:07.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.311+0000 7fc2c7e59640 1 --2- 172.21.15.73:0/1802970380 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2c015ab60 0x7fc2c006c240 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fc2c00b48c0 tx=0x7fc2bc02a1e0 comp rx=0 tx=0).stop 2024-04-26T20:52:07.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.311+0000 7fc2c7e59640 1 -- 172.21.15.73:0/1802970380 shutdown_connections 2024-04-26T20:52:07.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.311+0000 7fc2c7e59640 1 --2- 172.21.15.73:0/1802970380 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc2c015d5f0 0x7fc2c015daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:07.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.311+0000 7fc2c7e59640 1 --2- 172.21.15.73:0/1802970380 >> v2:172.21.15.120:3300/0 conn(0x7fc2c01617f0 0x7fc2c006c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:07.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.311+0000 7fc2c7e59640 1 --2- 172.21.15.73:0/1802970380 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2c015ab60 0x7fc2c006c240 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:07.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.311+0000 7fc2c7e59640 1 -- 172.21.15.73:0/1802970380 >> 172.21.15.73:0/1802970380 conn(0x7fc2c00d1cf0 msgr2=0x7fc2c00d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:07.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.311+0000 7fc2c7e59640 1 -- 172.21.15.73:0/1802970380 shutdown_connections 2024-04-26T20:52:07.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.311+0000 7fc2c7e59640 1 -- 172.21.15.73:0/1802970380 wait complete. 2024-04-26T20:52:07.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.312+0000 7fc2c7e59640 1 Processor -- start 2024-04-26T20:52:07.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.312+0000 7fc2c7e59640 1 -- start start 2024-04-26T20:52:07.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.313+0000 7fc2c7e59640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc2c015ab60 0x7fc2c014f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:07.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.313+0000 7fc2c7e59640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2c015d5f0 0x7fc2c0145cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:07.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.313+0000 7fc2c7e59640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc2c01617f0 0x7fc2c00af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:07.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.313+0000 7fc2c7e59640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c0072f70 con 0x7fc2c015d5f0 2024-04-26T20:52:07.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.313+0000 7fc2c7e59640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c015dfe0 con 0x7fc2c015ab60 2024-04-26T20:52:07.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.313+0000 7fc2c7e59640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fc2c00730f0 con 0x7fc2c01617f0 2024-04-26T20:52:07.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.313+0000 7fc2c53f4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2c015d5f0 0x7fc2c0145cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:07.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.313+0000 7fc2c53f4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2c015d5f0 0x7fc2c0145cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40010/0 (socket says 172.21.15.73:40010) 2024-04-26T20:52:07.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.313+0000 7fc2c53f4640 1 -- 172.21.15.73:0/188019917 learned_addr learned my addr 172.21.15.73:0/188019917 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:07.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.313+0000 7fc2c5bf5640 1 --2- 172.21.15.73:0/188019917 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc2c015ab60 0x7fc2c014f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:07.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.313+0000 7fc2c63f6640 1 --2- 172.21.15.73:0/188019917 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc2c01617f0 0x7fc2c00af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:07.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.313+0000 7fc2c53f4640 1 -- 172.21.15.73:0/188019917 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc2c01617f0 msgr2=0x7fc2c00af2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:07.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.313+0000 7fc2c53f4640 1 --2- 172.21.15.73:0/188019917 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc2c01617f0 0x7fc2c00af2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:07.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.313+0000 7fc2c53f4640 1 -- 172.21.15.73:0/188019917 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc2c015ab60 msgr2=0x7fc2c014f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:07.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.313+0000 7fc2c53f4640 1 --2- 172.21.15.73:0/188019917 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc2c015ab60 0x7fc2c014f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:07.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.313+0000 7fc2c53f4640 1 -- 172.21.15.73:0/188019917 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2c00cd9c0 con 0x7fc2c015d5f0 2024-04-26T20:52:07.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.313+0000 7fc2c5bf5640 1 --2- 172.21.15.73:0/188019917 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc2c015ab60 0x7fc2c014f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:52:07.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.314+0000 7fc2c53f4640 1 --2- 172.21.15.73:0/188019917 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2c015d5f0 0x7fc2c0145cc0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fc2b000c820 tx=0x7fc2b000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:07.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.314+0000 7fc2b67fc640 1 -- 172.21.15.73:0/188019917 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b0015070 con 0x7fc2c015d5f0 2024-04-26T20:52:07.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.314+0000 7fc2b67fc640 1 -- 172.21.15.73:0/188019917 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2b0004020 con 0x7fc2c015d5f0 2024-04-26T20:52:07.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.314+0000 7fc2c7e59640 1 -- 172.21.15.73:0/188019917 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2c00b0ee0 con 0x7fc2c015d5f0 2024-04-26T20:52:07.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.314+0000 7fc2b67fc640 1 -- 172.21.15.73:0/188019917 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b00043f0 con 0x7fc2c015d5f0 2024-04-26T20:52:07.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.314+0000 7fc2c7e59640 1 -- 172.21.15.73:0/188019917 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2c00d1660 con 0x7fc2c015d5f0 2024-04-26T20:52:07.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.315+0000 7fc2c7e59640 1 -- 172.21.15.73:0/188019917 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2c00694f0 con 0x7fc2c015d5f0 2024-04-26T20:52:07.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.315+0000 7fc2b67fc640 1 -- 172.21.15.73:0/188019917 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc2b0007830 con 0x7fc2c015d5f0 2024-04-26T20:52:07.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.316+0000 7fc2b67fc640 1 --2- 172.21.15.73:0/188019917 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc2a406c920 0x7fc2a406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:07.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.316+0000 7fc2c5bf5640 1 --2- 172.21.15.73:0/188019917 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc2a406c920 0x7fc2a406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:07.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.316+0000 7fc2b67fc640 1 -- 172.21.15.73:0/188019917 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6187+0+0 (secure 0 0 0) 0x7fc2b008baa0 con 0x7fc2c015d5f0 2024-04-26T20:52:07.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.316+0000 7fc2c5bf5640 1 --2- 172.21.15.73:0/188019917 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc2a406c920 0x7fc2a406ede0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc034000 tx=0x7fc2bc02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:07.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.320+0000 7fc2b67fc640 1 -- 172.21.15.73:0/188019917 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc2b0059cf0 con 0x7fc2c015d5f0 2024-04-26T20:52:07.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.494+0000 7fc2c7e59640 1 -- 172.21.15.73:0/188019917 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc2c00b2bb0 con 0x7fc2c015d5f0 2024-04-26T20:52:07.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.495+0000 7fc2b67fc640 1 -- 172.21.15.73:0/188019917 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+17071 (secure 0 0 0) 0x7fc2b00595f0 con 0x7fc2c015d5f0 2024-04-26T20:52:07.497 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:52:07.497 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":40,"fsid":"fbd3f548-b547-4713-856b-09f98f5d1764","created":"2024-04-26T20:50:07.876786+0000","modified":"2024-04-26T20:51:58.783253+0000","last_up_change":"2024-04-26T20:50:19.082741+0000","last_in_change":"2024-04-26T20:50:10.018400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:50:19.993731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:51:43.110774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-26T20:51:44.235846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d6f4f3-214a-4392-a02f-c33cbcab3c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6809","nonce":3461705124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6811","nonce":3461705124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6815","nonce":3461705124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6813","nonce":3461705124}]},"public_addr":"172.21.15.73:6809/3461705124","cluster_addr":"172.21.15.73:6811/3461705124","heartbeat_back_addr":"172.21.15.73:6815/3461705124","heartbeat_front_addr":"172.21.15.73:6813/3461705124","state":["exists","up"]},{"osd":1,"uuid":"786bf8ca-6ddc-4b3c-be6c-9152398bd5fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6817","nonce":2827619403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6819","nonce":2827619403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6825","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6827","nonce":2827619403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6821","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6823","nonce":2827619403}]},"public_addr":"172.21.15.73:6817/2827619403","cluster_addr":"172.21.15.73:6819/2827619403","heartbeat_back_addr":"172.21.15.73:6827/2827619403","heartbeat_front_addr":"172.21.15.73:6823/2827619403","state":["exists","up"]},{"osd":2,"uuid":"90963b1a-5437-4ef8-ad8b-6b74c534eeb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6801","nonce":2542681064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6803","nonce":2542681064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6808","nonce":2542681064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6805","nonce":2542681064}]},"public_addr":"172.21.15.73:6801/2542681064","cluster_addr":"172.21.15.73:6803/2542681064","heartbeat_back_addr":"172.21.15.73:6808/2542681064","heartbeat_front_addr":"172.21.15.73:6805/2542681064","state":["exists","up"]},{"osd":3,"uuid":"bd3d2148-6e38-41b4-a453-fe6e3b8b4ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6822","nonce":1604042718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6826","nonce":1604042718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6831","nonce":1604042718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6829","nonce":1604042718}]},"public_addr":"172.21.15.73:6822/1604042718","cluster_addr":"172.21.15.73:6826/1604042718","heartbeat_back_addr":"172.21.15.73:6831/1604042718","heartbeat_front_addr":"172.21.15.73:6829/1604042718","state":["exists","up"]},{"osd":4,"uuid":"e8668dc8-a23f-46d2-8311-cb882da08e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6817","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6819","nonce":3591055561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6824","nonce":3591055561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6831","nonce":3591055561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6828","nonce":3591055561}]},"public_addr":"172.21.15.120:6819/3591055561","cluster_addr":"172.21.15.120:6824/3591055561","heartbeat_back_addr":"172.21.15.120:6831/3591055561","heartbeat_front_addr":"172.21.15.120:6828/3591055561","state":["exists","up"]},{"osd":5,"uuid":"cbac0bdb-96bd-4852-bcce-e564495c3fcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6809","nonce":819450433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6811","nonce":819450433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6815","nonce":819450433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6813","nonce":819450433}]},"public_addr":"172.21.15.120:6809/819450433","cluster_addr":"172.21.15.120:6811/819450433","heartbeat_back_addr":"172.21.15.120:6815/819450433","heartbeat_front_addr":"172.21.15.120:6813/819450433","state":["exists","up"]},{"osd":6,"uuid":"1884b2a8-569f-4272-96a9-f25a09364dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6801","nonce":3946210727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6803","nonce":3946210727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6807","nonce":3946210727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6805","nonce":3946210727}]},"public_addr":"172.21.15.120:6801/3946210727","cluster_addr":"172.21.15.120:6803/3946210727","heartbeat_back_addr":"172.21.15.120:6807/3946210727","heartbeat_front_addr":"172.21.15.120:6805/3946210727","state":["exists","up"]},{"osd":7,"uuid":"c0c3bf6d-a4c5-4f1e-aa2d-330951830666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6818","nonce":2915574674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6821","nonce":2915574674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6827","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6829","nonce":2915574674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6823","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6825","nonce":2915574674}]},"public_addr":"172.21.15.120:6818/2915574674","cluster_addr":"172.21.15.120:6821/2915574674","heartbeat_back_addr":"172.21.15.120:6829/2915574674","heartbeat_front_addr":"172.21.15.120:6825/2915574674","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.218216+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.208710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.112705+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.182789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.164324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.174212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.225396+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.261559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.120:6835/761435580":"2024-04-27T20:51:58.783228+0000","172.21.15.120:6833/761435580":"2024-04-27T20:51:58.783228+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:52:07.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.498+0000 7fc2c7e59640 1 -- 172.21.15.73:0/188019917 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc2a406c920 msgr2=0x7fc2a406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:07.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.498+0000 7fc2c7e59640 1 --2- 172.21.15.73:0/188019917 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc2a406c920 0x7fc2a406ede0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc034000 tx=0x7fc2bc02e000 comp rx=0 tx=0).stop 2024-04-26T20:52:07.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.498+0000 7fc2c7e59640 1 -- 172.21.15.73:0/188019917 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2c015d5f0 msgr2=0x7fc2c0145cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:07.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.498+0000 7fc2c7e59640 1 --2- 172.21.15.73:0/188019917 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2c015d5f0 0x7fc2c0145cc0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fc2b000c820 tx=0x7fc2b000cce0 comp rx=0 tx=0).stop 2024-04-26T20:52:07.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.498+0000 7fc2c7e59640 1 -- 172.21.15.73:0/188019917 shutdown_connections 2024-04-26T20:52:07.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.498+0000 7fc2c7e59640 1 --2- 172.21.15.73:0/188019917 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2c015d5f0 0x7fc2c0145cc0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:07.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.498+0000 7fc2c7e59640 1 --2- 172.21.15.73:0/188019917 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc2a406c920 0x7fc2a406ede0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:07.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.498+0000 7fc2c7e59640 1 --2- 172.21.15.73:0/188019917 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc2c01617f0 0x7fc2c00af2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:07.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.498+0000 7fc2c7e59640 1 --2- 172.21.15.73:0/188019917 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc2c015ab60 0x7fc2c014f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:07.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.498+0000 7fc2c7e59640 1 -- 172.21.15.73:0/188019917 >> 172.21.15.73:0/188019917 conn(0x7fc2c00d1cf0 msgr2=0x7fc2c00ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:07.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.498+0000 7fc2c7e59640 1 -- 172.21.15.73:0/188019917 shutdown_connections 2024-04-26T20:52:07.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.498+0000 7fc2c7e59640 1 -- 172.21.15.73:0/188019917 wait complete. 2024-04-26T20:52:07.510 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:50:19.993731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:51:43.110774+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:51:44.235846+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:52:07.510 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:52:07.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.609+0000 7fc1da8f1640 1 Processor -- start 2024-04-26T20:52:07.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.609+0000 7fc1da8f1640 1 -- start start 2024-04-26T20:52:07.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.610+0000 7fc1da8f1640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc1d40cfde0 0x7fc1d40d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:07.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.610+0000 7fc1da8f1640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc1d4071260 0x7fc1d415e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:07.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.610+0000 7fc1da8f1640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1d40d2390 con 0x7fc1d4070350 2024-04-26T20:52:07.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.610+0000 7fc1da8f1640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7fc1d40afab0 con 0x7fc1d4071260 2024-04-26T20:52:07.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.610+0000 7fc1da8f1640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fc1d406dd20 con 0x7fc1d40cfde0 2024-04-26T20:52:07.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.610+0000 7fc1d8e8e640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc1d4071260 0x7fc1d415e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:07.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.610+0000 7fc1d8e8e640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc1d4071260 0x7fc1d415e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40018/0 (socket says 172.21.15.73:40018) 2024-04-26T20:52:07.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.610+0000 7fc1d8e8e640 1 -- 172.21.15.73:0/469927516 learned_addr learned my addr 172.21.15.73:0/469927516 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:07.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.610+0000 7fc1d3fff640 1 --2- 172.21.15.73:0/469927516 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc1d40cfde0 0x7fc1d40d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:07.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.610+0000 7fc1d8e8e640 1 -- 172.21.15.73:0/469927516 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc1d40cfde0 msgr2=0x7fc1d40d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:07.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.610+0000 7fc1d8e8e640 1 --2- 172.21.15.73:0/469927516 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc1d40cfde0 0x7fc1d40d1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:07.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.610+0000 7fc1d8e8e640 1 -- 172.21.15.73:0/469927516 >> v1:172.21.15.120:6789/0 conn(0x7fc1d4070350 legacy=0x7fc1d4070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:07.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.610+0000 7fc1d8e8e640 1 -- 172.21.15.73:0/469927516 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1d406d3f0 con 0x7fc1d4071260 2024-04-26T20:52:07.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.611+0000 7fc1d8e8e640 1 --2- 172.21.15.73:0/469927516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1d4071260 0x7fc1d415e9f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc009e50 tx=0x7fc1cc02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a137b7569ceef510 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:07.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.611+0000 7fc1d2ffd640 1 -- 172.21.15.73:0/469927516 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc036070 con 0x7fc1d4071260 2024-04-26T20:52:07.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.611+0000 7fc1d2ffd640 1 -- 172.21.15.73:0/469927516 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1cc031600 con 0x7fc1d4071260 2024-04-26T20:52:07.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.611+0000 7fc1d2ffd640 1 -- 172.21.15.73:0/469927516 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc0319d0 con 0x7fc1d4071260 2024-04-26T20:52:07.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.611+0000 7fc1da8f1640 1 -- 172.21.15.73:0/469927516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1d4071260 msgr2=0x7fc1d415e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:07.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.611+0000 7fc1da8f1640 1 --2- 172.21.15.73:0/469927516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1d4071260 0x7fc1d415e9f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc009e50 tx=0x7fc1cc02a680 comp rx=0 tx=0).stop 2024-04-26T20:52:07.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.612+0000 7fc1da8f1640 1 -- 172.21.15.73:0/469927516 shutdown_connections 2024-04-26T20:52:07.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.612+0000 7fc1da8f1640 1 --2- 172.21.15.73:0/469927516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1d4071260 0x7fc1d415e9f0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:07.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.612+0000 7fc1da8f1640 1 --2- 172.21.15.73:0/469927516 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc1d40cfde0 0x7fc1d40d1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:07.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.612+0000 7fc1da8f1640 1 -- 172.21.15.73:0/469927516 >> 172.21.15.73:0/469927516 conn(0x7fc1d40d2d80 msgr2=0x7fc1d40d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:07.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.612+0000 7fc1da8f1640 1 -- 172.21.15.73:0/469927516 shutdown_connections 2024-04-26T20:52:07.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.612+0000 7fc1da8f1640 1 -- 172.21.15.73:0/469927516 wait complete. 2024-04-26T20:52:07.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.613+0000 7fc1da8f1640 1 Processor -- start 2024-04-26T20:52:07.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.613+0000 7fc1da8f1640 1 -- start start 2024-04-26T20:52:07.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.614+0000 7fc1da8f1640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc1d4070350 0x7fc1d414c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:07.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.614+0000 7fc1da8f1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1d4071260 0x7fc1d409cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:07.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.614+0000 7fc1da8f1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1d40cfde0 0x7fc1d415bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:07.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.614+0000 7fc1da8f1640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d40d20a0 con 0x7fc1d40cfde0 2024-04-26T20:52:07.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.614+0000 7fc1da8f1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d406d9b0 con 0x7fc1d4071260 2024-04-26T20:52:07.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.614+0000 7fc1da8f1640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fc1d415ef30 con 0x7fc1d4070350 2024-04-26T20:52:07.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.614+0000 7fc1d8e8e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1d40cfde0 0x7fc1d415bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:07.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.614+0000 7fc1d8e8e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1d40cfde0 0x7fc1d415bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40028/0 (socket says 172.21.15.73:40028) 2024-04-26T20:52:07.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.614+0000 7fc1d8e8e640 1 -- 172.21.15.73:0/2200186402 learned_addr learned my addr 172.21.15.73:0/2200186402 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:07.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.614+0000 7fc1d37fe640 1 --2- 172.21.15.73:0/2200186402 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1d4071260 0x7fc1d409cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:07.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.614+0000 7fc1d3fff640 1 --2- 172.21.15.73:0/2200186402 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc1d4070350 0x7fc1d414c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:07.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.614+0000 7fc1d8e8e640 1 -- 172.21.15.73:0/2200186402 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc1d4070350 msgr2=0x7fc1d414c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:07.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.614+0000 7fc1d8e8e640 1 --2- 172.21.15.73:0/2200186402 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc1d4070350 0x7fc1d414c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:07.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.614+0000 7fc1d8e8e640 1 -- 172.21.15.73:0/2200186402 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1d4071260 msgr2=0x7fc1d409cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:07.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.614+0000 7fc1d8e8e640 1 --2- 172.21.15.73:0/2200186402 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1d4071260 0x7fc1d409cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:07.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.614+0000 7fc1d8e8e640 1 -- 172.21.15.73:0/2200186402 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1d4155c40 con 0x7fc1d40cfde0 2024-04-26T20:52:07.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.614+0000 7fc1d3fff640 1 --2- 172.21.15.73:0/2200186402 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc1d4070350 0x7fc1d414c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:52:07.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.614+0000 7fc1d8e8e640 1 --2- 172.21.15.73:0/2200186402 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1d40cfde0 0x7fc1d415bad0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc034040 tx=0x7fc1cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:07.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.614+0000 7fc1d0ff9640 1 -- 172.21.15.73:0/2200186402 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc03f020 con 0x7fc1d40cfde0 2024-04-26T20:52:07.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.614+0000 7fc1d0ff9640 1 -- 172.21.15.73:0/2200186402 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1cc037040 con 0x7fc1d40cfde0 2024-04-26T20:52:07.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.614+0000 7fc1da8f1640 1 -- 172.21.15.73:0/2200186402 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1d4155f20 con 0x7fc1d40cfde0 2024-04-26T20:52:07.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.614+0000 7fc1d0ff9640 1 -- 172.21.15.73:0/2200186402 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc036070 con 0x7fc1d40cfde0 2024-04-26T20:52:07.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.614+0000 7fc1da8f1640 1 -- 172.21.15.73:0/2200186402 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1d409d100 con 0x7fc1d40cfde0 2024-04-26T20:52:07.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.615+0000 7fc1da8f1640 1 -- 172.21.15.73:0/2200186402 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1d40694f0 con 0x7fc1d40cfde0 2024-04-26T20:52:07.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.616+0000 7fc1d0ff9640 1 -- 172.21.15.73:0/2200186402 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc1cc042020 con 0x7fc1d40cfde0 2024-04-26T20:52:07.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.616+0000 7fc1d0ff9640 1 --2- 172.21.15.73:0/2200186402 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc1b406c920 0x7fc1b406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:07.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.616+0000 7fc1d3fff640 1 --2- 172.21.15.73:0/2200186402 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc1b406c920 0x7fc1b406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:07.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.616+0000 7fc1d0ff9640 1 -- 172.21.15.73:0/2200186402 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6187+0+0 (secure 0 0 0) 0x7fc1cc0ac0e0 con 0x7fc1d40cfde0 2024-04-26T20:52:07.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.617+0000 7fc1d3fff640 1 --2- 172.21.15.73:0/2200186402 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc1b406c920 0x7fc1b406ede0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4006fd0 tx=0x7fc1c4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:07.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.621+0000 7fc1d0ff9640 1 -- 172.21.15.73:0/2200186402 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc1cc049030 con 0x7fc1d40cfde0 2024-04-26T20:52:07.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.797+0000 7fc1da8f1640 1 -- 172.21.15.73:0/2200186402 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc1d4071910 con 0x7fc1d40cfde0 2024-04-26T20:52:07.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.798+0000 7fc1d0ff9640 1 -- 172.21.15.73:0/2200186402 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) ==== 93+0+10 (secure 0 0 0) 0x7fc1cc07a3c0 con 0x7fc1d40cfde0 2024-04-26T20:52:07.799 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-26T20:52:07.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.800+0000 7fc1da8f1640 1 -- 172.21.15.73:0/2200186402 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc1b406c920 msgr2=0x7fc1b406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:07.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.800+0000 7fc1da8f1640 1 --2- 172.21.15.73:0/2200186402 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc1b406c920 0x7fc1b406ede0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4006fd0 tx=0x7fc1c4008040 comp rx=0 tx=0).stop 2024-04-26T20:52:07.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.800+0000 7fc1da8f1640 1 -- 172.21.15.73:0/2200186402 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1d40cfde0 msgr2=0x7fc1d415bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:07.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.800+0000 7fc1da8f1640 1 --2- 172.21.15.73:0/2200186402 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1d40cfde0 0x7fc1d415bad0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc034040 tx=0x7fc1cc004a40 comp rx=0 tx=0).stop 2024-04-26T20:52:07.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.800+0000 7fc1da8f1640 1 -- 172.21.15.73:0/2200186402 shutdown_connections 2024-04-26T20:52:07.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.800+0000 7fc1da8f1640 1 --2- 172.21.15.73:0/2200186402 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1d40cfde0 0x7fc1d415bad0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:07.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.800+0000 7fc1da8f1640 1 --2- 172.21.15.73:0/2200186402 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc1b406c920 0x7fc1b406ede0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:07.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.800+0000 7fc1da8f1640 1 --2- 172.21.15.73:0/2200186402 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1d4071260 0x7fc1d409cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:07.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.800+0000 7fc1da8f1640 1 --2- 172.21.15.73:0/2200186402 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc1d4070350 0x7fc1d414c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:07.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.800+0000 7fc1da8f1640 1 -- 172.21.15.73:0/2200186402 >> 172.21.15.73:0/2200186402 conn(0x7fc1d40d2d80 msgr2=0x7fc1d406e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:07.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.800+0000 7fc1da8f1640 1 -- 172.21.15.73:0/2200186402 shutdown_connections 2024-04-26T20:52:07.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.800+0000 7fc1da8f1640 1 -- 172.21.15.73:0/2200186402 wait complete. 2024-04-26T20:52:07.812 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:52:07.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.910+0000 7f4d2f539640 1 Processor -- start 2024-04-26T20:52:07.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.910+0000 7f4d2f539640 1 -- start start 2024-04-26T20:52:07.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.911+0000 7f4d2f539640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4d2815f2d0 0x7f4d281604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:07.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.911+0000 7f4d2f539640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f4d28072700 0x7f4d2806f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:07.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.911+0000 7f4d2f539640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d280700e0 con 0x7f4d2806fcd0 2024-04-26T20:52:07.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.911+0000 7f4d2f539640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f4d28074010 con 0x7f4d28072700 2024-04-26T20:52:07.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.911+0000 7f4d2f539640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f4d28072f70 con 0x7f4d2815f2d0 2024-04-26T20:52:07.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.911+0000 7f4d2dad6640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f4d2806fcd0 0x7f4d280cbb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:39588/0 (socket says 172.21.15.73:39588) 2024-04-26T20:52:07.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.911+0000 7f4d2dad6640 1 -- 172.21.15.73:0/2547274183 learned_addr learned my addr 172.21.15.73:0/2547274183 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:52:07.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.911+0000 7f4d2cad4640 1 --2- 172.21.15.73:0/2547274183 >> v2:172.21.15.120:3300/0 conn(0x7f4d28072700 0x7f4d2806f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:07.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.911+0000 7f4d2d2d5640 1 --2- 172.21.15.73:0/2547274183 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4d2815f2d0 0x7f4d281604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:07.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.911+0000 7f4d1ffff640 1 -- 172.21.15.73:0/2547274183 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3656028725 0 0) 0x7f4d20001490 con 0x7f4d2806fcd0 2024-04-26T20:52:07.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.911+0000 7f4d1ffff640 1 -- 172.21.15.73:0/2547274183 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d14003520 con 0x7f4d2806fcd0 2024-04-26T20:52:07.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.912+0000 7f4d1ffff640 1 -- 172.21.15.73:0/2547274183 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1976053974 0 0) 0x7f4d14003520 con 0x7f4d2806fcd0 2024-04-26T20:52:07.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.912+0000 7f4d2d2d5640 1 -- 172.21.15.73:0/2547274183 >> v2:172.21.15.120:3300/0 conn(0x7f4d28072700 msgr2=0x7f4d2806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:07.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.912+0000 7f4d2d2d5640 1 --2- 172.21.15.73:0/2547274183 >> v2:172.21.15.120:3300/0 conn(0x7f4d28072700 0x7f4d2806f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:07.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.912+0000 7f4d2d2d5640 1 -- 172.21.15.73:0/2547274183 >> v1:172.21.15.73:6789/0 conn(0x7f4d2806fcd0 legacy=0x7f4d280cbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:07.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.912+0000 7f4d2d2d5640 1 -- 172.21.15.73:0/2547274183 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d28073250 con 0x7f4d2815f2d0 2024-04-26T20:52:07.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.912+0000 7f4d2cad4640 1 --2- 172.21.15.73:0/2547274183 >> v2:172.21.15.120:3300/0 conn(0x7f4d28072700 0x7f4d2806f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:52:07.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.912+0000 7f4d2d2d5640 1 --2- 172.21.15.73:0/2547274183 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4d2815f2d0 0x7f4d281604e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4d240097a0 tx=0x7f4d2402a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d76a8335c5fab52 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:07.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.913+0000 7f4d1ffff640 1 -- 172.21.15.73:0/2547274183 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d24037070 con 0x7f4d2815f2d0 2024-04-26T20:52:07.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.913+0000 7f4d1ffff640 1 -- 172.21.15.73:0/2547274183 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d2402aba0 con 0x7f4d2815f2d0 2024-04-26T20:52:07.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.913+0000 7f4d1ffff640 1 -- 172.21.15.73:0/2547274183 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d20001bc0 con 0x7f4d2815f2d0 2024-04-26T20:52:07.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.913+0000 7f4d2f539640 1 -- 172.21.15.73:0/2547274183 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4d2815f2d0 msgr2=0x7f4d281604e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:07.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.913+0000 7f4d2f539640 1 --2- 172.21.15.73:0/2547274183 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4d2815f2d0 0x7f4d281604e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4d240097a0 tx=0x7f4d2402a110 comp rx=0 tx=0).stop 2024-04-26T20:52:07.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.913+0000 7f4d2f539640 1 -- 172.21.15.73:0/2547274183 shutdown_connections 2024-04-26T20:52:07.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.913+0000 7f4d2f539640 1 --2- 172.21.15.73:0/2547274183 >> v2:172.21.15.120:3300/0 conn(0x7f4d28072700 0x7f4d2806f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:07.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.913+0000 7f4d2f539640 1 --2- 172.21.15.73:0/2547274183 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4d2815f2d0 0x7f4d281604e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:07.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.913+0000 7f4d2f539640 1 -- 172.21.15.73:0/2547274183 >> 172.21.15.73:0/2547274183 conn(0x7f4d280d0be0 msgr2=0x7f4d280d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:07.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.913+0000 7f4d2f539640 1 -- 172.21.15.73:0/2547274183 shutdown_connections 2024-04-26T20:52:07.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.913+0000 7f4d2f539640 1 -- 172.21.15.73:0/2547274183 wait complete. 2024-04-26T20:52:07.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.914+0000 7f4d2f539640 1 Processor -- start 2024-04-26T20:52:07.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.914+0000 7f4d2f539640 1 -- start start 2024-04-26T20:52:07.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.915+0000 7f4d2f539640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4d2806fcd0 0x7f4d28148d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:07.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.915+0000 7f4d2f539640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d28072700 0x7f4d28149e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:07.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.915+0000 7f4d2f539640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d2815f2d0 0x7f4d2814a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:07.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.915+0000 7f4d2f539640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4d28073af0 con 0x7f4d28072700 2024-04-26T20:52:07.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.915+0000 7f4d2f539640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4d28073810 con 0x7f4d2815f2d0 2024-04-26T20:52:07.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.915+0000 7f4d2f539640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f4d2815f6b0 con 0x7f4d2806fcd0 2024-04-26T20:52:07.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.915+0000 7f4d2cad4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d28072700 0x7f4d28149e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:07.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.915+0000 7f4d2cad4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d28072700 0x7f4d28149e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40032/0 (socket says 172.21.15.73:40032) 2024-04-26T20:52:07.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.915+0000 7f4d2cad4640 1 -- 172.21.15.73:0/3120608331 learned_addr learned my addr 172.21.15.73:0/3120608331 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:07.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.915+0000 7f4d2d2d5640 1 --2- 172.21.15.73:0/3120608331 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4d2806fcd0 0x7f4d28148d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:07.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.915+0000 7f4d2dad6640 1 --2- 172.21.15.73:0/3120608331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d2815f2d0 0x7f4d2814a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:07.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.915+0000 7f4d2cad4640 1 -- 172.21.15.73:0/3120608331 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4d2806fcd0 msgr2=0x7f4d28148d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:07.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.915+0000 7f4d2cad4640 1 --2- 172.21.15.73:0/3120608331 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4d2806fcd0 0x7f4d28148d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:07.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.915+0000 7f4d2cad4640 1 -- 172.21.15.73:0/3120608331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d2815f2d0 msgr2=0x7f4d2814a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:07.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.915+0000 7f4d2cad4640 1 --2- 172.21.15.73:0/3120608331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d2815f2d0 0x7f4d2814a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:07.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.915+0000 7f4d2cad4640 1 -- 172.21.15.73:0/3120608331 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d280c9030 con 0x7f4d28072700 2024-04-26T20:52:07.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.915+0000 7f4d2d2d5640 1 --2- 172.21.15.73:0/3120608331 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4d2806fcd0 0x7f4d28148d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:52:07.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.916+0000 7f4d2cad4640 1 --2- 172.21.15.73:0/3120608331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d28072700 0x7f4d28149e50 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f4d1800ebe0 tx=0x7f4d1800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:07.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.916+0000 7f4d1dffb640 1 -- 172.21.15.73:0/3120608331 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d18017070 con 0x7f4d28072700 2024-04-26T20:52:07.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.916+0000 7f4d1dffb640 1 -- 172.21.15.73:0/3120608331 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d1800cbd0 con 0x7f4d28072700 2024-04-26T20:52:07.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.916+0000 7f4d2f539640 1 -- 172.21.15.73:0/3120608331 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d280c7cb0 con 0x7f4d28072700 2024-04-26T20:52:07.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.916+0000 7f4d1dffb640 1 -- 172.21.15.73:0/3120608331 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d18013470 con 0x7f4d28072700 2024-04-26T20:52:07.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.916+0000 7f4d2f539640 1 -- 172.21.15.73:0/3120608331 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d280c8270 con 0x7f4d28072700 2024-04-26T20:52:07.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.917+0000 7f4d2f539640 1 -- 172.21.15.73:0/3120608331 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d280694f0 con 0x7f4d28072700 2024-04-26T20:52:07.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.917+0000 7f4d1dffb640 1 -- 172.21.15.73:0/3120608331 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4d1801a070 con 0x7f4d28072700 2024-04-26T20:52:07.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.918+0000 7f4d1dffb640 1 --2- 172.21.15.73:0/3120608331 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f4d0c06c8f0 0x7f4d0c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:07.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.918+0000 7f4d1dffb640 1 -- 172.21.15.73:0/3120608331 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6187+0+0 (secure 0 0 0) 0x7f4d1808c090 con 0x7f4d28072700 2024-04-26T20:52:07.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.918+0000 7f4d2d2d5640 1 --2- 172.21.15.73:0/3120608331 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f4d0c06c8f0 0x7f4d0c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:07.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.918+0000 7f4d2d2d5640 1 --2- 172.21.15.73:0/3120608331 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f4d0c06c8f0 0x7f4d0c06edb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4d24034000 tx=0x7f4d2402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:07.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:07.922+0000 7f4d1dffb640 1 -- 172.21.15.73:0/3120608331 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4d1805a250 con 0x7f4d28072700 2024-04-26T20:52:08.097 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.096+0000 7f4d2f539640 1 -- 172.21.15.73:0/3120608331 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4d280b3c50 con 0x7f4d28072700 2024-04-26T20:52:08.097 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.096+0000 7f4d1dffb640 1 -- 172.21.15.73:0/3120608331 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) ==== 104+0+11 (secure 0 0 0) 0x7f4d18059b50 con 0x7f4d28072700 2024-04-26T20:52:08.097 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-26T20:52:08.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.098+0000 7f4d2f539640 1 -- 172.21.15.73:0/3120608331 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f4d0c06c8f0 msgr2=0x7f4d0c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:08.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.098+0000 7f4d2f539640 1 --2- 172.21.15.73:0/3120608331 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f4d0c06c8f0 0x7f4d0c06edb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4d24034000 tx=0x7f4d2402e000 comp rx=0 tx=0).stop 2024-04-26T20:52:08.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.098+0000 7f4d2f539640 1 -- 172.21.15.73:0/3120608331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d28072700 msgr2=0x7f4d28149e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:08.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.098+0000 7f4d2f539640 1 --2- 172.21.15.73:0/3120608331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d28072700 0x7f4d28149e50 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f4d1800ebe0 tx=0x7f4d1800c520 comp rx=0 tx=0).stop 2024-04-26T20:52:08.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.098+0000 7f4d2f539640 1 -- 172.21.15.73:0/3120608331 shutdown_connections 2024-04-26T20:52:08.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.098+0000 7f4d2f539640 1 --2- 172.21.15.73:0/3120608331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d28072700 0x7f4d28149e50 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:08.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.098+0000 7f4d2f539640 1 --2- 172.21.15.73:0/3120608331 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f4d0c06c8f0 0x7f4d0c06edb0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:08.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.098+0000 7f4d2f539640 1 --2- 172.21.15.73:0/3120608331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d2815f2d0 0x7f4d2814a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:08.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.098+0000 7f4d2f539640 1 --2- 172.21.15.73:0/3120608331 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4d2806fcd0 0x7f4d28148d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:08.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.098+0000 7f4d2f539640 1 -- 172.21.15.73:0/3120608331 >> 172.21.15.73:0/3120608331 conn(0x7f4d280d0be0 msgr2=0x7f4d280cae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:08.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.098+0000 7f4d2f539640 1 -- 172.21.15.73:0/3120608331 shutdown_connections 2024-04-26T20:52:08.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.098+0000 7f4d2f539640 1 -- 172.21.15.73:0/3120608331 wait complete. 2024-04-26T20:52:08.110 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:52:08.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.208+0000 7fea26b78640 1 Processor -- start 2024-04-26T20:52:08.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.208+0000 7fea26b78640 1 -- start start 2024-04-26T20:52:08.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.209+0000 7fea26b78640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fea20156460 0x7fea200cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:08.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.209+0000 7fea26b78640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fea200cba10 0x7fea2006f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:08.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.209+0000 7fea26b78640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea20156840 con 0x7fea2006fcd0 2024-04-26T20:52:08.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.209+0000 7fea26b78640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fea200cbf20 con 0x7fea200cba10 2024-04-26T20:52:08.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.209+0000 7fea26b78640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fea200cc200 con 0x7fea20156460 2024-04-26T20:52:08.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.209+0000 7fea1ffff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fea200cba10 0x7fea2006f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:08.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.209+0000 7fea24914640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fea20156460 0x7fea200cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:08.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.209+0000 7fea25115640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7fea2006fcd0 0x7fea2015f980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.73:35146/0 (socket says 172.21.15.73:35146) 2024-04-26T20:52:08.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.209+0000 7fea25115640 1 -- 172.21.15.73:0/851484982 learned_addr learned my addr 172.21.15.73:0/851484982 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:52:08.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.210+0000 7fea1ffff640 1 -- 172.21.15.73:0/851484982 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fea20156460 msgr2=0x7fea200cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:08.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.210+0000 7fea1ffff640 1 --2- 172.21.15.73:0/851484982 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fea20156460 0x7fea200cb4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:08.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.210+0000 7fea1ffff640 1 -- 172.21.15.73:0/851484982 >> v1:172.21.15.120:6789/0 conn(0x7fea2006fcd0 legacy=0x7fea2015f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:08.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.210+0000 7fea1ffff640 1 -- 172.21.15.73:0/851484982 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea200cc4e0 con 0x7fea200cba10 2024-04-26T20:52:08.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.210+0000 7fea1f7fe640 1 -- 172.21.15.73:0/851484982 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1852356568 0 0) 0x7fea20156840 con 0x7fea2006fcd0 2024-04-26T20:52:08.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.210+0000 7fea24914640 1 --2- 172.21.15.73:0/851484982 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fea20156460 0x7fea200cb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:52:08.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.210+0000 7fea1ffff640 1 --2- 172.21.15.73:0/851484982 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea200cba10 0x7fea2006f790 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fea0c0097a0 tx=0x7fea0c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f0cb1a67894f5b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:08.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.211+0000 7fea1f7fe640 1 -- 172.21.15.73:0/851484982 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0c037070 con 0x7fea200cba10 2024-04-26T20:52:08.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.211+0000 7fea1f7fe640 1 -- 172.21.15.73:0/851484982 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea0c032440 con 0x7fea200cba10 2024-04-26T20:52:08.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.211+0000 7fea1f7fe640 1 -- 172.21.15.73:0/851484982 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0c032810 con 0x7fea200cba10 2024-04-26T20:52:08.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.211+0000 7fea26b78640 1 -- 172.21.15.73:0/851484982 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea200cba10 msgr2=0x7fea2006f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:08.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.211+0000 7fea26b78640 1 --2- 172.21.15.73:0/851484982 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea200cba10 0x7fea2006f790 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fea0c0097a0 tx=0x7fea0c02a180 comp rx=0 tx=0).stop 2024-04-26T20:52:08.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.211+0000 7fea26b78640 1 -- 172.21.15.73:0/851484982 shutdown_connections 2024-04-26T20:52:08.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.211+0000 7fea26b78640 1 --2- 172.21.15.73:0/851484982 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea200cba10 0x7fea2006f790 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:08.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.211+0000 7fea26b78640 1 --2- 172.21.15.73:0/851484982 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fea20156460 0x7fea200cb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:08.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.211+0000 7fea26b78640 1 -- 172.21.15.73:0/851484982 >> 172.21.15.73:0/851484982 conn(0x7fea200d0be0 msgr2=0x7fea200d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:08.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.211+0000 7fea26b78640 1 -- 172.21.15.73:0/851484982 shutdown_connections 2024-04-26T20:52:08.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.211+0000 7fea26b78640 1 -- 172.21.15.73:0/851484982 wait complete. 2024-04-26T20:52:08.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.212+0000 7fea26b78640 1 Processor -- start 2024-04-26T20:52:08.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.212+0000 7fea26b78640 1 -- start start 2024-04-26T20:52:08.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.213+0000 7fea26b78640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fea2006fcd0 0x7fea200c9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:08.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.213+0000 7fea26b78640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea200cba10 0x7fea20151860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:08.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.213+0000 7fea26b78640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea20156460 0x7fea201f3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:08.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.213+0000 7fea26b78640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fea2006e260 con 0x7fea20156460 2024-04-26T20:52:08.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.213+0000 7fea26b78640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fea20072700 con 0x7fea200cba10 2024-04-26T20:52:08.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.213+0000 7fea26b78640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fea2006e3e0 con 0x7fea2006fcd0 2024-04-26T20:52:08.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.213+0000 7fea25115640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea20156460 0x7fea201f3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:08.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.213+0000 7fea25115640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea20156460 0x7fea201f3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40048/0 (socket says 172.21.15.73:40048) 2024-04-26T20:52:08.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.213+0000 7fea25115640 1 -- 172.21.15.73:0/741897364 learned_addr learned my addr 172.21.15.73:0/741897364 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:08.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.213+0000 7fea1ffff640 1 --2- 172.21.15.73:0/741897364 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea200cba10 0x7fea20151860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:08.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.213+0000 7fea24914640 1 --2- 172.21.15.73:0/741897364 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fea2006fcd0 0x7fea200c9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:08.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.213+0000 7fea25115640 1 -- 172.21.15.73:0/741897364 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fea2006fcd0 msgr2=0x7fea200c9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:08.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.213+0000 7fea25115640 1 --2- 172.21.15.73:0/741897364 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fea2006fcd0 0x7fea200c9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:08.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.213+0000 7fea25115640 1 -- 172.21.15.73:0/741897364 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea200cba10 msgr2=0x7fea20151860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:08.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.213+0000 7fea25115640 1 --2- 172.21.15.73:0/741897364 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea200cba10 0x7fea20151860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:08.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.214+0000 7fea25115640 1 -- 172.21.15.73:0/741897364 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea200cab40 con 0x7fea20156460 2024-04-26T20:52:08.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.214+0000 7fea25115640 1 --2- 172.21.15.73:0/741897364 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea20156460 0x7fea201f3110 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fea1800c7a0 tx=0x7fea1800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:08.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.214+0000 7fea1d7fa640 1 -- 172.21.15.73:0/741897364 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea18018070 con 0x7fea20156460 2024-04-26T20:52:08.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.214+0000 7fea1d7fa640 1 -- 172.21.15.73:0/741897364 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea1800a640 con 0x7fea20156460 2024-04-26T20:52:08.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.214+0000 7fea1d7fa640 1 -- 172.21.15.73:0/741897364 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1800aa70 con 0x7fea20156460 2024-04-26T20:52:08.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.214+0000 7fea26b78640 1 -- 172.21.15.73:0/741897364 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea200c9b00 con 0x7fea20156460 2024-04-26T20:52:08.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.214+0000 7fea26b78640 1 -- 172.21.15.73:0/741897364 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea200ca340 con 0x7fea20156460 2024-04-26T20:52:08.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.215+0000 7fea26b78640 1 -- 172.21.15.73:0/741897364 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea200694f0 con 0x7fea20156460 2024-04-26T20:52:08.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.215+0000 7fea1d7fa640 1 -- 172.21.15.73:0/741897364 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fea1801b070 con 0x7fea20156460 2024-04-26T20:52:08.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.216+0000 7fea1d7fa640 1 --2- 172.21.15.73:0/741897364 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fea0406c8f0 0x7fea0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:08.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.216+0000 7fea1d7fa640 1 -- 172.21.15.73:0/741897364 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6187+0+0 (secure 0 0 0) 0x7fea1808b1d0 con 0x7fea20156460 2024-04-26T20:52:08.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.216+0000 7fea24914640 1 --2- 172.21.15.73:0/741897364 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fea0406c8f0 0x7fea0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:08.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.216+0000 7fea24914640 1 --2- 172.21.15.73:0/741897364 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fea0406c8f0 0x7fea0406edb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fea14009560 tx=0x7fea14009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:08.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.220+0000 7fea1d7fa640 1 -- 172.21.15.73:0/741897364 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fea200c9b00 con 0x7fea20156460 2024-04-26T20:52:08.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.393+0000 7fea26b78640 1 -- 172.21.15.73:0/741897364 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fea20070770 con 0x7fea20156460 2024-04-26T20:52:08.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.394+0000 7fea1d7fa640 1 -- 172.21.15.73:0/741897364 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) ==== 100+0+11 (secure 0 0 0) 0x7fea18058e60 con 0x7fea20156460 2024-04-26T20:52:08.395 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-26T20:52:08.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.396+0000 7fea26b78640 1 -- 172.21.15.73:0/741897364 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fea0406c8f0 msgr2=0x7fea0406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:08.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.396+0000 7fea26b78640 1 --2- 172.21.15.73:0/741897364 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fea0406c8f0 0x7fea0406edb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fea14009560 tx=0x7fea14009190 comp rx=0 tx=0).stop 2024-04-26T20:52:08.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.396+0000 7fea26b78640 1 -- 172.21.15.73:0/741897364 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea20156460 msgr2=0x7fea201f3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:08.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.396+0000 7fea26b78640 1 --2- 172.21.15.73:0/741897364 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea20156460 0x7fea201f3110 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fea1800c7a0 tx=0x7fea1800cc60 comp rx=0 tx=0).stop 2024-04-26T20:52:08.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.396+0000 7fea26b78640 1 -- 172.21.15.73:0/741897364 shutdown_connections 2024-04-26T20:52:08.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.396+0000 7fea26b78640 1 --2- 172.21.15.73:0/741897364 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea20156460 0x7fea201f3110 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:08.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.396+0000 7fea26b78640 1 --2- 172.21.15.73:0/741897364 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fea0406c8f0 0x7fea0406edb0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:08.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.396+0000 7fea26b78640 1 --2- 172.21.15.73:0/741897364 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fea200cba10 0x7fea20151860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:08.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.396+0000 7fea26b78640 1 --2- 172.21.15.73:0/741897364 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fea2006fcd0 0x7fea200c9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:08.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.396+0000 7fea26b78640 1 -- 172.21.15.73:0/741897364 >> 172.21.15.73:0/741897364 conn(0x7fea200d0be0 msgr2=0x7fea200c6d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:08.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.396+0000 7fea26b78640 1 -- 172.21.15.73:0/741897364 shutdown_connections 2024-04-26T20:52:08.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.396+0000 7fea26b78640 1 -- 172.21.15.73:0/741897364 wait complete. 2024-04-26T20:52:08.408 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:52:08.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.503+0000 7f8056c15640 1 Processor -- start 2024-04-26T20:52:08.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.503+0000 7f8056c15640 1 -- start start 2024-04-26T20:52:08.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.504+0000 7f8056c15640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f80500d06f0 0x7f80500d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:08.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.504+0000 7f8056c15640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80500d0ba0 con 0x7f805015f3a0 2024-04-26T20:52:08.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.504+0000 7f8056c15640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80500d0e20 con 0x7f80500b2850 2024-04-26T20:52:08.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.504+0000 7f8056c15640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f80500b1d70 con 0x7f80500d06f0 2024-04-26T20:52:08.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.504+0000 7f804ffff640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f80500d06f0 0x7f80500d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:08.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.504+0000 7f80549b1640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f805015f3a0 0x7f80500d1770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:39592/0 (socket says 172.21.15.73:39592) 2024-04-26T20:52:08.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.504+0000 7f80549b1640 1 -- 172.21.15.73:0/3306266837 learned_addr learned my addr 172.21.15.73:0/3306266837 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:52:08.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.505+0000 7f804ffff640 1 -- 172.21.15.73:0/3306266837 >> v1:172.21.15.120:6789/0 conn(0x7f80500b2850 legacy=0x7f80500b2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:08.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.505+0000 7f804ffff640 1 -- 172.21.15.73:0/3306266837 >> v1:172.21.15.73:6789/0 conn(0x7f805015f3a0 legacy=0x7f80500d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:08.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.505+0000 7f804ffff640 1 -- 172.21.15.73:0/3306266837 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80500b2050 con 0x7f80500d06f0 2024-04-26T20:52:08.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.505+0000 7f804f7fe640 1 -- 172.21.15.73:0/3306266837 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 732533861 0 0) 0x7f8040001110 con 0x7f805015f3a0 2024-04-26T20:52:08.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.505+0000 7f804ffff640 1 --2- 172.21.15.73:0/3306266837 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80500d06f0 0x7f80500d1e80 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f80440097a0 tx=0x7f804402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91da07cfbb081347 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:08.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.505+0000 7f804f7fe640 1 -- 172.21.15.73:0/3306266837 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8044037070 con 0x7f80500d06f0 2024-04-26T20:52:08.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.505+0000 7f804f7fe640 1 -- 172.21.15.73:0/3306266837 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8044032440 con 0x7f80500d06f0 2024-04-26T20:52:08.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.506+0000 7f804f7fe640 1 -- 172.21.15.73:0/3306266837 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8044032810 con 0x7f80500d06f0 2024-04-26T20:52:08.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.506+0000 7f8056c15640 1 -- 172.21.15.73:0/3306266837 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80500d06f0 msgr2=0x7f80500d1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:08.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.506+0000 7f8056c15640 1 --2- 172.21.15.73:0/3306266837 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80500d06f0 0x7f80500d1e80 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f80440097a0 tx=0x7f804402a1e0 comp rx=0 tx=0).stop 2024-04-26T20:52:08.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.506+0000 7f8056c15640 1 -- 172.21.15.73:0/3306266837 shutdown_connections 2024-04-26T20:52:08.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.506+0000 7f8056c15640 1 --2- 172.21.15.73:0/3306266837 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80500d06f0 0x7f80500d1e80 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:08.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.506+0000 7f8056c15640 1 -- 172.21.15.73:0/3306266837 >> 172.21.15.73:0/3306266837 conn(0x7f80500d4f90 msgr2=0x7f80500d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:08.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.506+0000 7f8056c15640 1 -- 172.21.15.73:0/3306266837 shutdown_connections 2024-04-26T20:52:08.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.506+0000 7f8056c15640 1 -- 172.21.15.73:0/3306266837 wait complete. 2024-04-26T20:52:08.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.507+0000 7f8056c15640 1 Processor -- start 2024-04-26T20:52:08.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.507+0000 7f8056c15640 1 -- start start 2024-04-26T20:52:08.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.508+0000 7f8056c15640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f80500b2850 0x7f805015a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:08.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.508+0000 7f8056c15640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f80500d06f0 0x7f8050159700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:08.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.508+0000 7f8056c15640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f805015f3a0 0x7f8050161f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:08.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.508+0000 7f8056c15640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f805015f900 con 0x7f805015f3a0 2024-04-26T20:52:08.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.508+0000 7f8056c15640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f805015f780 con 0x7f80500b2850 2024-04-26T20:52:08.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.508+0000 7f8056c15640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f80500d23c0 con 0x7f80500d06f0 2024-04-26T20:52:08.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.508+0000 7f80551b2640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f805015f3a0 0x7f8050161f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:08.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.508+0000 7f80551b2640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f805015f3a0 0x7f8050161f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40062/0 (socket says 172.21.15.73:40062) 2024-04-26T20:52:08.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.508+0000 7f80551b2640 1 -- 172.21.15.73:0/2267020965 learned_addr learned my addr 172.21.15.73:0/2267020965 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:08.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.508+0000 7f80549b1640 1 --2- 172.21.15.73:0/2267020965 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f80500b2850 0x7f805015a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:08.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.508+0000 7f804ffff640 1 --2- 172.21.15.73:0/2267020965 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f80500d06f0 0x7f8050159700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:08.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.508+0000 7f80551b2640 1 -- 172.21.15.73:0/2267020965 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f80500d06f0 msgr2=0x7f8050159700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:08.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.508+0000 7f80551b2640 1 --2- 172.21.15.73:0/2267020965 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f80500d06f0 0x7f8050159700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:08.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.508+0000 7f80551b2640 1 -- 172.21.15.73:0/2267020965 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f80500b2850 msgr2=0x7f805015a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:08.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.508+0000 7f80551b2640 1 --2- 172.21.15.73:0/2267020965 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f80500b2850 0x7f805015a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:08.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.508+0000 7f80551b2640 1 -- 172.21.15.73:0/2267020965 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80500b9db0 con 0x7f805015f3a0 2024-04-26T20:52:08.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.509+0000 7f80551b2640 1 --2- 172.21.15.73:0/2267020965 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f805015f3a0 0x7f8050161f50 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f804800c780 tx=0x7f804800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:08.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.509+0000 7f804d7fa640 1 -- 172.21.15.73:0/2267020965 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8048004b60 con 0x7f805015f3a0 2024-04-26T20:52:08.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.509+0000 7f804d7fa640 1 -- 172.21.15.73:0/2267020965 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8048004d00 con 0x7f805015f3a0 2024-04-26T20:52:08.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.509+0000 7f804d7fa640 1 -- 172.21.15.73:0/2267020965 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f804800aa90 con 0x7f805015f3a0 2024-04-26T20:52:08.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.509+0000 7f8056c15640 1 -- 172.21.15.73:0/2267020965 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80500ba090 con 0x7f805015f3a0 2024-04-26T20:52:08.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.509+0000 7f8056c15640 1 -- 172.21.15.73:0/2267020965 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f805015fd60 con 0x7f805015f3a0 2024-04-26T20:52:08.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.510+0000 7f8056c15640 1 -- 172.21.15.73:0/2267020965 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f805006e1b0 con 0x7f805015f3a0 2024-04-26T20:52:08.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.510+0000 7f804d7fa640 1 -- 172.21.15.73:0/2267020965 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f804800ac30 con 0x7f805015f3a0 2024-04-26T20:52:08.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.510+0000 7f804d7fa640 1 --2- 172.21.15.73:0/2267020965 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f803c06c8f0 0x7f803c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:08.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.511+0000 7f80549b1640 1 --2- 172.21.15.73:0/2267020965 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f803c06c8f0 0x7f803c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:08.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.511+0000 7f804d7fa640 1 -- 172.21.15.73:0/2267020965 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6187+0+0 (secure 0 0 0) 0x7f804808bfd0 con 0x7f805015f3a0 2024-04-26T20:52:08.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.511+0000 7f80549b1640 1 --2- 172.21.15.73:0/2267020965 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f803c06c8f0 0x7f803c06edb0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8040000f60 tx=0x7f8040007c00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:08.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.515+0000 7f804d7fa640 1 -- 172.21.15.73:0/2267020965 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f805006e1b0 con 0x7f805015f3a0 2024-04-26T20:52:08.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.724+0000 7f8056c15640 1 -- 172.21.15.73:0/2267020965 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8050068d80 con 0x7f805015f3a0 2024-04-26T20:52:08.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.725+0000 7f804d7fa640 1 -- 172.21.15.73:0/2267020965 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5185 (secure 0 0 0) 0x7f8050068d80 con 0x7f805015f3a0 2024-04-26T20:52:08.726 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:52:08.726 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4802,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/272662913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":272662913},{"type":"v1","addr":"172.21.15.73:6836","nonce":272662913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4943,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2624899096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6833","nonce":2624899096},{"type":"v1","addr":"172.21.15.120:6835","nonce":2624899096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5047,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/1438858294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":1438858294},{"type":"v1","addr":"172.21.15.73:6837","nonce":1438858294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:51:45.273953+0000","modified":"2024-04-26T20:52:01.795359+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5050},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5050":{"gid":5050,"name":"b","rank":0,"incarnation":17,"state":"up:active","state_seq":9,"addr":"172.21.15.120:6834/3883540031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3883540031},{"type":"v1","addr":"172.21.15.120:6834","nonce":3883540031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5050,"qdb_cluster":[5050]},"id":2}]} 2024-04-26T20:52:08.726 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 20 2024-04-26T20:52:08.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.727+0000 7f8056c15640 1 -- 172.21.15.73:0/2267020965 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f803c06c8f0 msgr2=0x7f803c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:08.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.727+0000 7f8056c15640 1 --2- 172.21.15.73:0/2267020965 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f803c06c8f0 0x7f803c06edb0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8040000f60 tx=0x7f8040007c00 comp rx=0 tx=0).stop 2024-04-26T20:52:08.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.727+0000 7f8056c15640 1 -- 172.21.15.73:0/2267020965 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f805015f3a0 msgr2=0x7f8050161f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:08.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.727+0000 7f8056c15640 1 --2- 172.21.15.73:0/2267020965 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f805015f3a0 0x7f8050161f50 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f804800c780 tx=0x7f804800cc40 comp rx=0 tx=0).stop 2024-04-26T20:52:08.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.727+0000 7f8056c15640 1 -- 172.21.15.73:0/2267020965 shutdown_connections 2024-04-26T20:52:08.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.727+0000 7f8056c15640 1 --2- 172.21.15.73:0/2267020965 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f805015f3a0 0x7f8050161f50 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:08.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.727+0000 7f8056c15640 1 --2- 172.21.15.73:0/2267020965 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f803c06c8f0 0x7f803c06edb0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:08.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.727+0000 7f8056c15640 1 --2- 172.21.15.73:0/2267020965 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f80500d06f0 0x7f8050159700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:08.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.727+0000 7f8056c15640 1 --2- 172.21.15.73:0/2267020965 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f80500b2850 0x7f805015a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:08.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.727+0000 7f8056c15640 1 -- 172.21.15.73:0/2267020965 >> 172.21.15.73:0/2267020965 conn(0x7f80500d4f90 msgr2=0x7f805006e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:08.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.727+0000 7f8056c15640 1 -- 172.21.15.73:0/2267020965 shutdown_connections 2024-04-26T20:52:08.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.727+0000 7f8056c15640 1 -- 172.21.15.73:0/2267020965 wait complete. 2024-04-26T20:52:08.739 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:52:08.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.835+0000 7f49ee4ea640 1 Processor -- start 2024-04-26T20:52:08.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.835+0000 7f49ee4ea640 1 -- start start 2024-04-26T20:52:08.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.835+0000 7f49ee4ea640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f49e8162880 0x7f49e80d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:08.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.835+0000 7f49ee4ea640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f49e80b5f20 0x7f49e80d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:08.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.835+0000 7f49ee4ea640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f49e80d39f0 0x7f49e80b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:08.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.835+0000 7f49ee4ea640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f49e80d0580 con 0x7f49e80b5f20 2024-04-26T20:52:08.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.835+0000 7f49ee4ea640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f49e80b7e50 con 0x7f49e80d39f0 2024-04-26T20:52:08.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.835+0000 7f49ee4ea640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f49e80d1800 con 0x7f49e8162880 2024-04-26T20:52:08.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.836+0000 7f49e77fe640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f49e80b5f20 0x7f49e80d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:08.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.836+0000 7f49e77fe640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f49e80b5f20 0x7f49e80d34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40074/0 (socket says 172.21.15.73:40074) 2024-04-26T20:52:08.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.836+0000 7f49e77fe640 1 -- 172.21.15.73:0/2160804844 learned_addr learned my addr 172.21.15.73:0/2160804844 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:08.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.836+0000 7f49e7fff640 1 --2- 172.21.15.73:0/2160804844 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f49e8162880 0x7f49e80d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:08.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.836+0000 7f49eca87640 1 --2- 172.21.15.73:0/2160804844 >> v2:172.21.15.120:3300/0 conn(0x7f49e80d39f0 0x7f49e80b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:08.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.836+0000 7f49e77fe640 1 -- 172.21.15.73:0/2160804844 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f49e8162880 msgr2=0x7f49e80d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:08.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.836+0000 7f49e77fe640 1 --2- 172.21.15.73:0/2160804844 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f49e8162880 0x7f49e80d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:08.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.836+0000 7f49e77fe640 1 -- 172.21.15.73:0/2160804844 >> v2:172.21.15.120:3300/0 conn(0x7f49e80d39f0 msgr2=0x7f49e80b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:08.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.836+0000 7f49e77fe640 1 --2- 172.21.15.73:0/2160804844 >> v2:172.21.15.120:3300/0 conn(0x7f49e80d39f0 0x7f49e80b69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:08.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.836+0000 7f49e77fe640 1 -- 172.21.15.73:0/2160804844 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49e80d07d0 con 0x7f49e80b5f20 2024-04-26T20:52:08.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.836+0000 7f49e7fff640 1 --2- 172.21.15.73:0/2160804844 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f49e8162880 0x7f49e80d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:52:08.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.836+0000 7f49eca87640 1 --2- 172.21.15.73:0/2160804844 >> v2:172.21.15.120:3300/0 conn(0x7f49e80d39f0 0x7f49e80b69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:52:08.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.837+0000 7f49e77fe640 1 --2- 172.21.15.73:0/2160804844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49e80b5f20 0x7f49e80d34b0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f49e808f770 tx=0x7f49d802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8012c4139fc4f70d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:08.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.837+0000 7f49e6ffd640 1 -- 172.21.15.73:0/2160804844 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d8037070 con 0x7f49e80b5f20 2024-04-26T20:52:08.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.837+0000 7f49e6ffd640 1 -- 172.21.15.73:0/2160804844 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49d8032440 con 0x7f49e80b5f20 2024-04-26T20:52:08.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.837+0000 7f49e6ffd640 1 -- 172.21.15.73:0/2160804844 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d8032810 con 0x7f49e80b5f20 2024-04-26T20:52:08.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.837+0000 7f49ee4ea640 1 -- 172.21.15.73:0/2160804844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49e80b5f20 msgr2=0x7f49e80d34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:08.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.837+0000 7f49ee4ea640 1 --2- 172.21.15.73:0/2160804844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49e80b5f20 0x7f49e80d34b0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f49e808f770 tx=0x7f49d802a1e0 comp rx=0 tx=0).stop 2024-04-26T20:52:08.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.837+0000 7f49ee4ea640 1 -- 172.21.15.73:0/2160804844 shutdown_connections 2024-04-26T20:52:08.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.837+0000 7f49ee4ea640 1 --2- 172.21.15.73:0/2160804844 >> v2:172.21.15.120:3300/0 conn(0x7f49e80d39f0 0x7f49e80b69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:08.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.837+0000 7f49ee4ea640 1 --2- 172.21.15.73:0/2160804844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49e80b5f20 0x7f49e80d34b0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:08.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.837+0000 7f49ee4ea640 1 --2- 172.21.15.73:0/2160804844 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f49e8162880 0x7f49e80d2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:08.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.837+0000 7f49ee4ea640 1 -- 172.21.15.73:0/2160804844 >> 172.21.15.73:0/2160804844 conn(0x7f49e80d4f60 msgr2=0x7f49e80d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:08.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.837+0000 7f49ee4ea640 1 -- 172.21.15.73:0/2160804844 shutdown_connections 2024-04-26T20:52:08.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.837+0000 7f49ee4ea640 1 -- 172.21.15.73:0/2160804844 wait complete. 2024-04-26T20:52:08.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.838+0000 7f49ee4ea640 1 Processor -- start 2024-04-26T20:52:08.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.839+0000 7f49ee4ea640 1 -- start start 2024-04-26T20:52:08.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.839+0000 7f49ee4ea640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49e80b5f20 0x7f49e80b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:08.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.839+0000 7f49ee4ea640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f49e80d39f0 0x7f49e80b2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:08.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.839+0000 7f49ee4ea640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49e8162880 0x7f49e81f8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:08.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.839+0000 7f49ee4ea640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f49e8162c60 con 0x7f49e80b5f20 2024-04-26T20:52:08.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.839+0000 7f49ee4ea640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f49e80d0d90 con 0x7f49e8162880 2024-04-26T20:52:08.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.839+0000 7f49ee4ea640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f49e8162de0 con 0x7f49e80d39f0 2024-04-26T20:52:08.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.839+0000 7f49e7fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49e80b5f20 0x7f49e80b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:08.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.840+0000 7f49e7fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49e80b5f20 0x7f49e80b40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40078/0 (socket says 172.21.15.73:40078) 2024-04-26T20:52:08.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.840+0000 7f49e7fff640 1 -- 172.21.15.73:0/1707582989 learned_addr learned my addr 172.21.15.73:0/1707582989 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:08.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.840+0000 7f49e77fe640 1 --2- 172.21.15.73:0/1707582989 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f49e80d39f0 0x7f49e80b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:08.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.840+0000 7f49eca87640 1 --2- 172.21.15.73:0/1707582989 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49e8162880 0x7f49e81f8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:08.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.840+0000 7f49e7fff640 1 -- 172.21.15.73:0/1707582989 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f49e80d39f0 msgr2=0x7f49e80b2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:08.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.840+0000 7f49e7fff640 1 --2- 172.21.15.73:0/1707582989 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f49e80d39f0 0x7f49e80b2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:08.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.840+0000 7f49e7fff640 1 -- 172.21.15.73:0/1707582989 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49e8162880 msgr2=0x7f49e81f8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:08.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.840+0000 7f49e7fff640 1 --2- 172.21.15.73:0/1707582989 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49e8162880 0x7f49e81f8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:08.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.840+0000 7f49e7fff640 1 -- 172.21.15.73:0/1707582989 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49e80b3220 con 0x7f49e80b5f20 2024-04-26T20:52:08.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.840+0000 7f49eca87640 1 --2- 172.21.15.73:0/1707582989 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49e8162880 0x7f49e81f8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:52:08.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.840+0000 7f49e7fff640 1 --2- 172.21.15.73:0/1707582989 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49e80b5f20 0x7f49e80b40a0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f49dc00c820 tx=0x7f49dc00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:08.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.840+0000 7f49e4ff9640 1 -- 172.21.15.73:0/1707582989 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49dc015070 con 0x7f49e80b5f20 2024-04-26T20:52:08.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.840+0000 7f49e4ff9640 1 -- 172.21.15.73:0/1707582989 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49dc007a30 con 0x7f49e80b5f20 2024-04-26T20:52:08.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.840+0000 7f49e4ff9640 1 -- 172.21.15.73:0/1707582989 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49dc007e00 con 0x7f49e80b5f20 2024-04-26T20:52:08.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.840+0000 7f49ee4ea640 1 -- 172.21.15.73:0/1707582989 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49e80cc040 con 0x7f49e80b5f20 2024-04-26T20:52:08.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.840+0000 7f49ee4ea640 1 -- 172.21.15.73:0/1707582989 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49e80cc600 con 0x7f49e80b5f20 2024-04-26T20:52:08.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.841+0000 7f49ee4ea640 1 -- 172.21.15.73:0/1707582989 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49e80694f0 con 0x7f49e80b5f20 2024-04-26T20:52:08.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.842+0000 7f49e4ff9640 1 -- 172.21.15.73:0/1707582989 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f49dc018070 con 0x7f49e80b5f20 2024-04-26T20:52:08.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.842+0000 7f49e4ff9640 1 --2- 172.21.15.73:0/1707582989 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f49d406c8f0 0x7f49d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:08.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.842+0000 7f49e4ff9640 1 -- 172.21.15.73:0/1707582989 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6187+0+0 (secure 0 0 0) 0x7f49dc08bc50 con 0x7f49e80b5f20 2024-04-26T20:52:08.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.842+0000 7f49e77fe640 1 --2- 172.21.15.73:0/1707582989 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f49d406c8f0 0x7f49d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:08.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.842+0000 7f49e77fe640 1 --2- 172.21.15.73:0/1707582989 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f49d406c8f0 0x7f49d406edb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f49d8034000 tx=0x7f49d802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:08.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:08.846+0000 7f49e4ff9640 1 -- 172.21.15.73:0/1707582989 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f49dc091020 con 0x7f49e80b5f20 2024-04-26T20:52:09.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.025+0000 7f49ee4ea640 1 -- 172.21.15.73:0/1707582989 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f49e8069c20 con 0x7f49e80b5f20 2024-04-26T20:52:09.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.026+0000 7f49e4ff9640 1 -- 172.21.15.73:0/1707582989 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+17071 (secure 0 0 0) 0x7f49dc059850 con 0x7f49e80b5f20 2024-04-26T20:52:09.028 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:52:09.028 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":40,"fsid":"fbd3f548-b547-4713-856b-09f98f5d1764","created":"2024-04-26T20:50:07.876786+0000","modified":"2024-04-26T20:51:58.783253+0000","last_up_change":"2024-04-26T20:50:19.082741+0000","last_in_change":"2024-04-26T20:50:10.018400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:50:19.993731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:51:43.110774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-26T20:51:44.235846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d6f4f3-214a-4392-a02f-c33cbcab3c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6809","nonce":3461705124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6811","nonce":3461705124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6815","nonce":3461705124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6813","nonce":3461705124}]},"public_addr":"172.21.15.73:6809/3461705124","cluster_addr":"172.21.15.73:6811/3461705124","heartbeat_back_addr":"172.21.15.73:6815/3461705124","heartbeat_front_addr":"172.21.15.73:6813/3461705124","state":["exists","up"]},{"osd":1,"uuid":"786bf8ca-6ddc-4b3c-be6c-9152398bd5fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6817","nonce":2827619403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6819","nonce":2827619403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6825","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6827","nonce":2827619403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6821","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6823","nonce":2827619403}]},"public_addr":"172.21.15.73:6817/2827619403","cluster_addr":"172.21.15.73:6819/2827619403","heartbeat_back_addr":"172.21.15.73:6827/2827619403","heartbeat_front_addr":"172.21.15.73:6823/2827619403","state":["exists","up"]},{"osd":2,"uuid":"90963b1a-5437-4ef8-ad8b-6b74c534eeb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6801","nonce":2542681064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6803","nonce":2542681064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6808","nonce":2542681064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6805","nonce":2542681064}]},"public_addr":"172.21.15.73:6801/2542681064","cluster_addr":"172.21.15.73:6803/2542681064","heartbeat_back_addr":"172.21.15.73:6808/2542681064","heartbeat_front_addr":"172.21.15.73:6805/2542681064","state":["exists","up"]},{"osd":3,"uuid":"bd3d2148-6e38-41b4-a453-fe6e3b8b4ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6822","nonce":1604042718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6826","nonce":1604042718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6831","nonce":1604042718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6829","nonce":1604042718}]},"public_addr":"172.21.15.73:6822/1604042718","cluster_addr":"172.21.15.73:6826/1604042718","heartbeat_back_addr":"172.21.15.73:6831/1604042718","heartbeat_front_addr":"172.21.15.73:6829/1604042718","state":["exists","up"]},{"osd":4,"uuid":"e8668dc8-a23f-46d2-8311-cb882da08e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6817","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6819","nonce":3591055561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6824","nonce":3591055561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6831","nonce":3591055561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6828","nonce":3591055561}]},"public_addr":"172.21.15.120:6819/3591055561","cluster_addr":"172.21.15.120:6824/3591055561","heartbeat_back_addr":"172.21.15.120:6831/3591055561","heartbeat_front_addr":"172.21.15.120:6828/3591055561","state":["exists","up"]},{"osd":5,"uuid":"cbac0bdb-96bd-4852-bcce-e564495c3fcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6809","nonce":819450433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6811","nonce":819450433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6815","nonce":819450433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6813","nonce":819450433}]},"public_addr":"172.21.15.120:6809/819450433","cluster_addr":"172.21.15.120:6811/819450433","heartbeat_back_addr":"172.21.15.120:6815/819450433","heartbeat_front_addr":"172.21.15.120:6813/819450433","state":["exists","up"]},{"osd":6,"uuid":"1884b2a8-569f-4272-96a9-f25a09364dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6801","nonce":3946210727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6803","nonce":3946210727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6807","nonce":3946210727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6805","nonce":3946210727}]},"public_addr":"172.21.15.120:6801/3946210727","cluster_addr":"172.21.15.120:6803/3946210727","heartbeat_back_addr":"172.21.15.120:6807/3946210727","heartbeat_front_addr":"172.21.15.120:6805/3946210727","state":["exists","up"]},{"osd":7,"uuid":"c0c3bf6d-a4c5-4f1e-aa2d-330951830666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6818","nonce":2915574674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6821","nonce":2915574674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6827","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6829","nonce":2915574674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6823","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6825","nonce":2915574674}]},"public_addr":"172.21.15.120:6818/2915574674","cluster_addr":"172.21.15.120:6821/2915574674","heartbeat_back_addr":"172.21.15.120:6829/2915574674","heartbeat_front_addr":"172.21.15.120:6825/2915574674","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.218216+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.208710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.112705+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.182789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.164324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.174212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.225396+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.261559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.120:6835/761435580":"2024-04-27T20:51:58.783228+0000","172.21.15.120:6833/761435580":"2024-04-27T20:51:58.783228+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:52:09.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.028+0000 7f49ee4ea640 1 -- 172.21.15.73:0/1707582989 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f49d406c8f0 msgr2=0x7f49d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:09.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.028+0000 7f49ee4ea640 1 --2- 172.21.15.73:0/1707582989 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f49d406c8f0 0x7f49d406edb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f49d8034000 tx=0x7f49d802e000 comp rx=0 tx=0).stop 2024-04-26T20:52:09.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.028+0000 7f49ee4ea640 1 -- 172.21.15.73:0/1707582989 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49e80b5f20 msgr2=0x7f49e80b40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:09.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.028+0000 7f49ee4ea640 1 --2- 172.21.15.73:0/1707582989 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49e80b5f20 0x7f49e80b40a0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f49dc00c820 tx=0x7f49dc00cc20 comp rx=0 tx=0).stop 2024-04-26T20:52:09.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.028+0000 7f49ee4ea640 1 -- 172.21.15.73:0/1707582989 shutdown_connections 2024-04-26T20:52:09.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.028+0000 7f49ee4ea640 1 --2- 172.21.15.73:0/1707582989 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f49d406c8f0 0x7f49d406edb0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:09.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.028+0000 7f49ee4ea640 1 --2- 172.21.15.73:0/1707582989 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f49e8162880 0x7f49e81f8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:09.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.028+0000 7f49ee4ea640 1 --2- 172.21.15.73:0/1707582989 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f49e80d39f0 0x7f49e80b2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:09.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.028+0000 7f49ee4ea640 1 --2- 172.21.15.73:0/1707582989 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49e80b5f20 0x7f49e80b40a0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:09.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.028+0000 7f49ee4ea640 1 -- 172.21.15.73:0/1707582989 >> 172.21.15.73:0/1707582989 conn(0x7f49e80d4f60 msgr2=0x7f49e815e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:09.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.029+0000 7f49ee4ea640 1 -- 172.21.15.73:0/1707582989 shutdown_connections 2024-04-26T20:52:09.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.029+0000 7f49ee4ea640 1 -- 172.21.15.73:0/1707582989 wait complete. 2024-04-26T20:52:09.041 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T20:52:09.041 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:52:09.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.136+0000 7fa5222c4640 1 Processor -- start 2024-04-26T20:52:09.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.136+0000 7fa5222c4640 1 -- start start 2024-04-26T20:52:09.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.137+0000 7fa5222c4640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa51c15a9f0 0x7fa51c0b4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:09.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.137+0000 7fa5222c4640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fa51c0726e0 0x7fa51c072b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:09.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.137+0000 7fa5222c4640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa51c0ce060 con 0x7fa51c161510 2024-04-26T20:52:09.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.137+0000 7fa5222c4640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fa51c0ce620 con 0x7fa51c0726e0 2024-04-26T20:52:09.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.137+0000 7fa5222c4640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fa51c0ce340 con 0x7fa51c15a9f0 2024-04-26T20:52:09.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.137+0000 7fa51b7fe640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fa51c161510 0x7fa51c15dbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:39598/0 (socket says 172.21.15.73:39598) 2024-04-26T20:52:09.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.137+0000 7fa51b7fe640 1 -- 172.21.15.73:0/1863000521 learned_addr learned my addr 172.21.15.73:0/1863000521 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:52:09.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.137+0000 7fa51bfff640 1 --2- 172.21.15.73:0/1863000521 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa51c15a9f0 0x7fa51c0b4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:09.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.137+0000 7fa520861640 1 --2- 172.21.15.73:0/1863000521 >> v2:172.21.15.120:3300/0 conn(0x7fa51c0726e0 0x7fa51c072b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:09.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.137+0000 7fa51affd640 1 -- 172.21.15.73:0/1863000521 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4064687895 0 0) 0x7fa51c0ce060 con 0x7fa51c161510 2024-04-26T20:52:09.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.137+0000 7fa51affd640 1 -- 172.21.15.73:0/1863000521 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa504003520 con 0x7fa51c161510 2024-04-26T20:52:09.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.138+0000 7fa51affd640 1 -- 172.21.15.73:0/1863000521 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1430095963 0 0) 0x7fa504003520 con 0x7fa51c161510 2024-04-26T20:52:09.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.138+0000 7fa51bfff640 1 -- 172.21.15.73:0/1863000521 >> v2:172.21.15.120:3300/0 conn(0x7fa51c0726e0 msgr2=0x7fa51c072b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:09.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.138+0000 7fa51bfff640 1 --2- 172.21.15.73:0/1863000521 >> v2:172.21.15.120:3300/0 conn(0x7fa51c0726e0 0x7fa51c072b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:09.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.138+0000 7fa51bfff640 1 -- 172.21.15.73:0/1863000521 >> v1:172.21.15.73:6789/0 conn(0x7fa51c161510 legacy=0x7fa51c15dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:09.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.138+0000 7fa51bfff640 1 -- 172.21.15.73:0/1863000521 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa51c15d6b0 con 0x7fa51c15a9f0 2024-04-26T20:52:09.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.138+0000 7fa520861640 1 --2- 172.21.15.73:0/1863000521 >> v2:172.21.15.120:3300/0 conn(0x7fa51c0726e0 0x7fa51c072b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:52:09.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.138+0000 7fa51bfff640 1 --2- 172.21.15.73:0/1863000521 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa51c15a9f0 0x7fa51c0b4480 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa50c0097a0 tx=0x7fa50c02a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f1f516ec5bb21ba server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:09.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.139+0000 7fa51affd640 1 -- 172.21.15.73:0/1863000521 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50c037070 con 0x7fa51c15a9f0 2024-04-26T20:52:09.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.139+0000 7fa51affd640 1 -- 172.21.15.73:0/1863000521 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa50c02aba0 con 0x7fa51c15a9f0 2024-04-26T20:52:09.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.139+0000 7fa51affd640 1 -- 172.21.15.73:0/1863000521 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa510001670 con 0x7fa51c15a9f0 2024-04-26T20:52:09.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.139+0000 7fa5222c4640 1 -- 172.21.15.73:0/1863000521 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa51c15a9f0 msgr2=0x7fa51c0b4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:09.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.139+0000 7fa5222c4640 1 --2- 172.21.15.73:0/1863000521 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa51c15a9f0 0x7fa51c0b4480 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa50c0097a0 tx=0x7fa50c02a110 comp rx=0 tx=0).stop 2024-04-26T20:52:09.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.139+0000 7fa5222c4640 1 -- 172.21.15.73:0/1863000521 shutdown_connections 2024-04-26T20:52:09.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.139+0000 7fa5222c4640 1 --2- 172.21.15.73:0/1863000521 >> v2:172.21.15.120:3300/0 conn(0x7fa51c0726e0 0x7fa51c072b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:09.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.139+0000 7fa5222c4640 1 --2- 172.21.15.73:0/1863000521 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa51c15a9f0 0x7fa51c0b4480 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:09.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.139+0000 7fa5222c4640 1 -- 172.21.15.73:0/1863000521 >> 172.21.15.73:0/1863000521 conn(0x7fa51c0b3180 msgr2=0x7fa51c0b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:09.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.139+0000 7fa5222c4640 1 -- 172.21.15.73:0/1863000521 shutdown_connections 2024-04-26T20:52:09.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.139+0000 7fa5222c4640 1 -- 172.21.15.73:0/1863000521 wait complete. 2024-04-26T20:52:09.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.140+0000 7fa5222c4640 1 Processor -- start 2024-04-26T20:52:09.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.141+0000 7fa5222c4640 1 -- start start 2024-04-26T20:52:09.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.141+0000 7fa5222c4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa51c0726e0 0x7fa51c0af9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:09.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.141+0000 7fa5222c4640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa51c15a9f0 0x7fa51c0ad4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:09.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.141+0000 7fa5222c4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa51c161510 0x7fa51c146e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:09.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.141+0000 7fa5222c4640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa51c15e2e0 con 0x7fa51c0726e0 2024-04-26T20:52:09.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.141+0000 7fa5222c4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa51c15add0 con 0x7fa51c161510 2024-04-26T20:52:09.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.141+0000 7fa5222c4640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fa51c15e460 con 0x7fa51c15a9f0 2024-04-26T20:52:09.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.141+0000 7fa51bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa51c0726e0 0x7fa51c0af9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:09.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.141+0000 7fa51bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa51c0726e0 0x7fa51c0af9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40094/0 (socket says 172.21.15.73:40094) 2024-04-26T20:52:09.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.141+0000 7fa51bfff640 1 -- 172.21.15.73:0/3405135841 learned_addr learned my addr 172.21.15.73:0/3405135841 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:09.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.142+0000 7fa520861640 1 --2- 172.21.15.73:0/3405135841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa51c161510 0x7fa51c146e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:09.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.142+0000 7fa51bfff640 1 -- 172.21.15.73:0/3405135841 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa51c15a9f0 msgr2=0x7fa51c0ad4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:09.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.142+0000 7fa51b7fe640 1 --2- 172.21.15.73:0/3405135841 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa51c15a9f0 0x7fa51c0ad4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:09.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.142+0000 7fa51bfff640 1 --2- 172.21.15.73:0/3405135841 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa51c15a9f0 0x7fa51c0ad4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:09.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.142+0000 7fa51bfff640 1 -- 172.21.15.73:0/3405135841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa51c161510 msgr2=0x7fa51c146e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:09.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.142+0000 7fa51bfff640 1 --2- 172.21.15.73:0/3405135841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa51c161510 0x7fa51c146e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:09.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.142+0000 7fa51bfff640 1 -- 172.21.15.73:0/3405135841 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa51c0aa600 con 0x7fa51c0726e0 2024-04-26T20:52:09.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.142+0000 7fa51b7fe640 1 --2- 172.21.15.73:0/3405135841 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa51c15a9f0 0x7fa51c0ad4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:52:09.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.142+0000 7fa520861640 1 --2- 172.21.15.73:0/3405135841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa51c161510 0x7fa51c146e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:52:09.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.142+0000 7fa51bfff640 1 --2- 172.21.15.73:0/3405135841 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa51c0726e0 0x7fa51c0af9f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fa50c035040 tx=0x7fa50c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:09.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.142+0000 7fa518ff9640 1 -- 172.21.15.73:0/3405135841 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50c037050 con 0x7fa51c0726e0 2024-04-26T20:52:09.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.142+0000 7fa518ff9640 1 -- 172.21.15.73:0/3405135841 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa50c043070 con 0x7fa51c0726e0 2024-04-26T20:52:09.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.142+0000 7fa518ff9640 1 -- 172.21.15.73:0/3405135841 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50c006e10 con 0x7fa51c0726e0 2024-04-26T20:52:09.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.142+0000 7fa5222c4640 1 -- 172.21.15.73:0/3405135841 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa51c0b00d0 con 0x7fa51c0726e0 2024-04-26T20:52:09.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.142+0000 7fa5222c4640 1 -- 172.21.15.73:0/3405135841 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa51c0aab80 con 0x7fa51c0726e0 2024-04-26T20:52:09.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.143+0000 7fa5222c4640 1 -- 172.21.15.73:0/3405135841 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa51c068fc0 con 0x7fa51c0726e0 2024-04-26T20:52:09.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.144+0000 7fa518ff9640 1 -- 172.21.15.73:0/3405135841 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa50c03ca10 con 0x7fa51c0726e0 2024-04-26T20:52:09.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.144+0000 7fa518ff9640 1 --2- 172.21.15.73:0/3405135841 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa50806c8f0 0x7fa50806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:09.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.144+0000 7fa518ff9640 1 -- 172.21.15.73:0/3405135841 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6187+0+0 (secure 0 0 0) 0x7fa50c0ac7b0 con 0x7fa51c0726e0 2024-04-26T20:52:09.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.144+0000 7fa51b7fe640 1 --2- 172.21.15.73:0/3405135841 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa50806c8f0 0x7fa50806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:09.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.144+0000 7fa51b7fe640 1 --2- 172.21.15.73:0/3405135841 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa50806c8f0 0x7fa50806edb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa51c1f62e0 tx=0x7fa510009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:09.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.148+0000 7fa518ff9640 1 -- 172.21.15.73:0/3405135841 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa50c07aa90 con 0x7fa51c0726e0 2024-04-26T20:52:09.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.361+0000 7fa5222c4640 1 -- 172.21.15.73:0/3405135841 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa51c0694f0 con 0x7fa51c0726e0 2024-04-26T20:52:09.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.362+0000 7fa518ff9640 1 -- 172.21.15.73:0/3405135841 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5185 (secure 0 0 0) 0x7fa50c07a390 con 0x7fa51c0726e0 2024-04-26T20:52:09.363 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:52:09.363 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4802,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/272662913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":272662913},{"type":"v1","addr":"172.21.15.73:6836","nonce":272662913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4943,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2624899096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6833","nonce":2624899096},{"type":"v1","addr":"172.21.15.120:6835","nonce":2624899096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5047,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/1438858294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":1438858294},{"type":"v1","addr":"172.21.15.73:6837","nonce":1438858294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:51:45.273953+0000","modified":"2024-04-26T20:52:01.795359+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5050},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5050":{"gid":5050,"name":"b","rank":0,"incarnation":17,"state":"up:active","state_seq":9,"addr":"172.21.15.120:6834/3883540031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3883540031},{"type":"v1","addr":"172.21.15.120:6834","nonce":3883540031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5050,"qdb_cluster":[5050]},"id":2}]} 2024-04-26T20:52:09.363 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 20 2024-04-26T20:52:09.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.364+0000 7fa5222c4640 1 -- 172.21.15.73:0/3405135841 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa50806c8f0 msgr2=0x7fa50806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:09.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.364+0000 7fa5222c4640 1 --2- 172.21.15.73:0/3405135841 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa50806c8f0 0x7fa50806edb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa51c1f62e0 tx=0x7fa510009040 comp rx=0 tx=0).stop 2024-04-26T20:52:09.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.364+0000 7fa5222c4640 1 -- 172.21.15.73:0/3405135841 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa51c0726e0 msgr2=0x7fa51c0af9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:09.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.364+0000 7fa5222c4640 1 --2- 172.21.15.73:0/3405135841 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa51c0726e0 0x7fa51c0af9f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fa50c035040 tx=0x7fa50c0046c0 comp rx=0 tx=0).stop 2024-04-26T20:52:09.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.364+0000 7fa5222c4640 1 -- 172.21.15.73:0/3405135841 shutdown_connections 2024-04-26T20:52:09.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.364+0000 7fa5222c4640 1 --2- 172.21.15.73:0/3405135841 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa50806c8f0 0x7fa50806edb0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:09.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.364+0000 7fa5222c4640 1 --2- 172.21.15.73:0/3405135841 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa51c161510 0x7fa51c146e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:09.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.364+0000 7fa5222c4640 1 --2- 172.21.15.73:0/3405135841 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa51c15a9f0 0x7fa51c0ad4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:09.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.364+0000 7fa5222c4640 1 --2- 172.21.15.73:0/3405135841 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa51c0726e0 0x7fa51c0af9f0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:09.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.364+0000 7fa5222c4640 1 -- 172.21.15.73:0/3405135841 >> 172.21.15.73:0/3405135841 conn(0x7fa51c0b3180 msgr2=0x7fa51c0b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:09.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.364+0000 7fa5222c4640 1 -- 172.21.15.73:0/3405135841 shutdown_connections 2024-04-26T20:52:09.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.364+0000 7fa5222c4640 1 -- 172.21.15.73:0/3405135841 wait complete. 2024-04-26T20:52:09.376 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:52:09.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.473+0000 7f8691cfc640 1 Processor -- start 2024-04-26T20:52:09.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.474+0000 7f8691cfc640 1 -- start start 2024-04-26T20:52:09.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.474+0000 7f8691cfc640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f868c0cffa0 0x7f868c0d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:09.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.474+0000 7f8691cfc640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f868c0d0a90 0x7f868c0b2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:09.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.474+0000 7f8691cfc640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f868c0d0f00 con 0x7f868c1608f0 2024-04-26T20:52:09.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.474+0000 7f8691cfc640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f868c0cea30 con 0x7f868c0cffa0 2024-04-26T20:52:09.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.474+0000 7f8691cfc640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f868c0cde20 con 0x7f868c0d0a90 2024-04-26T20:52:09.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.475+0000 7f868affd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f868c0cffa0 0x7f868c0d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:09.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.475+0000 7f868affd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f868c0cffa0 0x7f868c0d1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.73:35430/0 (socket says 172.21.15.73:35430) 2024-04-26T20:52:09.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.475+0000 7f868affd640 1 -- 172.21.15.73:0/2756257349 learned_addr learned my addr 172.21.15.73:0/2756257349 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:09.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.475+0000 7f868b7fe640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f868c1608f0 0x7f868c0cf890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.73:55156/0 (socket says 172.21.15.73:55156) 2024-04-26T20:52:09.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.475+0000 7f868bfff640 1 --2- 172.21.15.73:0/2756257349 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f868c0d0a90 0x7f868c0b2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:09.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.475+0000 7f868affd640 1 -- 172.21.15.73:0/2756257349 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f868c0d0a90 msgr2=0x7f868c0b2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:09.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.475+0000 7f868affd640 1 --2- 172.21.15.73:0/2756257349 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f868c0d0a90 0x7f868c0b2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:09.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.475+0000 7f868affd640 1 -- 172.21.15.73:0/2756257349 >> v1:172.21.15.120:6789/0 conn(0x7f868c1608f0 legacy=0x7f868c0cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:09.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.475+0000 7f868affd640 1 -- 172.21.15.73:0/2756257349 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f868c0ce100 con 0x7f868c0cffa0 2024-04-26T20:52:09.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.475+0000 7f868a7fc640 1 -- 172.21.15.73:0/2756257349 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1363693229 0 0) 0x7f868c0d0f00 con 0x7f868c1608f0 2024-04-26T20:52:09.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.475+0000 7f868bfff640 1 --2- 172.21.15.73:0/2756257349 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f868c0d0a90 0x7f868c0b2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:52:09.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.476+0000 7f868affd640 1 --2- 172.21.15.73:0/2756257349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f868c0cffa0 0x7f868c0d1bf0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8680009870 tx=0x7f868002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c11ca5d97b3ac27 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:09.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.476+0000 7f868a7fc640 1 -- 172.21.15.73:0/2756257349 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8680037070 con 0x7f868c0cffa0 2024-04-26T20:52:09.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.476+0000 7f868a7fc640 1 -- 172.21.15.73:0/2756257349 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8680032440 con 0x7f868c0cffa0 2024-04-26T20:52:09.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.476+0000 7f868a7fc640 1 -- 172.21.15.73:0/2756257349 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8680032810 con 0x7f868c0cffa0 2024-04-26T20:52:09.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.476+0000 7f8691cfc640 1 -- 172.21.15.73:0/2756257349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f868c0cffa0 msgr2=0x7f868c0d1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:09.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.476+0000 7f8691cfc640 1 --2- 172.21.15.73:0/2756257349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f868c0cffa0 0x7f868c0d1bf0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8680009870 tx=0x7f868002a270 comp rx=0 tx=0).stop 2024-04-26T20:52:09.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.477+0000 7f8691cfc640 1 -- 172.21.15.73:0/2756257349 shutdown_connections 2024-04-26T20:52:09.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.477+0000 7f8691cfc640 1 --2- 172.21.15.73:0/2756257349 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f868c0d0a90 0x7f868c0b2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:09.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.477+0000 7f8691cfc640 1 --2- 172.21.15.73:0/2756257349 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f868c0cffa0 0x7f868c0d1bf0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:09.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.477+0000 7f8691cfc640 1 -- 172.21.15.73:0/2756257349 >> 172.21.15.73:0/2756257349 conn(0x7f868c0d2ec0 msgr2=0x7f868c0d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:09.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.477+0000 7f8691cfc640 1 -- 172.21.15.73:0/2756257349 shutdown_connections 2024-04-26T20:52:09.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.477+0000 7f8691cfc640 1 -- 172.21.15.73:0/2756257349 wait complete. 2024-04-26T20:52:09.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.478+0000 7f8691cfc640 1 Processor -- start 2024-04-26T20:52:09.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.478+0000 7f8691cfc640 1 -- start start 2024-04-26T20:52:09.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.479+0000 7f8691cfc640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f868c0cffa0 0x7f868c1503a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:09.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.479+0000 7f8691cfc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f868c0d0a90 0x7f868c0cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:09.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.479+0000 7f8691cfc640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f868c1608f0 0x7f868c0c6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:09.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.479+0000 7f8691cfc640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f868c0ceec0 con 0x7f868c0d0a90 2024-04-26T20:52:09.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.479+0000 7f8691cfc640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f868c0ce6c0 con 0x7f868c1608f0 2024-04-26T20:52:09.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.479+0000 7f8691cfc640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f868c0cda40 con 0x7f868c0cffa0 2024-04-26T20:52:09.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.479+0000 7f868affd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f868c0d0a90 0x7f868c0cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:09.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.479+0000 7f868affd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f868c0d0a90 0x7f868c0cafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43400/0 (socket says 172.21.15.73:43400) 2024-04-26T20:52:09.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.479+0000 7f868affd640 1 -- 172.21.15.73:0/4100605665 learned_addr learned my addr 172.21.15.73:0/4100605665 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:09.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.479+0000 7f868bfff640 1 --2- 172.21.15.73:0/4100605665 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f868c1608f0 0x7f868c0c6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:09.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.479+0000 7f868b7fe640 1 --2- 172.21.15.73:0/4100605665 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f868c0cffa0 0x7f868c1503a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:09.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.479+0000 7f868affd640 1 -- 172.21.15.73:0/4100605665 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f868c0cffa0 msgr2=0x7f868c1503a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:09.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.479+0000 7f868affd640 1 --2- 172.21.15.73:0/4100605665 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f868c0cffa0 0x7f868c1503a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:09.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.479+0000 7f868affd640 1 -- 172.21.15.73:0/4100605665 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f868c1608f0 msgr2=0x7f868c0c6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:09.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.479+0000 7f868affd640 1 --2- 172.21.15.73:0/4100605665 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f868c1608f0 0x7f868c0c6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:09.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.479+0000 7f868affd640 1 -- 172.21.15.73:0/4100605665 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f868c0cba30 con 0x7f868c0d0a90 2024-04-26T20:52:09.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.479+0000 7f868bfff640 1 --2- 172.21.15.73:0/4100605665 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f868c1608f0 0x7f868c0c6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:52:09.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.479+0000 7f868b7fe640 1 --2- 172.21.15.73:0/4100605665 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f868c0cffa0 0x7f868c1503a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:52:09.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.479+0000 7f868affd640 1 --2- 172.21.15.73:0/4100605665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f868c0d0a90 0x7f868c0cafe0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f8680035040 tx=0x7f86800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:09.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.479+0000 7f8673fff640 1 -- 172.21.15.73:0/4100605665 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8680037040 con 0x7f868c0d0a90 2024-04-26T20:52:09.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.479+0000 7f8673fff640 1 -- 172.21.15.73:0/4100605665 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8680004030 con 0x7f868c0d0a90 2024-04-26T20:52:09.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.479+0000 7f8691cfc640 1 -- 172.21.15.73:0/4100605665 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f868c0cbd10 con 0x7f868c0d0a90 2024-04-26T20:52:09.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.479+0000 7f8673fff640 1 -- 172.21.15.73:0/4100605665 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8680004420 con 0x7f868c0d0a90 2024-04-26T20:52:09.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.479+0000 7f8691cfc640 1 -- 172.21.15.73:0/4100605665 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f868c0cc2d0 con 0x7f868c0d0a90 2024-04-26T20:52:09.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.480+0000 7f8691cfc640 1 -- 172.21.15.73:0/4100605665 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f868c0694f0 con 0x7f868c0d0a90 2024-04-26T20:52:09.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.481+0000 7f8673fff640 1 -- 172.21.15.73:0/4100605665 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f868003a070 con 0x7f868c0d0a90 2024-04-26T20:52:09.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.481+0000 7f8673fff640 1 --2- 172.21.15.73:0/4100605665 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f867806c8f0 0x7f867806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:09.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.481+0000 7f8673fff640 1 -- 172.21.15.73:0/4100605665 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6187+0+0 (secure 0 0 0) 0x7f86800ac260 con 0x7f868c0d0a90 2024-04-26T20:52:09.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.481+0000 7f868b7fe640 1 --2- 172.21.15.73:0/4100605665 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f867806c8f0 0x7f867806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:09.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.482+0000 7f868b7fe640 1 --2- 172.21.15.73:0/4100605665 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f867806c8f0 0x7f867806edb0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f867c000f60 tx=0x7f867c007c10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:09.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.485+0000 7f8673fff640 1 -- 172.21.15.73:0/4100605665 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8680041020 con 0x7f868c0d0a90 2024-04-26T20:52:09.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.697+0000 7f8691cfc640 1 -- 172.21.15.73:0/4100605665 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f868c0c6600 con 0x7f868c0d0a90 2024-04-26T20:52:09.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.697+0000 7f8673fff640 1 -- 172.21.15.73:0/4100605665 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5185 (secure 0 0 0) 0x7f868007a5e0 con 0x7f868c0d0a90 2024-04-26T20:52:09.699 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:52:09.699 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4802,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/272662913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":272662913},{"type":"v1","addr":"172.21.15.73:6836","nonce":272662913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4943,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2624899096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6833","nonce":2624899096},{"type":"v1","addr":"172.21.15.120:6835","nonce":2624899096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5047,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/1438858294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":1438858294},{"type":"v1","addr":"172.21.15.73:6837","nonce":1438858294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:51:45.273953+0000","modified":"2024-04-26T20:52:01.795359+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5050},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5050":{"gid":5050,"name":"b","rank":0,"incarnation":17,"state":"up:active","state_seq":9,"addr":"172.21.15.120:6834/3883540031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3883540031},{"type":"v1","addr":"172.21.15.120:6834","nonce":3883540031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5050,"qdb_cluster":[5050]},"id":2}]} 2024-04-26T20:52:09.699 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 20 2024-04-26T20:52:09.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.699+0000 7f8691cfc640 1 -- 172.21.15.73:0/4100605665 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f867806c8f0 msgr2=0x7f867806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:09.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.699+0000 7f8691cfc640 1 --2- 172.21.15.73:0/4100605665 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f867806c8f0 0x7f867806edb0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f867c000f60 tx=0x7f867c007c10 comp rx=0 tx=0).stop 2024-04-26T20:52:09.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.699+0000 7f8691cfc640 1 -- 172.21.15.73:0/4100605665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f868c0d0a90 msgr2=0x7f868c0cafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:09.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.699+0000 7f8691cfc640 1 --2- 172.21.15.73:0/4100605665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f868c0d0a90 0x7f868c0cafe0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f8680035040 tx=0x7f86800046c0 comp rx=0 tx=0).stop 2024-04-26T20:52:09.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.699+0000 7f8691cfc640 1 -- 172.21.15.73:0/4100605665 shutdown_connections 2024-04-26T20:52:09.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.699+0000 7f8691cfc640 1 --2- 172.21.15.73:0/4100605665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f868c0d0a90 0x7f868c0cafe0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:09.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.699+0000 7f8691cfc640 1 --2- 172.21.15.73:0/4100605665 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f867806c8f0 0x7f867806edb0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:09.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.699+0000 7f8691cfc640 1 --2- 172.21.15.73:0/4100605665 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f868c1608f0 0x7f868c0c6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:09.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.699+0000 7f8691cfc640 1 --2- 172.21.15.73:0/4100605665 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f868c0cffa0 0x7f868c1503a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:09.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.699+0000 7f8691cfc640 1 -- 172.21.15.73:0/4100605665 >> 172.21.15.73:0/4100605665 conn(0x7f868c0d2ec0 msgr2=0x7f868c0b7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:09.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.700+0000 7f8691cfc640 1 -- 172.21.15.73:0/4100605665 shutdown_connections 2024-04-26T20:52:09.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.700+0000 7f8691cfc640 1 -- 172.21.15.73:0/4100605665 wait complete. 2024-04-26T20:52:09.711 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:52:09.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.807+0000 7f4f6f807640 1 Processor -- start 2024-04-26T20:52:09.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.807+0000 7f4f6f807640 1 -- start start 2024-04-26T20:52:09.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.808+0000 7f4f6f807640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4f680b4480 0x7f4f6806c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:09.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.808+0000 7f4f6f807640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f4f6806c780 0x7f4f680af030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:09.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.808+0000 7f4f6f807640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f680b49f0 con 0x7f4f6815f120 2024-04-26T20:52:09.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.808+0000 7f4f6f807640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f4f6815f500 con 0x7f4f6806c780 2024-04-26T20:52:09.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.808+0000 7f4f6f807640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f4f680ae8c0 con 0x7f4f680b4480 2024-04-26T20:52:09.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.808+0000 7f4f6dda4640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f4f6806c780 0x7f4f680af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:09.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.808+0000 7f4f6d5a3640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f4f6815f120 0x7f4f6806f110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.73:55168/0 (socket says 172.21.15.73:55168) 2024-04-26T20:52:09.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.808+0000 7f4f6dda4640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f4f6806c780 0x7f4f680af030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.73:35458/0 (socket says 172.21.15.73:35458) 2024-04-26T20:52:09.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.808+0000 7f4f6cda2640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4f680b4480 0x7f4f6806c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:09.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.808+0000 7f4f6d5a3640 1 -- 172.21.15.73:0/3151568039 learned_addr learned my addr 172.21.15.73:0/3151568039 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:52:09.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.809+0000 7f4f6cda2640 1 -- 172.21.15.73:0/3151568039 >> v2:172.21.15.120:3300/0 conn(0x7f4f6806c780 msgr2=0x7f4f680af030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:09.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.809+0000 7f4f6cda2640 1 --2- 172.21.15.73:0/3151568039 >> v2:172.21.15.120:3300/0 conn(0x7f4f6806c780 0x7f4f680af030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:09.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.809+0000 7f4f6cda2640 1 -- 172.21.15.73:0/3151568039 >> v1:172.21.15.120:6789/0 conn(0x7f4f6815f120 legacy=0x7f4f6806f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:09.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.809+0000 7f4f6cda2640 1 -- 172.21.15.73:0/3151568039 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f680b4cd0 con 0x7f4f680b4480 2024-04-26T20:52:09.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.809+0000 7f4f6dda4640 1 --2- 172.21.15.73:0/3151568039 >> v2:172.21.15.120:3300/0 conn(0x7f4f6806c780 0x7f4f680af030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:52:09.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.809+0000 7f4f6cda2640 1 --2- 172.21.15.73:0/3151568039 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4f680b4480 0x7f4f6806c240 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4f580097a0 tx=0x7f4f5802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d2847ceca6470de server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:09.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.809+0000 7f4f5ffff640 1 -- 172.21.15.73:0/3151568039 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f58037070 con 0x7f4f680b4480 2024-04-26T20:52:09.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.810+0000 7f4f5ffff640 1 -- 172.21.15.73:0/3151568039 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f5802ae50 con 0x7f4f680b4480 2024-04-26T20:52:09.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.810+0000 7f4f5ffff640 1 -- 172.21.15.73:0/3151568039 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f58032660 con 0x7f4f680b4480 2024-04-26T20:52:09.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.810+0000 7f4f6f807640 1 -- 172.21.15.73:0/3151568039 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4f680b4480 msgr2=0x7f4f6806c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:09.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.810+0000 7f4f6f807640 1 --2- 172.21.15.73:0/3151568039 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4f680b4480 0x7f4f6806c240 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4f580097a0 tx=0x7f4f5802a100 comp rx=0 tx=0).stop 2024-04-26T20:52:09.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.810+0000 7f4f6f807640 1 -- 172.21.15.73:0/3151568039 shutdown_connections 2024-04-26T20:52:09.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.810+0000 7f4f6f807640 1 --2- 172.21.15.73:0/3151568039 >> v2:172.21.15.120:3300/0 conn(0x7f4f6806c780 0x7f4f680af030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:09.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.810+0000 7f4f6f807640 1 --2- 172.21.15.73:0/3151568039 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4f680b4480 0x7f4f6806c240 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:09.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.810+0000 7f4f6f807640 1 -- 172.21.15.73:0/3151568039 >> 172.21.15.73:0/3151568039 conn(0x7f4f680b3180 msgr2=0x7f4f680b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:09.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.810+0000 7f4f6f807640 1 -- 172.21.15.73:0/3151568039 shutdown_connections 2024-04-26T20:52:09.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.810+0000 7f4f6f807640 1 -- 172.21.15.73:0/3151568039 wait complete. 2024-04-26T20:52:09.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.811+0000 7f4f6f807640 1 Processor -- start 2024-04-26T20:52:09.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.811+0000 7f4f6f807640 1 -- start start 2024-04-26T20:52:09.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.812+0000 7f4f6f807640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4f6806c780 0x7f4f680b2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:09.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.812+0000 7f4f6f807640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f680b4480 0x7f4f68151bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:09.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.812+0000 7f4f6f807640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4f6815f120 0x7f4f68152110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:09.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.812+0000 7f4f6f807640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4f680af570 con 0x7f4f6806c780 2024-04-26T20:52:09.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.812+0000 7f4f6f807640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4f680aeba0 con 0x7f4f680b4480 2024-04-26T20:52:09.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.812+0000 7f4f6f807640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f4f680af6f0 con 0x7f4f6815f120 2024-04-26T20:52:09.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.812+0000 7f4f6d5a3640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4f6806c780 0x7f4f680b2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:09.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.812+0000 7f4f6d5a3640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4f6806c780 0x7f4f680b2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43410/0 (socket says 172.21.15.73:43410) 2024-04-26T20:52:09.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.812+0000 7f4f6d5a3640 1 -- 172.21.15.73:0/989775595 learned_addr learned my addr 172.21.15.73:0/989775595 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:09.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.812+0000 7f4f6cda2640 1 --2- 172.21.15.73:0/989775595 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f680b4480 0x7f4f68151bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:09.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.812+0000 7f4f6dda4640 1 --2- 172.21.15.73:0/989775595 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4f6815f120 0x7f4f68152110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:09.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.812+0000 7f4f6d5a3640 1 -- 172.21.15.73:0/989775595 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4f6815f120 msgr2=0x7f4f68152110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:09.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.812+0000 7f4f6d5a3640 1 --2- 172.21.15.73:0/989775595 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4f6815f120 0x7f4f68152110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:09.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.812+0000 7f4f6d5a3640 1 -- 172.21.15.73:0/989775595 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f680b4480 msgr2=0x7f4f68151bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:09.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.812+0000 7f4f6d5a3640 1 --2- 172.21.15.73:0/989775595 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f680b4480 0x7f4f68151bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:09.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.812+0000 7f4f6d5a3640 1 -- 172.21.15.73:0/989775595 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f681f8520 con 0x7f4f6806c780 2024-04-26T20:52:09.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.813+0000 7f4f6d5a3640 1 --2- 172.21.15.73:0/989775595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4f6806c780 0x7f4f680b2710 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f4f6400ca70 tx=0x7f4f6400ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:09.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.813+0000 7f4f5dffb640 1 -- 172.21.15.73:0/989775595 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f64017070 con 0x7f4f6806c780 2024-04-26T20:52:09.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.813+0000 7f4f5dffb640 1 -- 172.21.15.73:0/989775595 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f6400ab10 con 0x7f4f6806c780 2024-04-26T20:52:09.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.813+0000 7f4f6f807640 1 -- 172.21.15.73:0/989775595 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f680d0ba0 con 0x7f4f6806c780 2024-04-26T20:52:09.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.813+0000 7f4f5dffb640 1 -- 172.21.15.73:0/989775595 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f64012400 con 0x7f4f6806c780 2024-04-26T20:52:09.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.813+0000 7f4f6f807640 1 -- 172.21.15.73:0/989775595 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f681f8aa0 con 0x7f4f6806c780 2024-04-26T20:52:09.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.814+0000 7f4f6f807640 1 -- 172.21.15.73:0/989775595 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f680694f0 con 0x7f4f6806c780 2024-04-26T20:52:09.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.814+0000 7f4f5dffb640 1 -- 172.21.15.73:0/989775595 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4f6401a070 con 0x7f4f6806c780 2024-04-26T20:52:09.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.815+0000 7f4f5dffb640 1 --2- 172.21.15.73:0/989775595 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f4f4c06c8f0 0x7f4f4c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:09.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.815+0000 7f4f5dffb640 1 -- 172.21.15.73:0/989775595 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6187+0+0 (secure 0 0 0) 0x7f4f6408b870 con 0x7f4f6806c780 2024-04-26T20:52:09.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.815+0000 7f4f6cda2640 1 --2- 172.21.15.73:0/989775595 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f4f4c06c8f0 0x7f4f4c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:09.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.815+0000 7f4f6cda2640 1 --2- 172.21.15.73:0/989775595 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f4f4c06c8f0 0x7f4f4c06edb0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4f58034000 tx=0x7f4f5802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:09.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:09.819+0000 7f4f5dffb640 1 -- 172.21.15.73:0/989775595 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4f64059500 con 0x7f4f6806c780 2024-04-26T20:52:10.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.000+0000 7f4f6f807640 1 -- 172.21.15.73:0/989775595 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4f68069b20 con 0x7f4f6806c780 2024-04-26T20:52:10.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.001+0000 7f4f5dffb640 1 -- 172.21.15.73:0/989775595 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+17071 (secure 0 0 0) 0x7f4f6405d800 con 0x7f4f6806c780 2024-04-26T20:52:10.002 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:52:10.002 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":40,"fsid":"fbd3f548-b547-4713-856b-09f98f5d1764","created":"2024-04-26T20:50:07.876786+0000","modified":"2024-04-26T20:51:58.783253+0000","last_up_change":"2024-04-26T20:50:19.082741+0000","last_in_change":"2024-04-26T20:50:10.018400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:50:19.993731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:51:43.110774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-26T20:51:44.235846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d6f4f3-214a-4392-a02f-c33cbcab3c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6809","nonce":3461705124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6811","nonce":3461705124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6815","nonce":3461705124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6813","nonce":3461705124}]},"public_addr":"172.21.15.73:6809/3461705124","cluster_addr":"172.21.15.73:6811/3461705124","heartbeat_back_addr":"172.21.15.73:6815/3461705124","heartbeat_front_addr":"172.21.15.73:6813/3461705124","state":["exists","up"]},{"osd":1,"uuid":"786bf8ca-6ddc-4b3c-be6c-9152398bd5fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6817","nonce":2827619403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6819","nonce":2827619403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6825","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6827","nonce":2827619403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6821","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6823","nonce":2827619403}]},"public_addr":"172.21.15.73:6817/2827619403","cluster_addr":"172.21.15.73:6819/2827619403","heartbeat_back_addr":"172.21.15.73:6827/2827619403","heartbeat_front_addr":"172.21.15.73:6823/2827619403","state":["exists","up"]},{"osd":2,"uuid":"90963b1a-5437-4ef8-ad8b-6b74c534eeb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6801","nonce":2542681064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6803","nonce":2542681064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6808","nonce":2542681064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6805","nonce":2542681064}]},"public_addr":"172.21.15.73:6801/2542681064","cluster_addr":"172.21.15.73:6803/2542681064","heartbeat_back_addr":"172.21.15.73:6808/2542681064","heartbeat_front_addr":"172.21.15.73:6805/2542681064","state":["exists","up"]},{"osd":3,"uuid":"bd3d2148-6e38-41b4-a453-fe6e3b8b4ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6822","nonce":1604042718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6826","nonce":1604042718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6831","nonce":1604042718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6829","nonce":1604042718}]},"public_addr":"172.21.15.73:6822/1604042718","cluster_addr":"172.21.15.73:6826/1604042718","heartbeat_back_addr":"172.21.15.73:6831/1604042718","heartbeat_front_addr":"172.21.15.73:6829/1604042718","state":["exists","up"]},{"osd":4,"uuid":"e8668dc8-a23f-46d2-8311-cb882da08e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6817","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6819","nonce":3591055561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6824","nonce":3591055561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6831","nonce":3591055561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6828","nonce":3591055561}]},"public_addr":"172.21.15.120:6819/3591055561","cluster_addr":"172.21.15.120:6824/3591055561","heartbeat_back_addr":"172.21.15.120:6831/3591055561","heartbeat_front_addr":"172.21.15.120:6828/3591055561","state":["exists","up"]},{"osd":5,"uuid":"cbac0bdb-96bd-4852-bcce-e564495c3fcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6809","nonce":819450433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6811","nonce":819450433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6815","nonce":819450433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6813","nonce":819450433}]},"public_addr":"172.21.15.120:6809/819450433","cluster_addr":"172.21.15.120:6811/819450433","heartbeat_back_addr":"172.21.15.120:6815/819450433","heartbeat_front_addr":"172.21.15.120:6813/819450433","state":["exists","up"]},{"osd":6,"uuid":"1884b2a8-569f-4272-96a9-f25a09364dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6801","nonce":3946210727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6803","nonce":3946210727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6807","nonce":3946210727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6805","nonce":3946210727}]},"public_addr":"172.21.15.120:6801/3946210727","cluster_addr":"172.21.15.120:6803/3946210727","heartbeat_back_addr":"172.21.15.120:6807/3946210727","heartbeat_front_addr":"172.21.15.120:6805/3946210727","state":["exists","up"]},{"osd":7,"uuid":"c0c3bf6d-a4c5-4f1e-aa2d-330951830666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6818","nonce":2915574674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6821","nonce":2915574674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6827","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6829","nonce":2915574674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6823","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6825","nonce":2915574674}]},"public_addr":"172.21.15.120:6818/2915574674","cluster_addr":"172.21.15.120:6821/2915574674","heartbeat_back_addr":"172.21.15.120:6829/2915574674","heartbeat_front_addr":"172.21.15.120:6825/2915574674","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.218216+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.208710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.112705+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.182789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.164324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.174212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.225396+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.261559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.120:6835/761435580":"2024-04-27T20:51:58.783228+0000","172.21.15.120:6833/761435580":"2024-04-27T20:51:58.783228+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:52:10.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.003+0000 7f4f6f807640 1 -- 172.21.15.73:0/989775595 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f4f4c06c8f0 msgr2=0x7f4f4c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:10.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.003+0000 7f4f6f807640 1 --2- 172.21.15.73:0/989775595 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f4f4c06c8f0 0x7f4f4c06edb0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4f58034000 tx=0x7f4f5802e000 comp rx=0 tx=0).stop 2024-04-26T20:52:10.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.003+0000 7f4f6f807640 1 -- 172.21.15.73:0/989775595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4f6806c780 msgr2=0x7f4f680b2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:10.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.003+0000 7f4f6f807640 1 --2- 172.21.15.73:0/989775595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4f6806c780 0x7f4f680b2710 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f4f6400ca70 tx=0x7f4f6400ce70 comp rx=0 tx=0).stop 2024-04-26T20:52:10.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.003+0000 7f4f6f807640 1 -- 172.21.15.73:0/989775595 shutdown_connections 2024-04-26T20:52:10.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.003+0000 7f4f6f807640 1 --2- 172.21.15.73:0/989775595 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f4f4c06c8f0 0x7f4f4c06edb0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:10.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.003+0000 7f4f6f807640 1 --2- 172.21.15.73:0/989775595 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4f6815f120 0x7f4f68152110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:10.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.003+0000 7f4f6f807640 1 --2- 172.21.15.73:0/989775595 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4f680b4480 0x7f4f68151bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:10.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.003+0000 7f4f6f807640 1 --2- 172.21.15.73:0/989775595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4f6806c780 0x7f4f680b2710 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:10.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.003+0000 7f4f6f807640 1 -- 172.21.15.73:0/989775595 >> 172.21.15.73:0/989775595 conn(0x7f4f680b3180 msgr2=0x7f4f68161690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:10.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.003+0000 7f4f6f807640 1 -- 172.21.15.73:0/989775595 shutdown_connections 2024-04-26T20:52:10.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.003+0000 7f4f6f807640 1 -- 172.21.15.73:0/989775595 wait complete. 2024-04-26T20:52:10.015 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T20:52:10.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.112+0000 7fc8b7919640 1 Processor -- start 2024-04-26T20:52:10.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.112+0000 7fc8b7919640 1 -- start start 2024-04-26T20:52:10.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.113+0000 7fc8b7919640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc8b015f290 0x7fc8b00cb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:10.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.113+0000 7fc8b7919640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fc8b00cd4a0 0x7fc8b00726c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:10.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.113+0000 7fc8b7919640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8b006f5a0 con 0x7fc8b00cd060 2024-04-26T20:52:10.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.113+0000 7fc8b7919640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fc8b006f350 con 0x7fc8b00cd4a0 2024-04-26T20:52:10.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.113+0000 7fc8b7919640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fc8b006f880 con 0x7fc8b015f290 2024-04-26T20:52:10.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.113+0000 7fc8b5eb6640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fc8b00cd4a0 0x7fc8b00726c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:10.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.113+0000 7fc8b56b5640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc8b015f290 0x7fc8b00cb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:10.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.113+0000 7fc8b4eb4640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7fc8b00cd060 0x7fc8b0162e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.73:55178/0 (socket says 172.21.15.73:55178) 2024-04-26T20:52:10.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.113+0000 7fc8b4eb4640 1 -- 172.21.15.73:0/368990724 learned_addr learned my addr 172.21.15.73:0/368990724 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:52:10.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.113+0000 7fc8b56b5640 1 -- 172.21.15.73:0/368990724 >> v2:172.21.15.120:3300/0 conn(0x7fc8b00cd4a0 msgr2=0x7fc8b00726c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:10.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.113+0000 7fc8b56b5640 1 --2- 172.21.15.73:0/368990724 >> v2:172.21.15.120:3300/0 conn(0x7fc8b00cd4a0 0x7fc8b00726c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:10.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.113+0000 7fc8b56b5640 1 -- 172.21.15.73:0/368990724 >> v1:172.21.15.120:6789/0 conn(0x7fc8b00cd060 legacy=0x7fc8b0162e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:10.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.113+0000 7fc8b56b5640 1 -- 172.21.15.73:0/368990724 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8b006fb60 con 0x7fc8b015f290 2024-04-26T20:52:10.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.113+0000 7fc8b5eb6640 1 --2- 172.21.15.73:0/368990724 >> v2:172.21.15.120:3300/0 conn(0x7fc8b00cd4a0 0x7fc8b00726c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:52:10.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.114+0000 7fc8b56b5640 1 --2- 172.21.15.73:0/368990724 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc8b015f290 0x7fc8b00cb7e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac0097a0 tx=0x7fc8ac02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=152944803bfbe601 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:10.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.114+0000 7fc8a7fff640 1 -- 172.21.15.73:0/368990724 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac037070 con 0x7fc8b015f290 2024-04-26T20:52:10.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.114+0000 7fc8a7fff640 1 -- 172.21.15.73:0/368990724 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8ac032440 con 0x7fc8b015f290 2024-04-26T20:52:10.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.114+0000 7fc8a7fff640 1 -- 172.21.15.73:0/368990724 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac032810 con 0x7fc8b015f290 2024-04-26T20:52:10.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.115+0000 7fc8b7919640 1 -- 172.21.15.73:0/368990724 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc8b015f290 msgr2=0x7fc8b00cb7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:10.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.115+0000 7fc8b7919640 1 --2- 172.21.15.73:0/368990724 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc8b015f290 0x7fc8b00cb7e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac0097a0 tx=0x7fc8ac02a210 comp rx=0 tx=0).stop 2024-04-26T20:52:10.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.115+0000 7fc8b7919640 1 -- 172.21.15.73:0/368990724 shutdown_connections 2024-04-26T20:52:10.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.115+0000 7fc8b7919640 1 --2- 172.21.15.73:0/368990724 >> v2:172.21.15.120:3300/0 conn(0x7fc8b00cd4a0 0x7fc8b00726c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:10.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.115+0000 7fc8b7919640 1 --2- 172.21.15.73:0/368990724 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc8b015f290 0x7fc8b00cb7e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:10.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.115+0000 7fc8b7919640 1 -- 172.21.15.73:0/368990724 >> 172.21.15.73:0/368990724 conn(0x7fc8b00d0fd0 msgr2=0x7fc8b00d13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:10.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.115+0000 7fc8b7919640 1 -- 172.21.15.73:0/368990724 shutdown_connections 2024-04-26T20:52:10.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.115+0000 7fc8b7919640 1 -- 172.21.15.73:0/368990724 wait complete. 2024-04-26T20:52:10.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.116+0000 7fc8b7919640 1 Processor -- start 2024-04-26T20:52:10.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.116+0000 7fc8b7919640 1 -- start start 2024-04-26T20:52:10.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.117+0000 7fc8b7919640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc8b00cd060 0x7fc8b0152510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:10.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.117+0000 7fc8b7919640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc8b00cd4a0 0x7fc8b00cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:10.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.117+0000 7fc8b7919640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8b015f290 0x7fc8b00cf2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:10.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.117+0000 7fc8b7919640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b0163710 con 0x7fc8b015f290 2024-04-26T20:52:10.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.117+0000 7fc8b7919640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b0163590 con 0x7fc8b00cd060 2024-04-26T20:52:10.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.117+0000 7fc8b7919640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fc8b0072c00 con 0x7fc8b00cd4a0 2024-04-26T20:52:10.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.117+0000 7fc8b5eb6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8b015f290 0x7fc8b00cf2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:10.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.117+0000 7fc8b5eb6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8b015f290 0x7fc8b00cf2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43414/0 (socket says 172.21.15.73:43414) 2024-04-26T20:52:10.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.117+0000 7fc8b5eb6640 1 -- 172.21.15.73:0/251195396 learned_addr learned my addr 172.21.15.73:0/251195396 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:10.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.117+0000 7fc8b4eb4640 1 --2- 172.21.15.73:0/251195396 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc8b00cd4a0 0x7fc8b00cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:10.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.117+0000 7fc8b56b5640 1 --2- 172.21.15.73:0/251195396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc8b00cd060 0x7fc8b0152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:10.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.117+0000 7fc8b5eb6640 1 -- 172.21.15.73:0/251195396 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc8b00cd4a0 msgr2=0x7fc8b00cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:10.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.117+0000 7fc8b5eb6640 1 --2- 172.21.15.73:0/251195396 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc8b00cd4a0 0x7fc8b00cedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:10.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.117+0000 7fc8b5eb6640 1 -- 172.21.15.73:0/251195396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc8b00cd060 msgr2=0x7fc8b0152510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:10.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.117+0000 7fc8b5eb6640 1 --2- 172.21.15.73:0/251195396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc8b00cd060 0x7fc8b0152510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:10.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.117+0000 7fc8b5eb6640 1 -- 172.21.15.73:0/251195396 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8b00b14f0 con 0x7fc8b015f290 2024-04-26T20:52:10.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.118+0000 7fc8b5eb6640 1 --2- 172.21.15.73:0/251195396 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8b015f290 0x7fc8b00cf2f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fc8a800efc0 tx=0x7fc8a8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:10.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.118+0000 7fc8a5ffb640 1 -- 172.21.15.73:0/251195396 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a8018030 con 0x7fc8b015f290 2024-04-26T20:52:10.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.118+0000 7fc8a5ffb640 1 -- 172.21.15.73:0/251195396 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8a800cde0 con 0x7fc8b015f290 2024-04-26T20:52:10.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.118+0000 7fc8b7919640 1 -- 172.21.15.73:0/251195396 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8b00b17d0 con 0x7fc8b015f290 2024-04-26T20:52:10.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.118+0000 7fc8a5ffb640 1 -- 172.21.15.73:0/251195396 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a8003bb0 con 0x7fc8b015f290 2024-04-26T20:52:10.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.118+0000 7fc8b7919640 1 -- 172.21.15.73:0/251195396 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8b009ce80 con 0x7fc8b015f290 2024-04-26T20:52:10.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.119+0000 7fc8b7919640 1 -- 172.21.15.73:0/251195396 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8b009d080 con 0x7fc8b015f290 2024-04-26T20:52:10.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.119+0000 7fc8a5ffb640 1 -- 172.21.15.73:0/251195396 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc8a801b070 con 0x7fc8b015f290 2024-04-26T20:52:10.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.119+0000 7fc8a5ffb640 1 --2- 172.21.15.73:0/251195396 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc89406c8f0 0x7fc89406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:10.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.120+0000 7fc8a5ffb640 1 -- 172.21.15.73:0/251195396 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6187+0+0 (secure 0 0 0) 0x7fc8a808c690 con 0x7fc8b015f290 2024-04-26T20:52:10.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.120+0000 7fc8b56b5640 1 --2- 172.21.15.73:0/251195396 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc89406c8f0 0x7fc89406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:10.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.120+0000 7fc8b56b5640 1 --2- 172.21.15.73:0/251195396 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc89406c8f0 0x7fc89406edb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac034000 tx=0x7fc8ac02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:10.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.124+0000 7fc8a5ffb640 1 -- 172.21.15.73:0/251195396 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc8b009d080 con 0x7fc8b015f290 2024-04-26T20:52:10.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.335+0000 7fc8b7919640 1 -- 172.21.15.73:0/251195396 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fc8b00694f0 con 0x7fc8b015f290 2024-04-26T20:52:10.336 INFO:tasks.ceph.mon.a.smithi073.stderr:2024-04-26T20:52:10.335+0000 7f712aaa2640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T20:52:10.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.337+0000 7fc8a5ffb640 1 -- 172.21.15.73:0/251195396 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v21) ==== 161+0+0 (secure 0 0 0) 0x7fc8a8010040 con 0x7fc8b015f290 2024-04-26T20:52:10.338 INFO:teuthology.orchestra.run.smithi073.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T20:52:10.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.340+0000 7fc8b7919640 1 -- 172.21.15.73:0/251195396 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc89406c8f0 msgr2=0x7fc89406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:10.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.340+0000 7fc8b7919640 1 --2- 172.21.15.73:0/251195396 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc89406c8f0 0x7fc89406edb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac034000 tx=0x7fc8ac02e000 comp rx=0 tx=0).stop 2024-04-26T20:52:10.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.340+0000 7fc8b7919640 1 -- 172.21.15.73:0/251195396 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8b015f290 msgr2=0x7fc8b00cf2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:10.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.340+0000 7fc8b7919640 1 --2- 172.21.15.73:0/251195396 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8b015f290 0x7fc8b00cf2f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fc8a800efc0 tx=0x7fc8a8009f90 comp rx=0 tx=0).stop 2024-04-26T20:52:10.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.340+0000 7fc8b7919640 1 -- 172.21.15.73:0/251195396 shutdown_connections 2024-04-26T20:52:10.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.340+0000 7fc8b7919640 1 --2- 172.21.15.73:0/251195396 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8b015f290 0x7fc8b00cf2f0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:10.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.340+0000 7fc8b7919640 1 --2- 172.21.15.73:0/251195396 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc89406c8f0 0x7fc89406edb0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:10.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.340+0000 7fc8b7919640 1 --2- 172.21.15.73:0/251195396 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc8b00cd4a0 0x7fc8b00cedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:10.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.340+0000 7fc8b7919640 1 --2- 172.21.15.73:0/251195396 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc8b00cd060 0x7fc8b0152510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:10.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.340+0000 7fc8b7919640 1 -- 172.21.15.73:0/251195396 >> 172.21.15.73:0/251195396 conn(0x7fc8b00d0fd0 msgr2=0x7fc8b00b47d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:10.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.340+0000 7fc8b7919640 1 -- 172.21.15.73:0/251195396 shutdown_connections 2024-04-26T20:52:10.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.340+0000 7fc8b7919640 1 -- 172.21.15.73:0/251195396 wait complete. 2024-04-26T20:52:10.343 INFO:tasks.ceph.mds.b.smithi120.stderr: -693> 2024-04-26T20:52:04.389+0000 7fbad0340640 -1 log_channel(cluster) log [ERR] : bad backtrace on directory inode 0x10000000000 2024-04-26T20:52:10.344 INFO:tasks.ceph.mds.b.smithi120.stderr: -233> 2024-04-26T20:52:05.311+0000 7fbad4348640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T20:52:10.352 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T20:52:10.370 INFO:tasks.ceph.mds.b.smithi120.stdout:starting mds.b at 2024-04-26T20:52:10.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.449+0000 7f51d6c25640 1 Processor -- start 2024-04-26T20:52:10.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.449+0000 7f51d6c25640 1 -- start start 2024-04-26T20:52:10.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.450+0000 7f51d6c25640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f51d006c950 0x7f51d00ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:10.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.450+0000 7f51d6c25640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51d00cfd80 con 0x7f51d00b10b0 2024-04-26T20:52:10.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.450+0000 7f51d6c25640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51d00d0020 con 0x7f51d00b0380 2024-04-26T20:52:10.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.450+0000 7f51d6c25640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f51d00af8e0 con 0x7f51d006c950 2024-04-26T20:52:10.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.450+0000 7f51cffff640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f51d00b10b0 0x7f51d015e970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40028/0 (socket says 172.21.15.73:40028) 2024-04-26T20:52:10.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.450+0000 7f51cffff640 1 -- 172.21.15.73:0/1306437086 learned_addr learned my addr 172.21.15.73:0/1306437086 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:52:10.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.450+0000 7f51d51c2640 1 --2- 172.21.15.73:0/1306437086 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f51d006c950 0x7f51d00ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:10.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.450+0000 7f51cf7fe640 1 -- 172.21.15.73:0/1306437086 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3457217802 0 0) 0x7f51d00cfd80 con 0x7f51d00b10b0 2024-04-26T20:52:10.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.450+0000 7f51cf7fe640 1 -- 172.21.15.73:0/1306437086 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f51c0003520 con 0x7f51d00b10b0 2024-04-26T20:52:10.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.450+0000 7f51cf7fe640 1 -- 172.21.15.73:0/1306437086 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1923810005 0 0) 0x7f51d00d0020 con 0x7f51d00b0380 2024-04-26T20:52:10.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.450+0000 7f51cf7fe640 1 -- 172.21.15.73:0/1306437086 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f51d00cfd80 con 0x7f51d00b0380 2024-04-26T20:52:10.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.450+0000 7f51cf7fe640 1 -- 172.21.15.73:0/1306437086 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3171777732 0 0) 0x7f51c0003520 con 0x7f51d00b10b0 2024-04-26T20:52:10.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.450+0000 7f51d51c2640 1 -- 172.21.15.73:0/1306437086 >> v1:172.21.15.120:6789/0 conn(0x7f51d00b0380 legacy=0x7f51d006c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:10.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.450+0000 7f51d51c2640 1 -- 172.21.15.73:0/1306437086 >> v1:172.21.15.73:6789/0 conn(0x7f51d00b10b0 legacy=0x7f51d015e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:10.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.450+0000 7f51d51c2640 1 -- 172.21.15.73:0/1306437086 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51d00d0300 con 0x7f51d006c950 2024-04-26T20:52:10.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.451+0000 7f51d51c2640 1 --2- 172.21.15.73:0/1306437086 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f51d006c950 0x7f51d00ccd50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f51c8009dc0 tx=0x7f51c802a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edf8f02c2cdfb6f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:10.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.451+0000 7f51cf7fe640 1 -- 172.21.15.73:0/1306437086 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51bc001690 con 0x7f51d006c950 2024-04-26T20:52:10.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.451+0000 7f51cf7fe640 1 -- 172.21.15.73:0/1306437086 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51c8007540 con 0x7f51d006c950 2024-04-26T20:52:10.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.451+0000 7f51cf7fe640 1 -- 172.21.15.73:0/1306437086 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c80078f0 con 0x7f51d006c950 2024-04-26T20:52:10.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.452+0000 7f51d6c25640 1 -- 172.21.15.73:0/1306437086 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f51d006c950 msgr2=0x7f51d00ccd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:10.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.452+0000 7f51d6c25640 1 --2- 172.21.15.73:0/1306437086 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f51d006c950 0x7f51d00ccd50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f51c8009dc0 tx=0x7f51c802a5d0 comp rx=0 tx=0).stop 2024-04-26T20:52:10.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.452+0000 7f51d6c25640 1 -- 172.21.15.73:0/1306437086 shutdown_connections 2024-04-26T20:52:10.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.452+0000 7f51d6c25640 1 --2- 172.21.15.73:0/1306437086 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f51d006c950 0x7f51d00ccd50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:10.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.452+0000 7f51d6c25640 1 -- 172.21.15.73:0/1306437086 >> 172.21.15.73:0/1306437086 conn(0x7f51d00d1cf0 msgr2=0x7f51d00d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:10.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.452+0000 7f51d6c25640 1 -- 172.21.15.73:0/1306437086 shutdown_connections 2024-04-26T20:52:10.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.452+0000 7f51d6c25640 1 -- 172.21.15.73:0/1306437086 wait complete. 2024-04-26T20:52:10.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.453+0000 7f51d6c25640 1 Processor -- start 2024-04-26T20:52:10.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.453+0000 7f51d6c25640 1 -- start start 2024-04-26T20:52:10.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.454+0000 7f51d6c25640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51d006c950 0x7f51d00b6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:10.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.454+0000 7f51d6c25640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f51d00b0380 0x7f51d015cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:10.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.454+0000 7f51d6c25640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f51d00b10b0 0x7f51d015d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:10.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.454+0000 7f51d6c25640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f51d00b0760 con 0x7f51d006c950 2024-04-26T20:52:10.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.454+0000 7f51d6c25640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f51d00d1980 con 0x7f51d00b10b0 2024-04-26T20:52:10.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.454+0000 7f51d6c25640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f51d00d15c0 con 0x7f51d00b0380 2024-04-26T20:52:10.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.454+0000 7f51d49c1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51d006c950 0x7f51d00b6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:10.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.454+0000 7f51d49c1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51d006c950 0x7f51d00b6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43418/0 (socket says 172.21.15.73:43418) 2024-04-26T20:52:10.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.454+0000 7f51d49c1640 1 -- 172.21.15.73:0/1109269207 learned_addr learned my addr 172.21.15.73:0/1109269207 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:10.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.454+0000 7f51cffff640 1 --2- 172.21.15.73:0/1109269207 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f51d00b0380 0x7f51d015cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:10.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.454+0000 7f51d51c2640 1 --2- 172.21.15.73:0/1109269207 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f51d00b10b0 0x7f51d015d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:10.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.454+0000 7f51d49c1640 1 -- 172.21.15.73:0/1109269207 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f51d00b0380 msgr2=0x7f51d015cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:10.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.454+0000 7f51d49c1640 1 --2- 172.21.15.73:0/1109269207 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f51d00b0380 0x7f51d015cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:10.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.454+0000 7f51d49c1640 1 -- 172.21.15.73:0/1109269207 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f51d00b10b0 msgr2=0x7f51d015d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:10.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.454+0000 7f51d49c1640 1 --2- 172.21.15.73:0/1109269207 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f51d00b10b0 0x7f51d015d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:10.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.454+0000 7f51d49c1640 1 -- 172.21.15.73:0/1109269207 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51d00b5ae0 con 0x7f51d006c950 2024-04-26T20:52:10.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.454+0000 7f51d51c2640 1 --2- 172.21.15.73:0/1109269207 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f51d00b10b0 0x7f51d015d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:52:10.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.454+0000 7f51d49c1640 1 --2- 172.21.15.73:0/1109269207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51d006c950 0x7f51d00b6b80 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f51c400c500 tx=0x7f51c400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:10.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.454+0000 7f51cd7fa640 1 -- 172.21.15.73:0/1109269207 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c40052d0 con 0x7f51d006c950 2024-04-26T20:52:10.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.454+0000 7f51cd7fa640 1 -- 172.21.15.73:0/1109269207 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51c4005490 con 0x7f51d006c950 2024-04-26T20:52:10.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.454+0000 7f51d6c25640 1 -- 172.21.15.73:0/1109269207 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51d00b5dc0 con 0x7f51d006c950 2024-04-26T20:52:10.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.454+0000 7f51cd7fa640 1 -- 172.21.15.73:0/1109269207 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c4012870 con 0x7f51d006c950 2024-04-26T20:52:10.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.454+0000 7f51d6c25640 1 -- 172.21.15.73:0/1109269207 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51d00b6380 con 0x7f51d006c950 2024-04-26T20:52:10.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.455+0000 7f51d6c25640 1 -- 172.21.15.73:0/1109269207 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51d00694f0 con 0x7f51d006c950 2024-04-26T20:52:10.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.456+0000 7f51cd7fa640 1 -- 172.21.15.73:0/1109269207 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f51c4012a10 con 0x7f51d006c950 2024-04-26T20:52:10.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.456+0000 7f51cd7fa640 1 --2- 172.21.15.73:0/1109269207 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f51b406c8f0 0x7f51b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:10.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.456+0000 7f51cd7fa640 1 -- 172.21.15.73:0/1109269207 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6273+0+0 (secure 0 0 0) 0x7f51c408b9c0 con 0x7f51d006c950 2024-04-26T20:52:10.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.456+0000 7f51cffff640 1 --2- 172.21.15.73:0/1109269207 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f51b406c8f0 0x7f51b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:10.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.457+0000 7f51cffff640 1 --2- 172.21.15.73:0/1109269207 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f51b406c8f0 0x7f51b406edb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f51bc001eb0 tx=0x7f51bc0014b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:10.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.460+0000 7f51cd7fa640 1 -- 172.21.15.73:0/1109269207 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f51c408de20 con 0x7f51d006c950 2024-04-26T20:52:10.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:10.675+0000 7f51d6c25640 1 -- 172.21.15.73:0/1109269207 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f51d00cf1a0 con 0x7f51d006c950 2024-04-26T20:52:11.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.338+0000 7f51cd7fa640 1 -- 172.21.15.73:0/1109269207 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v22) ==== 104+0+0 (secure 0 0 0) 0x7f51c4059bc0 con 0x7f51d006c950 2024-04-26T20:52:11.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.340+0000 7f51d6c25640 1 -- 172.21.15.73:0/1109269207 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f51b406c8f0 msgr2=0x7f51b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:11.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.340+0000 7f51d6c25640 1 --2- 172.21.15.73:0/1109269207 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f51b406c8f0 0x7f51b406edb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f51bc001eb0 tx=0x7f51bc0014b0 comp rx=0 tx=0).stop 2024-04-26T20:52:11.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.340+0000 7f51d6c25640 1 -- 172.21.15.73:0/1109269207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51d006c950 msgr2=0x7f51d00b6b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:11.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.340+0000 7f51d6c25640 1 --2- 172.21.15.73:0/1109269207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51d006c950 0x7f51d00b6b80 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f51c400c500 tx=0x7f51c400c9c0 comp rx=0 tx=0).stop 2024-04-26T20:52:11.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.341+0000 7f51d6c25640 1 -- 172.21.15.73:0/1109269207 shutdown_connections 2024-04-26T20:52:11.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.341+0000 7f51d6c25640 1 --2- 172.21.15.73:0/1109269207 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f51b406c8f0 0x7f51b406edb0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:11.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.341+0000 7f51d6c25640 1 --2- 172.21.15.73:0/1109269207 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f51d00b10b0 0x7f51d015d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:11.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.341+0000 7f51d6c25640 1 --2- 172.21.15.73:0/1109269207 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f51d00b0380 0x7f51d015cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:11.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.341+0000 7f51d6c25640 1 --2- 172.21.15.73:0/1109269207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51d006c950 0x7f51d00b6b80 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:11.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.341+0000 7f51d6c25640 1 -- 172.21.15.73:0/1109269207 >> 172.21.15.73:0/1109269207 conn(0x7f51d00d1cf0 msgr2=0x7f51d00d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:11.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.341+0000 7f51d6c25640 1 -- 172.21.15.73:0/1109269207 shutdown_connections 2024-04-26T20:52:11.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.341+0000 7f51d6c25640 1 -- 172.21.15.73:0/1109269207 wait complete. 2024-04-26T20:52:11.354 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T20:52:11.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.456+0000 7fe54f233640 1 Processor -- start 2024-04-26T20:52:11.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.456+0000 7fe54f233640 1 -- start start 2024-04-26T20:52:11.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.457+0000 7fe54f233640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe5481605d0 0x7fe5480b2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:11.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.457+0000 7fe54f233640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fe5480b5650 0x7fe5480d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:11.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.457+0000 7fe54f233640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe5480d1410 0x7fe5480d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:11.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.457+0000 7fe54f233640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7fe5480cd9c0 con 0x7fe5481605d0 2024-04-26T20:52:11.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.457+0000 7fe54f233640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fe5480d08a0 con 0x7fe5480b5650 2024-04-26T20:52:11.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.457+0000 7fe54f233640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fe5480cdc80 con 0x7fe5480d1410 2024-04-26T20:52:11.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.457+0000 7fe54cfcf640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe5481605d0 0x7fe5480b2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:11.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.457+0000 7fe54cfcf640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe5481605d0 0x7fe5480b2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43426/0 (socket says 172.21.15.73:43426) 2024-04-26T20:52:11.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.457+0000 7fe54cfcf640 1 -- 172.21.15.73:0/3765372590 learned_addr learned my addr 172.21.15.73:0/3765372590 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:11.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.457+0000 7fe54cfcf640 1 -- 172.21.15.73:0/3765372590 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe5480d1410 msgr2=0x7fe5480d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:11.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.457+0000 7fe54cfcf640 1 --2- 172.21.15.73:0/3765372590 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe5480d1410 0x7fe5480d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:11.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.457+0000 7fe54cfcf640 1 -- 172.21.15.73:0/3765372590 >> v2:172.21.15.120:3300/0 conn(0x7fe5480b5650 msgr2=0x7fe5480d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:11.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.457+0000 7fe54cfcf640 1 --2- 172.21.15.73:0/3765372590 >> v2:172.21.15.120:3300/0 conn(0x7fe5480b5650 0x7fe5480d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:11.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.457+0000 7fe54cfcf640 1 -- 172.21.15.73:0/3765372590 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5480d0000 con 0x7fe5481605d0 2024-04-26T20:52:11.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.458+0000 7fe54cfcf640 1 --2- 172.21.15.73:0/3765372590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5481605d0 0x7fe5480b2710 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fe548073160 tx=0x7fe53802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41e13af0cc456531 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:11.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.458+0000 7fe5477fe640 1 -- 172.21.15.73:0/3765372590 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe538037070 con 0x7fe5481605d0 2024-04-26T20:52:11.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.458+0000 7fe5477fe640 1 -- 172.21.15.73:0/3765372590 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe538032440 con 0x7fe5481605d0 2024-04-26T20:52:11.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.458+0000 7fe5477fe640 1 -- 172.21.15.73:0/3765372590 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe538032810 con 0x7fe5481605d0 2024-04-26T20:52:11.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.459+0000 7fe54f233640 1 -- 172.21.15.73:0/3765372590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5481605d0 msgr2=0x7fe5480b2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:11.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.459+0000 7fe54f233640 1 --2- 172.21.15.73:0/3765372590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5481605d0 0x7fe5480b2710 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fe548073160 tx=0x7fe53802a1e0 comp rx=0 tx=0).stop 2024-04-26T20:52:11.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.459+0000 7fe54f233640 1 -- 172.21.15.73:0/3765372590 shutdown_connections 2024-04-26T20:52:11.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.459+0000 7fe54f233640 1 --2- 172.21.15.73:0/3765372590 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe5480d1410 0x7fe5480d18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:11.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.459+0000 7fe54f233640 1 --2- 172.21.15.73:0/3765372590 >> v2:172.21.15.120:3300/0 conn(0x7fe5480b5650 0x7fe5480d0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:11.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.459+0000 7fe54f233640 1 --2- 172.21.15.73:0/3765372590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5481605d0 0x7fe5480b2710 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:11.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.459+0000 7fe54f233640 1 -- 172.21.15.73:0/3765372590 >> 172.21.15.73:0/3765372590 conn(0x7fe5480b49e0 msgr2=0x7fe5480b4df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:11.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.459+0000 7fe54f233640 1 -- 172.21.15.73:0/3765372590 shutdown_connections 2024-04-26T20:52:11.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.459+0000 7fe54f233640 1 -- 172.21.15.73:0/3765372590 wait complete. 2024-04-26T20:52:11.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.460+0000 7fe54f233640 1 Processor -- start 2024-04-26T20:52:11.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.460+0000 7fe54f233640 1 -- start start 2024-04-26T20:52:11.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.461+0000 7fe54f233640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5480b5650 0x7fe5481e89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:11.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.461+0000 7fe54f233640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5480d1410 0x7fe5481e8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:11.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.461+0000 7fe54f233640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe5481605d0 0x7fe5480b6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:11.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.461+0000 7fe54f233640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe5481609b0 con 0x7fe5480d1410 2024-04-26T20:52:11.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.461+0000 7fe54f233640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe5480d05c0 con 0x7fe5480b5650 2024-04-26T20:52:11.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.461+0000 7fe54f233640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fe5480d0fe0 con 0x7fe5481605d0 2024-04-26T20:52:11.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.461+0000 7fe547fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5480d1410 0x7fe5481e8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:11.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.461+0000 7fe547fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5480d1410 0x7fe5481e8ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43430/0 (socket says 172.21.15.73:43430) 2024-04-26T20:52:11.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.461+0000 7fe547fff640 1 -- 172.21.15.73:0/3783993953 learned_addr learned my addr 172.21.15.73:0/3783993953 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:11.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.461+0000 7fe54cfcf640 1 --2- 172.21.15.73:0/3783993953 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5480b5650 0x7fe5481e89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:11.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.461+0000 7fe54d7d0640 1 --2- 172.21.15.73:0/3783993953 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe5481605d0 0x7fe5480b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:11.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.461+0000 7fe547fff640 1 -- 172.21.15.73:0/3783993953 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe5481605d0 msgr2=0x7fe5480b6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:11.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.461+0000 7fe547fff640 1 --2- 172.21.15.73:0/3783993953 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe5481605d0 0x7fe5480b6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:11.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.461+0000 7fe547fff640 1 -- 172.21.15.73:0/3783993953 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5480b5650 msgr2=0x7fe5481e89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:11.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.461+0000 7fe547fff640 1 --2- 172.21.15.73:0/3783993953 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5480b5650 0x7fe5481e89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:11.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.461+0000 7fe547fff640 1 -- 172.21.15.73:0/3783993953 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5480b7950 con 0x7fe5480d1410 2024-04-26T20:52:11.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.461+0000 7fe54cfcf640 1 --2- 172.21.15.73:0/3783993953 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5480b5650 0x7fe5481e89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:52:11.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.461+0000 7fe547fff640 1 --2- 172.21.15.73:0/3783993953 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5480d1410 0x7fe5481e8ef0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fe53c00b4a0 tx=0x7fe53c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:11.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.462+0000 7fe5457fa640 1 -- 172.21.15.73:0/3783993953 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53c014070 con 0x7fe5480d1410 2024-04-26T20:52:11.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.462+0000 7fe5457fa640 1 -- 172.21.15.73:0/3783993953 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe53c010610 con 0x7fe5480d1410 2024-04-26T20:52:11.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.462+0000 7fe54f233640 1 -- 172.21.15.73:0/3783993953 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5480b7bf0 con 0x7fe5480d1410 2024-04-26T20:52:11.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.462+0000 7fe5457fa640 1 -- 172.21.15.73:0/3783993953 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53c010a00 con 0x7fe5480d1410 2024-04-26T20:52:11.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.462+0000 7fe54f233640 1 -- 172.21.15.73:0/3783993953 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5480b81b0 con 0x7fe5480d1410 2024-04-26T20:52:11.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.463+0000 7fe54f233640 1 -- 172.21.15.73:0/3783993953 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5480b5ac0 con 0x7fe5480d1410 2024-04-26T20:52:11.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.463+0000 7fe5457fa640 1 -- 172.21.15.73:0/3783993953 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe53c017070 con 0x7fe5480d1410 2024-04-26T20:52:11.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.463+0000 7fe5457fa640 1 --2- 172.21.15.73:0/3783993953 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe53406c920 0x7fe53406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:11.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.464+0000 7fe5457fa640 1 -- 172.21.15.73:0/3783993953 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6273+0+0 (secure 0 0 0) 0x7fe53c08b7e0 con 0x7fe5480d1410 2024-04-26T20:52:11.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.464+0000 7fe54cfcf640 1 --2- 172.21.15.73:0/3783993953 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe53406c920 0x7fe53406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:11.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.464+0000 7fe54cfcf640 1 --2- 172.21.15.73:0/3783993953 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe53406c920 0x7fe53406ede0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe538034000 tx=0x7fe53802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:11.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.468+0000 7fe5457fa640 1 -- 172.21.15.73:0/3783993953 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe5480b5ac0 con 0x7fe5480d1410 2024-04-26T20:52:11.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:11.641+0000 7fe54f233640 1 -- 172.21.15.73:0/3783993953 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe5480d2670 con 0x7fe5480d1410 2024-04-26T20:52:12.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.339+0000 7fe5457fa640 1 -- 172.21.15.73:0/3783993953 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v42) ==== 181+0+0 (secure 0 0 0) 0x7fe53c0599d0 con 0x7fe5480d1410 2024-04-26T20:52:12.341 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_metadata' removed 2024-04-26T20:52:12.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.342+0000 7fe54f233640 1 -- 172.21.15.73:0/3783993953 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe53406c920 msgr2=0x7fe53406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:12.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.342+0000 7fe54f233640 1 --2- 172.21.15.73:0/3783993953 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe53406c920 0x7fe53406ede0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe538034000 tx=0x7fe53802e000 comp rx=0 tx=0).stop 2024-04-26T20:52:12.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.342+0000 7fe54f233640 1 -- 172.21.15.73:0/3783993953 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5480d1410 msgr2=0x7fe5481e8ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:12.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.342+0000 7fe54f233640 1 --2- 172.21.15.73:0/3783993953 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5480d1410 0x7fe5481e8ef0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fe53c00b4a0 tx=0x7fe53c00b960 comp rx=0 tx=0).stop 2024-04-26T20:52:12.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.342+0000 7fe54f233640 1 -- 172.21.15.73:0/3783993953 shutdown_connections 2024-04-26T20:52:12.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.342+0000 7fe54f233640 1 --2- 172.21.15.73:0/3783993953 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5480d1410 0x7fe5481e8ef0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:12.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.342+0000 7fe54f233640 1 --2- 172.21.15.73:0/3783993953 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe53406c920 0x7fe53406ede0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:12.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.342+0000 7fe54f233640 1 --2- 172.21.15.73:0/3783993953 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe5481605d0 0x7fe5480b6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:12.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.342+0000 7fe54f233640 1 --2- 172.21.15.73:0/3783993953 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe5480b5650 0x7fe5481e89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:12.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.342+0000 7fe54f233640 1 -- 172.21.15.73:0/3783993953 >> 172.21.15.73:0/3783993953 conn(0x7fe5480b49e0 msgr2=0x7fe5480b2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:12.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.342+0000 7fe54f233640 1 -- 172.21.15.73:0/3783993953 shutdown_connections 2024-04-26T20:52:12.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.342+0000 7fe54f233640 1 -- 172.21.15.73:0/3783993953 wait complete. 2024-04-26T20:52:12.355 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T20:52:12.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.461+0000 7f32c6bc6640 1 Processor -- start 2024-04-26T20:52:12.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.461+0000 7f32c6bc6640 1 -- start start 2024-04-26T20:52:12.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.462+0000 7f32c6bc6640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f32c015d100 0x7f32c00d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:12.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.462+0000 7f32c6bc6640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f32c00b39e0 0x7f32c0161190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:12.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.462+0000 7f32c6bc6640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32c0161b30 con 0x7f32c0160d00 2024-04-26T20:52:12.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.462+0000 7f32c6bc6640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f32c00d31e0 con 0x7f32c015d100 2024-04-26T20:52:12.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.462+0000 7f32c6bc6640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f32c00d34c0 con 0x7f32c00b39e0 2024-04-26T20:52:12.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.462+0000 7f32c5163640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f32c00b39e0 0x7f32c0161190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:12.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.462+0000 7f32c4962640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f32c015d100 0x7f32c00d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:12.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.462+0000 7f32bffff640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f32c0160d00 0x7f32c015d4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.73:55186/0 (socket says 172.21.15.73:55186) 2024-04-26T20:52:12.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.462+0000 7f32bffff640 1 -- 172.21.15.73:0/4073925910 learned_addr learned my addr 172.21.15.73:0/4073925910 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:52:12.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.462+0000 7f32c4962640 1 -- 172.21.15.73:0/4073925910 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f32c00b39e0 msgr2=0x7f32c0161190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:12.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.462+0000 7f32c4962640 1 --2- 172.21.15.73:0/4073925910 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f32c00b39e0 0x7f32c0161190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:12.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.462+0000 7f32c4962640 1 -- 172.21.15.73:0/4073925910 >> v1:172.21.15.120:6789/0 conn(0x7f32c0160d00 legacy=0x7f32c015d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:12.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.462+0000 7f32c4962640 1 -- 172.21.15.73:0/4073925910 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32c00d37a0 con 0x7f32c015d100 2024-04-26T20:52:12.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.462+0000 7f32bf7fe640 1 -- 172.21.15.73:0/4073925910 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 734747588 0 0) 0x7f32c0161b30 con 0x7f32c0160d00 2024-04-26T20:52:12.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.462+0000 7f32c5163640 1 --2- 172.21.15.73:0/4073925910 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f32c00b39e0 0x7f32c0161190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:52:12.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.463+0000 7f32c4962640 1 --2- 172.21.15.73:0/4073925910 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32c015d100 0x7f32c00d2550 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f32b40097a0 tx=0x7f32b402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee78300def58d4cb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:12.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.463+0000 7f32bf7fe640 1 -- 172.21.15.73:0/4073925910 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b4037070 con 0x7f32c015d100 2024-04-26T20:52:12.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.463+0000 7f32bf7fe640 1 -- 172.21.15.73:0/4073925910 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32b402ae50 con 0x7f32c015d100 2024-04-26T20:52:12.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.463+0000 7f32bf7fe640 1 -- 172.21.15.73:0/4073925910 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b4032660 con 0x7f32c015d100 2024-04-26T20:52:12.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.464+0000 7f32c6bc6640 1 -- 172.21.15.73:0/4073925910 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32c015d100 msgr2=0x7f32c00d2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:12.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.464+0000 7f32c6bc6640 1 --2- 172.21.15.73:0/4073925910 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32c015d100 0x7f32c00d2550 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f32b40097a0 tx=0x7f32b402a100 comp rx=0 tx=0).stop 2024-04-26T20:52:12.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.464+0000 7f32c6bc6640 1 -- 172.21.15.73:0/4073925910 shutdown_connections 2024-04-26T20:52:12.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.464+0000 7f32c6bc6640 1 --2- 172.21.15.73:0/4073925910 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f32c00b39e0 0x7f32c0161190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:12.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.464+0000 7f32c6bc6640 1 --2- 172.21.15.73:0/4073925910 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32c015d100 0x7f32c00d2550 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:12.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.464+0000 7f32c6bc6640 1 -- 172.21.15.73:0/4073925910 >> 172.21.15.73:0/4073925910 conn(0x7f32c009ba50 msgr2=0x7f32c009be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:12.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.464+0000 7f32c6bc6640 1 -- 172.21.15.73:0/4073925910 shutdown_connections 2024-04-26T20:52:12.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.464+0000 7f32c6bc6640 1 -- 172.21.15.73:0/4073925910 wait complete. 2024-04-26T20:52:12.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.465+0000 7f32c6bc6640 1 Processor -- start 2024-04-26T20:52:12.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.465+0000 7f32c6bc6640 1 -- start start 2024-04-26T20:52:12.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.466+0000 7f32c6bc6640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f32c00b39e0 0x7f32c014f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:12.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.466+0000 7f32c6bc6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32c015d100 0x7f32c00cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:12.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.466+0000 7f32c6bc6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32c0160d00 0x7f32c006e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:12.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.466+0000 7f32c6bc6640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f32c00d2e00 con 0x7f32c015d100 2024-04-26T20:52:12.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.466+0000 7f32c6bc6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f32c00d2a90 con 0x7f32c0160d00 2024-04-26T20:52:12.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.466+0000 7f32c6bc6640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f32c00d2f80 con 0x7f32c00b39e0 2024-04-26T20:52:12.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.466+0000 7f32bffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32c015d100 0x7f32c00cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:12.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.466+0000 7f32bffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32c015d100 0x7f32c00cf8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43446/0 (socket says 172.21.15.73:43446) 2024-04-26T20:52:12.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.466+0000 7f32bffff640 1 -- 172.21.15.73:0/1625602330 learned_addr learned my addr 172.21.15.73:0/1625602330 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:12.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.466+0000 7f32c5163640 1 --2- 172.21.15.73:0/1625602330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32c0160d00 0x7f32c006e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:12.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.466+0000 7f32c4962640 1 --2- 172.21.15.73:0/1625602330 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f32c00b39e0 0x7f32c014f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:12.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.466+0000 7f32bffff640 1 -- 172.21.15.73:0/1625602330 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f32c00b39e0 msgr2=0x7f32c014f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:12.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.466+0000 7f32bffff640 1 --2- 172.21.15.73:0/1625602330 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f32c00b39e0 0x7f32c014f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:12.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.466+0000 7f32bffff640 1 -- 172.21.15.73:0/1625602330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32c0160d00 msgr2=0x7f32c006e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:12.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.466+0000 7f32bffff640 1 --2- 172.21.15.73:0/1625602330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32c0160d00 0x7f32c006e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:12.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.466+0000 7f32bffff640 1 -- 172.21.15.73:0/1625602330 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32c00cc0a0 con 0x7f32c015d100 2024-04-26T20:52:12.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.467+0000 7f32bffff640 1 --2- 172.21.15.73:0/1625602330 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32c015d100 0x7f32c00cf8a0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f32ac00c500 tx=0x7f32ac00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:12.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.467+0000 7f32bd7fa640 1 -- 172.21.15.73:0/1625602330 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ac018070 con 0x7f32c015d100 2024-04-26T20:52:12.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.467+0000 7f32bd7fa640 1 -- 172.21.15.73:0/1625602330 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32ac007500 con 0x7f32c015d100 2024-04-26T20:52:12.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.467+0000 7f32c6bc6640 1 -- 172.21.15.73:0/1625602330 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32c00cc380 con 0x7f32c015d100 2024-04-26T20:52:12.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.467+0000 7f32bd7fa640 1 -- 172.21.15.73:0/1625602330 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ac004a20 con 0x7f32c015d100 2024-04-26T20:52:12.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.467+0000 7f32c6bc6640 1 -- 172.21.15.73:0/1625602330 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32c00cc940 con 0x7f32c015d100 2024-04-26T20:52:12.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.468+0000 7f32c6bc6640 1 -- 172.21.15.73:0/1625602330 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32c0069600 con 0x7f32c015d100 2024-04-26T20:52:12.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.468+0000 7f32bd7fa640 1 -- 172.21.15.73:0/1625602330 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f32ac01b070 con 0x7f32c015d100 2024-04-26T20:52:12.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.469+0000 7f32bd7fa640 1 --2- 172.21.15.73:0/1625602330 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f32a406c8f0 0x7f32a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:12.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.469+0000 7f32bd7fa640 1 -- 172.21.15.73:0/1625602330 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5835+0+0 (secure 0 0 0) 0x7f32ac08ade0 con 0x7f32c015d100 2024-04-26T20:52:12.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.469+0000 7f32c4962640 1 --2- 172.21.15.73:0/1625602330 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f32a406c8f0 0x7f32a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:12.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.469+0000 7f32c4962640 1 --2- 172.21.15.73:0/1625602330 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f32a406c8f0 0x7f32a406edb0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f32b4034000 tx=0x7f32b402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:12.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.473+0000 7f32bd7fa640 1 -- 172.21.15.73:0/1625602330 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f32ac058b40 con 0x7f32c015d100 2024-04-26T20:52:12.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:12.646+0000 7f32c6bc6640 1 -- 172.21.15.73:0/1625602330 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f32c0072020 con 0x7f32c015d100 2024-04-26T20:52:13.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.347+0000 7f32bd7fa640 1 -- 172.21.15.73:0/1625602330 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v43) ==== 169+0+0 (secure 0 0 0) 0x7f32ac05ce40 con 0x7f32c015d100 2024-04-26T20:52:13.349 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data' removed 2024-04-26T20:52:13.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.350+0000 7f32c6bc6640 1 -- 172.21.15.73:0/1625602330 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f32a406c8f0 msgr2=0x7f32a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:13.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.350+0000 7f32c6bc6640 1 --2- 172.21.15.73:0/1625602330 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f32a406c8f0 0x7f32a406edb0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f32b4034000 tx=0x7f32b402e000 comp rx=0 tx=0).stop 2024-04-26T20:52:13.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.350+0000 7f32c6bc6640 1 -- 172.21.15.73:0/1625602330 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32c015d100 msgr2=0x7f32c00cf8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:13.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.350+0000 7f32c6bc6640 1 --2- 172.21.15.73:0/1625602330 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32c015d100 0x7f32c00cf8a0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f32ac00c500 tx=0x7f32ac00c900 comp rx=0 tx=0).stop 2024-04-26T20:52:13.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.351+0000 7f32c6bc6640 1 -- 172.21.15.73:0/1625602330 shutdown_connections 2024-04-26T20:52:13.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.351+0000 7f32c6bc6640 1 --2- 172.21.15.73:0/1625602330 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32c015d100 0x7f32c00cf8a0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:13.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.351+0000 7f32c6bc6640 1 --2- 172.21.15.73:0/1625602330 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f32a406c8f0 0x7f32a406edb0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:13.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.351+0000 7f32c6bc6640 1 --2- 172.21.15.73:0/1625602330 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f32c0160d00 0x7f32c006e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:13.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.351+0000 7f32c6bc6640 1 --2- 172.21.15.73:0/1625602330 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f32c00b39e0 0x7f32c014f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:13.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.351+0000 7f32c6bc6640 1 -- 172.21.15.73:0/1625602330 >> 172.21.15.73:0/1625602330 conn(0x7f32c009ba50 msgr2=0x7f32c015fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:13.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.351+0000 7f32c6bc6640 1 -- 172.21.15.73:0/1625602330 shutdown_connections 2024-04-26T20:52:13.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.351+0000 7f32c6bc6640 1 -- 172.21.15.73:0/1625602330 wait complete. 2024-04-26T20:52:13.366 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-26T20:52:13.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.466+0000 7f62e8ac8640 1 Processor -- start 2024-04-26T20:52:13.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.466+0000 7f62e8ac8640 1 -- start start 2024-04-26T20:52:13.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.467+0000 7f62e8ac8640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f62e41604e0 0x7f62e406f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:13.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.467+0000 7f62e8ac8640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62e4160970 con 0x7f62e406fcd0 2024-04-26T20:52:13.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.467+0000 7f62e8ac8640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62e4072f70 con 0x7f62e415f2d0 2024-04-26T20:52:13.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.467+0000 7f62e8ac8640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f62e4074010 con 0x7f62e41604e0 2024-04-26T20:52:13.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.468+0000 7f62e2d9d640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f62e406fcd0 0x7f62e40ce7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40044/0 (socket says 172.21.15.73:40044) 2024-04-26T20:52:13.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.468+0000 7f62e2d9d640 1 -- 172.21.15.73:0/4277997901 learned_addr learned my addr 172.21.15.73:0/4277997901 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:52:13.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.468+0000 7f62e1d9b640 1 --2- 172.21.15.73:0/4277997901 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f62e41604e0 0x7f62e406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:13.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.468+0000 7f62e159a640 1 -- 172.21.15.73:0/4277997901 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2566487407 0 0) 0x7f62e4160970 con 0x7f62e406fcd0 2024-04-26T20:52:13.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.468+0000 7f62e159a640 1 -- 172.21.15.73:0/4277997901 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f62d0003520 con 0x7f62e406fcd0 2024-04-26T20:52:13.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.468+0000 7f62e159a640 1 -- 172.21.15.73:0/4277997901 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2962940646 0 0) 0x7f62d0003520 con 0x7f62e406fcd0 2024-04-26T20:52:13.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.468+0000 7f62e159a640 1 -- 172.21.15.73:0/4277997901 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f62e41604e0 msgr2=0x7f62e406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:13.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.468+0000 7f62e159a640 1 --2- 172.21.15.73:0/4277997901 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f62e41604e0 0x7f62e406f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:13.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.468+0000 7f62e159a640 1 -- 172.21.15.73:0/4277997901 >> v1:172.21.15.120:6789/0 conn(0x7f62e415f2d0 legacy=0x7f62e4072700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:13.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.468+0000 7f62e159a640 1 -- 172.21.15.73:0/4277997901 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62e4073250 con 0x7f62e406fcd0 2024-04-26T20:52:13.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.468+0000 7f62e159a640 1 -- 172.21.15.73:0/4277997901 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7f62dc0019f0 con 0x7f62e406fcd0 2024-04-26T20:52:13.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.468+0000 7f62e159a640 1 -- 172.21.15.73:0/4277997901 >> v1:172.21.15.73:6789/0 conn(0x7f62e406fcd0 legacy=0x7f62e40ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:13.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.468+0000 7f62e159a640 1 --2- 172.21.15.73:0/4277997901 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62d00039e0 0x7f62d0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:13.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.468+0000 7f62e159a640 1 --2- 172.21.15.73:0/4277997901 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f62e41604e0 0x7f62d0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:13.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.468+0000 7f62e159a640 1 --2- 172.21.15.73:0/4277997901 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f62d0004d40 0x7f62d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:13.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.468+0000 7f62e159a640 1 -- 172.21.15.73:0/4277997901 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f62e4074010 con 0x7f62d00039e0 2024-04-26T20:52:13.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.468+0000 7f62e159a640 1 -- 172.21.15.73:0/4277997901 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f62d000baa0 con 0x7f62d0004d40 2024-04-26T20:52:13.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.468+0000 7f62e159a640 1 -- 172.21.15.73:0/4277997901 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f62d000bca0 con 0x7f62e41604e0 2024-04-26T20:52:13.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.468+0000 7f62e259c640 1 --2- 172.21.15.73:0/4277997901 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62d00039e0 0x7f62d0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:13.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.468+0000 7f62e1d9b640 1 --2- 172.21.15.73:0/4277997901 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f62e41604e0 0x7f62d0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:13.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.468+0000 7f62e2d9d640 1 --2- 172.21.15.73:0/4277997901 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f62d0004d40 0x7f62d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:13.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.468+0000 7f62e259c640 1 -- 172.21.15.73:0/4277997901 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f62e41604e0 msgr2=0x7f62d0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:13.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.469+0000 7f62e259c640 1 --2- 172.21.15.73:0/4277997901 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f62e41604e0 0x7f62d0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:13.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.469+0000 7f62e259c640 1 -- 172.21.15.73:0/4277997901 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f62d0004d40 msgr2=0x7f62d00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:13.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.469+0000 7f62e259c640 1 --2- 172.21.15.73:0/4277997901 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f62d0004d40 0x7f62d00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:13.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.469+0000 7f62e259c640 1 -- 172.21.15.73:0/4277997901 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62d0003520 con 0x7f62d00039e0 2024-04-26T20:52:13.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.469+0000 7f62e2d9d640 1 --2- 172.21.15.73:0/4277997901 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f62d0004d40 0x7f62d00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:52:13.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.469+0000 7f62e259c640 1 --2- 172.21.15.73:0/4277997901 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62d00039e0 0x7f62d0003e00 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f62d4001720 tx=0x7f62d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d3747895b97c285 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:13.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.469+0000 7f62e159a640 1 -- 172.21.15.73:0/4277997901 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d4001370 con 0x7f62d00039e0 2024-04-26T20:52:13.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.469+0000 7f62e159a640 1 -- 172.21.15.73:0/4277997901 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62d40052d0 con 0x7f62d00039e0 2024-04-26T20:52:13.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.469+0000 7f62e159a640 1 -- 172.21.15.73:0/4277997901 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d402abc0 con 0x7f62d00039e0 2024-04-26T20:52:13.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.470+0000 7f62e8ac8640 1 -- 172.21.15.73:0/4277997901 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62d00039e0 msgr2=0x7f62d0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:13.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.470+0000 7f62e8ac8640 1 --2- 172.21.15.73:0/4277997901 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62d00039e0 0x7f62d0003e00 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f62d4001720 tx=0x7f62d0005830 comp rx=0 tx=0).stop 2024-04-26T20:52:13.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.470+0000 7f62e2d9d640 1 -- 172.21.15.73:0/4277997901 reap_dead start 2024-04-26T20:52:13.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.470+0000 7f62e8ac8640 1 -- 172.21.15.73:0/4277997901 shutdown_connections 2024-04-26T20:52:13.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.470+0000 7f62e8ac8640 1 -- 172.21.15.73:0/4277997901 >> 172.21.15.73:0/4277997901 conn(0x7f62e40d0be0 msgr2=0x7f62e40d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:13.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.470+0000 7f62e8ac8640 1 -- 172.21.15.73:0/4277997901 shutdown_connections 2024-04-26T20:52:13.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.470+0000 7f62e8ac8640 1 -- 172.21.15.73:0/4277997901 wait complete. 2024-04-26T20:52:13.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.471+0000 7f62e8ac8640 1 Processor -- start 2024-04-26T20:52:13.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.471+0000 7f62e8ac8640 1 -- start start 2024-04-26T20:52:13.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.472+0000 7f62e8ac8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f62d00039e0 0x7f62e40ce730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:13.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.472+0000 7f62e8ac8640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62e41516f0 0x7f62e40cec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:13.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.472+0000 7f62e8ac8640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f62e4152cb0 0x7f62e40cf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:13.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.472+0000 7f62e8ac8640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f62e4073810 con 0x7f62e41516f0 2024-04-26T20:52:13.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.472+0000 7f62e8ac8640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f62e4070100 con 0x7f62d00039e0 2024-04-26T20:52:13.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.472+0000 7f62e8ac8640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f62e4073af0 con 0x7f62e4152cb0 2024-04-26T20:52:13.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.472+0000 7f62e1d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62e41516f0 0x7f62e40cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:13.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.472+0000 7f62e1d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62e41516f0 0x7f62e40cec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43460/0 (socket says 172.21.15.73:43460) 2024-04-26T20:52:13.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.472+0000 7f62e1d9b640 1 -- 172.21.15.73:0/2148989094 learned_addr learned my addr 172.21.15.73:0/2148989094 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:13.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.472+0000 7f62e2d9d640 1 --2- 172.21.15.73:0/2148989094 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f62e4152cb0 0x7f62e40cf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:13.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.472+0000 7f62e259c640 1 --2- 172.21.15.73:0/2148989094 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f62d00039e0 0x7f62e40ce730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:13.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.472+0000 7f62e1d9b640 1 -- 172.21.15.73:0/2148989094 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f62e4152cb0 msgr2=0x7f62e40cf1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:13.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.472+0000 7f62e1d9b640 1 --2- 172.21.15.73:0/2148989094 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f62e4152cb0 0x7f62e40cf1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:13.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.472+0000 7f62e1d9b640 1 -- 172.21.15.73:0/2148989094 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f62d00039e0 msgr2=0x7f62e40ce730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:13.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.472+0000 7f62e1d9b640 1 --2- 172.21.15.73:0/2148989094 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f62d00039e0 0x7f62e40ce730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:13.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.472+0000 7f62e1d9b640 1 -- 172.21.15.73:0/2148989094 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62e414ff70 con 0x7f62e41516f0 2024-04-26T20:52:13.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.473+0000 7f62e1d9b640 1 --2- 172.21.15.73:0/2148989094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62e41516f0 0x7f62e40cec70 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f62cc0119d0 tx=0x7f62cc011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:13.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.473+0000 7f62daffd640 1 -- 172.21.15.73:0/2148989094 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62cc00fb50 con 0x7f62e41516f0 2024-04-26T20:52:13.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.473+0000 7f62daffd640 1 -- 172.21.15.73:0/2148989094 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62cc00fcf0 con 0x7f62e41516f0 2024-04-26T20:52:13.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.473+0000 7f62e8ac8640 1 -- 172.21.15.73:0/2148989094 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62e4151c10 con 0x7f62e41516f0 2024-04-26T20:52:13.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.473+0000 7f62daffd640 1 -- 172.21.15.73:0/2148989094 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62cc00fea0 con 0x7f62e41516f0 2024-04-26T20:52:13.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.473+0000 7f62e8ac8640 1 -- 172.21.15.73:0/2148989094 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62e41521d0 con 0x7f62e41516f0 2024-04-26T20:52:13.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.474+0000 7f62e8ac8640 1 -- 172.21.15.73:0/2148989094 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62e40694f0 con 0x7f62e41516f0 2024-04-26T20:52:13.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.474+0000 7f62daffd640 1 -- 172.21.15.73:0/2148989094 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f62cc012040 con 0x7f62e41516f0 2024-04-26T20:52:13.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.475+0000 7f62daffd640 1 --2- 172.21.15.73:0/2148989094 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f62c406c920 0x7f62c406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:13.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.475+0000 7f62daffd640 1 -- 172.21.15.73:0/2148989094 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5421+0+0 (secure 0 0 0) 0x7f62cc08bed0 con 0x7f62e41516f0 2024-04-26T20:52:13.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.475+0000 7f62e259c640 1 --2- 172.21.15.73:0/2148989094 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f62c406c920 0x7f62c406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:13.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.475+0000 7f62e259c640 1 --2- 172.21.15.73:0/2148989094 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f62c406c920 0x7f62c406ede0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f62d4032000 tx=0x7f62d402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:13.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.479+0000 7f62daffd640 1 -- 172.21.15.73:0/2148989094 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f62e414ff70 con 0x7f62e41516f0 2024-04-26T20:52:13.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.652+0000 7f62e8ac8640 1 -- 172.21.15.73:0/2148989094 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f62e406de80 con 0x7f62e41516f0 2024-04-26T20:52:13.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.655+0000 7f62daffd640 1 -- 172.21.15.73:0/2148989094 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f62cc019030 con 0x7f62e41516f0 2024-04-26T20:52:13.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.657+0000 7f62e8ac8640 1 -- 172.21.15.73:0/2148989094 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f62c406c920 msgr2=0x7f62c406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:13.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.657+0000 7f62e8ac8640 1 --2- 172.21.15.73:0/2148989094 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f62c406c920 0x7f62c406ede0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f62d4032000 tx=0x7f62d402e000 comp rx=0 tx=0).stop 2024-04-26T20:52:13.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.657+0000 7f62e8ac8640 1 -- 172.21.15.73:0/2148989094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62e41516f0 msgr2=0x7f62e40cec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:13.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.657+0000 7f62e8ac8640 1 --2- 172.21.15.73:0/2148989094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62e41516f0 0x7f62e40cec70 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f62cc0119d0 tx=0x7f62cc011e90 comp rx=0 tx=0).stop 2024-04-26T20:52:13.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.657+0000 7f62e8ac8640 1 -- 172.21.15.73:0/2148989094 shutdown_connections 2024-04-26T20:52:13.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.657+0000 7f62e8ac8640 1 --2- 172.21.15.73:0/2148989094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62e41516f0 0x7f62e40cec70 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:13.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.657+0000 7f62e8ac8640 1 --2- 172.21.15.73:0/2148989094 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f62c406c920 0x7f62c406ede0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:13.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.657+0000 7f62e8ac8640 1 --2- 172.21.15.73:0/2148989094 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f62e4152cb0 0x7f62e40cf1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:13.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.657+0000 7f62e8ac8640 1 --2- 172.21.15.73:0/2148989094 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f62d00039e0 0x7f62e40ce730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:13.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.657+0000 7f62e8ac8640 1 -- 172.21.15.73:0/2148989094 >> 172.21.15.73:0/2148989094 conn(0x7f62e40d0be0 msgr2=0x7f62e40b3860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:13.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.657+0000 7f62e8ac8640 1 -- 172.21.15.73:0/2148989094 shutdown_connections 2024-04-26T20:52:13.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.657+0000 7f62e8ac8640 1 -- 172.21.15.73:0/2148989094 wait complete. 2024-04-26T20:52:13.669 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_auto_repair.TestMDSAutoRepair.test_backtrace_repair' 2024-04-26T20:52:13.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.767+0000 7fee3156a640 1 Processor -- start 2024-04-26T20:52:13.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.767+0000 7fee3156a640 1 -- start start 2024-04-26T20:52:13.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.768+0000 7fee3156a640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fee2c06c950 0x7fee2c15f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:13.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.768+0000 7fee3156a640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fee2c156340 0x7fee2c15fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:13.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.768+0000 7fee3156a640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee2c160530 con 0x7fee2c155f60 2024-04-26T20:52:13.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.768+0000 7fee3156a640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fee2c0d1c70 con 0x7fee2c06c950 2024-04-26T20:52:13.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.768+0000 7fee3156a640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fee2c0d1ec0 con 0x7fee2c156340 2024-04-26T20:52:13.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.768+0000 7fee2affd640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fee2c155f60 0x7fee2c06c240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40046/0 (socket says 172.21.15.73:40046) 2024-04-26T20:52:13.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.768+0000 7fee2affd640 1 -- 172.21.15.73:0/4217077618 learned_addr learned my addr 172.21.15.73:0/4217077618 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:52:13.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.768+0000 7fee2a7fc640 1 -- 172.21.15.73:0/4217077618 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1381343114 0 0) 0x7fee2c160530 con 0x7fee2c155f60 2024-04-26T20:52:13.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.769+0000 7fee2a7fc640 1 -- 172.21.15.73:0/4217077618 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee18003520 con 0x7fee2c155f60 2024-04-26T20:52:13.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.769+0000 7fee2b7fe640 1 --2- 172.21.15.73:0/4217077618 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fee2c156340 0x7fee2c15fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:13.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.769+0000 7fee227fc640 1 --2- 172.21.15.73:0/4217077618 >> v2:172.21.15.120:3300/0 conn(0x7fee2c06c950 0x7fee2c15f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:13.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.769+0000 7fee2a7fc640 1 -- 172.21.15.73:0/4217077618 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3571468089 0 0) 0x7fee18003520 con 0x7fee2c155f60 2024-04-26T20:52:13.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.769+0000 7fee2a7fc640 1 -- 172.21.15.73:0/4217077618 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fee2c156340 msgr2=0x7fee2c15fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:13.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.769+0000 7fee2a7fc640 1 --2- 172.21.15.73:0/4217077618 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fee2c156340 0x7fee2c15fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:13.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.769+0000 7fee2a7fc640 1 -- 172.21.15.73:0/4217077618 >> v2:172.21.15.120:3300/0 conn(0x7fee2c06c950 msgr2=0x7fee2c15f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:13.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.769+0000 7fee2a7fc640 1 --2- 172.21.15.73:0/4217077618 >> v2:172.21.15.120:3300/0 conn(0x7fee2c06c950 0x7fee2c15f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:13.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.769+0000 7fee2a7fc640 1 -- 172.21.15.73:0/4217077618 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee2c0d21a0 con 0x7fee2c155f60 2024-04-26T20:52:13.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.769+0000 7fee2a7fc640 1 -- 172.21.15.73:0/4217077618 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7fee1c001670 con 0x7fee2c155f60 2024-04-26T20:52:13.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.769+0000 7fee2a7fc640 1 -- 172.21.15.73:0/4217077618 >> v1:172.21.15.73:6789/0 conn(0x7fee2c155f60 legacy=0x7fee2c06c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:13.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.769+0000 7fee2a7fc640 1 --2- 172.21.15.73:0/4217077618 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fee2c156340 0x7fee180039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:13.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.769+0000 7fee2a7fc640 1 --2- 172.21.15.73:0/4217077618 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee18003fe0 0x7fee18004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:13.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.769+0000 7fee2a7fc640 1 --2- 172.21.15.73:0/4217077618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee18004d40 0x7fee180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:13.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.769+0000 7fee2a7fc640 1 -- 172.21.15.73:0/4217077618 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fee2c0d1c70 con 0x7fee18003fe0 2024-04-26T20:52:13.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.769+0000 7fee2a7fc640 1 -- 172.21.15.73:0/4217077618 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fee2c0d1ec0 con 0x7fee18004d40 2024-04-26T20:52:13.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.769+0000 7fee2a7fc640 1 -- 172.21.15.73:0/4217077618 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fee1800bac0 con 0x7fee2c156340 2024-04-26T20:52:13.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.769+0000 7fee227fc640 1 --2- 172.21.15.73:0/4217077618 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee18003fe0 0x7fee18004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:13.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.769+0000 7fee2affd640 1 --2- 172.21.15.73:0/4217077618 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fee2c156340 0x7fee180039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:13.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.769+0000 7fee2b7fe640 1 --2- 172.21.15.73:0/4217077618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee18004d40 0x7fee180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:13.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.769+0000 7fee227fc640 1 -- 172.21.15.73:0/4217077618 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fee2c156340 msgr2=0x7fee180039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:13.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.769+0000 7fee227fc640 1 --2- 172.21.15.73:0/4217077618 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fee2c156340 0x7fee180039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:13.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.769+0000 7fee227fc640 1 -- 172.21.15.73:0/4217077618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee18004d40 msgr2=0x7fee180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:13.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.769+0000 7fee227fc640 1 --2- 172.21.15.73:0/4217077618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee18004d40 0x7fee180051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:13.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.769+0000 7fee227fc640 1 -- 172.21.15.73:0/4217077618 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee18003520 con 0x7fee18003fe0 2024-04-26T20:52:13.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.769+0000 7fee2b7fe640 1 --2- 172.21.15.73:0/4217077618 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee18004d40 0x7fee180051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:52:13.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.770+0000 7fee227fc640 1 --2- 172.21.15.73:0/4217077618 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee18003fe0 0x7fee18004800 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fee1400ba30 tx=0x7fee18005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13ccd64c0adf2bee server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:13.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.770+0000 7fee2a7fc640 1 -- 172.21.15.73:0/4217077618 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee14030070 con 0x7fee18003fe0 2024-04-26T20:52:13.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.770+0000 7fee2a7fc640 1 -- 172.21.15.73:0/4217077618 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee1402ae10 con 0x7fee18003fe0 2024-04-26T20:52:13.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.770+0000 7fee2a7fc640 1 -- 172.21.15.73:0/4217077618 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1400b600 con 0x7fee18003fe0 2024-04-26T20:52:13.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.770+0000 7fee3156a640 1 -- 172.21.15.73:0/4217077618 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee18003fe0 msgr2=0x7fee18004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:13.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.770+0000 7fee3156a640 1 --2- 172.21.15.73:0/4217077618 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee18003fe0 0x7fee18004800 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fee1400ba30 tx=0x7fee18005830 comp rx=0 tx=0).stop 2024-04-26T20:52:13.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.771+0000 7fee2b7fe640 1 -- 172.21.15.73:0/4217077618 reap_dead start 2024-04-26T20:52:13.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.771+0000 7fee3156a640 1 -- 172.21.15.73:0/4217077618 shutdown_connections 2024-04-26T20:52:13.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.771+0000 7fee3156a640 1 -- 172.21.15.73:0/4217077618 >> 172.21.15.73:0/4217077618 conn(0x7fee2c068090 msgr2=0x7fee2c0684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:13.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.771+0000 7fee3156a640 1 -- 172.21.15.73:0/4217077618 shutdown_connections 2024-04-26T20:52:13.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.771+0000 7fee3156a640 1 -- 172.21.15.73:0/4217077618 wait complete. 2024-04-26T20:52:13.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.772+0000 7fee3156a640 1 Processor -- start 2024-04-26T20:52:13.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.772+0000 7fee3156a640 1 -- start start 2024-04-26T20:52:13.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.773+0000 7fee3156a640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fee18003fe0 0x7fee2c0b26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:13.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.773+0000 7fee3156a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee2c13e4a0 0x7fee2c06f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:13.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.773+0000 7fee3156a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee2c06f570 0x7fee2c0b3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:13.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.773+0000 7fee3156a640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fee2c0d2b20 con 0x7fee2c06f570 2024-04-26T20:52:13.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.773+0000 7fee3156a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fee2c0d29a0 con 0x7fee2c13e4a0 2024-04-26T20:52:13.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.773+0000 7fee3156a640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fee2c15fc80 con 0x7fee18003fe0 2024-04-26T20:52:13.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.773+0000 7fee2b7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee2c06f570 0x7fee2c0b3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:13.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.773+0000 7fee2b7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee2c06f570 0x7fee2c0b3360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43476/0 (socket says 172.21.15.73:43476) 2024-04-26T20:52:13.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.773+0000 7fee2b7fe640 1 -- 172.21.15.73:0/4231133917 learned_addr learned my addr 172.21.15.73:0/4231133917 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:13.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.773+0000 7fee2affd640 1 --2- 172.21.15.73:0/4231133917 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fee18003fe0 0x7fee2c0b26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:13.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.773+0000 7fee227fc640 1 --2- 172.21.15.73:0/4231133917 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee2c13e4a0 0x7fee2c06f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:13.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.773+0000 7fee2b7fe640 1 -- 172.21.15.73:0/4231133917 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fee18003fe0 msgr2=0x7fee2c0b26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:13.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.773+0000 7fee2b7fe640 1 --2- 172.21.15.73:0/4231133917 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fee18003fe0 0x7fee2c0b26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:13.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.773+0000 7fee2b7fe640 1 -- 172.21.15.73:0/4231133917 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee2c13e4a0 msgr2=0x7fee2c06f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:13.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.773+0000 7fee2b7fe640 1 --2- 172.21.15.73:0/4231133917 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee2c13e4a0 0x7fee2c06f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:13.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.773+0000 7fee2b7fe640 1 -- 172.21.15.73:0/4231133917 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee2c0b3fb0 con 0x7fee2c06f570 2024-04-26T20:52:13.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.773+0000 7fee2b7fe640 1 --2- 172.21.15.73:0/4231133917 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee2c06f570 0x7fee2c0b3360 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fee2c070e00 tx=0x7fee24004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:13.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.773+0000 7fee23fff640 1 -- 172.21.15.73:0/4231133917 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee24007c50 con 0x7fee2c06f570 2024-04-26T20:52:13.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.773+0000 7fee23fff640 1 -- 172.21.15.73:0/4231133917 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee24019070 con 0x7fee2c06f570 2024-04-26T20:52:13.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.773+0000 7fee3156a640 1 -- 172.21.15.73:0/4231133917 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee2c0b1c00 con 0x7fee2c06f570 2024-04-26T20:52:13.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.773+0000 7fee23fff640 1 -- 172.21.15.73:0/4231133917 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee24014650 con 0x7fee2c06f570 2024-04-26T20:52:13.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.773+0000 7fee3156a640 1 -- 172.21.15.73:0/4231133917 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee2c0b21c0 con 0x7fee2c06f570 2024-04-26T20:52:13.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.774+0000 7fee3156a640 1 -- 172.21.15.73:0/4231133917 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee2c0694f0 con 0x7fee2c06f570 2024-04-26T20:52:13.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.775+0000 7fee23fff640 1 -- 172.21.15.73:0/4231133917 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fee240147f0 con 0x7fee2c06f570 2024-04-26T20:52:13.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.775+0000 7fee23fff640 1 --2- 172.21.15.73:0/4231133917 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fee0c06c8f0 0x7fee0c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:13.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.775+0000 7fee23fff640 1 -- 172.21.15.73:0/4231133917 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5421+0+0 (secure 0 0 0) 0x7fee2408bed0 con 0x7fee2c06f570 2024-04-26T20:52:13.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.775+0000 7fee2affd640 1 --2- 172.21.15.73:0/4231133917 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fee0c06c8f0 0x7fee0c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:13.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.775+0000 7fee2affd640 1 --2- 172.21.15.73:0/4231133917 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fee0c06c8f0 0x7fee0c06edb0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fee1c007fd0 tx=0x7fee1c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:13.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:13.779+0000 7fee23fff640 1 -- 172.21.15.73:0/4231133917 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fee2405a420 con 0x7fee2c06f570 2024-04-26T20:52:14.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.027+0000 7fee3156a640 1 -- 172.21.15.73:0/4231133917 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_auto_repair.TestMDSAutoRepair.test_backtrace_repair"]} v 0) -- 0x7fee2c0cefc0 con 0x7fee2c06f570 2024-04-26T20:52:14.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.349+0000 7fee23fff640 1 -- 172.21.15.73:0/4231133917 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_auto_repair.TestMDSAutoRepair.test_backtrace_repair"]}]=0 v120) ==== 148+0+0 (secure 0 0 0) 0x7fee24059d20 con 0x7fee2c06f570 2024-04-26T20:52:14.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.351+0000 7fee3156a640 1 -- 172.21.15.73:0/4231133917 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fee0c06c8f0 msgr2=0x7fee0c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:14.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.351+0000 7fee3156a640 1 --2- 172.21.15.73:0/4231133917 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fee0c06c8f0 0x7fee0c06edb0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fee1c007fd0 tx=0x7fee1c009040 comp rx=0 tx=0).stop 2024-04-26T20:52:14.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.351+0000 7fee3156a640 1 -- 172.21.15.73:0/4231133917 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee2c06f570 msgr2=0x7fee2c0b3360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:14.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.351+0000 7fee3156a640 1 --2- 172.21.15.73:0/4231133917 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee2c06f570 0x7fee2c0b3360 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fee2c070e00 tx=0x7fee24004840 comp rx=0 tx=0).stop 2024-04-26T20:52:14.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.351+0000 7fee3156a640 1 -- 172.21.15.73:0/4231133917 shutdown_connections 2024-04-26T20:52:14.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.351+0000 7fee3156a640 1 --2- 172.21.15.73:0/4231133917 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee2c06f570 0x7fee2c0b3360 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:14.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.351+0000 7fee3156a640 1 --2- 172.21.15.73:0/4231133917 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fee0c06c8f0 0x7fee0c06edb0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:14.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.351+0000 7fee3156a640 1 --2- 172.21.15.73:0/4231133917 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fee2c13e4a0 0x7fee2c06f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:14.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.351+0000 7fee3156a640 1 --2- 172.21.15.73:0/4231133917 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fee18003fe0 0x7fee2c0b26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:14.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.351+0000 7fee3156a640 1 -- 172.21.15.73:0/4231133917 >> 172.21.15.73:0/4231133917 conn(0x7fee2c068090 msgr2=0x7fee2c162190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:14.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.351+0000 7fee3156a640 1 -- 172.21.15.73:0/4231133917 shutdown_connections 2024-04-26T20:52:14.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.351+0000 7fee3156a640 1 -- 172.21.15.73:0/4231133917 wait complete. 2024-04-26T20:52:14.364 INFO:tasks.cephfs_test_runner:MDS should verify/fix backtrace on fetch dirfrag ... ok 2024-04-26T20:52:14.364 INFO:tasks.cephfs_test_runner:Starting test: test_mds_readonly (tasks.cephfs.test_auto_repair.TestMDSAutoRepair) test if MDS behave correct when it's readonly 2024-04-26T20:52:14.364 INFO:tasks.cephfs_test_runner:test_mds_readonly (tasks.cephfs.test_auto_repair.TestMDSAutoRepair) 2024-04-26T20:52:14.364 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:52:14.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.461+0000 7f97034ea640 1 Processor -- start 2024-04-26T20:52:14.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.462+0000 7f97034ea640 1 -- start start 2024-04-26T20:52:14.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.462+0000 7f97034ea640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f96fc15f3a0 0x7f96fc0d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:14.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.462+0000 7f97034ea640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f96fc0d1770 0x7f96fc0d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:14.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.462+0000 7f97034ea640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96fc0d25a0 con 0x7f96fc0d2130 2024-04-26T20:52:14.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.462+0000 7f97034ea640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f96fc0d0e20 con 0x7f96fc0d1770 2024-04-26T20:52:14.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.462+0000 7f97034ea640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f96fc0b1d70 con 0x7f96fc15f3a0 2024-04-26T20:52:14.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.463+0000 7f9701286640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f96fc15f3a0 0x7f96fc0d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:14.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.463+0000 7f9700a85640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f96fc0d1770 0x7f96fc0d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:14.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.463+0000 7f9701286640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f96fc15f3a0 0x7f96fc0d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.73:50538/0 (socket says 172.21.15.73:50538) 2024-04-26T20:52:14.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.463+0000 7f9701a87640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f96fc0d2130 0x7f96fc0d2e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.73:55206/0 (socket says 172.21.15.73:55206) 2024-04-26T20:52:14.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.463+0000 7f9701286640 1 -- 172.21.15.73:0/102251393 learned_addr learned my addr 172.21.15.73:0/102251393 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:14.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.463+0000 7f9701286640 1 -- 172.21.15.73:0/102251393 >> v2:172.21.15.120:3300/0 conn(0x7f96fc0d1770 msgr2=0x7f96fc0d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:14.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.463+0000 7f9701286640 1 --2- 172.21.15.73:0/102251393 >> v2:172.21.15.120:3300/0 conn(0x7f96fc0d1770 0x7f96fc0d1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:14.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.463+0000 7f9701286640 1 -- 172.21.15.73:0/102251393 >> v1:172.21.15.120:6789/0 conn(0x7f96fc0d2130 legacy=0x7f96fc0d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:14.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.463+0000 7f9701286640 1 -- 172.21.15.73:0/102251393 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96fc0b2050 con 0x7f96fc15f3a0 2024-04-26T20:52:14.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.463+0000 7f9700a85640 1 --2- 172.21.15.73:0/102251393 >> v2:172.21.15.120:3300/0 conn(0x7f96fc0d1770 0x7f96fc0d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:52:14.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.464+0000 7f9701286640 1 --2- 172.21.15.73:0/102251393 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f96fc15f3a0 0x7f96fc0d06f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f96f4009870 tx=0x7f96f402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=797738741e5bb404 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:14.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.464+0000 7f96fbfff640 1 -- 172.21.15.73:0/102251393 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f4037070 con 0x7f96fc15f3a0 2024-04-26T20:52:14.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.464+0000 7f96fbfff640 1 -- 172.21.15.73:0/102251393 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96f4032440 con 0x7f96fc15f3a0 2024-04-26T20:52:14.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.464+0000 7f96fbfff640 1 -- 172.21.15.73:0/102251393 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f4032810 con 0x7f96fc15f3a0 2024-04-26T20:52:14.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.464+0000 7f97034ea640 1 -- 172.21.15.73:0/102251393 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f96fc15f3a0 msgr2=0x7f96fc0d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:14.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.464+0000 7f97034ea640 1 --2- 172.21.15.73:0/102251393 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f96fc15f3a0 0x7f96fc0d06f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f96f4009870 tx=0x7f96f402a270 comp rx=0 tx=0).stop 2024-04-26T20:52:14.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.464+0000 7f97034ea640 1 -- 172.21.15.73:0/102251393 shutdown_connections 2024-04-26T20:52:14.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.464+0000 7f97034ea640 1 --2- 172.21.15.73:0/102251393 >> v2:172.21.15.120:3300/0 conn(0x7f96fc0d1770 0x7f96fc0d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:14.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.464+0000 7f97034ea640 1 --2- 172.21.15.73:0/102251393 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f96fc15f3a0 0x7f96fc0d06f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:14.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.464+0000 7f97034ea640 1 -- 172.21.15.73:0/102251393 >> 172.21.15.73:0/102251393 conn(0x7f96fc0d4f90 msgr2=0x7f96fc0d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:14.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.465+0000 7f97034ea640 1 -- 172.21.15.73:0/102251393 shutdown_connections 2024-04-26T20:52:14.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.465+0000 7f97034ea640 1 -- 172.21.15.73:0/102251393 wait complete. 2024-04-26T20:52:14.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.466+0000 7f97034ea640 1 Processor -- start 2024-04-26T20:52:14.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.466+0000 7f97034ea640 1 -- start start 2024-04-26T20:52:14.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.466+0000 7f97034ea640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96fc0d1770 0x7f96fc0ce9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:14.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.466+0000 7f97034ea640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f96fc0d2130 0x7f96fc0cef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:14.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.466+0000 7f97034ea640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96fc15f3a0 0x7f96fc157b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:14.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.466+0000 7f97034ea640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f96fc15f900 con 0x7f96fc15f3a0 2024-04-26T20:52:14.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.466+0000 7f97034ea640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f96fc15f780 con 0x7f96fc0d1770 2024-04-26T20:52:14.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.466+0000 7f97034ea640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f96fc0b3860 con 0x7f96fc0d2130 2024-04-26T20:52:14.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.466+0000 7f9701a87640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96fc15f3a0 0x7f96fc157b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:14.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.466+0000 7f9701a87640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96fc15f3a0 0x7f96fc157b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43480/0 (socket says 172.21.15.73:43480) 2024-04-26T20:52:14.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.466+0000 7f9701a87640 1 -- 172.21.15.73:0/2044851256 learned_addr learned my addr 172.21.15.73:0/2044851256 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:14.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.467+0000 7f9700a85640 1 --2- 172.21.15.73:0/2044851256 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f96fc0d2130 0x7f96fc0cef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:14.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.467+0000 7f9701286640 1 --2- 172.21.15.73:0/2044851256 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96fc0d1770 0x7f96fc0ce9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:14.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.467+0000 7f9701a87640 1 -- 172.21.15.73:0/2044851256 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f96fc0d2130 msgr2=0x7f96fc0cef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:14.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.467+0000 7f9701a87640 1 --2- 172.21.15.73:0/2044851256 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f96fc0d2130 0x7f96fc0cef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:14.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.467+0000 7f9701a87640 1 -- 172.21.15.73:0/2044851256 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96fc0d1770 msgr2=0x7f96fc0ce9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:14.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.467+0000 7f9701a87640 1 --2- 172.21.15.73:0/2044851256 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96fc0d1770 0x7f96fc0ce9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:14.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.467+0000 7f9701a87640 1 -- 172.21.15.73:0/2044851256 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96fc152370 con 0x7f96fc15f3a0 2024-04-26T20:52:14.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.467+0000 7f9700a85640 1 --2- 172.21.15.73:0/2044851256 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f96fc0d2130 0x7f96fc0cef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:52:14.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.467+0000 7f9701286640 1 --2- 172.21.15.73:0/2044851256 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96fc0d1770 0x7f96fc0ce9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:52:14.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.467+0000 7f9701a87640 1 --2- 172.21.15.73:0/2044851256 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96fc15f3a0 0x7f96fc157b70 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f96ec00c740 tx=0x7f96ec00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:14.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.467+0000 7f96f9ffb640 1 -- 172.21.15.73:0/2044851256 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ec017070 con 0x7f96fc15f3a0 2024-04-26T20:52:14.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.467+0000 7f96f9ffb640 1 -- 172.21.15.73:0/2044851256 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96ec00ce50 con 0x7f96fc15f3a0 2024-04-26T20:52:14.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.467+0000 7f97034ea640 1 -- 172.21.15.73:0/2044851256 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96fc0b9b10 con 0x7f96fc15f3a0 2024-04-26T20:52:14.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.467+0000 7f96f9ffb640 1 -- 172.21.15.73:0/2044851256 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ec00a860 con 0x7f96fc15f3a0 2024-04-26T20:52:14.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.467+0000 7f97034ea640 1 -- 172.21.15.73:0/2044851256 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96fc152ac0 con 0x7f96fc15f3a0 2024-04-26T20:52:14.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.468+0000 7f97034ea640 1 -- 172.21.15.73:0/2044851256 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96fc0d1bf0 con 0x7f96fc15f3a0 2024-04-26T20:52:14.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.468+0000 7f96f9ffb640 1 -- 172.21.15.73:0/2044851256 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f96ec01a070 con 0x7f96fc15f3a0 2024-04-26T20:52:14.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.469+0000 7f96f9ffb640 1 --2- 172.21.15.73:0/2044851256 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f96e806c8f0 0x7f96e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:14.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.469+0000 7f96f9ffb640 1 -- 172.21.15.73:0/2044851256 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5421+0+0 (secure 0 0 0) 0x7f96ec08af80 con 0x7f96fc15f3a0 2024-04-26T20:52:14.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.469+0000 7f9701286640 1 --2- 172.21.15.73:0/2044851256 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f96e806c8f0 0x7f96e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:14.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.469+0000 7f9701286640 1 --2- 172.21.15.73:0/2044851256 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f96e806c8f0 0x7f96e806edb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f96f4034000 tx=0x7f96f402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:14.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.473+0000 7f96f9ffb640 1 -- 172.21.15.73:0/2044851256 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f96fc0d1bf0 con 0x7f96fc15f3a0 2024-04-26T20:52:14.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.649+0000 7f97034ea640 1 -- 172.21.15.73:0/2044851256 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f96fc0b4290 con 0x7f96fc15f3a0 2024-04-26T20:52:14.650 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.649+0000 7f96f9ffb640 1 -- 172.21.15.73:0/2044851256 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) ==== 74+0+13425 (secure 0 0 0) 0x7f96fc0b4290 con 0x7f96fc15f3a0 2024-04-26T20:52:14.650 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:52:14.651 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":43,"fsid":"fbd3f548-b547-4713-856b-09f98f5d1764","created":"2024-04-26T20:50:07.876786+0000","modified":"2024-04-26T20:52:13.341131+0000","last_up_change":"2024-04-26T20:50:19.082741+0000","last_in_change":"2024-04-26T20:50:10.018400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:50:19.993731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d6f4f3-214a-4392-a02f-c33cbcab3c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6809","nonce":3461705124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6811","nonce":3461705124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6815","nonce":3461705124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6813","nonce":3461705124}]},"public_addr":"172.21.15.73:6809/3461705124","cluster_addr":"172.21.15.73:6811/3461705124","heartbeat_back_addr":"172.21.15.73:6815/3461705124","heartbeat_front_addr":"172.21.15.73:6813/3461705124","state":["exists","up"]},{"osd":1,"uuid":"786bf8ca-6ddc-4b3c-be6c-9152398bd5fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6817","nonce":2827619403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6819","nonce":2827619403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6825","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6827","nonce":2827619403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6821","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6823","nonce":2827619403}]},"public_addr":"172.21.15.73:6817/2827619403","cluster_addr":"172.21.15.73:6819/2827619403","heartbeat_back_addr":"172.21.15.73:6827/2827619403","heartbeat_front_addr":"172.21.15.73:6823/2827619403","state":["exists","up"]},{"osd":2,"uuid":"90963b1a-5437-4ef8-ad8b-6b74c534eeb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6801","nonce":2542681064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6803","nonce":2542681064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6808","nonce":2542681064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6805","nonce":2542681064}]},"public_addr":"172.21.15.73:6801/2542681064","cluster_addr":"172.21.15.73:6803/2542681064","heartbeat_back_addr":"172.21.15.73:6808/2542681064","heartbeat_front_addr":"172.21.15.73:6805/2542681064","state":["exists","up"]},{"osd":3,"uuid":"bd3d2148-6e38-41b4-a453-fe6e3b8b4ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6822","nonce":1604042718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6826","nonce":1604042718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6831","nonce":1604042718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6829","nonce":1604042718}]},"public_addr":"172.21.15.73:6822/1604042718","cluster_addr":"172.21.15.73:6826/1604042718","heartbeat_back_addr":"172.21.15.73:6831/1604042718","heartbeat_front_addr":"172.21.15.73:6829/1604042718","state":["exists","up"]},{"osd":4,"uuid":"e8668dc8-a23f-46d2-8311-cb882da08e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6817","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6819","nonce":3591055561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6824","nonce":3591055561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6831","nonce":3591055561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6828","nonce":3591055561}]},"public_addr":"172.21.15.120:6819/3591055561","cluster_addr":"172.21.15.120:6824/3591055561","heartbeat_back_addr":"172.21.15.120:6831/3591055561","heartbeat_front_addr":"172.21.15.120:6828/3591055561","state":["exists","up"]},{"osd":5,"uuid":"cbac0bdb-96bd-4852-bcce-e564495c3fcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6809","nonce":819450433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6811","nonce":819450433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6815","nonce":819450433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6813","nonce":819450433}]},"public_addr":"172.21.15.120:6809/819450433","cluster_addr":"172.21.15.120:6811/819450433","heartbeat_back_addr":"172.21.15.120:6815/819450433","heartbeat_front_addr":"172.21.15.120:6813/819450433","state":["exists","up"]},{"osd":6,"uuid":"1884b2a8-569f-4272-96a9-f25a09364dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6801","nonce":3946210727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6803","nonce":3946210727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6807","nonce":3946210727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6805","nonce":3946210727}]},"public_addr":"172.21.15.120:6801/3946210727","cluster_addr":"172.21.15.120:6803/3946210727","heartbeat_back_addr":"172.21.15.120:6807/3946210727","heartbeat_front_addr":"172.21.15.120:6805/3946210727","state":["exists","up"]},{"osd":7,"uuid":"c0c3bf6d-a4c5-4f1e-aa2d-330951830666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6818","nonce":2915574674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6821","nonce":2915574674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6827","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6829","nonce":2915574674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6823","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6825","nonce":2915574674}]},"public_addr":"172.21.15.120:6818/2915574674","cluster_addr":"172.21.15.120:6821/2915574674","heartbeat_back_addr":"172.21.15.120:6829/2915574674","heartbeat_front_addr":"172.21.15.120:6825/2915574674","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.218216+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.208710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.112705+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.182789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.164324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.174212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.225396+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.261559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.120:6832/3883540031":"2024-04-27T20:52:10.336630+0000","172.21.15.120:6835/761435580":"2024-04-27T20:51:58.783228+0000","172.21.15.120:6834/3883540031":"2024-04-27T20:52:10.336630+0000","172.21.15.120:6833/761435580":"2024-04-27T20:51:58.783228+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:52:14.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.651+0000 7f97034ea640 1 -- 172.21.15.73:0/2044851256 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f96e806c8f0 msgr2=0x7f96e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:14.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.651+0000 7f97034ea640 1 --2- 172.21.15.73:0/2044851256 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f96e806c8f0 0x7f96e806edb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f96f4034000 tx=0x7f96f402e000 comp rx=0 tx=0).stop 2024-04-26T20:52:14.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.651+0000 7f97034ea640 1 -- 172.21.15.73:0/2044851256 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96fc15f3a0 msgr2=0x7f96fc157b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:14.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.651+0000 7f97034ea640 1 --2- 172.21.15.73:0/2044851256 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96fc15f3a0 0x7f96fc157b70 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f96ec00c740 tx=0x7f96ec00cc00 comp rx=0 tx=0).stop 2024-04-26T20:52:14.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.652+0000 7f97034ea640 1 -- 172.21.15.73:0/2044851256 shutdown_connections 2024-04-26T20:52:14.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.652+0000 7f97034ea640 1 --2- 172.21.15.73:0/2044851256 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96fc15f3a0 0x7f96fc157b70 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:14.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.652+0000 7f97034ea640 1 --2- 172.21.15.73:0/2044851256 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f96e806c8f0 0x7f96e806edb0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:14.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.652+0000 7f97034ea640 1 --2- 172.21.15.73:0/2044851256 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f96fc0d2130 0x7f96fc0cef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:14.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.652+0000 7f97034ea640 1 --2- 172.21.15.73:0/2044851256 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f96fc0d1770 0x7f96fc0ce9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:14.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.652+0000 7f97034ea640 1 -- 172.21.15.73:0/2044851256 >> 172.21.15.73:0/2044851256 conn(0x7f96fc0d4f90 msgr2=0x7f96fc0d2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:14.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.652+0000 7f97034ea640 1 -- 172.21.15.73:0/2044851256 shutdown_connections 2024-04-26T20:52:14.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.652+0000 7f97034ea640 1 -- 172.21.15.73:0/2044851256 wait complete. 2024-04-26T20:52:14.664 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:50:19.993731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:52:14.664 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:52:14.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.763+0000 7f7303f9d640 1 Processor -- start 2024-04-26T20:52:14.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.763+0000 7f7303f9d640 1 -- start start 2024-04-26T20:52:14.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.764+0000 7f7303f9d640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f72fc070350 0x7f72fc070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:14.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.764+0000 7f7303f9d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f72fc071090 0x7f72fc15e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:14.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.764+0000 7f7303f9d640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72fc0d2390 con 0x7f72fc0cfde0 2024-04-26T20:52:14.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.764+0000 7f7303f9d640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f72fc0afab0 con 0x7f72fc070350 2024-04-26T20:52:14.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.764+0000 7f7303f9d640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f72fc06dd20 con 0x7f72fc071090 2024-04-26T20:52:14.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.764+0000 7f7301538640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f72fc070350 0x7f72fc070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:14.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.764+0000 7f7301d39640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f72fc0cfde0 0x7f72fc0d1b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40062/0 (socket says 172.21.15.73:40062) 2024-04-26T20:52:14.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.764+0000 7f7301d39640 1 -- 172.21.15.73:0/2869638182 learned_addr learned my addr 172.21.15.73:0/2869638182 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:52:14.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.764+0000 7f7300d37640 1 -- 172.21.15.73:0/2869638182 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3585631593 0 0) 0x7f72fc0d2390 con 0x7f72fc0cfde0 2024-04-26T20:52:14.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.764+0000 7f7300d37640 1 -- 172.21.15.73:0/2869638182 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f72e8003520 con 0x7f72fc0cfde0 2024-04-26T20:52:14.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.764+0000 7f730253a640 1 --2- 172.21.15.73:0/2869638182 >> v2:172.21.15.120:3300/0 conn(0x7f72fc071090 0x7f72fc15e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:14.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.764+0000 7f7301538640 1 -- 172.21.15.73:0/2869638182 >> v2:172.21.15.120:3300/0 conn(0x7f72fc071090 msgr2=0x7f72fc15e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:14.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.764+0000 7f7301538640 1 --2- 172.21.15.73:0/2869638182 >> v2:172.21.15.120:3300/0 conn(0x7f72fc071090 0x7f72fc15e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:14.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.764+0000 7f7301538640 1 -- 172.21.15.73:0/2869638182 >> v1:172.21.15.73:6789/0 conn(0x7f72fc0cfde0 legacy=0x7f72fc0d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:14.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.764+0000 7f7301538640 1 -- 172.21.15.73:0/2869638182 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72fc06d3f0 con 0x7f72fc070350 2024-04-26T20:52:14.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.764+0000 7f730253a640 1 --2- 172.21.15.73:0/2869638182 >> v2:172.21.15.120:3300/0 conn(0x7f72fc071090 0x7f72fc15e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:52:14.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.765+0000 7f7301538640 1 --2- 172.21.15.73:0/2869638182 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72fc070350 0x7f72fc070b50 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f72fc0c6860 tx=0x7f72ec02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99703a91c86c0d23 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:14.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.765+0000 7f7300d37640 1 -- 172.21.15.73:0/2869638182 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ec037070 con 0x7f72fc070350 2024-04-26T20:52:14.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.765+0000 7f7300d37640 1 -- 172.21.15.73:0/2869638182 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72ec032440 con 0x7f72fc070350 2024-04-26T20:52:14.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.765+0000 7f7300d37640 1 -- 172.21.15.73:0/2869638182 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ec02ae60 con 0x7f72fc070350 2024-04-26T20:52:14.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.766+0000 7f7303f9d640 1 -- 172.21.15.73:0/2869638182 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72fc070350 msgr2=0x7f72fc070b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:14.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.766+0000 7f7303f9d640 1 --2- 172.21.15.73:0/2869638182 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72fc070350 0x7f72fc070b50 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f72fc0c6860 tx=0x7f72ec02a110 comp rx=0 tx=0).stop 2024-04-26T20:52:14.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.766+0000 7f7303f9d640 1 -- 172.21.15.73:0/2869638182 shutdown_connections 2024-04-26T20:52:14.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.766+0000 7f7303f9d640 1 --2- 172.21.15.73:0/2869638182 >> v2:172.21.15.120:3300/0 conn(0x7f72fc071090 0x7f72fc15e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:14.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.766+0000 7f7303f9d640 1 --2- 172.21.15.73:0/2869638182 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72fc070350 0x7f72fc070b50 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:14.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.766+0000 7f7303f9d640 1 -- 172.21.15.73:0/2869638182 >> 172.21.15.73:0/2869638182 conn(0x7f72fc0d2d80 msgr2=0x7f72fc0d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:14.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.766+0000 7f7303f9d640 1 -- 172.21.15.73:0/2869638182 shutdown_connections 2024-04-26T20:52:14.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.766+0000 7f7303f9d640 1 -- 172.21.15.73:0/2869638182 wait complete. 2024-04-26T20:52:14.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.767+0000 7f7303f9d640 1 Processor -- start 2024-04-26T20:52:14.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.767+0000 7f7303f9d640 1 -- start start 2024-04-26T20:52:14.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.768+0000 7f7303f9d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f72fc070350 0x7f72fc1f8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:14.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.768+0000 7f7303f9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72fc071090 0x7f72fc0b3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:14.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.768+0000 7f7303f9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f72fc0cfde0 0x7f72fc09ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:14.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.768+0000 7f7303f9d640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f72fc06d9b0 con 0x7f72fc071090 2024-04-26T20:52:14.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.768+0000 7f7303f9d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f72fc0714c0 con 0x7f72fc0cfde0 2024-04-26T20:52:14.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.768+0000 7f7303f9d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f72fc15ef30 con 0x7f72fc070350 2024-04-26T20:52:14.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.768+0000 7f7301538640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72fc071090 0x7f72fc0b3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:14.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.768+0000 7f7301538640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72fc071090 0x7f72fc0b3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43508/0 (socket says 172.21.15.73:43508) 2024-04-26T20:52:14.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.768+0000 7f7301538640 1 -- 172.21.15.73:0/2725043562 learned_addr learned my addr 172.21.15.73:0/2725043562 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:14.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.768+0000 7f730253a640 1 --2- 172.21.15.73:0/2725043562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f72fc0cfde0 0x7f72fc09ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:14.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.768+0000 7f7301d39640 1 --2- 172.21.15.73:0/2725043562 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f72fc070350 0x7f72fc1f8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:14.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.768+0000 7f7301538640 1 -- 172.21.15.73:0/2725043562 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f72fc070350 msgr2=0x7f72fc1f8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:14.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.768+0000 7f7301538640 1 --2- 172.21.15.73:0/2725043562 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f72fc070350 0x7f72fc1f8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:14.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.768+0000 7f7301538640 1 -- 172.21.15.73:0/2725043562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f72fc0cfde0 msgr2=0x7f72fc09ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:14.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.768+0000 7f7301538640 1 --2- 172.21.15.73:0/2725043562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f72fc0cfde0 0x7f72fc09ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:14.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.768+0000 7f7301538640 1 -- 172.21.15.73:0/2725043562 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72fc14dff0 con 0x7f72fc071090 2024-04-26T20:52:14.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.768+0000 7f7301d39640 1 --2- 172.21.15.73:0/2725043562 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f72fc070350 0x7f72fc1f8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:52:14.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.768+0000 7f730253a640 1 --2- 172.21.15.73:0/2725043562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f72fc0cfde0 0x7f72fc09ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:52:14.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.768+0000 7f7301538640 1 --2- 172.21.15.73:0/2725043562 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72fc071090 0x7f72fc0b3a60 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f72ec009890 tx=0x7f72ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:14.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.768+0000 7f72f27fc640 1 -- 172.21.15.73:0/2725043562 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ec037040 con 0x7f72fc071090 2024-04-26T20:52:14.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.769+0000 7f72f27fc640 1 -- 172.21.15.73:0/2725043562 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72ec042070 con 0x7f72fc071090 2024-04-26T20:52:14.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.769+0000 7f72f27fc640 1 -- 172.21.15.73:0/2725043562 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ec030960 con 0x7f72fc071090 2024-04-26T20:52:14.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.769+0000 7f7303f9d640 1 -- 172.21.15.73:0/2725043562 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72fc14e2d0 con 0x7f72fc071090 2024-04-26T20:52:14.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.769+0000 7f7303f9d640 1 -- 172.21.15.73:0/2725043562 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72fc14e890 con 0x7f72fc071090 2024-04-26T20:52:14.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.769+0000 7f7303f9d640 1 -- 172.21.15.73:0/2725043562 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72fc0694f0 con 0x7f72fc071090 2024-04-26T20:52:14.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.770+0000 7f72f27fc640 1 -- 172.21.15.73:0/2725043562 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f72ec030b00 con 0x7f72fc071090 2024-04-26T20:52:14.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.770+0000 7f72f27fc640 1 --2- 172.21.15.73:0/2725043562 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f72e006c8f0 0x7f72e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:14.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.770+0000 7f7301d39640 1 --2- 172.21.15.73:0/2725043562 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f72e006c8f0 0x7f72e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:14.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.770+0000 7f72f27fc640 1 -- 172.21.15.73:0/2725043562 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5421+0+0 (secure 0 0 0) 0x7f72ec0ac330 con 0x7f72fc071090 2024-04-26T20:52:14.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.771+0000 7f7301d39640 1 --2- 172.21.15.73:0/2725043562 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f72e006c8f0 0x7f72e006edb0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f72f8005880 tx=0x7f72f8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:14.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.774+0000 7f72f27fc640 1 -- 172.21.15.73:0/2725043562 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f72ec045360 con 0x7f72fc071090 2024-04-26T20:52:14.953 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.952+0000 7f7303f9d640 1 -- 172.21.15.73:0/2725043562 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f72fc15f320 con 0x7f72fc071090 2024-04-26T20:52:14.953 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.952+0000 7f72f27fc640 1 -- 172.21.15.73:0/2725043562 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) ==== 93+0+10 (secure 0 0 0) 0x7f72ec07a910 con 0x7f72fc071090 2024-04-26T20:52:14.953 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-26T20:52:14.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.954+0000 7f7303f9d640 1 -- 172.21.15.73:0/2725043562 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f72e006c8f0 msgr2=0x7f72e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:14.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.954+0000 7f7303f9d640 1 --2- 172.21.15.73:0/2725043562 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f72e006c8f0 0x7f72e006edb0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f72f8005880 tx=0x7f72f8009040 comp rx=0 tx=0).stop 2024-04-26T20:52:14.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.954+0000 7f7303f9d640 1 -- 172.21.15.73:0/2725043562 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72fc071090 msgr2=0x7f72fc0b3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:14.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.954+0000 7f7303f9d640 1 --2- 172.21.15.73:0/2725043562 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72fc071090 0x7f72fc0b3a60 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f72ec009890 tx=0x7f72ec0046c0 comp rx=0 tx=0).stop 2024-04-26T20:52:14.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.954+0000 7f7303f9d640 1 -- 172.21.15.73:0/2725043562 shutdown_connections 2024-04-26T20:52:14.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.954+0000 7f7303f9d640 1 --2- 172.21.15.73:0/2725043562 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72fc071090 0x7f72fc0b3a60 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:14.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.954+0000 7f7303f9d640 1 --2- 172.21.15.73:0/2725043562 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f72e006c8f0 0x7f72e006edb0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:14.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.954+0000 7f7303f9d640 1 --2- 172.21.15.73:0/2725043562 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f72fc0cfde0 0x7f72fc09ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:14.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.954+0000 7f7303f9d640 1 --2- 172.21.15.73:0/2725043562 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f72fc070350 0x7f72fc1f8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:14.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.954+0000 7f7303f9d640 1 -- 172.21.15.73:0/2725043562 >> 172.21.15.73:0/2725043562 conn(0x7f72fc0d2d80 msgr2=0x7f72fc0b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:14.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.954+0000 7f7303f9d640 1 -- 172.21.15.73:0/2725043562 shutdown_connections 2024-04-26T20:52:14.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:14.954+0000 7f7303f9d640 1 -- 172.21.15.73:0/2725043562 wait complete. 2024-04-26T20:52:14.966 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_auto_repair.TestMDSAutoRepair.test_mds_readonly' 2024-04-26T20:52:15.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.062+0000 7f08699ec640 1 Processor -- start 2024-04-26T20:52:15.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.062+0000 7f08699ec640 1 -- start start 2024-04-26T20:52:15.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.063+0000 7f08699ec640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f086415f3a0 0x7f08640d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:15.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.063+0000 7f08699ec640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f08640d1770 0x7f08640d1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:15.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.063+0000 7f08699ec640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08640d2590 con 0x7f08640d2160 2024-04-26T20:52:15.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.063+0000 7f08699ec640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f08640d0e20 con 0x7f086415f3a0 2024-04-26T20:52:15.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.063+0000 7f08699ec640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f08640b1d70 con 0x7f08640d1770 2024-04-26T20:52:15.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.063+0000 7f0862ffd640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f086415f3a0 0x7f08640d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:15.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.063+0000 7f0862ffd640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f086415f3a0 0x7f08640d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43514/0 (socket says 172.21.15.73:43514) 2024-04-26T20:52:15.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.063+0000 7f0862ffd640 1 -- 172.21.15.73:0/3864811595 learned_addr learned my addr 172.21.15.73:0/3864811595 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:15.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.063+0000 7f08627fc640 1 --2- 172.21.15.73:0/3864811595 >> v2:172.21.15.120:3300/0 conn(0x7f08640d1770 0x7f08640d1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:15.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.063+0000 7f0862ffd640 1 -- 172.21.15.73:0/3864811595 >> v2:172.21.15.120:3300/0 conn(0x7f08640d1770 msgr2=0x7f08640d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:15.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.063+0000 7f0862ffd640 1 --2- 172.21.15.73:0/3864811595 >> v2:172.21.15.120:3300/0 conn(0x7f08640d1770 0x7f08640d1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:15.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.063+0000 7f0862ffd640 1 -- 172.21.15.73:0/3864811595 >> v1:172.21.15.120:6789/0 conn(0x7f08640d2160 legacy=0x7f08640d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:15.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.063+0000 7f0862ffd640 1 -- 172.21.15.73:0/3864811595 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08640b2050 con 0x7f086415f3a0 2024-04-26T20:52:15.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.064+0000 7f0862ffd640 1 --2- 172.21.15.73:0/3864811595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f086415f3a0 0x7f08640d06f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f08640b7b80 tx=0x7f085402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69c9bf615141a9cb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:15.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.064+0000 7f0861ffb640 1 -- 172.21.15.73:0/3864811595 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0854036070 con 0x7f086415f3a0 2024-04-26T20:52:15.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.064+0000 7f0861ffb640 1 -- 172.21.15.73:0/3864811595 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0854031600 con 0x7f086415f3a0 2024-04-26T20:52:15.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.064+0000 7f0861ffb640 1 -- 172.21.15.73:0/3864811595 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08540319d0 con 0x7f086415f3a0 2024-04-26T20:52:15.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.065+0000 7f08699ec640 1 -- 172.21.15.73:0/3864811595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f086415f3a0 msgr2=0x7f08640d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:15.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.065+0000 7f08699ec640 1 --2- 172.21.15.73:0/3864811595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f086415f3a0 0x7f08640d06f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f08640b7b80 tx=0x7f085402a6f0 comp rx=0 tx=0).stop 2024-04-26T20:52:15.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.065+0000 7f08699ec640 1 -- 172.21.15.73:0/3864811595 shutdown_connections 2024-04-26T20:52:15.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.065+0000 7f08699ec640 1 --2- 172.21.15.73:0/3864811595 >> v2:172.21.15.120:3300/0 conn(0x7f08640d1770 0x7f08640d1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:15.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.065+0000 7f08699ec640 1 --2- 172.21.15.73:0/3864811595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f086415f3a0 0x7f08640d06f0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:15.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.065+0000 7f08699ec640 1 -- 172.21.15.73:0/3864811595 >> 172.21.15.73:0/3864811595 conn(0x7f08640d4f90 msgr2=0x7f08640d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:15.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.065+0000 7f08699ec640 1 -- 172.21.15.73:0/3864811595 shutdown_connections 2024-04-26T20:52:15.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.065+0000 7f08699ec640 1 -- 172.21.15.73:0/3864811595 wait complete. 2024-04-26T20:52:15.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.066+0000 7f08699ec640 1 Processor -- start 2024-04-26T20:52:15.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.066+0000 7f08699ec640 1 -- start start 2024-04-26T20:52:15.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.067+0000 7f08699ec640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f08640d1770 0x7f08641f22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:15.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.067+0000 7f08699ec640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08640d2160 0x7f08641f5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:15.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.067+0000 7f08699ec640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f086415f3a0 0x7f08641f5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:15.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.067+0000 7f08699ec640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f086415f900 con 0x7f08640d2160 2024-04-26T20:52:15.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.067+0000 7f08699ec640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f086415f780 con 0x7f086415f3a0 2024-04-26T20:52:15.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.067+0000 7f08699ec640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f08640b3860 con 0x7f08640d1770 2024-04-26T20:52:15.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.067+0000 7f08627fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08640d2160 0x7f08641f5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:15.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.067+0000 7f08627fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08640d2160 0x7f08641f5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43522/0 (socket says 172.21.15.73:43522) 2024-04-26T20:52:15.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.067+0000 7f08627fc640 1 -- 172.21.15.73:0/421463646 learned_addr learned my addr 172.21.15.73:0/421463646 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:15.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.067+0000 7f0862ffd640 1 --2- 172.21.15.73:0/421463646 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f08640d1770 0x7f08641f22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:15.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.067+0000 7f08637fe640 1 --2- 172.21.15.73:0/421463646 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f086415f3a0 0x7f08641f5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:15.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.067+0000 7f08627fc640 1 -- 172.21.15.73:0/421463646 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f08640d1770 msgr2=0x7f08641f22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:15.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.067+0000 7f08627fc640 1 --2- 172.21.15.73:0/421463646 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f08640d1770 0x7f08641f22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:15.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.067+0000 7f08627fc640 1 -- 172.21.15.73:0/421463646 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f086415f3a0 msgr2=0x7f08641f5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:15.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.067+0000 7f08627fc640 1 --2- 172.21.15.73:0/421463646 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f086415f3a0 0x7f08641f5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:15.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.067+0000 7f08627fc640 1 -- 172.21.15.73:0/421463646 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08641f5c40 con 0x7f08640d2160 2024-04-26T20:52:15.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.067+0000 7f08627fc640 1 --2- 172.21.15.73:0/421463646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08640d2160 0x7f08641f5700 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f085800c820 tx=0x7f085800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:15.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.068+0000 7f084bfff640 1 -- 172.21.15.73:0/421463646 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0858015070 con 0x7f08640d2160 2024-04-26T20:52:15.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.068+0000 7f084bfff640 1 -- 172.21.15.73:0/421463646 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0858004240 con 0x7f08640d2160 2024-04-26T20:52:15.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.068+0000 7f08699ec640 1 -- 172.21.15.73:0/421463646 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f086415fc90 con 0x7f08640d2160 2024-04-26T20:52:15.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.068+0000 7f084bfff640 1 -- 172.21.15.73:0/421463646 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085800ce10 con 0x7f08640d2160 2024-04-26T20:52:15.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.068+0000 7f08699ec640 1 -- 172.21.15.73:0/421463646 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0864160250 con 0x7f08640d2160 2024-04-26T20:52:15.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.068+0000 7f08699ec640 1 -- 172.21.15.73:0/421463646 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08640694f0 con 0x7f08640d2160 2024-04-26T20:52:15.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.069+0000 7f084bfff640 1 -- 172.21.15.73:0/421463646 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f08580077d0 con 0x7f08640d2160 2024-04-26T20:52:15.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.069+0000 7f084bfff640 1 --2- 172.21.15.73:0/421463646 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f085006c8f0 0x7f085006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:15.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.070+0000 7f0862ffd640 1 --2- 172.21.15.73:0/421463646 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f085006c8f0 0x7f085006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:15.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.070+0000 7f084bfff640 1 -- 172.21.15.73:0/421463646 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5421+0+0 (secure 0 0 0) 0x7f085808b8a0 con 0x7f08640d2160 2024-04-26T20:52:15.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.070+0000 7f0862ffd640 1 --2- 172.21.15.73:0/421463646 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f085006c8f0 0x7f085006edb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0854033000 tx=0x7f0854009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:15.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.074+0000 7f084bfff640 1 -- 172.21.15.73:0/421463646 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0858059df0 con 0x7f08640d2160 2024-04-26T20:52:15.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.316+0000 7f08699ec640 1 -- 172.21.15.73:0/421463646 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_auto_repair.TestMDSAutoRepair.test_mds_readonly"]} v 0) -- 0x7f08640cbe20 con 0x7f08640d2160 2024-04-26T20:52:15.351 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.350+0000 7f084bfff640 1 -- 172.21.15.73:0/421463646 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_auto_repair.TestMDSAutoRepair.test_mds_readonly"]}]=0 v121) ==== 147+0+0 (secure 0 0 0) 0x7f08580596f0 con 0x7f08640d2160 2024-04-26T20:52:15.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.352+0000 7f08699ec640 1 -- 172.21.15.73:0/421463646 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f085006c8f0 msgr2=0x7f085006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:15.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.352+0000 7f08699ec640 1 --2- 172.21.15.73:0/421463646 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f085006c8f0 0x7f085006edb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0854033000 tx=0x7f0854009000 comp rx=0 tx=0).stop 2024-04-26T20:52:15.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.352+0000 7f08699ec640 1 -- 172.21.15.73:0/421463646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08640d2160 msgr2=0x7f08641f5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:15.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.352+0000 7f08699ec640 1 --2- 172.21.15.73:0/421463646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08640d2160 0x7f08641f5700 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f085800c820 tx=0x7f085800cce0 comp rx=0 tx=0).stop 2024-04-26T20:52:15.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.352+0000 7f08699ec640 1 -- 172.21.15.73:0/421463646 shutdown_connections 2024-04-26T20:52:15.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.352+0000 7f08699ec640 1 --2- 172.21.15.73:0/421463646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08640d2160 0x7f08641f5700 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:15.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.352+0000 7f08699ec640 1 --2- 172.21.15.73:0/421463646 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f085006c8f0 0x7f085006edb0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:15.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.352+0000 7f08699ec640 1 --2- 172.21.15.73:0/421463646 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f086415f3a0 0x7f08641f5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:15.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.352+0000 7f08699ec640 1 --2- 172.21.15.73:0/421463646 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f08640d1770 0x7f08641f22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:15.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.352+0000 7f08699ec640 1 -- 172.21.15.73:0/421463646 >> 172.21.15.73:0/421463646 conn(0x7f08640d4f90 msgr2=0x7f086406d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:15.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.353+0000 7f08699ec640 1 -- 172.21.15.73:0/421463646 shutdown_connections 2024-04-26T20:52:15.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.353+0000 7f08699ec640 1 -- 172.21.15.73:0/421463646 wait complete. 2024-04-26T20:52:15.364 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T20:52:15.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.463+0000 7fa6f4425640 1 Processor -- start 2024-04-26T20:52:15.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.463+0000 7fa6f4425640 1 -- start start 2024-04-26T20:52:15.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.464+0000 7fa6f4425640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fa6ec071e60 0x7fa6ec06f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:15.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.464+0000 7fa6f4425640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa6ec0cc480 0x7fa6ec166ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:15.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.464+0000 7fa6f4425640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6ec15d7c0 con 0x7fa6ec06fe20 2024-04-26T20:52:15.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.464+0000 7fa6f4425640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fa6ec15daa0 con 0x7fa6ec071e60 2024-04-26T20:52:15.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.464+0000 7fa6f4425640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fa6ec15dd80 con 0x7fa6ec0cc480 2024-04-26T20:52:15.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.464+0000 7fa6f19c0640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fa6ec06fe20 0x7fa6ec072240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40076/0 (socket says 172.21.15.73:40076) 2024-04-26T20:52:15.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.464+0000 7fa6f19c0640 1 -- 172.21.15.73:0/1556227080 learned_addr learned my addr 172.21.15.73:0/1556227080 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:52:15.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.464+0000 7fa6f11bf640 1 -- 172.21.15.73:0/1556227080 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 709197992 0 0) 0x7fa6ec15d7c0 con 0x7fa6ec06fe20 2024-04-26T20:52:15.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.464+0000 7fa6f11bf640 1 -- 172.21.15.73:0/1556227080 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa6d8003520 con 0x7fa6ec06fe20 2024-04-26T20:52:15.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.464+0000 7fa6f29c2640 1 --2- 172.21.15.73:0/1556227080 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa6ec0cc480 0x7fa6ec166ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:15.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.464+0000 7fa6f21c1640 1 --2- 172.21.15.73:0/1556227080 >> v2:172.21.15.120:3300/0 conn(0x7fa6ec071e60 0x7fa6ec06f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:15.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.464+0000 7fa6f11bf640 1 -- 172.21.15.73:0/1556227080 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2404966309 0 0) 0x7fa6d8003520 con 0x7fa6ec06fe20 2024-04-26T20:52:15.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.464+0000 7fa6f11bf640 1 -- 172.21.15.73:0/1556227080 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa6ec0cc480 msgr2=0x7fa6ec166ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:15.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.464+0000 7fa6f11bf640 1 --2- 172.21.15.73:0/1556227080 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa6ec0cc480 0x7fa6ec166ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:15.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.464+0000 7fa6f11bf640 1 -- 172.21.15.73:0/1556227080 >> v2:172.21.15.120:3300/0 conn(0x7fa6ec071e60 msgr2=0x7fa6ec06f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:15.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.464+0000 7fa6f11bf640 1 --2- 172.21.15.73:0/1556227080 >> v2:172.21.15.120:3300/0 conn(0x7fa6ec071e60 0x7fa6ec06f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:15.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.464+0000 7fa6f11bf640 1 -- 172.21.15.73:0/1556227080 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6ec15e060 con 0x7fa6ec06fe20 2024-04-26T20:52:15.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.465+0000 7fa6f11bf640 1 -- 172.21.15.73:0/1556227080 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7fa6dc001670 con 0x7fa6ec06fe20 2024-04-26T20:52:15.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.465+0000 7fa6f11bf640 1 -- 172.21.15.73:0/1556227080 >> v1:172.21.15.73:6789/0 conn(0x7fa6ec06fe20 legacy=0x7fa6ec072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:15.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.465+0000 7fa6f11bf640 1 --2- 172.21.15.73:0/1556227080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa6d80039b0 0x7fa6d8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:15.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.465+0000 7fa6f11bf640 1 --2- 172.21.15.73:0/1556227080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6d8004800 0x7fa6d8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:15.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.465+0000 7fa6f11bf640 1 --2- 172.21.15.73:0/1556227080 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa6ec0cc480 0x7fa6d80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:15.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.465+0000 7fa6f11bf640 1 -- 172.21.15.73:0/1556227080 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ec15daa0 con 0x7fa6d8004800 2024-04-26T20:52:15.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.465+0000 7fa6f11bf640 1 -- 172.21.15.73:0/1556227080 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ec15dd80 con 0x7fa6d80039b0 2024-04-26T20:52:15.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.465+0000 7fa6f11bf640 1 -- 172.21.15.73:0/1556227080 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fa6d800ba70 con 0x7fa6ec0cc480 2024-04-26T20:52:15.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.465+0000 7fa6f19c0640 1 --2- 172.21.15.73:0/1556227080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6d8004800 0x7fa6d8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:15.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.465+0000 7fa6f21c1640 1 --2- 172.21.15.73:0/1556227080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa6d80039b0 0x7fa6d8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:15.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.465+0000 7fa6f29c2640 1 --2- 172.21.15.73:0/1556227080 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa6ec0cc480 0x7fa6d80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:15.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.465+0000 7fa6f19c0640 1 -- 172.21.15.73:0/1556227080 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa6ec0cc480 msgr2=0x7fa6d80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:15.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.465+0000 7fa6f19c0640 1 --2- 172.21.15.73:0/1556227080 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa6ec0cc480 0x7fa6d80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:15.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.465+0000 7fa6f19c0640 1 -- 172.21.15.73:0/1556227080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa6d80039b0 msgr2=0x7fa6d8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:15.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.465+0000 7fa6f19c0640 1 --2- 172.21.15.73:0/1556227080 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa6d80039b0 0x7fa6d8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:15.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.465+0000 7fa6f19c0640 1 -- 172.21.15.73:0/1556227080 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6d8003520 con 0x7fa6d8004800 2024-04-26T20:52:15.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.466+0000 7fa6f19c0640 1 --2- 172.21.15.73:0/1556227080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6d8004800 0x7fa6d8004c80 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc00c210 tx=0x7fa6d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=335f42e018c2acbe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:15.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.466+0000 7fa6f11bf640 1 -- 172.21.15.73:0/1556227080 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6dc007d30 con 0x7fa6d8004800 2024-04-26T20:52:15.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.466+0000 7fa6f11bf640 1 -- 172.21.15.73:0/1556227080 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6dc0058d0 con 0x7fa6d8004800 2024-04-26T20:52:15.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.466+0000 7fa6f11bf640 1 -- 172.21.15.73:0/1556227080 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6dc005ca0 con 0x7fa6d8004800 2024-04-26T20:52:15.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.466+0000 7fa6f4425640 1 -- 172.21.15.73:0/1556227080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6d8004800 msgr2=0x7fa6d8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:15.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.466+0000 7fa6f4425640 1 --2- 172.21.15.73:0/1556227080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6d8004800 0x7fa6d8004c80 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc00c210 tx=0x7fa6d8005830 comp rx=0 tx=0).stop 2024-04-26T20:52:15.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.466+0000 7fa6f29c2640 1 -- 172.21.15.73:0/1556227080 reap_dead start 2024-04-26T20:52:15.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.466+0000 7fa6f4425640 1 -- 172.21.15.73:0/1556227080 shutdown_connections 2024-04-26T20:52:15.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.466+0000 7fa6f4425640 1 -- 172.21.15.73:0/1556227080 >> 172.21.15.73:0/1556227080 conn(0x7fa6ec0d0be0 msgr2=0x7fa6ec0d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:15.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.466+0000 7fa6f4425640 1 -- 172.21.15.73:0/1556227080 shutdown_connections 2024-04-26T20:52:15.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.466+0000 7fa6f4425640 1 -- 172.21.15.73:0/1556227080 wait complete. 2024-04-26T20:52:15.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.468+0000 7fa6f4425640 1 Processor -- start 2024-04-26T20:52:15.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.468+0000 7fa6f4425640 1 -- start start 2024-04-26T20:52:15.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.468+0000 7fa6f4425640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa6d8004800 0x7fa6ec14e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:15.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.468+0000 7fa6f4425640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa6ec0ae8d0 0x7fa6ec0cff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:15.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.468+0000 7fa6f4425640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6ec15b660 0x7fa6ec15bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:15.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.468+0000 7fa6f4425640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ec15d570 con 0x7fa6ec15b660 2024-04-26T20:52:15.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.468+0000 7fa6f4425640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ec15d3f0 con 0x7fa6ec0ae8d0 2024-04-26T20:52:15.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.468+0000 7fa6f4425640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fa6ec072950 con 0x7fa6d8004800 2024-04-26T20:52:15.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.468+0000 7fa6f29c2640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6ec15b660 0x7fa6ec15bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:15.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.468+0000 7fa6f29c2640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6ec15b660 0x7fa6ec15bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43536/0 (socket says 172.21.15.73:43536) 2024-04-26T20:52:15.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.468+0000 7fa6f29c2640 1 -- 172.21.15.73:0/3561395079 learned_addr learned my addr 172.21.15.73:0/3561395079 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:15.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.469+0000 7fa6f21c1640 1 --2- 172.21.15.73:0/3561395079 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa6d8004800 0x7fa6ec14e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:15.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.469+0000 7fa6f19c0640 1 --2- 172.21.15.73:0/3561395079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa6ec0ae8d0 0x7fa6ec0cff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:15.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.469+0000 7fa6f29c2640 1 -- 172.21.15.73:0/3561395079 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa6d8004800 msgr2=0x7fa6ec14e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:15.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.469+0000 7fa6f29c2640 1 --2- 172.21.15.73:0/3561395079 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa6d8004800 0x7fa6ec14e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:15.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.469+0000 7fa6f29c2640 1 -- 172.21.15.73:0/3561395079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa6ec0ae8d0 msgr2=0x7fa6ec0cff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:15.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.469+0000 7fa6f29c2640 1 --2- 172.21.15.73:0/3561395079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa6ec0ae8d0 0x7fa6ec0cff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:15.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.469+0000 7fa6f29c2640 1 -- 172.21.15.73:0/3561395079 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6ec0aed50 con 0x7fa6ec15b660 2024-04-26T20:52:15.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.469+0000 7fa6f19c0640 1 --2- 172.21.15.73:0/3561395079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa6ec0ae8d0 0x7fa6ec0cff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:52:15.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.469+0000 7fa6f29c2640 1 --2- 172.21.15.73:0/3561395079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6ec15b660 0x7fa6ec15bae0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fa6e40107f0 tx=0x7fa6e4010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:15.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.469+0000 7fa6e2ffd640 1 -- 172.21.15.73:0/3561395079 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e4010ea0 con 0x7fa6ec15b660 2024-04-26T20:52:15.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.469+0000 7fa6e2ffd640 1 -- 172.21.15.73:0/3561395079 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6e401c070 con 0x7fa6ec15b660 2024-04-26T20:52:15.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.469+0000 7fa6f4425640 1 -- 172.21.15.73:0/3561395079 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6ec0aefd0 con 0x7fa6ec15b660 2024-04-26T20:52:15.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.469+0000 7fa6e2ffd640 1 -- 172.21.15.73:0/3561395079 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e400eb20 con 0x7fa6ec15b660 2024-04-26T20:52:15.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.469+0000 7fa6f4425640 1 -- 172.21.15.73:0/3561395079 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6ec0af590 con 0x7fa6ec15b660 2024-04-26T20:52:15.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.470+0000 7fa6f4425640 1 -- 172.21.15.73:0/3561395079 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6ec0694f0 con 0x7fa6ec15b660 2024-04-26T20:52:15.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.470+0000 7fa6e2ffd640 1 -- 172.21.15.73:0/3561395079 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa6e4012040 con 0x7fa6ec15b660 2024-04-26T20:52:15.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.471+0000 7fa6e2ffd640 1 --2- 172.21.15.73:0/3561395079 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa6d006c8f0 0x7fa6d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:15.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.471+0000 7fa6e2ffd640 1 -- 172.21.15.73:0/3561395079 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5421+0+0 (secure 0 0 0) 0x7fa6e408bab0 con 0x7fa6ec15b660 2024-04-26T20:52:15.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.471+0000 7fa6f21c1640 1 --2- 172.21.15.73:0/3561395079 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa6d006c8f0 0x7fa6d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:15.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.471+0000 7fa6f21c1640 1 --2- 172.21.15.73:0/3561395079 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa6d006c8f0 0x7fa6d006edb0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8003aa0 tx=0x7fa6e8009760 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:15.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.475+0000 7fa6e2ffd640 1 -- 172.21.15.73:0/3561395079 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa6ec0aed50 con 0x7fa6ec15b660 2024-04-26T20:52:15.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.650+0000 7fa6f4425640 1 -- 172.21.15.73:0/3561395079 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fa6ec06e9b0 con 0x7fa6ec15b660 2024-04-26T20:52:15.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.653+0000 7fa6e2ffd640 1 -- 172.21.15.73:0/3561395079 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7fa6e4012330 con 0x7fa6ec15b660 2024-04-26T20:52:15.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.655+0000 7fa6f4425640 1 -- 172.21.15.73:0/3561395079 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa6d006c8f0 msgr2=0x7fa6d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:15.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.655+0000 7fa6f4425640 1 --2- 172.21.15.73:0/3561395079 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa6d006c8f0 0x7fa6d006edb0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8003aa0 tx=0x7fa6e8009760 comp rx=0 tx=0).stop 2024-04-26T20:52:15.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.655+0000 7fa6f4425640 1 -- 172.21.15.73:0/3561395079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6ec15b660 msgr2=0x7fa6ec15bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:15.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.655+0000 7fa6f4425640 1 --2- 172.21.15.73:0/3561395079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6ec15b660 0x7fa6ec15bae0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fa6e40107f0 tx=0x7fa6e4010cb0 comp rx=0 tx=0).stop 2024-04-26T20:52:15.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.656+0000 7fa6f4425640 1 -- 172.21.15.73:0/3561395079 shutdown_connections 2024-04-26T20:52:15.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.656+0000 7fa6f4425640 1 --2- 172.21.15.73:0/3561395079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6ec15b660 0x7fa6ec15bae0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:15.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.656+0000 7fa6f4425640 1 --2- 172.21.15.73:0/3561395079 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa6d006c8f0 0x7fa6d006edb0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:15.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.656+0000 7fa6f4425640 1 --2- 172.21.15.73:0/3561395079 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa6ec0ae8d0 0x7fa6ec0cff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:15.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.656+0000 7fa6f4425640 1 --2- 172.21.15.73:0/3561395079 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa6d8004800 0x7fa6ec14e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:15.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.656+0000 7fa6f4425640 1 -- 172.21.15.73:0/3561395079 >> 172.21.15.73:0/3561395079 conn(0x7fa6ec0d0be0 msgr2=0x7fa6ec0b0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:15.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.656+0000 7fa6f4425640 1 -- 172.21.15.73:0/3561395079 shutdown_connections 2024-04-26T20:52:15.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.656+0000 7fa6f4425640 1 -- 172.21.15.73:0/3561395079 wait complete. 2024-04-26T20:52:15.667 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T20:52:15.667 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T20:52:15.709 DEBUG:teuthology.orchestra.run.smithi120:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T20:52:15.752 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T20:52:15.753 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-26T20:52:15.753 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:52:15.767 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19637 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T20:52:15.768 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-26T20:52:15.768 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:52:15.811 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056097,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19070 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T20:52:15.811 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-26T20:52:15.811 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:52:15.826 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19637 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T20:52:15.826 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-26T20:52:15.826 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:52:15.868 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056097,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19070 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T20:52:15.868 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-26T20:52:15.868 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:52:15.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.964+0000 7f14ffd46640 1 Processor -- start 2024-04-26T20:52:15.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.964+0000 7f14ffd46640 1 -- start start 2024-04-26T20:52:15.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.965+0000 7f14ffd46640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f14f80b2340 0x7f14f80cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:15.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.965+0000 7f14ffd46640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f14f80cd930 0x7f14f80d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:15.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.965+0000 7f14ffd46640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14f80d10d0 con 0x7f14f8162810 2024-04-26T20:52:15.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.965+0000 7f14ffd46640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f14f80d0750 con 0x7f14f80cd930 2024-04-26T20:52:15.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.965+0000 7f14ffd46640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f14f80d16a0 con 0x7f14f80b2340 2024-04-26T20:52:15.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.965+0000 7f14fe2e3640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f14f80cd930 0x7f14f80d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:15.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.965+0000 7f14fe2e3640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f14f80cd930 0x7f14f80d1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43552/0 (socket says 172.21.15.73:43552) 2024-04-26T20:52:15.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.965+0000 7f14fe2e3640 1 -- 172.21.15.73:0/123212021 learned_addr learned my addr 172.21.15.73:0/123212021 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:15.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.965+0000 7f14fd2e1640 1 --2- 172.21.15.73:0/123212021 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f14f80b2340 0x7f14f80cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:15.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.965+0000 7f14fe2e3640 1 -- 172.21.15.73:0/123212021 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f14f80b2340 msgr2=0x7f14f80cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:15.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.965+0000 7f14fe2e3640 1 --2- 172.21.15.73:0/123212021 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f14f80b2340 0x7f14f80cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:15.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.965+0000 7f14fe2e3640 1 -- 172.21.15.73:0/123212021 >> v1:172.21.15.120:6789/0 conn(0x7f14f8162810 legacy=0x7f14f8070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:15.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.965+0000 7f14fe2e3640 1 -- 172.21.15.73:0/123212021 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14f80d1370 con 0x7f14f80cd930 2024-04-26T20:52:15.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.966+0000 7f14fe2e3640 1 --2- 172.21.15.73:0/123212021 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14f80cd930 0x7f14f80d1b30 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f14f4009e50 tx=0x7f14f402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e20df656fd1b7429 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:15.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.966+0000 7f14fcae0640 1 -- 172.21.15.73:0/123212021 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f4036070 con 0x7f14f80cd930 2024-04-26T20:52:15.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.966+0000 7f14fcae0640 1 -- 172.21.15.73:0/123212021 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14f4031600 con 0x7f14f80cd930 2024-04-26T20:52:15.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.966+0000 7f14fcae0640 1 -- 172.21.15.73:0/123212021 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f40319d0 con 0x7f14f80cd930 2024-04-26T20:52:15.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.967+0000 7f14ffd46640 1 -- 172.21.15.73:0/123212021 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14f80cd930 msgr2=0x7f14f80d1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:15.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.967+0000 7f14ffd46640 1 --2- 172.21.15.73:0/123212021 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14f80cd930 0x7f14f80d1b30 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f14f4009e50 tx=0x7f14f402a6c0 comp rx=0 tx=0).stop 2024-04-26T20:52:15.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.967+0000 7f14ffd46640 1 -- 172.21.15.73:0/123212021 shutdown_connections 2024-04-26T20:52:15.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.967+0000 7f14ffd46640 1 --2- 172.21.15.73:0/123212021 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14f80cd930 0x7f14f80d1b30 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:15.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.967+0000 7f14ffd46640 1 --2- 172.21.15.73:0/123212021 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f14f80b2340 0x7f14f80cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:15.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.967+0000 7f14ffd46640 1 -- 172.21.15.73:0/123212021 >> 172.21.15.73:0/123212021 conn(0x7f14f80d2da0 msgr2=0x7f14f80d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:15.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.967+0000 7f14ffd46640 1 -- 172.21.15.73:0/123212021 shutdown_connections 2024-04-26T20:52:15.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.967+0000 7f14ffd46640 1 -- 172.21.15.73:0/123212021 wait complete. 2024-04-26T20:52:15.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.968+0000 7f14ffd46640 1 Processor -- start 2024-04-26T20:52:15.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.968+0000 7f14ffd46640 1 -- start start 2024-04-26T20:52:15.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.969+0000 7f14ffd46640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f14f80b2340 0x7f14f814a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:15.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.969+0000 7f14ffd46640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14f80cd930 0x7f14f80c9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:15.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.969+0000 7f14ffd46640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14f8162810 0x7f14f80ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:15.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.969+0000 7f14ffd46640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f14f80d2070 con 0x7f14f80cd930 2024-04-26T20:52:15.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.969+0000 7f14ffd46640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f14f80d0470 con 0x7f14f8162810 2024-04-26T20:52:15.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.969+0000 7f14ffd46640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f14f80d21f0 con 0x7f14f80b2340 2024-04-26T20:52:15.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.969+0000 7f14fd2e1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14f80cd930 0x7f14f80c9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:15.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.969+0000 7f14fd2e1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14f80cd930 0x7f14f80c9e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43560/0 (socket says 172.21.15.73:43560) 2024-04-26T20:52:15.973 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.969+0000 7f14fd2e1640 1 -- 172.21.15.73:0/2462878075 learned_addr learned my addr 172.21.15.73:0/2462878075 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:15.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.969+0000 7f14fe2e3640 1 --2- 172.21.15.73:0/2462878075 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14f8162810 0x7f14f80ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:15.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.969+0000 7f14fdae2640 1 --2- 172.21.15.73:0/2462878075 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f14f80b2340 0x7f14f814a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:15.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.969+0000 7f14fd2e1640 1 -- 172.21.15.73:0/2462878075 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f14f80b2340 msgr2=0x7f14f814a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:15.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.969+0000 7f14fd2e1640 1 --2- 172.21.15.73:0/2462878075 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f14f80b2340 0x7f14f814a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:15.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.969+0000 7f14fd2e1640 1 -- 172.21.15.73:0/2462878075 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14f8162810 msgr2=0x7f14f80ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:15.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.969+0000 7f14fd2e1640 1 --2- 172.21.15.73:0/2462878075 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14f8162810 0x7f14f80ca3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:15.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.969+0000 7f14fd2e1640 1 -- 172.21.15.73:0/2462878075 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14f80c6bc0 con 0x7f14f80cd930 2024-04-26T20:52:15.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.969+0000 7f14fd2e1640 1 --2- 172.21.15.73:0/2462878075 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14f80cd930 0x7f14f80c9e70 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f14e800d800 tx=0x7f14e800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:15.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.969+0000 7f14ee7fc640 1 -- 172.21.15.73:0/2462878075 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e800b6c0 con 0x7f14f80cd930 2024-04-26T20:52:15.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.969+0000 7f14ee7fc640 1 -- 172.21.15.73:0/2462878075 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14e8019070 con 0x7f14f80cd930 2024-04-26T20:52:15.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.969+0000 7f14ffd46640 1 -- 172.21.15.73:0/2462878075 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14f80ca9a0 con 0x7f14f80cd930 2024-04-26T20:52:15.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.969+0000 7f14ee7fc640 1 -- 172.21.15.73:0/2462878075 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e800ba80 con 0x7f14f80cd930 2024-04-26T20:52:15.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.969+0000 7f14ffd46640 1 -- 172.21.15.73:0/2462878075 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14f80caf60 con 0x7f14f80cd930 2024-04-26T20:52:15.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.970+0000 7f14ffd46640 1 -- 172.21.15.73:0/2462878075 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14f80cf130 con 0x7f14f80cd930 2024-04-26T20:52:15.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.971+0000 7f14ee7fc640 1 -- 172.21.15.73:0/2462878075 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f14e800bc20 con 0x7f14f80cd930 2024-04-26T20:52:15.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.971+0000 7f14ee7fc640 1 --2- 172.21.15.73:0/2462878075 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f14dc06c8f0 0x7f14dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:15.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.971+0000 7f14fdae2640 1 --2- 172.21.15.73:0/2462878075 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f14dc06c8f0 0x7f14dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:15.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.971+0000 7f14ee7fc640 1 -- 172.21.15.73:0/2462878075 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5421+0+0 (secure 0 0 0) 0x7f14e808be90 con 0x7f14f80cd930 2024-04-26T20:52:15.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.972+0000 7f14fdae2640 1 --2- 172.21.15.73:0/2462878075 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f14dc06c8f0 0x7f14dc06edb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f14f0006fd0 tx=0x7f14f0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:15.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:15.975+0000 7f14ee7fc640 1 -- 172.21.15.73:0/2462878075 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f14f80cf130 con 0x7f14f80cd930 2024-04-26T20:52:16.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.189+0000 7f14ffd46640 1 -- 172.21.15.73:0/2462878075 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f14f80694f0 con 0x7f14f80cd930 2024-04-26T20:52:16.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.190+0000 7f14ee7fc640 1 -- 172.21.15.73:0/2462878075 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+3872 (secure 0 0 0) 0x7f14e801c360 con 0x7f14f80cd930 2024-04-26T20:52:16.191 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:52:16.191 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":22,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4802,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/272662913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":272662913},{"type":"v1","addr":"172.21.15.73:6836","nonce":272662913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4943,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/2624899096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6833","nonce":2624899096},{"type":"v1","addr":"172.21.15.120:6835","nonce":2624899096}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5047,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/1438858294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":1438858294},{"type":"v1","addr":"172.21.15.73:6837","nonce":1438858294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5377,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6834/3288098358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":3288098358},{"type":"v1","addr":"172.21.15.120:6834","nonce":3288098358}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[]} 2024-04-26T20:52:16.191 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 22 2024-04-26T20:52:16.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.192+0000 7f14ffd46640 1 -- 172.21.15.73:0/2462878075 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f14dc06c8f0 msgr2=0x7f14dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:16.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.192+0000 7f14ffd46640 1 --2- 172.21.15.73:0/2462878075 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f14dc06c8f0 0x7f14dc06edb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f14f0006fd0 tx=0x7f14f0008040 comp rx=0 tx=0).stop 2024-04-26T20:52:16.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.192+0000 7f14ffd46640 1 -- 172.21.15.73:0/2462878075 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14f80cd930 msgr2=0x7f14f80c9e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:16.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.192+0000 7f14ffd46640 1 --2- 172.21.15.73:0/2462878075 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14f80cd930 0x7f14f80c9e70 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f14e800d800 tx=0x7f14e800dcc0 comp rx=0 tx=0).stop 2024-04-26T20:52:16.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.192+0000 7f14ffd46640 1 -- 172.21.15.73:0/2462878075 shutdown_connections 2024-04-26T20:52:16.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.192+0000 7f14ffd46640 1 --2- 172.21.15.73:0/2462878075 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14f80cd930 0x7f14f80c9e70 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:16.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.192+0000 7f14ffd46640 1 --2- 172.21.15.73:0/2462878075 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f14dc06c8f0 0x7f14dc06edb0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:16.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.192+0000 7f14ffd46640 1 --2- 172.21.15.73:0/2462878075 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f14f8162810 0x7f14f80ca3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:16.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.192+0000 7f14ffd46640 1 --2- 172.21.15.73:0/2462878075 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f14f80b2340 0x7f14f814a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:16.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.192+0000 7f14ffd46640 1 -- 172.21.15.73:0/2462878075 >> 172.21.15.73:0/2462878075 conn(0x7f14f80d2da0 msgr2=0x7f14f80cf3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:16.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.192+0000 7f14ffd46640 1 -- 172.21.15.73:0/2462878075 shutdown_connections 2024-04-26T20:52:16.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.192+0000 7f14ffd46640 1 -- 172.21.15.73:0/2462878075 wait complete. 2024-04-26T20:52:16.204 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T20:52:16.204 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-26T20:52:16.204 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T20:52:16.204 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:52:16.204 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T20:52:16.204 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-26T20:52:16.204 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T20:52:16.205 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:52:16.205 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T20:52:16.205 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-26T20:52:16.205 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T20:52:16.205 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:52:16.205 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T20:52:16.205 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-26T20:52:16.206 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T20:52:16.206 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:52:16.214 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T20:52:16.214 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T20:52:16.215 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T20:52:16.215 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T20:52:16.217 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T20:52:16.218 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T20:52:16.218 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T20:52:16.219 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T20:52:16.257 INFO:tasks.ceph.mds.a:Started 2024-04-26T20:52:16.257 DEBUG:teuthology.parallel:result is None 2024-04-26T20:52:16.258 INFO:tasks.ceph.mds.c:Started 2024-04-26T20:52:16.258 DEBUG:teuthology.parallel:result is None 2024-04-26T20:52:16.260 INFO:tasks.ceph.mds.b:Started 2024-04-26T20:52:16.260 DEBUG:teuthology.parallel:result is None 2024-04-26T20:52:16.261 INFO:tasks.ceph.mds.d:Started 2024-04-26T20:52:16.261 DEBUG:teuthology.parallel:result is None 2024-04-26T20:52:16.261 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-26T20:52:16.329 INFO:tasks.ceph.mds.a.smithi073.stdout:starting mds.a at 2024-04-26T20:52:16.330 INFO:tasks.ceph.mds.c.smithi073.stdout:starting mds.c at 2024-04-26T20:52:16.340 INFO:tasks.ceph.mds.b.smithi120.stdout:starting mds.b at 2024-04-26T20:52:16.340 INFO:tasks.ceph.mds.d.smithi120.stdout:starting mds.d at 2024-04-26T20:52:16.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.365+0000 7f6361cf1640 1 Processor -- start 2024-04-26T20:52:16.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.365+0000 7f6361cf1640 1 -- start start 2024-04-26T20:52:16.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.365+0000 7f6361cf1640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f635c15f2d0 0x7f635c1604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:16.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.365+0000 7f6361cf1640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f635c072700 0x7f635c06f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:16.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.366+0000 7f6361cf1640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f635c072f70 con 0x7f635c06fcd0 2024-04-26T20:52:16.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.366+0000 7f6361cf1640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f635c074010 con 0x7f635c15f2d0 2024-04-26T20:52:16.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.366+0000 7f6361cf1640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f635c070100 con 0x7f635c072700 2024-04-26T20:52:16.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.366+0000 7f635b7fe640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f635c15f2d0 0x7f635c1604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:16.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.366+0000 7f635bfff640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f635c06fcd0 0x7f635c0cbb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40096/0 (socket says 172.21.15.73:40096) 2024-04-26T20:52:16.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.366+0000 7f635b7fe640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f635c15f2d0 0x7f635c1604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43608/0 (socket says 172.21.15.73:43608) 2024-04-26T20:52:16.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.366+0000 7f635bfff640 1 -- 172.21.15.73:0/3641416287 learned_addr learned my addr 172.21.15.73:0/3641416287 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:52:16.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.366+0000 7f635affd640 1 --2- 172.21.15.73:0/3641416287 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f635c072700 0x7f635c06f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:16.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.366+0000 7f635b7fe640 1 -- 172.21.15.73:0/3641416287 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f635c072700 msgr2=0x7f635c06f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:16.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.366+0000 7f635b7fe640 1 --2- 172.21.15.73:0/3641416287 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f635c072700 0x7f635c06f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:16.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.366+0000 7f635b7fe640 1 -- 172.21.15.73:0/3641416287 >> v1:172.21.15.73:6789/0 conn(0x7f635c06fcd0 legacy=0x7f635c0cbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:16.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.366+0000 7f635a7fc640 1 -- 172.21.15.73:0/3641416287 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 46421529 0 0) 0x7f635c072f70 con 0x7f635c06fcd0 2024-04-26T20:52:16.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.366+0000 7f635b7fe640 1 -- 172.21.15.73:0/3641416287 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f635c073250 con 0x7f635c15f2d0 2024-04-26T20:52:16.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.366+0000 7f635affd640 1 --2- 172.21.15.73:0/3641416287 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f635c072700 0x7f635c06f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:52:16.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.367+0000 7f635b7fe640 1 --2- 172.21.15.73:0/3641416287 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f635c15f2d0 0x7f635c1604e0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f63440097a0 tx=0x7f634402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=201dcb6207b09492 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:16.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.367+0000 7f635a7fc640 1 -- 172.21.15.73:0/3641416287 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6344037070 con 0x7f635c15f2d0 2024-04-26T20:52:16.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.367+0000 7f635a7fc640 1 -- 172.21.15.73:0/3641416287 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6344032440 con 0x7f635c15f2d0 2024-04-26T20:52:16.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.367+0000 7f635a7fc640 1 -- 172.21.15.73:0/3641416287 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6344032810 con 0x7f635c15f2d0 2024-04-26T20:52:16.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.367+0000 7f6361cf1640 1 -- 172.21.15.73:0/3641416287 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f635c15f2d0 msgr2=0x7f635c1604e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:16.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.367+0000 7f6361cf1640 1 --2- 172.21.15.73:0/3641416287 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f635c15f2d0 0x7f635c1604e0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f63440097a0 tx=0x7f634402a180 comp rx=0 tx=0).stop 2024-04-26T20:52:16.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.367+0000 7f6361cf1640 1 -- 172.21.15.73:0/3641416287 shutdown_connections 2024-04-26T20:52:16.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.367+0000 7f6361cf1640 1 --2- 172.21.15.73:0/3641416287 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f635c072700 0x7f635c06f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:16.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.367+0000 7f6361cf1640 1 --2- 172.21.15.73:0/3641416287 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f635c15f2d0 0x7f635c1604e0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:16.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.367+0000 7f6361cf1640 1 -- 172.21.15.73:0/3641416287 >> 172.21.15.73:0/3641416287 conn(0x7f635c0d0be0 msgr2=0x7f635c0d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:16.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.368+0000 7f6361cf1640 1 -- 172.21.15.73:0/3641416287 shutdown_connections 2024-04-26T20:52:16.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.368+0000 7f6361cf1640 1 -- 172.21.15.73:0/3641416287 wait complete. 2024-04-26T20:52:16.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.369+0000 7f6361cf1640 1 Processor -- start 2024-04-26T20:52:16.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.369+0000 7f6361cf1640 1 -- start start 2024-04-26T20:52:16.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.369+0000 7f6361cf1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f635c06fcd0 0x7f635c147a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:16.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.369+0000 7f6361cf1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f635c072700 0x7f635c1f5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:16.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.369+0000 7f6361cf1640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f635c15f2d0 0x7f635c1f59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:16.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.369+0000 7f6361cf1640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f635c073af0 con 0x7f635c06fcd0 2024-04-26T20:52:16.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.369+0000 7f6361cf1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f635c073810 con 0x7f635c072700 2024-04-26T20:52:16.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.369+0000 7f6361cf1640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f635c15f6b0 con 0x7f635c15f2d0 2024-04-26T20:52:16.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.370+0000 7f635b7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f635c06fcd0 0x7f635c147a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:16.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.370+0000 7f635b7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f635c06fcd0 0x7f635c147a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43622/0 (socket says 172.21.15.73:43622) 2024-04-26T20:52:16.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.370+0000 7f635b7fe640 1 -- 172.21.15.73:0/968078660 learned_addr learned my addr 172.21.15.73:0/968078660 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:16.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.370+0000 7f635affd640 1 --2- 172.21.15.73:0/968078660 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f635c072700 0x7f635c1f5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:16.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.370+0000 7f635bfff640 1 --2- 172.21.15.73:0/968078660 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f635c15f2d0 0x7f635c1f59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:16.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.370+0000 7f635b7fe640 1 -- 172.21.15.73:0/968078660 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f635c15f2d0 msgr2=0x7f635c1f59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:16.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.370+0000 7f635b7fe640 1 --2- 172.21.15.73:0/968078660 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f635c15f2d0 0x7f635c1f59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:16.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.370+0000 7f635b7fe640 1 -- 172.21.15.73:0/968078660 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f635c072700 msgr2=0x7f635c1f5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:16.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.370+0000 7f635b7fe640 1 --2- 172.21.15.73:0/968078660 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f635c072700 0x7f635c1f5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:16.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.370+0000 7f635b7fe640 1 -- 172.21.15.73:0/968078660 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f635c0c6d50 con 0x7f635c06fcd0 2024-04-26T20:52:16.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.370+0000 7f635b7fe640 1 --2- 172.21.15.73:0/968078660 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f635c06fcd0 0x7f635c147a10 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f63440097a0 tx=0x7f63440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:16.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.370+0000 7f6353fff640 1 -- 172.21.15.73:0/968078660 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6344040070 con 0x7f635c06fcd0 2024-04-26T20:52:16.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.370+0000 7f6353fff640 1 -- 172.21.15.73:0/968078660 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6344039bf0 con 0x7f635c06fcd0 2024-04-26T20:52:16.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.370+0000 7f6353fff640 1 -- 172.21.15.73:0/968078660 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6344032e40 con 0x7f635c06fcd0 2024-04-26T20:52:16.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.370+0000 7f6361cf1640 1 -- 172.21.15.73:0/968078660 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f635c15c0b0 con 0x7f635c06fcd0 2024-04-26T20:52:16.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.370+0000 7f6361cf1640 1 -- 172.21.15.73:0/968078660 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f635c15c670 con 0x7f635c06fcd0 2024-04-26T20:52:16.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.371+0000 7f6361cf1640 1 -- 172.21.15.73:0/968078660 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f635c069cd0 con 0x7f635c06fcd0 2024-04-26T20:52:16.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.372+0000 7f6353fff640 1 -- 172.21.15.73:0/968078660 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6344004030 con 0x7f635c06fcd0 2024-04-26T20:52:16.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.372+0000 7f6353fff640 1 --2- 172.21.15.73:0/968078660 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f633c06c8f0 0x7f633c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:16.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.373+0000 7f635affd640 1 --2- 172.21.15.73:0/968078660 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f633c06c8f0 0x7f633c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:16.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.373+0000 7f6353fff640 1 -- 172.21.15.73:0/968078660 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5421+0+0 (secure 0 0 0) 0x7f63440adf10 con 0x7f635c06fcd0 2024-04-26T20:52:16.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.373+0000 7f635affd640 1 --2- 172.21.15.73:0/968078660 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f633c06c8f0 0x7f633c06edb0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f635c0c7090 tx=0x7f634c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:16.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.378+0000 7f6353fff640 1 -- 172.21.15.73:0/968078660 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f635c069cd0 con 0x7f635c06fcd0 2024-04-26T20:52:16.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:16.564+0000 7f6361cf1640 1 -- 172.21.15.73:0/968078660 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f635c0694f0 con 0x7f635c06fcd0 2024-04-26T20:52:17.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.376+0000 7f6353fff640 1 -- 172.21.15.73:0/968078660 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v48) ==== 97+0+0 (secure 0 0 0) 0x7f634407bea0 con 0x7f635c06fcd0 2024-04-26T20:52:17.376 INFO:teuthology.orchestra.run.smithi073.stderr: removed all blocklist entries 2024-04-26T20:52:17.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.378+0000 7f6361cf1640 1 -- 172.21.15.73:0/968078660 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f633c06c8f0 msgr2=0x7f633c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:17.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.378+0000 7f6361cf1640 1 --2- 172.21.15.73:0/968078660 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f633c06c8f0 0x7f633c06edb0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f635c0c7090 tx=0x7f634c008040 comp rx=0 tx=0).stop 2024-04-26T20:52:17.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.378+0000 7f6361cf1640 1 -- 172.21.15.73:0/968078660 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f635c06fcd0 msgr2=0x7f635c147a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:17.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.378+0000 7f6361cf1640 1 --2- 172.21.15.73:0/968078660 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f635c06fcd0 0x7f635c147a10 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f63440097a0 tx=0x7f63440046c0 comp rx=0 tx=0).stop 2024-04-26T20:52:17.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.378+0000 7f6361cf1640 1 -- 172.21.15.73:0/968078660 shutdown_connections 2024-04-26T20:52:17.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.378+0000 7f6361cf1640 1 --2- 172.21.15.73:0/968078660 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f633c06c8f0 0x7f633c06edb0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:17.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.378+0000 7f6361cf1640 1 --2- 172.21.15.73:0/968078660 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f635c15f2d0 0x7f635c1f59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:17.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.378+0000 7f6361cf1640 1 --2- 172.21.15.73:0/968078660 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f635c072700 0x7f635c1f5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:17.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.378+0000 7f6361cf1640 1 --2- 172.21.15.73:0/968078660 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f635c06fcd0 0x7f635c147a10 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:17.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.378+0000 7f6361cf1640 1 -- 172.21.15.73:0/968078660 >> 172.21.15.73:0/968078660 conn(0x7f635c0d0be0 msgr2=0x7f635c0b3860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:17.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.379+0000 7f6361cf1640 1 -- 172.21.15.73:0/968078660 shutdown_connections 2024-04-26T20:52:17.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.379+0000 7f6361cf1640 1 -- 172.21.15.73:0/968078660 wait complete. 2024-04-26T20:52:17.393 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-26T20:52:17.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.493+0000 7f294f445640 1 Processor -- start 2024-04-26T20:52:17.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.493+0000 7f294f445640 1 -- start start 2024-04-26T20:52:17.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.494+0000 7f294f445640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f29480b4d40 0x7f29480b6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:17.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.494+0000 7f294f445640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2948072fb0 con 0x7f29480b6c50 2024-04-26T20:52:17.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.494+0000 7f294f445640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2948073290 con 0x7f29480b0b70 2024-04-26T20:52:17.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.494+0000 7f294f445640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f2948073bc0 con 0x7f29480b4d40 2024-04-26T20:52:17.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.494+0000 7f294d9e2640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f29480b6c50 0x7f29480aed90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40100/0 (socket says 172.21.15.73:40100) 2024-04-26T20:52:17.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.494+0000 7f294d9e2640 1 -- 172.21.15.73:0/4006168931 learned_addr learned my addr 172.21.15.73:0/4006168931 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:52:17.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.495+0000 7f294c9e0640 1 --2- 172.21.15.73:0/4006168931 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f29480b4d40 0x7f29480b6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:17.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.495+0000 7f293ffff640 1 -- 172.21.15.73:0/4006168931 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1073234645 0 0) 0x7f2948072fb0 con 0x7f29480b6c50 2024-04-26T20:52:17.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.495+0000 7f293ffff640 1 -- 172.21.15.73:0/4006168931 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2934003520 con 0x7f29480b6c50 2024-04-26T20:52:17.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.495+0000 7f293ffff640 1 -- 172.21.15.73:0/4006168931 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3576911829 0 0) 0x7f2934003520 con 0x7f29480b6c50 2024-04-26T20:52:17.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.495+0000 7f293ffff640 1 -- 172.21.15.73:0/4006168931 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f29480b4d40 msgr2=0x7f29480b6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:17.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.495+0000 7f293ffff640 1 --2- 172.21.15.73:0/4006168931 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f29480b4d40 0x7f29480b6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:17.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.495+0000 7f293ffff640 1 -- 172.21.15.73:0/4006168931 >> v1:172.21.15.120:6789/0 conn(0x7f29480b0b70 legacy=0x7f29480b0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:17.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.495+0000 7f293ffff640 1 -- 172.21.15.73:0/4006168931 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2948073570 con 0x7f29480b6c50 2024-04-26T20:52:17.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.495+0000 7f293ffff640 1 -- 172.21.15.73:0/4006168931 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7f2940001670 con 0x7f29480b6c50 2024-04-26T20:52:17.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.495+0000 7f293ffff640 1 -- 172.21.15.73:0/4006168931 >> v1:172.21.15.73:6789/0 conn(0x7f29480b6c50 legacy=0x7f29480aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:17.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.495+0000 7f293ffff640 1 --2- 172.21.15.73:0/4006168931 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29340039e0 0x7f2934003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:17.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.495+0000 7f293ffff640 1 --2- 172.21.15.73:0/4006168931 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f29480b4d40 0x7f2934004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:17.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.495+0000 7f293ffff640 1 --2- 172.21.15.73:0/4006168931 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2934004d40 0x7f29340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:17.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.495+0000 7f293ffff640 1 -- 172.21.15.73:0/4006168931 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2948073bc0 con 0x7f29340039e0 2024-04-26T20:52:17.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.495+0000 7f293ffff640 1 -- 172.21.15.73:0/4006168931 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f293400ba70 con 0x7f2934004d40 2024-04-26T20:52:17.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.495+0000 7f293ffff640 1 -- 172.21.15.73:0/4006168931 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f293400bc70 con 0x7f29480b4d40 2024-04-26T20:52:17.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.495+0000 7f294d1e1640 1 --2- 172.21.15.73:0/4006168931 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29340039e0 0x7f2934003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:17.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.495+0000 7f294c9e0640 1 --2- 172.21.15.73:0/4006168931 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f29480b4d40 0x7f2934004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:17.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.495+0000 7f294d9e2640 1 --2- 172.21.15.73:0/4006168931 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2934004d40 0x7f29340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:17.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.495+0000 7f294d1e1640 1 -- 172.21.15.73:0/4006168931 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f29480b4d40 msgr2=0x7f2934004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:17.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.495+0000 7f294d1e1640 1 --2- 172.21.15.73:0/4006168931 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f29480b4d40 0x7f2934004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:17.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.495+0000 7f294d1e1640 1 -- 172.21.15.73:0/4006168931 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2934004d40 msgr2=0x7f29340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:17.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.495+0000 7f294d1e1640 1 --2- 172.21.15.73:0/4006168931 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2934004d40 0x7f29340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:17.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.495+0000 7f294d1e1640 1 -- 172.21.15.73:0/4006168931 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2934003520 con 0x7f29340039e0 2024-04-26T20:52:17.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.495+0000 7f294c9e0640 1 --2- 172.21.15.73:0/4006168931 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f29480b4d40 0x7f2934004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:52:17.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.496+0000 7f294d1e1640 1 --2- 172.21.15.73:0/4006168931 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29340039e0 0x7f2934003e20 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f2944001e20 tx=0x7f2934005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb4fb0240c2eb771 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:17.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.496+0000 7f293ffff640 1 -- 172.21.15.73:0/4006168931 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2944001a70 con 0x7f29340039e0 2024-04-26T20:52:17.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.496+0000 7f293ffff640 1 -- 172.21.15.73:0/4006168931 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f294402ae40 con 0x7f29340039e0 2024-04-26T20:52:17.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.496+0000 7f293ffff640 1 -- 172.21.15.73:0/4006168931 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2944003bb0 con 0x7f29340039e0 2024-04-26T20:52:17.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.497+0000 7f294f445640 1 -- 172.21.15.73:0/4006168931 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29340039e0 msgr2=0x7f2934003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:17.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.497+0000 7f294f445640 1 --2- 172.21.15.73:0/4006168931 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29340039e0 0x7f2934003e20 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f2944001e20 tx=0x7f2934005830 comp rx=0 tx=0).stop 2024-04-26T20:52:17.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.497+0000 7f294d9e2640 1 -- 172.21.15.73:0/4006168931 reap_dead start 2024-04-26T20:52:17.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.497+0000 7f294f445640 1 -- 172.21.15.73:0/4006168931 shutdown_connections 2024-04-26T20:52:17.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.497+0000 7f294f445640 1 -- 172.21.15.73:0/4006168931 >> 172.21.15.73:0/4006168931 conn(0x7f29480b39b0 msgr2=0x7f29480b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:17.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.497+0000 7f294f445640 1 -- 172.21.15.73:0/4006168931 shutdown_connections 2024-04-26T20:52:17.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.497+0000 7f294f445640 1 -- 172.21.15.73:0/4006168931 wait complete. 2024-04-26T20:52:17.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.498+0000 7f294f445640 1 Processor -- start 2024-04-26T20:52:17.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.498+0000 7f294f445640 1 -- start start 2024-04-26T20:52:17.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.499+0000 7f294f445640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29340039e0 0x7f2948146cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:17.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.499+0000 7f294f445640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2948149730 0x7f2948161a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:17.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.499+0000 7f294f445640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2948148360 0x7f2948155690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:17.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.499+0000 7f294f445640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f29481604e0 con 0x7f29340039e0 2024-04-26T20:52:17.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.499+0000 7f294f445640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f2948160360 con 0x7f2948149730 2024-04-26T20:52:17.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.499+0000 7f294f445640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f29480b6710 con 0x7f2948148360 2024-04-26T20:52:17.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.499+0000 7f294d1e1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29340039e0 0x7f2948146cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:17.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.499+0000 7f294d1e1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29340039e0 0x7f2948146cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43654/0 (socket says 172.21.15.73:43654) 2024-04-26T20:52:17.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.499+0000 7f294d1e1640 1 -- 172.21.15.73:0/1260035142 learned_addr learned my addr 172.21.15.73:0/1260035142 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:17.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.500+0000 7f294d9e2640 1 --2- 172.21.15.73:0/1260035142 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2948148360 0x7f2948155690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:17.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.500+0000 7f294c9e0640 1 --2- 172.21.15.73:0/1260035142 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2948149730 0x7f2948161a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:17.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.500+0000 7f294d1e1640 1 -- 172.21.15.73:0/1260035142 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2948148360 msgr2=0x7f2948155690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:17.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.500+0000 7f294d1e1640 1 --2- 172.21.15.73:0/1260035142 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2948148360 0x7f2948155690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:17.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.500+0000 7f294d1e1640 1 -- 172.21.15.73:0/1260035142 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2948149730 msgr2=0x7f2948161a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:17.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.500+0000 7f294d1e1640 1 --2- 172.21.15.73:0/1260035142 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2948149730 0x7f2948161a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:17.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.500+0000 7f294d1e1640 1 -- 172.21.15.73:0/1260035142 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29481472c0 con 0x7f29340039e0 2024-04-26T20:52:17.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.500+0000 7f294d1e1640 1 --2- 172.21.15.73:0/1260035142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29340039e0 0x7f2948146cc0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f2944001f40 tx=0x7f29440014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:17.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.500+0000 7f293dffb640 1 -- 172.21.15.73:0/1260035142 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294403f030 con 0x7f29340039e0 2024-04-26T20:52:17.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.500+0000 7f293dffb640 1 -- 172.21.15.73:0/1260035142 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f294402ac40 con 0x7f29340039e0 2024-04-26T20:52:17.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.500+0000 7f294f445640 1 -- 172.21.15.73:0/1260035142 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29481475a0 con 0x7f29340039e0 2024-04-26T20:52:17.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.500+0000 7f293dffb640 1 -- 172.21.15.73:0/1260035142 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2944007db0 con 0x7f29340039e0 2024-04-26T20:52:17.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.500+0000 7f294f445640 1 -- 172.21.15.73:0/1260035142 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2948147b60 con 0x7f29340039e0 2024-04-26T20:52:17.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.501+0000 7f294f445640 1 -- 172.21.15.73:0/1260035142 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29480694f0 con 0x7f29340039e0 2024-04-26T20:52:17.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.502+0000 7f293dffb640 1 -- 172.21.15.73:0/1260035142 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2944042070 con 0x7f29340039e0 2024-04-26T20:52:17.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.502+0000 7f293dffb640 1 --2- 172.21.15.73:0/1260035142 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f292c06c920 0x7f292c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:17.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.502+0000 7f293dffb640 1 -- 172.21.15.73:0/1260035142 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5249+0+0 (secure 0 0 0) 0x7f29440ab3f0 con 0x7f29340039e0 2024-04-26T20:52:17.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.502+0000 7f294c9e0640 1 --2- 172.21.15.73:0/1260035142 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f292c06c920 0x7f292c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:17.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.503+0000 7f294c9e0640 1 --2- 172.21.15.73:0/1260035142 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f292c06c920 0x7f292c06ede0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f29480d2940 tx=0x7f2938008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:17.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.507+0000 7f293dffb640 1 -- 172.21.15.73:0/1260035142 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2944079500 con 0x7f29340039e0 2024-04-26T20:52:17.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.751+0000 7f294f445640 1 -- 172.21.15.73:0/1260035142 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f294806fbb0 con 0x7f29340039e0 2024-04-26T20:52:17.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.752+0000 7f293dffb640 1 -- 172.21.15.73:0/1260035142 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) ==== 80+0+5304 (secure 0 0 0) 0x7f294407d800 con 0x7f29340039e0 2024-04-26T20:52:17.753 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:52:17.753 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-26T20:52:17.753 INFO:teuthology.orchestra.run.smithi073.stdout: "auth_dump": [ 2024-04-26T20:52:17.753 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-26T20:52:17.753 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.a", 2024-04-26T20:52:17.753 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQBUEyxmAE6mGhAAe52HRhOCVM94+tYM6+D8CQ==", 2024-04-26T20:52:17.753 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-26T20:52:17.753 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-26T20:52:17.754 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-26T20:52:17.754 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-26T20:52:17.754 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-26T20:52:17.754 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-26T20:52:17.754 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-26T20:52:17.754 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-26T20:52:17.754 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.b", 2024-04-26T20:52:17.754 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQBUEyxmS7XbJRAAqlMo8v1BqQcHUrDIScQh/Q==", 2024-04-26T20:52:17.754 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-26T20:52:17.754 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-26T20:52:17.754 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-26T20:52:17.754 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-26T20:52:17.754 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-26T20:52:17.754 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-26T20:52:17.755 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-26T20:52:17.755 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-26T20:52:17.755 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.c", 2024-04-26T20:52:17.755 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQBUEyxmpyRtIRAAfZQyLswSVyCCJ/G/i2R+XQ==", 2024-04-26T20:52:17.755 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-26T20:52:17.755 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-26T20:52:17.755 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-26T20:52:17.755 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-26T20:52:17.755 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-26T20:52:17.756 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-26T20:52:17.756 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-26T20:52:17.756 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-26T20:52:17.756 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.d", 2024-04-26T20:52:17.756 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQBUEyxmzMqgLBAAF33EqesRSqAWg392qc9ZIQ==", 2024-04-26T20:52:17.756 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-26T20:52:17.756 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-26T20:52:17.756 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-26T20:52:17.756 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-26T20:52:17.756 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-26T20:52:17.756 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-26T20:52:17.756 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-26T20:52:17.756 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-26T20:52:17.756 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.0", 2024-04-26T20:52:17.756 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQBYEyxmGSyTCRAAHyan+CyQu5SN2n3m7nvHKg==", 2024-04-26T20:52:17.757 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-26T20:52:17.757 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-26T20:52:17.757 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-26T20:52:17.757 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-26T20:52:17.757 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-26T20:52:17.757 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-26T20:52:17.757 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-26T20:52:17.757 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.1", 2024-04-26T20:52:17.757 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQBbEyxmD2XdBxAA80mAzCXbmZGi80w3Dx6CHg==", 2024-04-26T20:52:17.757 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-26T20:52:17.758 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-26T20:52:17.758 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-26T20:52:17.758 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-26T20:52:17.758 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-26T20:52:17.758 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-26T20:52:17.758 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-26T20:52:17.758 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.2", 2024-04-26T20:52:17.758 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQBeEyxmxfWzBRAAufFGKiy5pPW++6QRW3PHgQ==", 2024-04-26T20:52:17.758 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-26T20:52:17.758 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-26T20:52:17.758 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-26T20:52:17.758 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-26T20:52:17.758 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-26T20:52:17.758 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-26T20:52:17.759 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-26T20:52:17.759 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.3", 2024-04-26T20:52:17.759 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQBhEyxmT1dmAxAANZmJAcTSTsZUotOhTQiTKA==", 2024-04-26T20:52:17.759 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-26T20:52:17.759 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-26T20:52:17.759 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-26T20:52:17.759 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-26T20:52:17.759 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-26T20:52:17.759 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-26T20:52:17.759 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-26T20:52:17.759 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.4", 2024-04-26T20:52:17.759 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQBnEyxmjwN8ChAA4uHxtDWrxi+NX5t0VCNGag==", 2024-04-26T20:52:17.759 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-26T20:52:17.759 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-26T20:52:17.760 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-26T20:52:17.760 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-26T20:52:17.760 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-26T20:52:17.760 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-26T20:52:17.760 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-26T20:52:17.760 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.5", 2024-04-26T20:52:17.760 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQBqEyxmnCEwCBAAGsOlPpl0/bxbbcDlxufpBw==", 2024-04-26T20:52:17.760 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-26T20:52:17.760 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-26T20:52:17.760 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-26T20:52:17.760 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-26T20:52:17.760 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-26T20:52:17.760 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-26T20:52:17.760 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-26T20:52:17.761 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.6", 2024-04-26T20:52:17.761 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQBtEyxmRmZ5BhAAG/JdOd2r48xb8wHeXJNK0A==", 2024-04-26T20:52:17.761 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-26T20:52:17.761 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-26T20:52:17.761 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-26T20:52:17.761 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-26T20:52:17.761 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-26T20:52:17.761 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-26T20:52:17.761 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-26T20:52:17.761 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.7", 2024-04-26T20:52:17.761 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQBwEyxmtK3mBhAAEe5+8WbXSYIWx2YstWwE6g==", 2024-04-26T20:52:17.761 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-26T20:52:17.761 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2024-04-26T20:52:17.761 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2024-04-26T20:52:17.761 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-26T20:52:17.762 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-26T20:52:17.762 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-26T20:52:17.762 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-26T20:52:17.762 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.0", 2024-04-26T20:52:17.762 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQBUEyxmch+BMBAAap3kvW3+/FlCuAkhRtbWxQ==", 2024-04-26T20:52:17.762 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-26T20:52:17.762 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-26T20:52:17.762 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow r", 2024-04-26T20:52:17.762 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-26T20:52:17.762 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-26T20:52:17.762 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-26T20:52:17.762 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-26T20:52:17.762 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.1", 2024-04-26T20:52:17.763 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQBUEyxmg2vgOBAAPAq4hzJxsRywJoOawl7r7g==", 2024-04-26T20:52:17.763 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-26T20:52:17.763 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-26T20:52:17.763 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2024-04-26T20:52:17.763 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2024-04-26T20:52:17.763 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2024-04-26T20:52:17.763 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-26T20:52:17.763 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-26T20:52:17.763 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-26T20:52:17.763 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.2", 2024-04-26T20:52:17.763 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQBUEyxmHJ7fNRAAbjIeTP+R5KsjQ5IYlmpqFg==", 2024-04-26T20:52:17.763 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-26T20:52:17.763 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-26T20:52:17.763 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2024-04-26T20:52:17.764 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2024-04-26T20:52:17.764 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2024-04-26T20:52:17.764 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-26T20:52:17.764 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-26T20:52:17.764 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-26T20:52:17.764 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.3", 2024-04-26T20:52:17.764 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQBVEyxmt+GfAhAA5qvKp2+NlXcGj975bUQS9Q==", 2024-04-26T20:52:17.764 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-26T20:52:17.764 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2024-04-26T20:52:17.764 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2024-04-26T20:52:17.764 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2024-04-26T20:52:17.764 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2024-04-26T20:52:17.764 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-26T20:52:17.765 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-26T20:52:17.765 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-26T20:52:17.765 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.admin", 2024-04-26T20:52:17.765 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQBUEyxmPhJRABAAwL7GwQXnfU+Erheps382PQ==", 2024-04-26T20:52:17.765 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-26T20:52:17.765 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2024-04-26T20:52:17.765 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2024-04-26T20:52:17.765 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2024-04-26T20:52:17.765 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-26T20:52:17.765 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-26T20:52:17.765 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-26T20:52:17.765 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-26T20:52:17.765 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mgr.x", 2024-04-26T20:52:17.766 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQBUEyxmLbWjFxAAo2evZCpMUZUHkU6N+eDa4A==", 2024-04-26T20:52:17.766 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-26T20:52:17.766 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2024-04-26T20:52:17.766 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile mgr", 2024-04-26T20:52:17.766 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-26T20:52:17.766 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-26T20:52:17.766 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-26T20:52:17.766 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-26T20:52:17.766 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mgr.y", 2024-04-26T20:52:17.766 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQBUEyxmZnhRFBAAB2V6HQEbpoUNUbAflZok8A==", 2024-04-26T20:52:17.766 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2024-04-26T20:52:17.766 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2024-04-26T20:52:17.766 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile mgr", 2024-04-26T20:52:17.766 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2024-04-26T20:52:17.766 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-26T20:52:17.767 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-26T20:52:17.767 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-26T20:52:17.767 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-26T20:52:17.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.754+0000 7f294f445640 1 -- 172.21.15.73:0/1260035142 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f292c06c920 msgr2=0x7f292c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:17.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.754+0000 7f294f445640 1 --2- 172.21.15.73:0/1260035142 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f292c06c920 0x7f292c06ede0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f29480d2940 tx=0x7f2938008040 comp rx=0 tx=0).stop 2024-04-26T20:52:17.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.754+0000 7f294f445640 1 -- 172.21.15.73:0/1260035142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29340039e0 msgr2=0x7f2948146cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:17.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.754+0000 7f294f445640 1 --2- 172.21.15.73:0/1260035142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29340039e0 0x7f2948146cc0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f2944001f40 tx=0x7f29440014a0 comp rx=0 tx=0).stop 2024-04-26T20:52:17.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.754+0000 7f294f445640 1 -- 172.21.15.73:0/1260035142 shutdown_connections 2024-04-26T20:52:17.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.754+0000 7f294f445640 1 --2- 172.21.15.73:0/1260035142 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f292c06c920 0x7f292c06ede0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:17.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.754+0000 7f294f445640 1 --2- 172.21.15.73:0/1260035142 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f2948148360 0x7f2948155690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:17.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.754+0000 7f294f445640 1 --2- 172.21.15.73:0/1260035142 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2948149730 0x7f2948161a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:17.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.754+0000 7f294f445640 1 --2- 172.21.15.73:0/1260035142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29340039e0 0x7f2948146cc0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:17.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.754+0000 7f294f445640 1 -- 172.21.15.73:0/1260035142 >> 172.21.15.73:0/1260035142 conn(0x7f29480b39b0 msgr2=0x7f294806c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:17.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.754+0000 7f294f445640 1 -- 172.21.15.73:0/1260035142 shutdown_connections 2024-04-26T20:52:17.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.754+0000 7f294f445640 1 -- 172.21.15.73:0/1260035142 wait complete. 2024-04-26T20:52:17.769 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:52:17.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.864+0000 7f48fb047640 1 Processor -- start 2024-04-26T20:52:17.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.864+0000 7f48fb047640 1 -- start start 2024-04-26T20:52:17.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.865+0000 7f48fb047640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f48f406f790 0x7f48f406fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:17.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.865+0000 7f48fb047640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48f4156840 con 0x7f48f4156460 2024-04-26T20:52:17.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.865+0000 7f48fb047640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48f40cbf20 con 0x7f48f4072c50 2024-04-26T20:52:17.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.865+0000 7f48fb047640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f48f40cc200 con 0x7f48f406f790 2024-04-26T20:52:17.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.865+0000 7f48f8de3640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f48f4156460 0x7f48f40cb4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40106/0 (socket says 172.21.15.73:40106) 2024-04-26T20:52:17.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.865+0000 7f48f8de3640 1 -- 172.21.15.73:0/2578911806 learned_addr learned my addr 172.21.15.73:0/2578911806 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:52:17.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.865+0000 7f48f3fff640 1 --2- 172.21.15.73:0/2578911806 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f48f406f790 0x7f48f406fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:17.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.865+0000 7f48f37fe640 1 -- 172.21.15.73:0/2578911806 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2262006081 0 0) 0x7f48f4156840 con 0x7f48f4156460 2024-04-26T20:52:17.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.865+0000 7f48f37fe640 1 -- 172.21.15.73:0/2578911806 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f48dc003520 con 0x7f48f4156460 2024-04-26T20:52:17.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.866+0000 7f48f37fe640 1 -- 172.21.15.73:0/2578911806 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1308131592 0 0) 0x7f48f40cbf20 con 0x7f48f4072c50 2024-04-26T20:52:17.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.866+0000 7f48f37fe640 1 -- 172.21.15.73:0/2578911806 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f48f4156840 con 0x7f48f4072c50 2024-04-26T20:52:17.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.866+0000 7f48f3fff640 1 -- 172.21.15.73:0/2578911806 >> v1:172.21.15.120:6789/0 conn(0x7f48f4072c50 legacy=0x7f48f40730c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:17.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.866+0000 7f48f3fff640 1 -- 172.21.15.73:0/2578911806 >> v1:172.21.15.73:6789/0 conn(0x7f48f4156460 legacy=0x7f48f40cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:17.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.866+0000 7f48f3fff640 1 -- 172.21.15.73:0/2578911806 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48f40cc4e0 con 0x7f48f406f790 2024-04-26T20:52:17.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.866+0000 7f48f3fff640 1 --2- 172.21.15.73:0/2578911806 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f48f406f790 0x7f48f406fc10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f48e80097d0 tx=0x7f48e802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11a64a64322300a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:17.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.867+0000 7f48f37fe640 1 -- 172.21.15.73:0/2578911806 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e8037070 con 0x7f48f406f790 2024-04-26T20:52:17.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.867+0000 7f48f37fe640 1 -- 172.21.15.73:0/2578911806 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48e8032440 con 0x7f48f406f790 2024-04-26T20:52:17.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.867+0000 7f48f37fe640 1 -- 172.21.15.73:0/2578911806 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e80327f0 con 0x7f48f406f790 2024-04-26T20:52:17.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.867+0000 7f48fb047640 1 -- 172.21.15.73:0/2578911806 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f48f406f790 msgr2=0x7f48f406fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:17.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.867+0000 7f48fb047640 1 --2- 172.21.15.73:0/2578911806 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f48f406f790 0x7f48f406fc10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f48e80097d0 tx=0x7f48e802a210 comp rx=0 tx=0).stop 2024-04-26T20:52:17.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.867+0000 7f48fb047640 1 -- 172.21.15.73:0/2578911806 shutdown_connections 2024-04-26T20:52:17.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.867+0000 7f48fb047640 1 --2- 172.21.15.73:0/2578911806 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f48f406f790 0x7f48f406fc10 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:17.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.867+0000 7f48fb047640 1 -- 172.21.15.73:0/2578911806 >> 172.21.15.73:0/2578911806 conn(0x7f48f40d0be0 msgr2=0x7f48f40d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:17.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.867+0000 7f48fb047640 1 -- 172.21.15.73:0/2578911806 shutdown_connections 2024-04-26T20:52:17.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.867+0000 7f48fb047640 1 -- 172.21.15.73:0/2578911806 wait complete. 2024-04-26T20:52:17.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.868+0000 7f48fb047640 1 Processor -- start 2024-04-26T20:52:17.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.868+0000 7f48fb047640 1 -- start start 2024-04-26T20:52:17.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.869+0000 7f48fb047640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48f406f790 0x7f48f41f09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:17.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.869+0000 7f48fb047640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f48f4072c50 0x7f48f40d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:17.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.869+0000 7f48fb047640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f48f4156460 0x7f48f40c9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:17.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.869+0000 7f48fb047640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f48f40cbbe0 con 0x7f48f406f790 2024-04-26T20:52:17.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.869+0000 7f48fb047640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f48f4072700 con 0x7f48f4156460 2024-04-26T20:52:17.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.869+0000 7f48fb047640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f48f406e260 con 0x7f48f4072c50 2024-04-26T20:52:17.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.869+0000 7f48f8de3640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48f406f790 0x7f48f41f09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:17.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.869+0000 7f48f8de3640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48f406f790 0x7f48f41f09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43662/0 (socket says 172.21.15.73:43662) 2024-04-26T20:52:17.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.869+0000 7f48f8de3640 1 -- 172.21.15.73:0/1757477125 learned_addr learned my addr 172.21.15.73:0/1757477125 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:17.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.869+0000 7f48f3fff640 1 --2- 172.21.15.73:0/1757477125 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f48f4072c50 0x7f48f40d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:17.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.869+0000 7f48f95e4640 1 --2- 172.21.15.73:0/1757477125 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f48f4156460 0x7f48f40c9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:17.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.870+0000 7f48f8de3640 1 -- 172.21.15.73:0/1757477125 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f48f4072c50 msgr2=0x7f48f40d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:17.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.870+0000 7f48f8de3640 1 --2- 172.21.15.73:0/1757477125 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f48f4072c50 0x7f48f40d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:17.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.870+0000 7f48f8de3640 1 -- 172.21.15.73:0/1757477125 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f48f4156460 msgr2=0x7f48f40c9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:17.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.870+0000 7f48f8de3640 1 --2- 172.21.15.73:0/1757477125 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f48f4156460 0x7f48f40c9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:17.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.870+0000 7f48f8de3640 1 -- 172.21.15.73:0/1757477125 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48f4154c30 con 0x7f48f406f790 2024-04-26T20:52:17.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.870+0000 7f48f8de3640 1 --2- 172.21.15.73:0/1757477125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48f406f790 0x7f48f41f09f0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f48e400c500 tx=0x7f48e400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:17.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.870+0000 7f48f17fa640 1 -- 172.21.15.73:0/1757477125 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e4017070 con 0x7f48f406f790 2024-04-26T20:52:17.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.870+0000 7f48f17fa640 1 -- 172.21.15.73:0/1757477125 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48e4005ce0 con 0x7f48f406f790 2024-04-26T20:52:17.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.870+0000 7f48f17fa640 1 -- 172.21.15.73:0/1757477125 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e40072f0 con 0x7f48f406f790 2024-04-26T20:52:17.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.870+0000 7f48fb047640 1 -- 172.21.15.73:0/1757477125 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48f4154f10 con 0x7f48f406f790 2024-04-26T20:52:17.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.870+0000 7f48fb047640 1 -- 172.21.15.73:0/1757477125 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48f41554d0 con 0x7f48f406f790 2024-04-26T20:52:17.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.871+0000 7f48fb047640 1 -- 172.21.15.73:0/1757477125 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48f40c4690 con 0x7f48f406f790 2024-04-26T20:52:17.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.871+0000 7f48f17fa640 1 -- 172.21.15.73:0/1757477125 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f48e401a070 con 0x7f48f406f790 2024-04-26T20:52:17.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.872+0000 7f48f17fa640 1 --2- 172.21.15.73:0/1757477125 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f48e006c8f0 0x7f48e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:17.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.872+0000 7f48f17fa640 1 -- 172.21.15.73:0/1757477125 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5249+0+0 (secure 0 0 0) 0x7f48e408ac90 con 0x7f48f406f790 2024-04-26T20:52:17.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.872+0000 7f48f3fff640 1 --2- 172.21.15.73:0/1757477125 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f48e006c8f0 0x7f48e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:17.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.872+0000 7f48f3fff640 1 --2- 172.21.15.73:0/1757477125 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f48e006c8f0 0x7f48e006edb0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f48e8034000 tx=0x7f48e802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:17.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:17.876+0000 7f48f17fa640 1 -- 172.21.15.73:0/1757477125 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f48e4058cc0 con 0x7f48f406f790 2024-04-26T20:52:18.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.054+0000 7f48fb047640 1 -- 172.21.15.73:0/1757477125 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f48f40cafa0 con 0x7f48f406f790 2024-04-26T20:52:18.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.054+0000 7f48f17fa640 1 -- 172.21.15.73:0/1757477125 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) ==== 74+0+13164 (secure 0 0 0) 0x7f48e405cfc0 con 0x7f48f406f790 2024-04-26T20:52:18.055 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:52:18.056 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":48,"fsid":"fbd3f548-b547-4713-856b-09f98f5d1764","created":"2024-04-26T20:50:07.876786+0000","modified":"2024-04-26T20:52:17.373305+0000","last_up_change":"2024-04-26T20:50:19.082741+0000","last_in_change":"2024-04-26T20:50:10.018400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:50:19.993731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d6f4f3-214a-4392-a02f-c33cbcab3c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6809","nonce":3461705124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6811","nonce":3461705124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6815","nonce":3461705124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6813","nonce":3461705124}]},"public_addr":"172.21.15.73:6809/3461705124","cluster_addr":"172.21.15.73:6811/3461705124","heartbeat_back_addr":"172.21.15.73:6815/3461705124","heartbeat_front_addr":"172.21.15.73:6813/3461705124","state":["exists","up"]},{"osd":1,"uuid":"786bf8ca-6ddc-4b3c-be6c-9152398bd5fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6817","nonce":2827619403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6819","nonce":2827619403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6825","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6827","nonce":2827619403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6821","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6823","nonce":2827619403}]},"public_addr":"172.21.15.73:6817/2827619403","cluster_addr":"172.21.15.73:6819/2827619403","heartbeat_back_addr":"172.21.15.73:6827/2827619403","heartbeat_front_addr":"172.21.15.73:6823/2827619403","state":["exists","up"]},{"osd":2,"uuid":"90963b1a-5437-4ef8-ad8b-6b74c534eeb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6801","nonce":2542681064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6803","nonce":2542681064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6808","nonce":2542681064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6805","nonce":2542681064}]},"public_addr":"172.21.15.73:6801/2542681064","cluster_addr":"172.21.15.73:6803/2542681064","heartbeat_back_addr":"172.21.15.73:6808/2542681064","heartbeat_front_addr":"172.21.15.73:6805/2542681064","state":["exists","up"]},{"osd":3,"uuid":"bd3d2148-6e38-41b4-a453-fe6e3b8b4ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6822","nonce":1604042718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6826","nonce":1604042718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6831","nonce":1604042718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6829","nonce":1604042718}]},"public_addr":"172.21.15.73:6822/1604042718","cluster_addr":"172.21.15.73:6826/1604042718","heartbeat_back_addr":"172.21.15.73:6831/1604042718","heartbeat_front_addr":"172.21.15.73:6829/1604042718","state":["exists","up"]},{"osd":4,"uuid":"e8668dc8-a23f-46d2-8311-cb882da08e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6817","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6819","nonce":3591055561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6824","nonce":3591055561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6831","nonce":3591055561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6828","nonce":3591055561}]},"public_addr":"172.21.15.120:6819/3591055561","cluster_addr":"172.21.15.120:6824/3591055561","heartbeat_back_addr":"172.21.15.120:6831/3591055561","heartbeat_front_addr":"172.21.15.120:6828/3591055561","state":["exists","up"]},{"osd":5,"uuid":"cbac0bdb-96bd-4852-bcce-e564495c3fcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6809","nonce":819450433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6811","nonce":819450433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6815","nonce":819450433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6813","nonce":819450433}]},"public_addr":"172.21.15.120:6809/819450433","cluster_addr":"172.21.15.120:6811/819450433","heartbeat_back_addr":"172.21.15.120:6815/819450433","heartbeat_front_addr":"172.21.15.120:6813/819450433","state":["exists","up"]},{"osd":6,"uuid":"1884b2a8-569f-4272-96a9-f25a09364dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6801","nonce":3946210727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6803","nonce":3946210727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6807","nonce":3946210727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6805","nonce":3946210727}]},"public_addr":"172.21.15.120:6801/3946210727","cluster_addr":"172.21.15.120:6803/3946210727","heartbeat_back_addr":"172.21.15.120:6807/3946210727","heartbeat_front_addr":"172.21.15.120:6805/3946210727","state":["exists","up"]},{"osd":7,"uuid":"c0c3bf6d-a4c5-4f1e-aa2d-330951830666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6818","nonce":2915574674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6821","nonce":2915574674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6827","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6829","nonce":2915574674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6823","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6825","nonce":2915574674}]},"public_addr":"172.21.15.120:6818/2915574674","cluster_addr":"172.21.15.120:6821/2915574674","heartbeat_back_addr":"172.21.15.120:6829/2915574674","heartbeat_front_addr":"172.21.15.120:6825/2915574674","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.218216+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.208710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.112705+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.182789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.164324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.174212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.225396+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.261559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:52:18.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.056+0000 7f48fb047640 1 -- 172.21.15.73:0/1757477125 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f48e006c8f0 msgr2=0x7f48e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:18.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.056+0000 7f48fb047640 1 --2- 172.21.15.73:0/1757477125 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f48e006c8f0 0x7f48e006edb0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f48e8034000 tx=0x7f48e802e000 comp rx=0 tx=0).stop 2024-04-26T20:52:18.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.056+0000 7f48fb047640 1 -- 172.21.15.73:0/1757477125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48f406f790 msgr2=0x7f48f41f09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:18.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.056+0000 7f48fb047640 1 --2- 172.21.15.73:0/1757477125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48f406f790 0x7f48f41f09f0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f48e400c500 tx=0x7f48e400c9c0 comp rx=0 tx=0).stop 2024-04-26T20:52:18.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.056+0000 7f48fb047640 1 -- 172.21.15.73:0/1757477125 shutdown_connections 2024-04-26T20:52:18.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.056+0000 7f48fb047640 1 --2- 172.21.15.73:0/1757477125 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f48e006c8f0 0x7f48e006edb0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:18.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.056+0000 7f48fb047640 1 --2- 172.21.15.73:0/1757477125 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f48f4156460 0x7f48f40c9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:18.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.056+0000 7f48fb047640 1 --2- 172.21.15.73:0/1757477125 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f48f4072c50 0x7f48f40d00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:18.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.056+0000 7f48fb047640 1 --2- 172.21.15.73:0/1757477125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48f406f790 0x7f48f41f09f0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:18.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.057+0000 7f48fb047640 1 -- 172.21.15.73:0/1757477125 >> 172.21.15.73:0/1757477125 conn(0x7f48f40d0be0 msgr2=0x7f48f40b6270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:18.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.057+0000 7f48fb047640 1 -- 172.21.15.73:0/1757477125 shutdown_connections 2024-04-26T20:52:18.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.057+0000 7f48fb047640 1 -- 172.21.15.73:0/1757477125 wait complete. 2024-04-26T20:52:18.068 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:50:19.993731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:52:18.069 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:52:18.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.166+0000 7f4e5f9da640 1 Processor -- start 2024-04-26T20:52:18.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.166+0000 7f4e5f9da640 1 -- start start 2024-04-26T20:52:18.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.167+0000 7f4e5f9da640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4e580d0d00 0x7f4e580d11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:18.169 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.167+0000 7f4e5f9da640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e580d0710 con 0x7f4e58160720 2024-04-26T20:52:18.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.167+0000 7f4e5f9da640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e580d0430 con 0x7f4e580cf180 2024-04-26T20:52:18.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.167+0000 7f4e5f9da640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f4e580d01e0 con 0x7f4e580d0d00 2024-04-26T20:52:18.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.167+0000 7f4e5cf75640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4e580d0d00 0x7f4e580d11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:18.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.167+0000 7f4e5d776640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f4e58160720 0x7f4e580ce620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40110/0 (socket says 172.21.15.73:40110) 2024-04-26T20:52:18.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.167+0000 7f4e5d776640 1 -- 172.21.15.73:0/1659471973 learned_addr learned my addr 172.21.15.73:0/1659471973 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:52:18.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.168+0000 7f4e5cf75640 1 -- 172.21.15.73:0/1659471973 >> v1:172.21.15.120:6789/0 conn(0x7f4e580cf180 legacy=0x7f4e580b2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:18.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.168+0000 7f4e5cf75640 1 -- 172.21.15.73:0/1659471973 >> v1:172.21.15.73:6789/0 conn(0x7f4e58160720 legacy=0x7f4e580ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:18.170 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.168+0000 7f4e5cf75640 1 -- 172.21.15.73:0/1659471973 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e580cdb40 con 0x7f4e580d0d00 2024-04-26T20:52:18.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.168+0000 7f4e5cf75640 1 --2- 172.21.15.73:0/1659471973 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e580d0d00 0x7f4e580d11e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f4e480097a0 tx=0x7f4e4802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5739f0d1dd36eaa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:18.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.168+0000 7f4e4ffff640 1 -- 172.21.15.73:0/1659471973 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e48037070 con 0x7f4e580d0d00 2024-04-26T20:52:18.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.168+0000 7f4e4ffff640 1 -- 172.21.15.73:0/1659471973 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e48032440 con 0x7f4e580d0d00 2024-04-26T20:52:18.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.168+0000 7f4e4ffff640 1 -- 172.21.15.73:0/1659471973 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e48032810 con 0x7f4e580d0d00 2024-04-26T20:52:18.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.169+0000 7f4e5f9da640 1 -- 172.21.15.73:0/1659471973 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e580d0d00 msgr2=0x7f4e580d11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:18.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.169+0000 7f4e5f9da640 1 --2- 172.21.15.73:0/1659471973 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e580d0d00 0x7f4e580d11e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f4e480097a0 tx=0x7f4e4802a1e0 comp rx=0 tx=0).stop 2024-04-26T20:52:18.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.169+0000 7f4e5f9da640 1 -- 172.21.15.73:0/1659471973 shutdown_connections 2024-04-26T20:52:18.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.169+0000 7f4e5f9da640 1 --2- 172.21.15.73:0/1659471973 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e580d0d00 0x7f4e580d11e0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:18.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.169+0000 7f4e5f9da640 1 -- 172.21.15.73:0/1659471973 >> 172.21.15.73:0/1659471973 conn(0x7f4e580d2e00 msgr2=0x7f4e580d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:18.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.169+0000 7f4e5f9da640 1 -- 172.21.15.73:0/1659471973 shutdown_connections 2024-04-26T20:52:18.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.169+0000 7f4e5f9da640 1 -- 172.21.15.73:0/1659471973 wait complete. 2024-04-26T20:52:18.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.170+0000 7f4e5f9da640 1 Processor -- start 2024-04-26T20:52:18.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.170+0000 7f4e5f9da640 1 -- start start 2024-04-26T20:52:18.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.171+0000 7f4e5f9da640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e580cf180 0x7f4e58157370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:18.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.171+0000 7f4e5f9da640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4e580d0d00 0x7f4e581f3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:18.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.171+0000 7f4e5f9da640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e58160720 0x7f4e5815a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:18.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.171+0000 7f4e5f9da640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4e580ced30 con 0x7f4e58160720 2024-04-26T20:52:18.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.171+0000 7f4e5f9da640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4e580ce100 con 0x7f4e580cf180 2024-04-26T20:52:18.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.171+0000 7f4e5f9da640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f4e58160b00 con 0x7f4e580d0d00 2024-04-26T20:52:18.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.171+0000 7f4e5df77640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e58160720 0x7f4e5815a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:18.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.171+0000 7f4e5df77640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e58160720 0x7f4e5815a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43692/0 (socket says 172.21.15.73:43692) 2024-04-26T20:52:18.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.171+0000 7f4e5df77640 1 -- 172.21.15.73:0/593219295 learned_addr learned my addr 172.21.15.73:0/593219295 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:18.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.171+0000 7f4e5d776640 1 --2- 172.21.15.73:0/593219295 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e580cf180 0x7f4e58157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:18.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.171+0000 7f4e5cf75640 1 --2- 172.21.15.73:0/593219295 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4e580d0d00 0x7f4e581f3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:18.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.171+0000 7f4e5df77640 1 -- 172.21.15.73:0/593219295 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4e580d0d00 msgr2=0x7f4e581f3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:18.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.171+0000 7f4e5df77640 1 --2- 172.21.15.73:0/593219295 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4e580d0d00 0x7f4e581f3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:18.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.171+0000 7f4e5df77640 1 -- 172.21.15.73:0/593219295 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e580cf180 msgr2=0x7f4e58157370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:18.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.171+0000 7f4e5df77640 1 --2- 172.21.15.73:0/593219295 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e580cf180 0x7f4e58157370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:18.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.171+0000 7f4e5df77640 1 -- 172.21.15.73:0/593219295 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e580ca8b0 con 0x7f4e58160720 2024-04-26T20:52:18.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.171+0000 7f4e5cf75640 1 --2- 172.21.15.73:0/593219295 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4e580d0d00 0x7f4e581f3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:52:18.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.172+0000 7f4e5df77640 1 --2- 172.21.15.73:0/593219295 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e58160720 0x7f4e5815a110 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f4e5000b410 tx=0x7f4e5000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:18.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.172+0000 7f4e4dffb640 1 -- 172.21.15.73:0/593219295 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e50015030 con 0x7f4e58160720 2024-04-26T20:52:18.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.172+0000 7f4e4dffb640 1 -- 172.21.15.73:0/593219295 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e50004020 con 0x7f4e58160720 2024-04-26T20:52:18.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.172+0000 7f4e4dffb640 1 -- 172.21.15.73:0/593219295 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e50004410 con 0x7f4e58160720 2024-04-26T20:52:18.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.172+0000 7f4e5f9da640 1 -- 172.21.15.73:0/593219295 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e580cab90 con 0x7f4e58160720 2024-04-26T20:52:18.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.172+0000 7f4e5f9da640 1 -- 172.21.15.73:0/593219295 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e581f3840 con 0x7f4e58160720 2024-04-26T20:52:18.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.173+0000 7f4e5f9da640 1 -- 172.21.15.73:0/593219295 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e580694f0 con 0x7f4e58160720 2024-04-26T20:52:18.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.173+0000 7f4e4dffb640 1 -- 172.21.15.73:0/593219295 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4e50018070 con 0x7f4e58160720 2024-04-26T20:52:18.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.174+0000 7f4e4dffb640 1 --2- 172.21.15.73:0/593219295 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f4e3c06c8f0 0x7f4e3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:18.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.174+0000 7f4e5d776640 1 --2- 172.21.15.73:0/593219295 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f4e3c06c8f0 0x7f4e3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:18.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.174+0000 7f4e4dffb640 1 -- 172.21.15.73:0/593219295 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5249+0+0 (secure 0 0 0) 0x7f4e5008b2d0 con 0x7f4e58160720 2024-04-26T20:52:18.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.174+0000 7f4e5d776640 1 --2- 172.21.15.73:0/593219295 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f4e3c06c8f0 0x7f4e3c06edb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4e540012e0 tx=0x7f4e54001a30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:18.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.178+0000 7f4e4dffb640 1 -- 172.21.15.73:0/593219295 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4e50059950 con 0x7f4e58160720 2024-04-26T20:52:18.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.352+0000 7f4e5f9da640 1 -- 172.21.15.73:0/593219295 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4e580b8d80 con 0x7f4e58160720 2024-04-26T20:52:18.354 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.352+0000 7f4e4dffb640 1 -- 172.21.15.73:0/593219295 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v48) ==== 93+0+10 (secure 0 0 0) 0x7f4e50059250 con 0x7f4e58160720 2024-04-26T20:52:18.355 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-26T20:52:18.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.354+0000 7f4e5f9da640 1 -- 172.21.15.73:0/593219295 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f4e3c06c8f0 msgr2=0x7f4e3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:18.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.354+0000 7f4e5f9da640 1 --2- 172.21.15.73:0/593219295 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f4e3c06c8f0 0x7f4e3c06edb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4e540012e0 tx=0x7f4e54001a30 comp rx=0 tx=0).stop 2024-04-26T20:52:18.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.354+0000 7f4e5f9da640 1 -- 172.21.15.73:0/593219295 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e58160720 msgr2=0x7f4e5815a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:18.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.354+0000 7f4e5f9da640 1 --2- 172.21.15.73:0/593219295 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e58160720 0x7f4e5815a110 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f4e5000b410 tx=0x7f4e5000b8d0 comp rx=0 tx=0).stop 2024-04-26T20:52:18.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.354+0000 7f4e5f9da640 1 -- 172.21.15.73:0/593219295 shutdown_connections 2024-04-26T20:52:18.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.354+0000 7f4e5f9da640 1 --2- 172.21.15.73:0/593219295 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e58160720 0x7f4e5815a110 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:18.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.354+0000 7f4e5f9da640 1 --2- 172.21.15.73:0/593219295 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f4e3c06c8f0 0x7f4e3c06edb0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:18.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.354+0000 7f4e5f9da640 1 --2- 172.21.15.73:0/593219295 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4e580d0d00 0x7f4e581f3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:18.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.354+0000 7f4e5f9da640 1 --2- 172.21.15.73:0/593219295 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4e580cf180 0x7f4e58157370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:18.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.354+0000 7f4e5f9da640 1 -- 172.21.15.73:0/593219295 >> 172.21.15.73:0/593219295 conn(0x7f4e580d2e00 msgr2=0x7f4e5815e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:18.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.354+0000 7f4e5f9da640 1 -- 172.21.15.73:0/593219295 shutdown_connections 2024-04-26T20:52:18.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.354+0000 7f4e5f9da640 1 -- 172.21.15.73:0/593219295 wait complete. 2024-04-26T20:52:18.367 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T20:52:18.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.462+0000 7f1f5d0a9640 1 Processor -- start 2024-04-26T20:52:18.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.462+0000 7f1f5d0a9640 1 -- start start 2024-04-26T20:52:18.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.463+0000 7f1f5d0a9640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1f58156460 0x7f1f580cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:18.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.463+0000 7f1f5d0a9640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f1f580cba10 0x7f1f5806f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:18.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.463+0000 7f1f5d0a9640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f58156840 con 0x7f1f5806fcd0 2024-04-26T20:52:18.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.463+0000 7f1f5d0a9640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f1f580cbf20 con 0x7f1f580cba10 2024-04-26T20:52:18.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.463+0000 7f1f5d0a9640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f1f580cc200 con 0x7f1f58156460 2024-04-26T20:52:18.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.463+0000 7f1f4ffff640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f1f580cba10 0x7f1f5806f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:18.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.463+0000 7f1f56d9d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1f58156460 0x7f1f580cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:18.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.463+0000 7f1f5759e640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f1f5806fcd0 0x7f1f5815f980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.73:55236/0 (socket says 172.21.15.73:55236) 2024-04-26T20:52:18.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.463+0000 7f1f5759e640 1 -- 172.21.15.73:0/2448444717 learned_addr learned my addr 172.21.15.73:0/2448444717 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:52:18.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.463+0000 7f1f4ffff640 1 --2- 172.21.15.73:0/2448444717 >> v2:172.21.15.120:3300/0 conn(0x7f1f580cba10 0x7f1f5806f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3300/0 says I am v2:172.21.15.73:35748/0 (socket says 172.21.15.73:35748) 2024-04-26T20:52:18.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.463+0000 7f1f4ffff640 1 -- 172.21.15.73:0/2448444717 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1f58156460 msgr2=0x7f1f580cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:18.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.463+0000 7f1f4ffff640 1 --2- 172.21.15.73:0/2448444717 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1f58156460 0x7f1f580cb4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:18.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.463+0000 7f1f4ffff640 1 -- 172.21.15.73:0/2448444717 >> v1:172.21.15.120:6789/0 conn(0x7f1f5806fcd0 legacy=0x7f1f5815f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:18.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.463+0000 7f1f4ffff640 1 -- 172.21.15.73:0/2448444717 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f580cc4e0 con 0x7f1f580cba10 2024-04-26T20:52:18.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.463+0000 7f1f5659c640 1 -- 172.21.15.73:0/2448444717 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3796327356 0 0) 0x7f1f58156840 con 0x7f1f5806fcd0 2024-04-26T20:52:18.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.463+0000 7f1f56d9d640 1 --2- 172.21.15.73:0/2448444717 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1f58156460 0x7f1f580cb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:52:18.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.464+0000 7f1f4ffff640 1 --2- 172.21.15.73:0/2448444717 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f580cba10 0x7f1f5806f790 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1f400097a0 tx=0x7f1f4002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1caa809389484df server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:18.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.464+0000 7f1f5659c640 1 -- 172.21.15.73:0/2448444717 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f40037070 con 0x7f1f580cba10 2024-04-26T20:52:18.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.464+0000 7f1f5659c640 1 -- 172.21.15.73:0/2448444717 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f40032440 con 0x7f1f580cba10 2024-04-26T20:52:18.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.464+0000 7f1f5659c640 1 -- 172.21.15.73:0/2448444717 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f40032810 con 0x7f1f580cba10 2024-04-26T20:52:18.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.465+0000 7f1f5d0a9640 1 -- 172.21.15.73:0/2448444717 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f580cba10 msgr2=0x7f1f5806f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:18.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.465+0000 7f1f5d0a9640 1 --2- 172.21.15.73:0/2448444717 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f580cba10 0x7f1f5806f790 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1f400097a0 tx=0x7f1f4002a180 comp rx=0 tx=0).stop 2024-04-26T20:52:18.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.465+0000 7f1f5d0a9640 1 -- 172.21.15.73:0/2448444717 shutdown_connections 2024-04-26T20:52:18.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.465+0000 7f1f5d0a9640 1 --2- 172.21.15.73:0/2448444717 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f580cba10 0x7f1f5806f790 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:18.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.465+0000 7f1f5d0a9640 1 --2- 172.21.15.73:0/2448444717 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1f58156460 0x7f1f580cb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:18.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.465+0000 7f1f5d0a9640 1 -- 172.21.15.73:0/2448444717 >> 172.21.15.73:0/2448444717 conn(0x7f1f580d0be0 msgr2=0x7f1f580d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:18.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.465+0000 7f1f5d0a9640 1 -- 172.21.15.73:0/2448444717 shutdown_connections 2024-04-26T20:52:18.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.465+0000 7f1f5d0a9640 1 -- 172.21.15.73:0/2448444717 wait complete. 2024-04-26T20:52:18.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.466+0000 7f1f5d0a9640 1 Processor -- start 2024-04-26T20:52:18.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.466+0000 7f1f5d0a9640 1 -- start start 2024-04-26T20:52:18.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.467+0000 7f1f5d0a9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f5806fcd0 0x7f1f580c9930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:18.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.467+0000 7f1f5d0a9640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f580cba10 0x7f1f581e5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:18.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.467+0000 7f1f5d0a9640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1f58156460 0x7f1f581e4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:18.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.467+0000 7f1f5d0a9640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5806e260 con 0x7f1f5806fcd0 2024-04-26T20:52:18.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.467+0000 7f1f5d0a9640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1f58072700 con 0x7f1f580cba10 2024-04-26T20:52:18.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.467+0000 7f1f5d0a9640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f1f5806e3e0 con 0x7f1f58156460 2024-04-26T20:52:18.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.467+0000 7f1f56d9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f5806fcd0 0x7f1f580c9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:18.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.467+0000 7f1f56d9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f5806fcd0 0x7f1f580c9930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43708/0 (socket says 172.21.15.73:43708) 2024-04-26T20:52:18.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.467+0000 7f1f56d9d640 1 -- 172.21.15.73:0/1619849726 learned_addr learned my addr 172.21.15.73:0/1619849726 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:18.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.467+0000 7f1f4ffff640 1 --2- 172.21.15.73:0/1619849726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f580cba10 0x7f1f581e5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:18.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.467+0000 7f1f5759e640 1 --2- 172.21.15.73:0/1619849726 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1f58156460 0x7f1f581e4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:18.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.467+0000 7f1f56d9d640 1 -- 172.21.15.73:0/1619849726 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1f58156460 msgr2=0x7f1f581e4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:18.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.467+0000 7f1f56d9d640 1 --2- 172.21.15.73:0/1619849726 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1f58156460 0x7f1f581e4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:18.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.467+0000 7f1f56d9d640 1 -- 172.21.15.73:0/1619849726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f580cba10 msgr2=0x7f1f581e5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:18.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.467+0000 7f1f56d9d640 1 --2- 172.21.15.73:0/1619849726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f580cba10 0x7f1f581e5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:18.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.467+0000 7f1f56d9d640 1 -- 172.21.15.73:0/1619849726 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f580cab40 con 0x7f1f5806fcd0 2024-04-26T20:52:18.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.467+0000 7f1f4ffff640 1 --2- 172.21.15.73:0/1619849726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f580cba10 0x7f1f581e5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:52:18.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.467+0000 7f1f56d9d640 1 --2- 172.21.15.73:0/1619849726 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f5806fcd0 0x7f1f580c9930 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f1f4800e990 tx=0x7f1f4800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:18.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.468+0000 7f1f4f7fe640 1 -- 172.21.15.73:0/1619849726 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f48017070 con 0x7f1f5806fcd0 2024-04-26T20:52:18.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.468+0000 7f1f4f7fe640 1 -- 172.21.15.73:0/1619849726 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f4800c9e0 con 0x7f1f5806fcd0 2024-04-26T20:52:18.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.468+0000 7f1f4f7fe640 1 -- 172.21.15.73:0/1619849726 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4800cdb0 con 0x7f1f5806fcd0 2024-04-26T20:52:18.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.468+0000 7f1f5d0a9640 1 -- 172.21.15.73:0/1619849726 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f580ca580 con 0x7f1f5806fcd0 2024-04-26T20:52:18.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.468+0000 7f1f5d0a9640 1 -- 172.21.15.73:0/1619849726 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f580c8ab0 con 0x7f1f5806fcd0 2024-04-26T20:52:18.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.469+0000 7f1f5d0a9640 1 -- 172.21.15.73:0/1619849726 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f580694f0 con 0x7f1f5806fcd0 2024-04-26T20:52:18.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.469+0000 7f1f4f7fe640 1 -- 172.21.15.73:0/1619849726 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1f4801a070 con 0x7f1f5806fcd0 2024-04-26T20:52:18.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.469+0000 7f1f4f7fe640 1 --2- 172.21.15.73:0/1619849726 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1f3806c8f0 0x7f1f3806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:18.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.469+0000 7f1f4f7fe640 1 -- 172.21.15.73:0/1619849726 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5249+0+0 (secure 0 0 0) 0x7f1f4808b730 con 0x7f1f5806fcd0 2024-04-26T20:52:18.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.469+0000 7f1f4ffff640 1 --2- 172.21.15.73:0/1619849726 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1f3806c8f0 0x7f1f3806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:18.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.470+0000 7f1f4ffff640 1 --2- 172.21.15.73:0/1619849726 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1f3806c8f0 0x7f1f3806edb0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1f40034000 tx=0x7f1f4002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:18.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.474+0000 7f1f4f7fe640 1 -- 172.21.15.73:0/1619849726 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1f48059760 con 0x7f1f5806fcd0 2024-04-26T20:52:18.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.652+0000 7f1f5d0a9640 1 -- 172.21.15.73:0/1619849726 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f1f5815f350 con 0x7f1f5806fcd0 2024-04-26T20:52:18.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.653+0000 7f1f4f7fe640 1 -- 172.21.15.73:0/1619849726 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v48) ==== 84+0+65 (secure 0 0 0) 0x7f1f4805da60 con 0x7f1f5806fcd0 2024-04-26T20:52:18.655 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:52:18.655 INFO:teuthology.orchestra.run.smithi073.stdout:[ 2024-04-26T20:52:18.655 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-26T20:52:18.655 INFO:teuthology.orchestra.run.smithi073.stdout: "poolnum": 1, 2024-04-26T20:52:18.655 INFO:teuthology.orchestra.run.smithi073.stdout: "poolname": ".mgr" 2024-04-26T20:52:18.655 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-26T20:52:18.655 INFO:teuthology.orchestra.run.smithi073.stdout:] 2024-04-26T20:52:18.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.654+0000 7f1f5d0a9640 1 -- 172.21.15.73:0/1619849726 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1f3806c8f0 msgr2=0x7f1f3806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:18.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.654+0000 7f1f5d0a9640 1 --2- 172.21.15.73:0/1619849726 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1f3806c8f0 0x7f1f3806edb0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1f40034000 tx=0x7f1f4002e000 comp rx=0 tx=0).stop 2024-04-26T20:52:18.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.654+0000 7f1f5d0a9640 1 -- 172.21.15.73:0/1619849726 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f5806fcd0 msgr2=0x7f1f580c9930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:18.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.654+0000 7f1f5d0a9640 1 --2- 172.21.15.73:0/1619849726 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f5806fcd0 0x7f1f580c9930 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f1f4800e990 tx=0x7f1f4800ee50 comp rx=0 tx=0).stop 2024-04-26T20:52:18.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.655+0000 7f1f5d0a9640 1 -- 172.21.15.73:0/1619849726 shutdown_connections 2024-04-26T20:52:18.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.655+0000 7f1f5d0a9640 1 --2- 172.21.15.73:0/1619849726 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1f3806c8f0 0x7f1f3806edb0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:18.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.655+0000 7f1f5d0a9640 1 --2- 172.21.15.73:0/1619849726 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1f58156460 0x7f1f581e4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:18.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.655+0000 7f1f5d0a9640 1 --2- 172.21.15.73:0/1619849726 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1f580cba10 0x7f1f581e5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:18.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.655+0000 7f1f5d0a9640 1 --2- 172.21.15.73:0/1619849726 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f5806fcd0 0x7f1f580c9930 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:18.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.655+0000 7f1f5d0a9640 1 -- 172.21.15.73:0/1619849726 >> 172.21.15.73:0/1619849726 conn(0x7f1f580d0be0 msgr2=0x7f1f580d0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:18.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.655+0000 7f1f5d0a9640 1 -- 172.21.15.73:0/1619849726 shutdown_connections 2024-04-26T20:52:18.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.655+0000 7f1f5d0a9640 1 -- 172.21.15.73:0/1619849726 wait complete. 2024-04-26T20:52:18.667 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T20:52:18.667 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T20:52:18.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.766+0000 7f3b74a15640 1 Processor -- start 2024-04-26T20:52:18.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.766+0000 7f3b74a15640 1 -- start start 2024-04-26T20:52:18.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.767+0000 7f3b74a15640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3b70070fa0 0x7f3b700713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:18.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.767+0000 7f3b74a15640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b700718f0 con 0x7f3b70159860 2024-04-26T20:52:18.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.767+0000 7f3b74a15640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b700b6cc0 con 0x7f3b70073680 2024-04-26T20:52:18.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.767+0000 7f3b74a15640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f3b7015f830 con 0x7f3b70070fa0 2024-04-26T20:52:18.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.767+0000 7f3b6e59c640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f3b70159860 0x7f3b70160290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40118/0 (socket says 172.21.15.73:40118) 2024-04-26T20:52:18.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.767+0000 7f3b6e59c640 1 -- 172.21.15.73:0/4224966131 learned_addr learned my addr 172.21.15.73:0/4224966131 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:52:18.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.767+0000 7f3b6ed9d640 1 --2- 172.21.15.73:0/4224966131 >> v2:172.21.15.73:3300/0 conn(0x7f3b70070fa0 0x7f3b700713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:18.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.768+0000 7f3b6d59a640 1 -- 172.21.15.73:0/4224966131 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1015500427 0 0) 0x7f3b60001490 con 0x7f3b70159860 2024-04-26T20:52:18.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.768+0000 7f3b6d59a640 1 -- 172.21.15.73:0/4224966131 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b58003520 con 0x7f3b70159860 2024-04-26T20:52:18.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.768+0000 7f3b6ed9d640 1 -- 172.21.15.73:0/4224966131 >> v1:172.21.15.120:6789/0 conn(0x7f3b70073680 legacy=0x7f3b700b63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:18.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.768+0000 7f3b6ed9d640 1 -- 172.21.15.73:0/4224966131 >> v1:172.21.15.73:6789/0 conn(0x7f3b70159860 legacy=0x7f3b70160290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:18.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.768+0000 7f3b6ed9d640 1 -- 172.21.15.73:0/4224966131 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b700b6fa0 con 0x7f3b70070fa0 2024-04-26T20:52:18.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.768+0000 7f3b6ed9d640 1 --2- 172.21.15.73:0/4224966131 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b70070fa0 0x7f3b700713b0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f3b680097a0 tx=0x7f3b6802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1319d6ebbe733977 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:18.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.768+0000 7f3b6d59a640 1 -- 172.21.15.73:0/4224966131 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b68037070 con 0x7f3b70070fa0 2024-04-26T20:52:18.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.769+0000 7f3b6d59a640 1 -- 172.21.15.73:0/4224966131 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b6802ae50 con 0x7f3b70070fa0 2024-04-26T20:52:18.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.769+0000 7f3b6d59a640 1 -- 172.21.15.73:0/4224966131 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b68032640 con 0x7f3b70070fa0 2024-04-26T20:52:18.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.769+0000 7f3b74a15640 1 -- 172.21.15.73:0/4224966131 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b70070fa0 msgr2=0x7f3b700713b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:18.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.769+0000 7f3b74a15640 1 --2- 172.21.15.73:0/4224966131 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b70070fa0 0x7f3b700713b0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f3b680097a0 tx=0x7f3b6802a140 comp rx=0 tx=0).stop 2024-04-26T20:52:18.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.769+0000 7f3b74a15640 1 -- 172.21.15.73:0/4224966131 shutdown_connections 2024-04-26T20:52:18.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.769+0000 7f3b74a15640 1 --2- 172.21.15.73:0/4224966131 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b70070fa0 0x7f3b700713b0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:18.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.769+0000 7f3b74a15640 1 -- 172.21.15.73:0/4224966131 >> 172.21.15.73:0/4224966131 conn(0x7f3b700d1d60 msgr2=0x7f3b700d2140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:18.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.769+0000 7f3b74a15640 1 -- 172.21.15.73:0/4224966131 shutdown_connections 2024-04-26T20:52:18.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.769+0000 7f3b74a15640 1 -- 172.21.15.73:0/4224966131 wait complete. 2024-04-26T20:52:18.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.770+0000 7f3b74a15640 1 Processor -- start 2024-04-26T20:52:18.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.770+0000 7f3b74a15640 1 -- start start 2024-04-26T20:52:18.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.771+0000 7f3b74a15640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b70070fa0 0x7f3b70156690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:18.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.771+0000 7f3b74a15640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b70073680 0x7f3b700affb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:18.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.771+0000 7f3b74a15640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3b70159860 0x7f3b70153a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:18.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.771+0000 7f3b74a15640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3b7006ed70 con 0x7f3b70073680 2024-04-26T20:52:18.774 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.771+0000 7f3b74a15640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3b70159c40 con 0x7f3b70070fa0 2024-04-26T20:52:18.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.771+0000 7f3b74a15640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f3b7006eef0 con 0x7f3b70159860 2024-04-26T20:52:18.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.771+0000 7f3b6dd9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b70073680 0x7f3b700affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:18.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.771+0000 7f3b6dd9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b70073680 0x7f3b700affb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43724/0 (socket says 172.21.15.73:43724) 2024-04-26T20:52:18.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.771+0000 7f3b6dd9b640 1 -- 172.21.15.73:0/2884667816 learned_addr learned my addr 172.21.15.73:0/2884667816 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:18.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.771+0000 7f3b6dd9b640 1 -- 172.21.15.73:0/2884667816 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3b70159860 msgr2=0x7f3b70153a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:18.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.771+0000 7f3b6dd9b640 1 --2- 172.21.15.73:0/2884667816 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3b70159860 0x7f3b70153a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:18.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.771+0000 7f3b6dd9b640 1 -- 172.21.15.73:0/2884667816 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b70070fa0 msgr2=0x7f3b70156690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:18.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.771+0000 7f3b6dd9b640 1 --2- 172.21.15.73:0/2884667816 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b70070fa0 0x7f3b70156690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:18.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.771+0000 7f3b6dd9b640 1 -- 172.21.15.73:0/2884667816 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b700b04f0 con 0x7f3b70073680 2024-04-26T20:52:18.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.772+0000 7f3b6dd9b640 1 --2- 172.21.15.73:0/2884667816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b70073680 0x7f3b700affb0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f3b6400c400 tx=0x7f3b6400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:18.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.772+0000 7f3b5effd640 1 -- 172.21.15.73:0/2884667816 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b64011790 con 0x7f3b70073680 2024-04-26T20:52:18.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.772+0000 7f3b5effd640 1 -- 172.21.15.73:0/2884667816 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b64017070 con 0x7f3b70073680 2024-04-26T20:52:18.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.772+0000 7f3b74a15640 1 -- 172.21.15.73:0/2884667816 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b70153f60 con 0x7f3b70073680 2024-04-26T20:52:18.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.772+0000 7f3b5effd640 1 -- 172.21.15.73:0/2884667816 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b64011bb0 con 0x7f3b70073680 2024-04-26T20:52:18.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.772+0000 7f3b74a15640 1 -- 172.21.15.73:0/2884667816 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b700b0a70 con 0x7f3b70073680 2024-04-26T20:52:18.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.773+0000 7f3b74a15640 1 -- 172.21.15.73:0/2884667816 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b700b5fc0 con 0x7f3b70073680 2024-04-26T20:52:18.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.773+0000 7f3b5effd640 1 -- 172.21.15.73:0/2884667816 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3b6401c460 con 0x7f3b70073680 2024-04-26T20:52:18.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.774+0000 7f3b5effd640 1 --2- 172.21.15.73:0/2884667816 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f3b5006c8f0 0x7f3b5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:18.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.774+0000 7f3b5effd640 1 -- 172.21.15.73:0/2884667816 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5249+0+0 (secure 0 0 0) 0x7f3b6408b7a0 con 0x7f3b70073680 2024-04-26T20:52:18.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.774+0000 7f3b6e59c640 1 --2- 172.21.15.73:0/2884667816 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f3b5006c8f0 0x7f3b5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:18.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.774+0000 7f3b6e59c640 1 --2- 172.21.15.73:0/2884667816 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f3b5006c8f0 0x7f3b5006edb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3b600012e0 tx=0x7f3b60001490 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:18.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.778+0000 7f3b5effd640 1 -- 172.21.15.73:0/2884667816 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3b700b5fc0 con 0x7f3b70073680 2024-04-26T20:52:18.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:18.955+0000 7f3b74a15640 1 -- 172.21.15.73:0/2884667816 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f3b70069600 con 0x7f3b70073680 2024-04-26T20:52:19.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.389+0000 7f3b5effd640 1 -- 172.21.15.73:0/2884667816 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v49) ==== 138+0+0 (secure 0 0 0) 0x7f3b64059e20 con 0x7f3b70073680 2024-04-26T20:52:19.391 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_metadata' created 2024-04-26T20:52:19.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.391+0000 7f3b74a15640 1 -- 172.21.15.73:0/2884667816 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f3b5006c8f0 msgr2=0x7f3b5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:19.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.391+0000 7f3b74a15640 1 --2- 172.21.15.73:0/2884667816 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f3b5006c8f0 0x7f3b5006edb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3b600012e0 tx=0x7f3b60001490 comp rx=0 tx=0).stop 2024-04-26T20:52:19.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.391+0000 7f3b74a15640 1 -- 172.21.15.73:0/2884667816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b70073680 msgr2=0x7f3b700affb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:19.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.391+0000 7f3b74a15640 1 --2- 172.21.15.73:0/2884667816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b70073680 0x7f3b700affb0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f3b6400c400 tx=0x7f3b6400c8c0 comp rx=0 tx=0).stop 2024-04-26T20:52:19.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.391+0000 7f3b74a15640 1 -- 172.21.15.73:0/2884667816 shutdown_connections 2024-04-26T20:52:19.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.391+0000 7f3b74a15640 1 --2- 172.21.15.73:0/2884667816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b70073680 0x7f3b700affb0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:19.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.391+0000 7f3b74a15640 1 --2- 172.21.15.73:0/2884667816 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f3b5006c8f0 0x7f3b5006edb0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:19.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.391+0000 7f3b74a15640 1 --2- 172.21.15.73:0/2884667816 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f3b70159860 0x7f3b70153a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:19.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.391+0000 7f3b74a15640 1 --2- 172.21.15.73:0/2884667816 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3b70070fa0 0x7f3b70156690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:19.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.391+0000 7f3b74a15640 1 -- 172.21.15.73:0/2884667816 >> 172.21.15.73:0/2884667816 conn(0x7f3b700d1d60 msgr2=0x7f3b700d2140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:19.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.391+0000 7f3b74a15640 1 -- 172.21.15.73:0/2884667816 shutdown_connections 2024-04-26T20:52:19.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.391+0000 7f3b74a15640 1 -- 172.21.15.73:0/2884667816 wait complete. 2024-04-26T20:52:19.408 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T20:52:19.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.511+0000 7f1bd43da640 1 Processor -- start 2024-04-26T20:52:19.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.511+0000 7f1bd43da640 1 -- start start 2024-04-26T20:52:19.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.511+0000 7f1bd43da640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1bcc0cfde0 0x7f1bcc0d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:19.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.512+0000 7f1bd43da640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1bcc071260 0x7f1bcc15e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:19.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.512+0000 7f1bd43da640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bcc0d2390 con 0x7f1bcc070350 2024-04-26T20:52:19.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.512+0000 7f1bd43da640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f1bcc0afab0 con 0x7f1bcc071260 2024-04-26T20:52:19.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.512+0000 7f1bd43da640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f1bcc06dd20 con 0x7f1bcc0cfde0 2024-04-26T20:52:19.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.512+0000 7f1bd2977640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1bcc071260 0x7f1bcc15e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:19.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.512+0000 7f1bd2977640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1bcc071260 0x7f1bcc15e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38016/0 (socket says 172.21.15.73:38016) 2024-04-26T20:52:19.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.512+0000 7f1bd2977640 1 -- 172.21.15.73:0/4095556411 learned_addr learned my addr 172.21.15.73:0/4095556411 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:19.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.512+0000 7f1bd2176640 1 --2- 172.21.15.73:0/4095556411 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1bcc0cfde0 0x7f1bcc0d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:19.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.512+0000 7f1bd2977640 1 -- 172.21.15.73:0/4095556411 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1bcc0cfde0 msgr2=0x7f1bcc0d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:19.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.512+0000 7f1bd2977640 1 --2- 172.21.15.73:0/4095556411 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1bcc0cfde0 0x7f1bcc0d1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:19.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.512+0000 7f1bd2977640 1 -- 172.21.15.73:0/4095556411 >> v1:172.21.15.120:6789/0 conn(0x7f1bcc070350 legacy=0x7f1bcc070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:19.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.512+0000 7f1bd2977640 1 -- 172.21.15.73:0/4095556411 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bcc06d3f0 con 0x7f1bcc071260 2024-04-26T20:52:19.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.512+0000 7f1bd2176640 1 --2- 172.21.15.73:0/4095556411 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1bcc0cfde0 0x7f1bcc0d1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:52:19.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.513+0000 7f1bd2977640 1 --2- 172.21.15.73:0/4095556411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1bcc071260 0x7f1bcc15e9f0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f1bc4009e50 tx=0x7f1bc402a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8850f60bd5515931 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:19.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.513+0000 7f1bd1174640 1 -- 172.21.15.73:0/4095556411 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc4036070 con 0x7f1bcc071260 2024-04-26T20:52:19.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.513+0000 7f1bd1174640 1 -- 172.21.15.73:0/4095556411 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bc4031600 con 0x7f1bcc071260 2024-04-26T20:52:19.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.513+0000 7f1bd1174640 1 -- 172.21.15.73:0/4095556411 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc40319d0 con 0x7f1bcc071260 2024-04-26T20:52:19.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.513+0000 7f1bd43da640 1 -- 172.21.15.73:0/4095556411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1bcc071260 msgr2=0x7f1bcc15e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:19.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.513+0000 7f1bd43da640 1 --2- 172.21.15.73:0/4095556411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1bcc071260 0x7f1bcc15e9f0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f1bc4009e50 tx=0x7f1bc402a630 comp rx=0 tx=0).stop 2024-04-26T20:52:19.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.513+0000 7f1bd43da640 1 -- 172.21.15.73:0/4095556411 shutdown_connections 2024-04-26T20:52:19.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.513+0000 7f1bd43da640 1 --2- 172.21.15.73:0/4095556411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1bcc071260 0x7f1bcc15e9f0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:19.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.513+0000 7f1bd43da640 1 --2- 172.21.15.73:0/4095556411 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1bcc0cfde0 0x7f1bcc0d1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:19.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.513+0000 7f1bd43da640 1 -- 172.21.15.73:0/4095556411 >> 172.21.15.73:0/4095556411 conn(0x7f1bcc0d2d80 msgr2=0x7f1bcc0d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:19.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.513+0000 7f1bd43da640 1 -- 172.21.15.73:0/4095556411 shutdown_connections 2024-04-26T20:52:19.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.513+0000 7f1bd43da640 1 -- 172.21.15.73:0/4095556411 wait complete. 2024-04-26T20:52:19.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.515+0000 7f1bd43da640 1 Processor -- start 2024-04-26T20:52:19.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.515+0000 7f1bd43da640 1 -- start start 2024-04-26T20:52:19.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.515+0000 7f1bd43da640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1bcc070350 0x7f1bcc161510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:19.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.515+0000 7f1bd43da640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1bcc071260 0x7f1bcc161a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:19.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.515+0000 7f1bd43da640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1bcc0cfde0 0x7f1bcc1f85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:19.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.515+0000 7f1bd43da640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1bcc0d20a0 con 0x7f1bcc070350 2024-04-26T20:52:19.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.515+0000 7f1bd43da640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1bcc06d9b0 con 0x7f1bcc071260 2024-04-26T20:52:19.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.515+0000 7f1bd43da640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f1bcc15ef30 con 0x7f1bcc0cfde0 2024-04-26T20:52:19.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.515+0000 7f1bd2176640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1bcc070350 0x7f1bcc161510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:19.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.515+0000 7f1bd2176640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1bcc070350 0x7f1bcc161510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38030/0 (socket says 172.21.15.73:38030) 2024-04-26T20:52:19.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.515+0000 7f1bd2176640 1 -- 172.21.15.73:0/3838754473 learned_addr learned my addr 172.21.15.73:0/3838754473 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:19.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.516+0000 7f1bd2176640 1 -- 172.21.15.73:0/3838754473 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1bcc0cfde0 msgr2=0x7f1bcc1f85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:19.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.516+0000 7f1bd1975640 1 --2- 172.21.15.73:0/3838754473 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1bcc071260 0x7f1bcc161a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:19.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.516+0000 7f1bd2176640 1 --2- 172.21.15.73:0/3838754473 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1bcc0cfde0 0x7f1bcc1f85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:19.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.516+0000 7f1bd2176640 1 -- 172.21.15.73:0/3838754473 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1bcc071260 msgr2=0x7f1bcc161a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:19.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.516+0000 7f1bd2176640 1 --2- 172.21.15.73:0/3838754473 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1bcc071260 0x7f1bcc161a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:19.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.516+0000 7f1bd2176640 1 -- 172.21.15.73:0/3838754473 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bcc1f5110 con 0x7f1bcc070350 2024-04-26T20:52:19.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.516+0000 7f1bd1975640 1 --2- 172.21.15.73:0/3838754473 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1bcc071260 0x7f1bcc161a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:52:19.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.516+0000 7f1bd2176640 1 --2- 172.21.15.73:0/3838754473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1bcc070350 0x7f1bcc161510 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f1bc800c7f0 tx=0x7f1bc800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:19.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.516+0000 7f1bc2ffd640 1 -- 172.21.15.73:0/3838754473 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc8015070 con 0x7f1bcc070350 2024-04-26T20:52:19.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.516+0000 7f1bc2ffd640 1 -- 172.21.15.73:0/3838754473 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bc8007820 con 0x7f1bcc070350 2024-04-26T20:52:19.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.516+0000 7f1bd43da640 1 -- 172.21.15.73:0/3838754473 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bcc1f53f0 con 0x7f1bcc070350 2024-04-26T20:52:19.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.516+0000 7f1bc2ffd640 1 -- 172.21.15.73:0/3838754473 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc8007bd0 con 0x7f1bcc070350 2024-04-26T20:52:19.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.516+0000 7f1bd43da640 1 -- 172.21.15.73:0/3838754473 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bcc1f59b0 con 0x7f1bcc070350 2024-04-26T20:52:19.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.517+0000 7f1bd43da640 1 -- 172.21.15.73:0/3838754473 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bcc0694f0 con 0x7f1bcc070350 2024-04-26T20:52:19.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.517+0000 7f1bc2ffd640 1 -- 172.21.15.73:0/3838754473 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1bc8018070 con 0x7f1bcc070350 2024-04-26T20:52:19.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.518+0000 7f1bc2ffd640 1 --2- 172.21.15.73:0/3838754473 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1bb006c8f0 0x7f1bb006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:19.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.518+0000 7f1bc2ffd640 1 -- 172.21.15.73:0/3838754473 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5619+0+0 (secure 0 0 0) 0x7f1bc808bc10 con 0x7f1bcc070350 2024-04-26T20:52:19.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.518+0000 7f1bd1975640 1 --2- 172.21.15.73:0/3838754473 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1bb006c8f0 0x7f1bb006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:19.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.518+0000 7f1bd1975640 1 --2- 172.21.15.73:0/3838754473 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1bb006c8f0 0x7f1bb006edb0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc1622f0 tx=0x7f1bbc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:19.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.522+0000 7f1bc2ffd640 1 -- 172.21.15.73:0/3838754473 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1bc805a090 con 0x7f1bcc070350 2024-04-26T20:52:19.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:19.698+0000 7f1bd43da640 1 -- 172.21.15.73:0/3838754473 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f1bcc0d2590 con 0x7f1bcc070350 2024-04-26T20:52:20.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.396+0000 7f1bc2ffd640 1 -- 172.21.15.73:0/3838754473 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v50) ==== 170+0+0 (secure 0 0 0) 0x7f1bc8059990 con 0x7f1bcc070350 2024-04-26T20:52:20.398 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data' created 2024-04-26T20:52:20.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.398+0000 7f1bd43da640 1 -- 172.21.15.73:0/3838754473 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1bb006c8f0 msgr2=0x7f1bb006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:20.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.398+0000 7f1bd43da640 1 --2- 172.21.15.73:0/3838754473 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1bb006c8f0 0x7f1bb006edb0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc1622f0 tx=0x7f1bbc009040 comp rx=0 tx=0).stop 2024-04-26T20:52:20.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.399+0000 7f1bd43da640 1 -- 172.21.15.73:0/3838754473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1bcc070350 msgr2=0x7f1bcc161510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:20.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.399+0000 7f1bd43da640 1 --2- 172.21.15.73:0/3838754473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1bcc070350 0x7f1bcc161510 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f1bc800c7f0 tx=0x7f1bc800ccb0 comp rx=0 tx=0).stop 2024-04-26T20:52:20.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.399+0000 7f1bd43da640 1 -- 172.21.15.73:0/3838754473 shutdown_connections 2024-04-26T20:52:20.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.399+0000 7f1bd43da640 1 --2- 172.21.15.73:0/3838754473 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1bb006c8f0 0x7f1bb006edb0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:20.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.399+0000 7f1bd43da640 1 --2- 172.21.15.73:0/3838754473 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1bcc0cfde0 0x7f1bcc1f85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:20.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.399+0000 7f1bd43da640 1 --2- 172.21.15.73:0/3838754473 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1bcc071260 0x7f1bcc161a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:20.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.399+0000 7f1bd43da640 1 --2- 172.21.15.73:0/3838754473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1bcc070350 0x7f1bcc161510 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:20.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.399+0000 7f1bd43da640 1 -- 172.21.15.73:0/3838754473 >> 172.21.15.73:0/3838754473 conn(0x7f1bcc0d2d80 msgr2=0x7f1bcc0b40e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:20.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.400+0000 7f1bd43da640 1 -- 172.21.15.73:0/3838754473 shutdown_connections 2024-04-26T20:52:20.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.401+0000 7f1bd43da640 1 -- 172.21.15.73:0/3838754473 wait complete. 2024-04-26T20:52:20.427 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T20:52:20.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.528+0000 7fe88002a640 1 Processor -- start 2024-04-26T20:52:20.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.529+0000 7fe88002a640 1 -- start start 2024-04-26T20:52:20.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.529+0000 7fe88002a640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe8780b40d0 0x7fe8781625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:20.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.529+0000 7fe88002a640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe878162b30 0x7fe8780b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:20.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.529+0000 7fe88002a640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8780d0a10 con 0x7fe8780d1a00 2024-04-26T20:52:20.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.529+0000 7fe88002a640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7fe8780d0cf0 con 0x7fe878162b30 2024-04-26T20:52:20.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.529+0000 7fe88002a640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fe8780d0fd0 con 0x7fe8780b40d0 2024-04-26T20:52:20.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.530+0000 7fe87e5c7640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe878162b30 0x7fe8780b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:20.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.530+0000 7fe87e5c7640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe878162b30 0x7fe8780b4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38036/0 (socket says 172.21.15.73:38036) 2024-04-26T20:52:20.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.530+0000 7fe87e5c7640 1 -- 172.21.15.73:0/2890614209 learned_addr learned my addr 172.21.15.73:0/2890614209 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:20.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.530+0000 7fe87d5c5640 1 --2- 172.21.15.73:0/2890614209 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe8780b40d0 0x7fe8781625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:20.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.530+0000 7fe87e5c7640 1 -- 172.21.15.73:0/2890614209 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe8780b40d0 msgr2=0x7fe8781625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:20.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.530+0000 7fe87e5c7640 1 --2- 172.21.15.73:0/2890614209 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe8780b40d0 0x7fe8781625f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:20.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.530+0000 7fe87e5c7640 1 -- 172.21.15.73:0/2890614209 >> v1:172.21.15.120:6789/0 conn(0x7fe8780d1a00 legacy=0x7fe8780d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:20.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.530+0000 7fe87e5c7640 1 -- 172.21.15.73:0/2890614209 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe878162fa0 con 0x7fe878162b30 2024-04-26T20:52:20.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.531+0000 7fe87e5c7640 1 --2- 172.21.15.73:0/2890614209 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe878162b30 0x7fe8780b4b00 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fe870009e50 tx=0x7fe87002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d499fc7cc0ba5db3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:20.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.531+0000 7fe87cdc4640 1 -- 172.21.15.73:0/2890614209 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe870036070 con 0x7fe878162b30 2024-04-26T20:52:20.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.531+0000 7fe87cdc4640 1 -- 172.21.15.73:0/2890614209 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe870031600 con 0x7fe878162b30 2024-04-26T20:52:20.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.531+0000 7fe87cdc4640 1 -- 172.21.15.73:0/2890614209 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8700319d0 con 0x7fe878162b30 2024-04-26T20:52:20.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.531+0000 7fe88002a640 1 -- 172.21.15.73:0/2890614209 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe878162b30 msgr2=0x7fe8780b4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:20.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.531+0000 7fe88002a640 1 --2- 172.21.15.73:0/2890614209 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe878162b30 0x7fe8780b4b00 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fe870009e50 tx=0x7fe87002a680 comp rx=0 tx=0).stop 2024-04-26T20:52:20.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.531+0000 7fe88002a640 1 -- 172.21.15.73:0/2890614209 shutdown_connections 2024-04-26T20:52:20.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.531+0000 7fe88002a640 1 --2- 172.21.15.73:0/2890614209 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe878162b30 0x7fe8780b4b00 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:20.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.531+0000 7fe88002a640 1 --2- 172.21.15.73:0/2890614209 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe8780b40d0 0x7fe8781625f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:20.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.531+0000 7fe88002a640 1 -- 172.21.15.73:0/2890614209 >> 172.21.15.73:0/2890614209 conn(0x7fe8780d4fd0 msgr2=0x7fe8780d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:20.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.531+0000 7fe88002a640 1 -- 172.21.15.73:0/2890614209 shutdown_connections 2024-04-26T20:52:20.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.531+0000 7fe88002a640 1 -- 172.21.15.73:0/2890614209 wait complete. 2024-04-26T20:52:20.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.532+0000 7fe88002a640 1 Processor -- start 2024-04-26T20:52:20.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.533+0000 7fe88002a640 1 -- start start 2024-04-26T20:52:20.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.533+0000 7fe88002a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe8780b40d0 0x7fe8781f5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:20.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.533+0000 7fe88002a640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe8780d1a00 0x7fe8781f47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:20.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.533+0000 7fe88002a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe878162b30 0x7fe8780cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:20.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.533+0000 7fe88002a640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe8780d2670 con 0x7fe878162b30 2024-04-26T20:52:20.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.533+0000 7fe88002a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe8780d24f0 con 0x7fe8780b40d0 2024-04-26T20:52:20.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.533+0000 7fe88002a640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fe8780b5040 con 0x7fe8780d1a00 2024-04-26T20:52:20.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.533+0000 7fe87e5c7640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe878162b30 0x7fe8780cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:20.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.533+0000 7fe87e5c7640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe878162b30 0x7fe8780cdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38046/0 (socket says 172.21.15.73:38046) 2024-04-26T20:52:20.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.533+0000 7fe87e5c7640 1 -- 172.21.15.73:0/1429627655 learned_addr learned my addr 172.21.15.73:0/1429627655 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:20.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.533+0000 7fe87d5c5640 1 --2- 172.21.15.73:0/1429627655 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe8780d1a00 0x7fe8781f47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:20.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.533+0000 7fe87ddc6640 1 --2- 172.21.15.73:0/1429627655 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe8780b40d0 0x7fe8781f5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:20.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.534+0000 7fe87e5c7640 1 -- 172.21.15.73:0/1429627655 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe8780d1a00 msgr2=0x7fe8781f47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:20.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.534+0000 7fe87e5c7640 1 --2- 172.21.15.73:0/1429627655 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe8780d1a00 0x7fe8781f47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:20.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.534+0000 7fe87e5c7640 1 -- 172.21.15.73:0/1429627655 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe8780b40d0 msgr2=0x7fe8781f5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:20.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.534+0000 7fe87e5c7640 1 --2- 172.21.15.73:0/1429627655 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe8780b40d0 0x7fe8781f5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:20.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.534+0000 7fe87e5c7640 1 -- 172.21.15.73:0/1429627655 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8781f5ed0 con 0x7fe878162b30 2024-04-26T20:52:20.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.534+0000 7fe87e5c7640 1 --2- 172.21.15.73:0/1429627655 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe878162b30 0x7fe8780cdc00 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fe870034040 tx=0x7fe870004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:20.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.534+0000 7fe86e7fc640 1 -- 172.21.15.73:0/1429627655 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe870036050 con 0x7fe878162b30 2024-04-26T20:52:20.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.534+0000 7fe86e7fc640 1 -- 172.21.15.73:0/1429627655 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe870042070 con 0x7fe878162b30 2024-04-26T20:52:20.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.534+0000 7fe88002a640 1 -- 172.21.15.73:0/1429627655 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8780cc2f0 con 0x7fe878162b30 2024-04-26T20:52:20.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.534+0000 7fe86e7fc640 1 -- 172.21.15.73:0/1429627655 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe870009510 con 0x7fe878162b30 2024-04-26T20:52:20.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.534+0000 7fe88002a640 1 -- 172.21.15.73:0/1429627655 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8780cc8b0 con 0x7fe878162b30 2024-04-26T20:52:20.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.535+0000 7fe88002a640 1 -- 172.21.15.73:0/1429627655 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8780694f0 con 0x7fe878162b30 2024-04-26T20:52:20.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.535+0000 7fe86e7fc640 1 -- 172.21.15.73:0/1429627655 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe870007c70 con 0x7fe878162b30 2024-04-26T20:52:20.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.536+0000 7fe86e7fc640 1 --2- 172.21.15.73:0/1429627655 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe85c06c8f0 0x7fe85c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:20.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.536+0000 7fe86e7fc640 1 -- 172.21.15.73:0/1429627655 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6001+0+0 (secure 0 0 0) 0x7fe8700ac980 con 0x7fe878162b30 2024-04-26T20:52:20.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.536+0000 7fe87ddc6640 1 --2- 172.21.15.73:0/1429627655 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe85c06c8f0 0x7fe85c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:20.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.536+0000 7fe87ddc6640 1 --2- 172.21.15.73:0/1429627655 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe85c06c8f0 0x7fe85c06edb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe87806df20 tx=0x7fe868009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:20.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.540+0000 7fe86e7fc640 1 -- 172.21.15.73:0/1429627655 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe8781f5ed0 con 0x7fe878162b30 2024-04-26T20:52:20.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:20.752+0000 7fe88002a640 1 -- 172.21.15.73:0/1429627655 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fe8780b4ba0 con 0x7fe878162b30 2024-04-26T20:52:21.399 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.397+0000 7fe86e7fc640 1 -- 172.21.15.73:0/1429627655 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v24) ==== 172+0+0 (secure 0 0 0) 0x7fe87007a630 con 0x7fe878162b30 2024-04-26T20:52:21.399 INFO:teuthology.orchestra.run.smithi073.stderr:new fs with metadata pool 7 and data pool 8 2024-04-26T20:52:21.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.401+0000 7fe88002a640 1 -- 172.21.15.73:0/1429627655 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe85c06c8f0 msgr2=0x7fe85c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:21.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.401+0000 7fe88002a640 1 --2- 172.21.15.73:0/1429627655 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe85c06c8f0 0x7fe85c06edb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe87806df20 tx=0x7fe868009040 comp rx=0 tx=0).stop 2024-04-26T20:52:21.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.401+0000 7fe88002a640 1 -- 172.21.15.73:0/1429627655 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe878162b30 msgr2=0x7fe8780cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:21.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.401+0000 7fe88002a640 1 --2- 172.21.15.73:0/1429627655 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe878162b30 0x7fe8780cdc00 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fe870034040 tx=0x7fe870004a40 comp rx=0 tx=0).stop 2024-04-26T20:52:21.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.401+0000 7fe88002a640 1 -- 172.21.15.73:0/1429627655 shutdown_connections 2024-04-26T20:52:21.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.401+0000 7fe88002a640 1 --2- 172.21.15.73:0/1429627655 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe878162b30 0x7fe8780cdc00 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:21.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.401+0000 7fe88002a640 1 --2- 172.21.15.73:0/1429627655 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe85c06c8f0 0x7fe85c06edb0 secure :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe87806df20 tx=0x7fe868009040 comp rx=0 tx=0).stop 2024-04-26T20:52:21.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.401+0000 7fe88002a640 1 --2- 172.21.15.73:0/1429627655 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe8780d1a00 0x7fe8781f47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:21.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.401+0000 7fe88002a640 1 --2- 172.21.15.73:0/1429627655 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe8780b40d0 0x7fe8781f5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:21.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.401+0000 7fe88002a640 1 -- 172.21.15.73:0/1429627655 >> 172.21.15.73:0/1429627655 conn(0x7fe8780d4fd0 msgr2=0x7fe8780d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:21.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.402+0000 7fe88002a640 1 -- 172.21.15.73:0/1429627655 shutdown_connections 2024-04-26T20:52:21.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.402+0000 7fe88002a640 1 -- 172.21.15.73:0/1429627655 wait complete. 2024-04-26T20:52:21.434 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:52:21.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.530+0000 7fe7a3264640 1 Processor -- start 2024-04-26T20:52:21.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.530+0000 7fe7a3264640 1 -- start start 2024-04-26T20:52:21.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.531+0000 7fe7a3264640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fe79c162810 0x7fe79c070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:21.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.531+0000 7fe7a3264640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe79c0d1b30 0x7fe79c0d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:21.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.531+0000 7fe7a3264640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe79c0d10d0 con 0x7fe79c0b2340 2024-04-26T20:52:21.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.531+0000 7fe7a3264640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7fe79c0d0750 con 0x7fe79c0d1b30 2024-04-26T20:52:21.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.531+0000 7fe7a3264640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fe79c0d16a0 con 0x7fe79c162810 2024-04-26T20:52:21.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.531+0000 7fe7a1801640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe79c0d1b30 0x7fe79c0d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:21.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.531+0000 7fe7a1801640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe79c0d1b30 0x7fe79c0d1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38048/0 (socket says 172.21.15.73:38048) 2024-04-26T20:52:21.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.531+0000 7fe7a1801640 1 -- 172.21.15.73:0/2688178044 learned_addr learned my addr 172.21.15.73:0/2688178044 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:21.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.532+0000 7fe7a1000640 1 --2- 172.21.15.73:0/2688178044 >> v2:172.21.15.120:3300/0 conn(0x7fe79c162810 0x7fe79c070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:21.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.532+0000 7fe7a1801640 1 -- 172.21.15.73:0/2688178044 >> v2:172.21.15.120:3300/0 conn(0x7fe79c162810 msgr2=0x7fe79c070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:21.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.532+0000 7fe7a1801640 1 --2- 172.21.15.73:0/2688178044 >> v2:172.21.15.120:3300/0 conn(0x7fe79c162810 0x7fe79c070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:21.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.532+0000 7fe7a1801640 1 -- 172.21.15.73:0/2688178044 >> v1:172.21.15.120:6789/0 conn(0x7fe79c0b2340 legacy=0x7fe79c0cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:21.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.532+0000 7fe7a1801640 1 -- 172.21.15.73:0/2688178044 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe79c0d1370 con 0x7fe79c0d1b30 2024-04-26T20:52:21.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.532+0000 7fe7a1801640 1 --2- 172.21.15.73:0/2688178044 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe79c0d1b30 0x7fe79c0d1fc0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fe79c073d90 tx=0x7fe79402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd1842fe534f0c58 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:21.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.533+0000 7fe79b7fe640 1 -- 172.21.15.73:0/2688178044 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe794037070 con 0x7fe79c0d1b30 2024-04-26T20:52:21.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.533+0000 7fe79b7fe640 1 -- 172.21.15.73:0/2688178044 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe794032440 con 0x7fe79c0d1b30 2024-04-26T20:52:21.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.533+0000 7fe79b7fe640 1 -- 172.21.15.73:0/2688178044 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe794032810 con 0x7fe79c0d1b30 2024-04-26T20:52:21.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.533+0000 7fe7a3264640 1 -- 172.21.15.73:0/2688178044 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe79c0d1b30 msgr2=0x7fe79c0d1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:21.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.533+0000 7fe7a3264640 1 --2- 172.21.15.73:0/2688178044 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe79c0d1b30 0x7fe79c0d1fc0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fe79c073d90 tx=0x7fe79402a270 comp rx=0 tx=0).stop 2024-04-26T20:52:21.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.533+0000 7fe7a3264640 1 -- 172.21.15.73:0/2688178044 shutdown_connections 2024-04-26T20:52:21.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.533+0000 7fe7a3264640 1 --2- 172.21.15.73:0/2688178044 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe79c0d1b30 0x7fe79c0d1fc0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:21.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.533+0000 7fe7a3264640 1 --2- 172.21.15.73:0/2688178044 >> v2:172.21.15.120:3300/0 conn(0x7fe79c162810 0x7fe79c070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:21.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.533+0000 7fe7a3264640 1 -- 172.21.15.73:0/2688178044 >> 172.21.15.73:0/2688178044 conn(0x7fe79c0d2da0 msgr2=0x7fe79c0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:21.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.533+0000 7fe7a3264640 1 -- 172.21.15.73:0/2688178044 shutdown_connections 2024-04-26T20:52:21.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.533+0000 7fe7a3264640 1 -- 172.21.15.73:0/2688178044 wait complete. 2024-04-26T20:52:21.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.534+0000 7fe7a3264640 1 Processor -- start 2024-04-26T20:52:21.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.534+0000 7fe7a3264640 1 -- start start 2024-04-26T20:52:21.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.535+0000 7fe7a3264640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe79c0b2340 0x7fe79c0cac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:21.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.535+0000 7fe7a3264640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe79c0d1b30 0x7fe79c0cb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:21.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.535+0000 7fe7a3264640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe79c162810 0x7fe79c14e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:21.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.535+0000 7fe7a3264640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c070890 con 0x7fe79c0d1b30 2024-04-26T20:52:21.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.535+0000 7fe7a3264640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe79c0d0470 con 0x7fe79c0b2340 2024-04-26T20:52:21.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.535+0000 7fe7a3264640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fe79c0cdb00 con 0x7fe79c162810 2024-04-26T20:52:21.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.535+0000 7fe79bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe79c0d1b30 0x7fe79c0cb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:21.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.535+0000 7fe79bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe79c0d1b30 0x7fe79c0cb8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38060/0 (socket says 172.21.15.73:38060) 2024-04-26T20:52:21.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.535+0000 7fe79bfff640 1 -- 172.21.15.73:0/1690967809 learned_addr learned my addr 172.21.15.73:0/1690967809 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:21.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.536+0000 7fe7a1801640 1 --2- 172.21.15.73:0/1690967809 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe79c162810 0x7fe79c14e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:21.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.536+0000 7fe7a1000640 1 --2- 172.21.15.73:0/1690967809 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe79c0b2340 0x7fe79c0cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:21.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.536+0000 7fe79bfff640 1 -- 172.21.15.73:0/1690967809 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe79c162810 msgr2=0x7fe79c14e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:21.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.536+0000 7fe79bfff640 1 --2- 172.21.15.73:0/1690967809 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe79c162810 0x7fe79c14e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:21.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.536+0000 7fe79bfff640 1 -- 172.21.15.73:0/1690967809 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe79c0b2340 msgr2=0x7fe79c0cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:21.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.536+0000 7fe79bfff640 1 --2- 172.21.15.73:0/1690967809 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe79c0b2340 0x7fe79c0cac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:21.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.536+0000 7fe79bfff640 1 -- 172.21.15.73:0/1690967809 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe79c0cbe00 con 0x7fe79c0d1b30 2024-04-26T20:52:21.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.536+0000 7fe79bfff640 1 --2- 172.21.15.73:0/1690967809 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe79c0d1b30 0x7fe79c0cb8c0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fe78400c400 tx=0x7fe78400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:21.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.536+0000 7fe7997fa640 1 -- 172.21.15.73:0/1690967809 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7840052d0 con 0x7fe79c0d1b30 2024-04-26T20:52:21.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.536+0000 7fe7997fa640 1 -- 172.21.15.73:0/1690967809 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe784005490 con 0x7fe79c0d1b30 2024-04-26T20:52:21.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.536+0000 7fe7a3264640 1 -- 172.21.15.73:0/1690967809 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe79c0cc0d0 con 0x7fe79c0d1b30 2024-04-26T20:52:21.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.536+0000 7fe7997fa640 1 -- 172.21.15.73:0/1690967809 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe784007260 con 0x7fe79c0d1b30 2024-04-26T20:52:21.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.536+0000 7fe7a3264640 1 -- 172.21.15.73:0/1690967809 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe79c0cc690 con 0x7fe79c0d1b30 2024-04-26T20:52:21.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.537+0000 7fe7a3264640 1 -- 172.21.15.73:0/1690967809 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe79c0694f0 con 0x7fe79c0d1b30 2024-04-26T20:52:21.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.537+0000 7fe7997fa640 1 -- 172.21.15.73:0/1690967809 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe784007400 con 0x7fe79c0d1b30 2024-04-26T20:52:21.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.538+0000 7fe7997fa640 1 --2- 172.21.15.73:0/1690967809 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe77c06c8f0 0x7fe77c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:21.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.538+0000 7fe7a1000640 1 --2- 172.21.15.73:0/1690967809 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe77c06c8f0 0x7fe77c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:21.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.538+0000 7fe7997fa640 1 -- 172.21.15.73:0/1690967809 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7fe78408b180 con 0x7fe79c0d1b30 2024-04-26T20:52:21.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.538+0000 7fe7a1000640 1 --2- 172.21.15.73:0/1690967809 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe77c06c8f0 0x7fe77c06edb0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe78c007380 tx=0x7fe78c00b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:21.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.542+0000 7fe7997fa640 1 -- 172.21.15.73:0/1690967809 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe7840594b0 con 0x7fe79c0d1b30 2024-04-26T20:52:21.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.718+0000 7fe7a3264640 1 -- 172.21.15.73:0/1690967809 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe79c070560 con 0x7fe79c0d1b30 2024-04-26T20:52:21.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.719+0000 7fe7997fa640 1 -- 172.21.15.73:0/1690967809 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) ==== 74+0+16955 (secure 0 0 0) 0x7fe784058db0 con 0x7fe79c0d1b30 2024-04-26T20:52:21.722 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:52:21.722 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":51,"fsid":"fbd3f548-b547-4713-856b-09f98f5d1764","created":"2024-04-26T20:50:07.876786+0000","modified":"2024-04-26T20:52:20.754772+0000","last_up_change":"2024-04-26T20:50:19.082741+0000","last_in_change":"2024-04-26T20:50:10.018400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:50:19.993731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:52:18.960897+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-26T20:52:19.704261+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d6f4f3-214a-4392-a02f-c33cbcab3c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6809","nonce":3461705124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6811","nonce":3461705124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6815","nonce":3461705124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6813","nonce":3461705124}]},"public_addr":"172.21.15.73:6809/3461705124","cluster_addr":"172.21.15.73:6811/3461705124","heartbeat_back_addr":"172.21.15.73:6815/3461705124","heartbeat_front_addr":"172.21.15.73:6813/3461705124","state":["exists","up"]},{"osd":1,"uuid":"786bf8ca-6ddc-4b3c-be6c-9152398bd5fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6817","nonce":2827619403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6819","nonce":2827619403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6825","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6827","nonce":2827619403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6821","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6823","nonce":2827619403}]},"public_addr":"172.21.15.73:6817/2827619403","cluster_addr":"172.21.15.73:6819/2827619403","heartbeat_back_addr":"172.21.15.73:6827/2827619403","heartbeat_front_addr":"172.21.15.73:6823/2827619403","state":["exists","up"]},{"osd":2,"uuid":"90963b1a-5437-4ef8-ad8b-6b74c534eeb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6801","nonce":2542681064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6803","nonce":2542681064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6808","nonce":2542681064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6805","nonce":2542681064}]},"public_addr":"172.21.15.73:6801/2542681064","cluster_addr":"172.21.15.73:6803/2542681064","heartbeat_back_addr":"172.21.15.73:6808/2542681064","heartbeat_front_addr":"172.21.15.73:6805/2542681064","state":["exists","up"]},{"osd":3,"uuid":"bd3d2148-6e38-41b4-a453-fe6e3b8b4ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6822","nonce":1604042718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6826","nonce":1604042718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6831","nonce":1604042718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6829","nonce":1604042718}]},"public_addr":"172.21.15.73:6822/1604042718","cluster_addr":"172.21.15.73:6826/1604042718","heartbeat_back_addr":"172.21.15.73:6831/1604042718","heartbeat_front_addr":"172.21.15.73:6829/1604042718","state":["exists","up"]},{"osd":4,"uuid":"e8668dc8-a23f-46d2-8311-cb882da08e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6817","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6819","nonce":3591055561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6824","nonce":3591055561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6831","nonce":3591055561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6828","nonce":3591055561}]},"public_addr":"172.21.15.120:6819/3591055561","cluster_addr":"172.21.15.120:6824/3591055561","heartbeat_back_addr":"172.21.15.120:6831/3591055561","heartbeat_front_addr":"172.21.15.120:6828/3591055561","state":["exists","up"]},{"osd":5,"uuid":"cbac0bdb-96bd-4852-bcce-e564495c3fcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6809","nonce":819450433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6811","nonce":819450433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6815","nonce":819450433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6813","nonce":819450433}]},"public_addr":"172.21.15.120:6809/819450433","cluster_addr":"172.21.15.120:6811/819450433","heartbeat_back_addr":"172.21.15.120:6815/819450433","heartbeat_front_addr":"172.21.15.120:6813/819450433","state":["exists","up"]},{"osd":6,"uuid":"1884b2a8-569f-4272-96a9-f25a09364dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6801","nonce":3946210727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6803","nonce":3946210727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6807","nonce":3946210727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6805","nonce":3946210727}]},"public_addr":"172.21.15.120:6801/3946210727","cluster_addr":"172.21.15.120:6803/3946210727","heartbeat_back_addr":"172.21.15.120:6807/3946210727","heartbeat_front_addr":"172.21.15.120:6805/3946210727","state":["exists","up"]},{"osd":7,"uuid":"c0c3bf6d-a4c5-4f1e-aa2d-330951830666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6818","nonce":2915574674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6821","nonce":2915574674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6827","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6829","nonce":2915574674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6823","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6825","nonce":2915574674}]},"public_addr":"172.21.15.120:6818/2915574674","cluster_addr":"172.21.15.120:6821/2915574674","heartbeat_back_addr":"172.21.15.120:6829/2915574674","heartbeat_front_addr":"172.21.15.120:6825/2915574674","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.218216+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.208710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.112705+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.182789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.164324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.174212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.225396+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.261559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:52:21.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.721+0000 7fe7a3264640 1 -- 172.21.15.73:0/1690967809 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe77c06c8f0 msgr2=0x7fe77c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:21.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.721+0000 7fe7a3264640 1 --2- 172.21.15.73:0/1690967809 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe77c06c8f0 0x7fe77c06edb0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe78c007380 tx=0x7fe78c00b040 comp rx=0 tx=0).stop 2024-04-26T20:52:21.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.721+0000 7fe7a3264640 1 -- 172.21.15.73:0/1690967809 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe79c0d1b30 msgr2=0x7fe79c0cb8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:21.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.721+0000 7fe7a3264640 1 --2- 172.21.15.73:0/1690967809 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe79c0d1b30 0x7fe79c0cb8c0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fe78400c400 tx=0x7fe78400c8c0 comp rx=0 tx=0).stop 2024-04-26T20:52:21.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.721+0000 7fe7a3264640 1 -- 172.21.15.73:0/1690967809 shutdown_connections 2024-04-26T20:52:21.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.721+0000 7fe7a3264640 1 --2- 172.21.15.73:0/1690967809 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe79c0d1b30 0x7fe79c0cb8c0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:21.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.721+0000 7fe7a3264640 1 --2- 172.21.15.73:0/1690967809 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe77c06c8f0 0x7fe77c06edb0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:21.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.721+0000 7fe7a3264640 1 --2- 172.21.15.73:0/1690967809 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe79c162810 0x7fe79c14e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:21.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.721+0000 7fe7a3264640 1 --2- 172.21.15.73:0/1690967809 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe79c0b2340 0x7fe79c0cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:21.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.721+0000 7fe7a3264640 1 -- 172.21.15.73:0/1690967809 >> 172.21.15.73:0/1690967809 conn(0x7fe79c0d2da0 msgr2=0x7fe79c0b29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:21.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.721+0000 7fe7a3264640 1 -- 172.21.15.73:0/1690967809 shutdown_connections 2024-04-26T20:52:21.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.721+0000 7fe7a3264640 1 -- 172.21.15.73:0/1690967809 wait complete. 2024-04-26T20:52:21.735 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:52:21.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.828+0000 7f57d4ebd640 1 Processor -- start 2024-04-26T20:52:21.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.828+0000 7f57d4ebd640 1 -- start start 2024-04-26T20:52:21.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.829+0000 7f57d4ebd640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f57d0162810 0x7f57d0070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:21.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.829+0000 7f57d4ebd640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57d00d10d0 con 0x7f57d00b2340 2024-04-26T20:52:21.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.829+0000 7f57d4ebd640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57d00d1370 con 0x7f57d00d1b30 2024-04-26T20:52:21.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.829+0000 7f57d4ebd640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f57d00d0750 con 0x7f57d0162810 2024-04-26T20:52:21.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.829+0000 7f57c7fff640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f57d00b2340 0x7f57d00cd3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:36592/0 (socket says 172.21.15.73:36592) 2024-04-26T20:52:21.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.829+0000 7f57c7fff640 1 -- 172.21.15.73:0/222227442 learned_addr learned my addr 172.21.15.73:0/222227442 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:52:21.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.829+0000 7f57ce59c640 1 --2- 172.21.15.73:0/222227442 >> v2:172.21.15.120:3300/0 conn(0x7f57d0162810 0x7f57d0070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:21.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.829+0000 7f57cdd9b640 1 -- 172.21.15.73:0/222227442 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 572383031 0 0) 0x7f57d00d10d0 con 0x7f57d00b2340 2024-04-26T20:52:21.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.829+0000 7f57cdd9b640 1 -- 172.21.15.73:0/222227442 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f57bc003520 con 0x7f57d00b2340 2024-04-26T20:52:21.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.829+0000 7f57cdd9b640 1 -- 172.21.15.73:0/222227442 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 271610636 0 0) 0x7f57bc003520 con 0x7f57d00b2340 2024-04-26T20:52:21.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.829+0000 7f57cdd9b640 1 -- 172.21.15.73:0/222227442 >> v2:172.21.15.120:3300/0 conn(0x7f57d0162810 msgr2=0x7f57d0070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:21.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.829+0000 7f57cdd9b640 1 --2- 172.21.15.73:0/222227442 >> v2:172.21.15.120:3300/0 conn(0x7f57d0162810 0x7f57d0070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:21.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.829+0000 7f57cdd9b640 1 -- 172.21.15.73:0/222227442 >> v1:172.21.15.120:6789/0 conn(0x7f57d00d1b30 legacy=0x7f57d0070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:21.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.829+0000 7f57cdd9b640 1 -- 172.21.15.73:0/222227442 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57d00d01b0 con 0x7f57d00b2340 2024-04-26T20:52:21.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.829+0000 7f57cdd9b640 1 -- 172.21.15.73:0/222227442 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7f57b8001670 con 0x7f57d00b2340 2024-04-26T20:52:21.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.829+0000 7f57cdd9b640 1 -- 172.21.15.73:0/222227442 >> v1:172.21.15.73:6789/0 conn(0x7f57d00b2340 legacy=0x7f57d00cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:21.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.829+0000 7f57cdd9b640 1 --2- 172.21.15.73:0/222227442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57bc0039b0 0x7f57bc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:21.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.829+0000 7f57cdd9b640 1 --2- 172.21.15.73:0/222227442 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57bc0043b0 0x7f57bc004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:21.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.830+0000 7f57cdd9b640 1 --2- 172.21.15.73:0/222227442 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f57bc005080 0x7f57bc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:21.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.830+0000 7f57cdd9b640 1 -- 172.21.15.73:0/222227442 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f57d00d0750 con 0x7f57bc0039b0 2024-04-26T20:52:21.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.830+0000 7f57cdd9b640 1 -- 172.21.15.73:0/222227442 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f57bc00bfe0 con 0x7f57bc0043b0 2024-04-26T20:52:21.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.830+0000 7f57cdd9b640 1 -- 172.21.15.73:0/222227442 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f57bc00c1e0 con 0x7f57bc005080 2024-04-26T20:52:21.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.830+0000 7f57ce59c640 1 --2- 172.21.15.73:0/222227442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57bc0039b0 0x7f57bc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:21.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.830+0000 7f57c7fff640 1 --2- 172.21.15.73:0/222227442 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57bc0043b0 0x7f57bc004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:21.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.830+0000 7f57ced9d640 1 --2- 172.21.15.73:0/222227442 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f57bc005080 0x7f57bc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:21.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.830+0000 7f57ce59c640 1 -- 172.21.15.73:0/222227442 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f57bc005080 msgr2=0x7f57bc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:21.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.830+0000 7f57ce59c640 1 --2- 172.21.15.73:0/222227442 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f57bc005080 0x7f57bc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:21.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.830+0000 7f57ce59c640 1 -- 172.21.15.73:0/222227442 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57bc0043b0 msgr2=0x7f57bc004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:21.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.830+0000 7f57ce59c640 1 --2- 172.21.15.73:0/222227442 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57bc0043b0 0x7f57bc004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:21.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.830+0000 7f57ce59c640 1 -- 172.21.15.73:0/222227442 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57bc003520 con 0x7f57bc0039b0 2024-04-26T20:52:21.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.830+0000 7f57ced9d640 1 --2- 172.21.15.73:0/222227442 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f57bc005080 0x7f57bc00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:52:21.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.830+0000 7f57ced9d640 1 -- 172.21.15.73:0/222227442 reap_dead start 2024-04-26T20:52:21.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.831+0000 7f57ce59c640 1 --2- 172.21.15.73:0/222227442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57bc0039b0 0x7f57bc003d90 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f57c0009ac0 tx=0x7f57bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab49a35cb1e44b20 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:21.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.831+0000 7f57cdd9b640 1 -- 172.21.15.73:0/222227442 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c0036070 con 0x7f57bc0039b0 2024-04-26T20:52:21.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.831+0000 7f57cdd9b640 1 -- 172.21.15.73:0/222227442 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57c0031400 con 0x7f57bc0039b0 2024-04-26T20:52:21.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.831+0000 7f57cdd9b640 1 -- 172.21.15.73:0/222227442 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c0009690 con 0x7f57bc0039b0 2024-04-26T20:52:21.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.831+0000 7f57d4ebd640 1 -- 172.21.15.73:0/222227442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57bc0039b0 msgr2=0x7f57bc003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:21.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.831+0000 7f57d4ebd640 1 --2- 172.21.15.73:0/222227442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57bc0039b0 0x7f57bc003d90 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f57c0009ac0 tx=0x7f57bc005830 comp rx=0 tx=0).stop 2024-04-26T20:52:21.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.831+0000 7f57d4ebd640 1 -- 172.21.15.73:0/222227442 shutdown_connections 2024-04-26T20:52:21.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.831+0000 7f57d4ebd640 1 --2- 172.21.15.73:0/222227442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57bc0039b0 0x7f57bc003d90 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:21.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.831+0000 7f57d4ebd640 1 -- 172.21.15.73:0/222227442 >> 172.21.15.73:0/222227442 conn(0x7f57d00d2da0 msgr2=0x7f57d00d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:21.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.831+0000 7f57d4ebd640 1 -- 172.21.15.73:0/222227442 shutdown_connections 2024-04-26T20:52:21.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.831+0000 7f57d4ebd640 1 -- 172.21.15.73:0/222227442 wait complete. 2024-04-26T20:52:21.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.833+0000 7f57d4ebd640 1 Processor -- start 2024-04-26T20:52:21.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.833+0000 7f57d4ebd640 1 -- start start 2024-04-26T20:52:21.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.833+0000 7f57d4ebd640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57bc0039b0 0x7f57d00b8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:21.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.833+0000 7f57d4ebd640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f57d00cc040 0x7f57d00cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:21.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.833+0000 7f57d4ebd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57d00c8430 0x7f57d00c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:21.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.833+0000 7f57d4ebd640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f57d00d1fc0 con 0x7f57d00c8430 2024-04-26T20:52:21.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.833+0000 7f57d4ebd640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f57d00d16a0 con 0x7f57bc0039b0 2024-04-26T20:52:21.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.833+0000 7f57d4ebd640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f57d00d2140 con 0x7f57d00cc040 2024-04-26T20:52:21.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.833+0000 7f57ced9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57d00c8430 0x7f57d00c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:21.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.833+0000 7f57ced9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57d00c8430 0x7f57d00c88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38078/0 (socket says 172.21.15.73:38078) 2024-04-26T20:52:21.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.833+0000 7f57ced9d640 1 -- 172.21.15.73:0/2341012454 learned_addr learned my addr 172.21.15.73:0/2341012454 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:21.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.834+0000 7f57ce59c640 1 --2- 172.21.15.73:0/2341012454 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57bc0039b0 0x7f57d00b8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:21.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.834+0000 7f57c7fff640 1 --2- 172.21.15.73:0/2341012454 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f57d00cc040 0x7f57d00cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:21.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.834+0000 7f57ced9d640 1 -- 172.21.15.73:0/2341012454 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f57d00cc040 msgr2=0x7f57d00cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:21.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.834+0000 7f57ced9d640 1 --2- 172.21.15.73:0/2341012454 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f57d00cc040 0x7f57d00cc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:21.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.834+0000 7f57ced9d640 1 -- 172.21.15.73:0/2341012454 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57bc0039b0 msgr2=0x7f57d00b8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:21.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.834+0000 7f57ced9d640 1 --2- 172.21.15.73:0/2341012454 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57bc0039b0 0x7f57d00b8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:21.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.834+0000 7f57ced9d640 1 -- 172.21.15.73:0/2341012454 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57d01f50d0 con 0x7f57d00c8430 2024-04-26T20:52:21.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.834+0000 7f57c7fff640 1 --2- 172.21.15.73:0/2341012454 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f57d00cc040 0x7f57d00cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:52:21.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.834+0000 7f57ce59c640 1 --2- 172.21.15.73:0/2341012454 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57bc0039b0 0x7f57d00b8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:52:21.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.834+0000 7f57ced9d640 1 --2- 172.21.15.73:0/2341012454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57d00c8430 0x7f57d00c88b0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f57d0073000 tx=0x7f57c800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:21.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.834+0000 7f57c6ffd640 1 -- 172.21.15.73:0/2341012454 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c800ee50 con 0x7f57d00c8430 2024-04-26T20:52:21.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.834+0000 7f57c6ffd640 1 -- 172.21.15.73:0/2341012454 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57c8018070 con 0x7f57d00c8430 2024-04-26T20:52:21.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.834+0000 7f57c6ffd640 1 -- 172.21.15.73:0/2341012454 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c8003dc0 con 0x7f57d00c8430 2024-04-26T20:52:21.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.834+0000 7f57d4ebd640 1 -- 172.21.15.73:0/2341012454 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57d01f5370 con 0x7f57d00c8430 2024-04-26T20:52:21.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.834+0000 7f57d4ebd640 1 -- 172.21.15.73:0/2341012454 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57d00c9160 con 0x7f57d00c8430 2024-04-26T20:52:21.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.835+0000 7f57d4ebd640 1 -- 172.21.15.73:0/2341012454 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57d00694f0 con 0x7f57d00c8430 2024-04-26T20:52:21.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.835+0000 7f57c6ffd640 1 -- 172.21.15.73:0/2341012454 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f57c80050d0 con 0x7f57d00c8430 2024-04-26T20:52:21.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.836+0000 7f57c6ffd640 1 --2- 172.21.15.73:0/2341012454 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f57b006c920 0x7f57b006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:21.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.836+0000 7f57c6ffd640 1 -- 172.21.15.73:0/2341012454 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7f57c808ba00 con 0x7f57d00c8430 2024-04-26T20:52:21.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.836+0000 7f57ce59c640 1 --2- 172.21.15.73:0/2341012454 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f57b006c920 0x7f57b006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:21.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.836+0000 7f57ce59c640 1 --2- 172.21.15.73:0/2341012454 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f57b006c920 0x7f57b006ede0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f57c0033000 tx=0x7f57c0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:21.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:21.840+0000 7f57c6ffd640 1 -- 172.21.15.73:0/2341012454 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f57c8059d30 con 0x7f57d00c8430 2024-04-26T20:52:22.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.018+0000 7f57d4ebd640 1 -- 172.21.15.73:0/2341012454 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f57d00d2470 con 0x7f57d00c8430 2024-04-26T20:52:22.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.019+0000 7f57c6ffd640 1 -- 172.21.15.73:0/2341012454 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) ==== 74+0+16955 (secure 0 0 0) 0x7f57c801be40 con 0x7f57d00c8430 2024-04-26T20:52:22.021 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:52:22.021 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":51,"fsid":"fbd3f548-b547-4713-856b-09f98f5d1764","created":"2024-04-26T20:50:07.876786+0000","modified":"2024-04-26T20:52:20.754772+0000","last_up_change":"2024-04-26T20:50:19.082741+0000","last_in_change":"2024-04-26T20:50:10.018400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:50:19.993731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:52:18.960897+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-26T20:52:19.704261+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d6f4f3-214a-4392-a02f-c33cbcab3c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6809","nonce":3461705124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6811","nonce":3461705124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6815","nonce":3461705124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6813","nonce":3461705124}]},"public_addr":"172.21.15.73:6809/3461705124","cluster_addr":"172.21.15.73:6811/3461705124","heartbeat_back_addr":"172.21.15.73:6815/3461705124","heartbeat_front_addr":"172.21.15.73:6813/3461705124","state":["exists","up"]},{"osd":1,"uuid":"786bf8ca-6ddc-4b3c-be6c-9152398bd5fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6817","nonce":2827619403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6819","nonce":2827619403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6825","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6827","nonce":2827619403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6821","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6823","nonce":2827619403}]},"public_addr":"172.21.15.73:6817/2827619403","cluster_addr":"172.21.15.73:6819/2827619403","heartbeat_back_addr":"172.21.15.73:6827/2827619403","heartbeat_front_addr":"172.21.15.73:6823/2827619403","state":["exists","up"]},{"osd":2,"uuid":"90963b1a-5437-4ef8-ad8b-6b74c534eeb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6801","nonce":2542681064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6803","nonce":2542681064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6808","nonce":2542681064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6805","nonce":2542681064}]},"public_addr":"172.21.15.73:6801/2542681064","cluster_addr":"172.21.15.73:6803/2542681064","heartbeat_back_addr":"172.21.15.73:6808/2542681064","heartbeat_front_addr":"172.21.15.73:6805/2542681064","state":["exists","up"]},{"osd":3,"uuid":"bd3d2148-6e38-41b4-a453-fe6e3b8b4ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6822","nonce":1604042718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6826","nonce":1604042718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6831","nonce":1604042718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6829","nonce":1604042718}]},"public_addr":"172.21.15.73:6822/1604042718","cluster_addr":"172.21.15.73:6826/1604042718","heartbeat_back_addr":"172.21.15.73:6831/1604042718","heartbeat_front_addr":"172.21.15.73:6829/1604042718","state":["exists","up"]},{"osd":4,"uuid":"e8668dc8-a23f-46d2-8311-cb882da08e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6817","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6819","nonce":3591055561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6824","nonce":3591055561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6831","nonce":3591055561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6828","nonce":3591055561}]},"public_addr":"172.21.15.120:6819/3591055561","cluster_addr":"172.21.15.120:6824/3591055561","heartbeat_back_addr":"172.21.15.120:6831/3591055561","heartbeat_front_addr":"172.21.15.120:6828/3591055561","state":["exists","up"]},{"osd":5,"uuid":"cbac0bdb-96bd-4852-bcce-e564495c3fcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6809","nonce":819450433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6811","nonce":819450433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6815","nonce":819450433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6813","nonce":819450433}]},"public_addr":"172.21.15.120:6809/819450433","cluster_addr":"172.21.15.120:6811/819450433","heartbeat_back_addr":"172.21.15.120:6815/819450433","heartbeat_front_addr":"172.21.15.120:6813/819450433","state":["exists","up"]},{"osd":6,"uuid":"1884b2a8-569f-4272-96a9-f25a09364dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6801","nonce":3946210727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6803","nonce":3946210727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6807","nonce":3946210727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6805","nonce":3946210727}]},"public_addr":"172.21.15.120:6801/3946210727","cluster_addr":"172.21.15.120:6803/3946210727","heartbeat_back_addr":"172.21.15.120:6807/3946210727","heartbeat_front_addr":"172.21.15.120:6805/3946210727","state":["exists","up"]},{"osd":7,"uuid":"c0c3bf6d-a4c5-4f1e-aa2d-330951830666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6818","nonce":2915574674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6821","nonce":2915574674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6827","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6829","nonce":2915574674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6823","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6825","nonce":2915574674}]},"public_addr":"172.21.15.120:6818/2915574674","cluster_addr":"172.21.15.120:6821/2915574674","heartbeat_back_addr":"172.21.15.120:6829/2915574674","heartbeat_front_addr":"172.21.15.120:6825/2915574674","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.218216+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.208710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.112705+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.182789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.164324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.174212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.225396+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.261559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:52:22.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.021+0000 7f57d4ebd640 1 -- 172.21.15.73:0/2341012454 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f57b006c920 msgr2=0x7f57b006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:22.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.021+0000 7f57d4ebd640 1 --2- 172.21.15.73:0/2341012454 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f57b006c920 0x7f57b006ede0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f57c0033000 tx=0x7f57c0008030 comp rx=0 tx=0).stop 2024-04-26T20:52:22.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.021+0000 7f57d4ebd640 1 -- 172.21.15.73:0/2341012454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57d00c8430 msgr2=0x7f57d00c88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:22.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.021+0000 7f57d4ebd640 1 --2- 172.21.15.73:0/2341012454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57d00c8430 0x7f57d00c88b0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f57d0073000 tx=0x7f57c800ec00 comp rx=0 tx=0).stop 2024-04-26T20:52:22.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.021+0000 7f57d4ebd640 1 -- 172.21.15.73:0/2341012454 shutdown_connections 2024-04-26T20:52:22.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.021+0000 7f57d4ebd640 1 --2- 172.21.15.73:0/2341012454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57d00c8430 0x7f57d00c88b0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:22.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.021+0000 7f57d4ebd640 1 --2- 172.21.15.73:0/2341012454 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f57b006c920 0x7f57b006ede0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:22.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.021+0000 7f57d4ebd640 1 --2- 172.21.15.73:0/2341012454 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f57d00cc040 0x7f57d00cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:22.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.021+0000 7f57d4ebd640 1 --2- 172.21.15.73:0/2341012454 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57bc0039b0 0x7f57d00b8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:22.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.021+0000 7f57d4ebd640 1 -- 172.21.15.73:0/2341012454 >> 172.21.15.73:0/2341012454 conn(0x7f57d00d2da0 msgr2=0x7f57d00d3180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:22.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.021+0000 7f57d4ebd640 1 -- 172.21.15.73:0/2341012454 shutdown_connections 2024-04-26T20:52:22.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.021+0000 7f57d4ebd640 1 -- 172.21.15.73:0/2341012454 wait complete. 2024-04-26T20:52:22.034 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T20:52:22.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.132+0000 7fd473122640 1 Processor -- start 2024-04-26T20:52:22.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.132+0000 7fd473122640 1 -- start start 2024-04-26T20:52:22.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.132+0000 7fd473122640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd46c072700 0x7fd46c072b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:22.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.132+0000 7fd473122640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fd46c06ee70 0x7fd46c06f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:22.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.132+0000 7fd473122640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd46c0b5830 con 0x7fd46c159c00 2024-04-26T20:52:22.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.132+0000 7fd473122640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7fd46c15d3f0 con 0x7fd46c072700 2024-04-26T20:52:22.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.132+0000 7fd473122640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fd46c06e9e0 con 0x7fd46c06ee70 2024-04-26T20:52:22.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.133+0000 7fd46bfff640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd46c072700 0x7fd46c072b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:22.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.133+0000 7fd470ebe640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fd46c159c00 0x7fd46c0b4d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:36596/0 (socket says 172.21.15.73:36596) 2024-04-26T20:52:22.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.133+0000 7fd470ebe640 1 -- 172.21.15.73:0/1368377888 learned_addr learned my addr 172.21.15.73:0/1368377888 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:52:22.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.133+0000 7fd46bfff640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd46c072700 0x7fd46c072b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38092/0 (socket says 172.21.15.73:38092) 2024-04-26T20:52:22.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.133+0000 7fd46bfff640 1 -- 172.21.15.73:0/1368377888 >> v2:172.21.15.120:3300/0 conn(0x7fd46c06ee70 msgr2=0x7fd46c06f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:22.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.133+0000 7fd46bfff640 1 --2- 172.21.15.73:0/1368377888 >> v2:172.21.15.120:3300/0 conn(0x7fd46c06ee70 0x7fd46c06f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:22.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.133+0000 7fd46bfff640 1 -- 172.21.15.73:0/1368377888 >> v1:172.21.15.73:6789/0 conn(0x7fd46c159c00 legacy=0x7fd46c0b4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:22.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.133+0000 7fd46bfff640 1 -- 172.21.15.73:0/1368377888 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd46c0b5b10 con 0x7fd46c072700 2024-04-26T20:52:22.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.134+0000 7fd46bfff640 1 --2- 172.21.15.73:0/1368377888 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd46c072700 0x7fd46c072b90 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fd46c06d3c0 tx=0x7fd45c02a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d548d30feb2abf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:22.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.134+0000 7fd46b7fe640 1 -- 172.21.15.73:0/1368377888 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45c037070 con 0x7fd46c072700 2024-04-26T20:52:22.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.134+0000 7fd46b7fe640 1 -- 172.21.15.73:0/1368377888 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd45c02ad30 con 0x7fd46c072700 2024-04-26T20:52:22.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.134+0000 7fd46b7fe640 1 -- 172.21.15.73:0/1368377888 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45c032660 con 0x7fd46c072700 2024-04-26T20:52:22.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.134+0000 7fd473122640 1 -- 172.21.15.73:0/1368377888 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd46c072700 msgr2=0x7fd46c072b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:22.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.134+0000 7fd473122640 1 --2- 172.21.15.73:0/1368377888 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd46c072700 0x7fd46c072b90 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fd46c06d3c0 tx=0x7fd45c02a0a0 comp rx=0 tx=0).stop 2024-04-26T20:52:22.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.134+0000 7fd473122640 1 -- 172.21.15.73:0/1368377888 shutdown_connections 2024-04-26T20:52:22.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.134+0000 7fd473122640 1 --2- 172.21.15.73:0/1368377888 >> v2:172.21.15.120:3300/0 conn(0x7fd46c06ee70 0x7fd46c06f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:22.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.134+0000 7fd473122640 1 --2- 172.21.15.73:0/1368377888 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd46c072700 0x7fd46c072b90 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:22.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.134+0000 7fd473122640 1 -- 172.21.15.73:0/1368377888 >> 172.21.15.73:0/1368377888 conn(0x7fd46c0b3110 msgr2=0x7fd46c0b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:22.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.134+0000 7fd473122640 1 -- 172.21.15.73:0/1368377888 shutdown_connections 2024-04-26T20:52:22.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.134+0000 7fd473122640 1 -- 172.21.15.73:0/1368377888 wait complete. 2024-04-26T20:52:22.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.135+0000 7fd473122640 1 Processor -- start 2024-04-26T20:52:22.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.135+0000 7fd473122640 1 -- start start 2024-04-26T20:52:22.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.136+0000 7fd473122640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd46c06ee70 0x7fd46c158d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:22.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.136+0000 7fd473122640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd46c072700 0x7fd46c0afc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:22.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.136+0000 7fd473122640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd46c0ad810 0x7fd46c1e7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:22.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.136+0000 7fd473122640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd46c0b5480 con 0x7fd46c06ee70 2024-04-26T20:52:22.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.136+0000 7fd473122640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd46c15c760 con 0x7fd46c0ad810 2024-04-26T20:52:22.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.136+0000 7fd473122640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fd46c0adcc0 con 0x7fd46c072700 2024-04-26T20:52:22.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.136+0000 7fd470ebe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd46c06ee70 0x7fd46c158d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:22.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.136+0000 7fd470ebe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd46c06ee70 0x7fd46c158d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38100/0 (socket says 172.21.15.73:38100) 2024-04-26T20:52:22.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.136+0000 7fd470ebe640 1 -- 172.21.15.73:0/4142614674 learned_addr learned my addr 172.21.15.73:0/4142614674 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:22.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.137+0000 7fd46bfff640 1 --2- 172.21.15.73:0/4142614674 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd46c072700 0x7fd46c0afc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:22.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.137+0000 7fd4716bf640 1 --2- 172.21.15.73:0/4142614674 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd46c0ad810 0x7fd46c1e7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:22.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.137+0000 7fd470ebe640 1 -- 172.21.15.73:0/4142614674 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd46c072700 msgr2=0x7fd46c0afc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:22.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.137+0000 7fd470ebe640 1 --2- 172.21.15.73:0/4142614674 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd46c072700 0x7fd46c0afc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:22.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.137+0000 7fd470ebe640 1 -- 172.21.15.73:0/4142614674 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd46c0ad810 msgr2=0x7fd46c1e7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:22.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.137+0000 7fd470ebe640 1 --2- 172.21.15.73:0/4142614674 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd46c0ad810 0x7fd46c1e7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:22.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.137+0000 7fd470ebe640 1 -- 172.21.15.73:0/4142614674 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd46c0b03a0 con 0x7fd46c06ee70 2024-04-26T20:52:22.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.137+0000 7fd470ebe640 1 --2- 172.21.15.73:0/4142614674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd46c06ee70 0x7fd46c158d80 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fd46400c910 tx=0x7fd46400cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:22.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.137+0000 7fd4697fa640 1 -- 172.21.15.73:0/4142614674 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd464018070 con 0x7fd46c06ee70 2024-04-26T20:52:22.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.137+0000 7fd4697fa640 1 -- 172.21.15.73:0/4142614674 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd46400a700 con 0x7fd46c06ee70 2024-04-26T20:52:22.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.137+0000 7fd473122640 1 -- 172.21.15.73:0/4142614674 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd46c0b0680 con 0x7fd46c06ee70 2024-04-26T20:52:22.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.137+0000 7fd4697fa640 1 -- 172.21.15.73:0/4142614674 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd46400aaf0 con 0x7fd46c06ee70 2024-04-26T20:52:22.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.137+0000 7fd473122640 1 -- 172.21.15.73:0/4142614674 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd46c0aa940 con 0x7fd46c06ee70 2024-04-26T20:52:22.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.138+0000 7fd473122640 1 -- 172.21.15.73:0/4142614674 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd46c068fc0 con 0x7fd46c06ee70 2024-04-26T20:52:22.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.138+0000 7fd4697fa640 1 -- 172.21.15.73:0/4142614674 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd46401b070 con 0x7fd46c06ee70 2024-04-26T20:52:22.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.139+0000 7fd4697fa640 1 --2- 172.21.15.73:0/4142614674 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd44c06c920 0x7fd44c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:22.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.139+0000 7fd46bfff640 1 --2- 172.21.15.73:0/4142614674 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd44c06c920 0x7fd44c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:22.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.139+0000 7fd4697fa640 1 -- 172.21.15.73:0/4142614674 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7fd46408b3b0 con 0x7fd46c06ee70 2024-04-26T20:52:22.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.139+0000 7fd46bfff640 1 --2- 172.21.15.73:0/4142614674 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd44c06c920 0x7fd44c06ede0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd45c034000 tx=0x7fd45c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:22.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.143+0000 7fd4697fa640 1 -- 172.21.15.73:0/4142614674 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd4640596e0 con 0x7fd46c06ee70 2024-04-26T20:52:22.277 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:52:22.279 DEBUG:teuthology.orchestra.run.smithi120:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:52:22.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.360+0000 7fd473122640 1 -- 172.21.15.73:0/4142614674 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fd46c0694f0 con 0x7fd46c06ee70 2024-04-26T20:52:22.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.411+0000 7fd4697fa640 1 -- 172.21.15.73:0/4142614674 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v25) ==== 118+0+0 (secure 0 0 0) 0x7fd464058fe0 con 0x7fd46c06ee70 2024-04-26T20:52:22.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.413+0000 7fd473122640 1 -- 172.21.15.73:0/4142614674 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd44c06c920 msgr2=0x7fd44c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:22.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.413+0000 7fd473122640 1 --2- 172.21.15.73:0/4142614674 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd44c06c920 0x7fd44c06ede0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd45c034000 tx=0x7fd45c02e000 comp rx=0 tx=0).stop 2024-04-26T20:52:22.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.413+0000 7fd473122640 1 -- 172.21.15.73:0/4142614674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd46c06ee70 msgr2=0x7fd46c158d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:22.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.413+0000 7fd473122640 1 --2- 172.21.15.73:0/4142614674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd46c06ee70 0x7fd46c158d80 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fd46400c910 tx=0x7fd46400cdd0 comp rx=0 tx=0).stop 2024-04-26T20:52:22.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.413+0000 7fd473122640 1 -- 172.21.15.73:0/4142614674 shutdown_connections 2024-04-26T20:52:22.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.413+0000 7fd473122640 1 --2- 172.21.15.73:0/4142614674 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd44c06c920 0x7fd44c06ede0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:22.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.413+0000 7fd473122640 1 --2- 172.21.15.73:0/4142614674 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd46c0ad810 0x7fd46c1e7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:22.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.413+0000 7fd473122640 1 --2- 172.21.15.73:0/4142614674 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd46c072700 0x7fd46c0afc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:22.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.413+0000 7fd473122640 1 --2- 172.21.15.73:0/4142614674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd46c06ee70 0x7fd46c158d80 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:22.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.413+0000 7fd473122640 1 -- 172.21.15.73:0/4142614674 >> 172.21.15.73:0/4142614674 conn(0x7fd46c0b3110 msgr2=0x7fd46c06f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:22.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.413+0000 7fd473122640 1 -- 172.21.15.73:0/4142614674 shutdown_connections 2024-04-26T20:52:22.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.413+0000 7fd473122640 1 -- 172.21.15.73:0/4142614674 wait complete. 2024-04-26T20:52:22.426 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-26T20:52:22.426 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T20:52:22.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.523+0000 7f8217e45640 1 Processor -- start 2024-04-26T20:52:22.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.524+0000 7f8217e45640 1 -- start start 2024-04-26T20:52:22.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.524+0000 7f8217e45640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f82100b40d0 0x7f82101625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:22.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.524+0000 7f8217e45640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f8210162b30 0x7f82100b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:22.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.524+0000 7f8217e45640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82100d0a10 con 0x7f82100d1a00 2024-04-26T20:52:22.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.524+0000 7f8217e45640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f82100d0cf0 con 0x7f8210162b30 2024-04-26T20:52:22.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.524+0000 7f8217e45640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f82100d0fd0 con 0x7f82100b40d0 2024-04-26T20:52:22.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.525+0000 7f82153e0640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f82100b40d0 0x7f82101625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:22.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.525+0000 7f82163e2640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f8210162b30 0x7f82100b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:22.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.525+0000 7f82153e0640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f82100b40d0 0x7f82101625f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.73:41126/0 (socket says 172.21.15.73:41126) 2024-04-26T20:52:22.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.525+0000 7f82153e0640 1 -- 172.21.15.73:0/44934427 learned_addr learned my addr 172.21.15.73:0/44934427 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:22.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.525+0000 7f82153e0640 1 -- 172.21.15.73:0/44934427 >> v2:172.21.15.120:3300/0 conn(0x7f8210162b30 msgr2=0x7f82100b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:22.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.525+0000 7f82153e0640 1 --2- 172.21.15.73:0/44934427 >> v2:172.21.15.120:3300/0 conn(0x7f8210162b30 0x7f82100b4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:22.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.525+0000 7f82153e0640 1 -- 172.21.15.73:0/44934427 >> v1:172.21.15.120:6789/0 conn(0x7f82100d1a00 legacy=0x7f82100d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:22.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.525+0000 7f82153e0640 1 -- 172.21.15.73:0/44934427 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8210162fa0 con 0x7f82100b40d0 2024-04-26T20:52:22.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.525+0000 7f82163e2640 1 --2- 172.21.15.73:0/44934427 >> v2:172.21.15.120:3300/0 conn(0x7f8210162b30 0x7f82100b4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:52:22.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.526+0000 7f82153e0640 1 --2- 172.21.15.73:0/44934427 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f82100b40d0 0x7f82101625f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f820c009870 tx=0x7f820c02a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19ee21b2da14b2fe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:22.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.526+0000 7f8214bdf640 1 -- 172.21.15.73:0/44934427 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820c037070 con 0x7f82100b40d0 2024-04-26T20:52:22.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.526+0000 7f8214bdf640 1 -- 172.21.15.73:0/44934427 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f820c032440 con 0x7f82100b40d0 2024-04-26T20:52:22.529 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.526+0000 7f8214bdf640 1 -- 172.21.15.73:0/44934427 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820c032810 con 0x7f82100b40d0 2024-04-26T20:52:22.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.527+0000 7f8217e45640 1 -- 172.21.15.73:0/44934427 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f82100b40d0 msgr2=0x7f82101625f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:22.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.527+0000 7f8217e45640 1 --2- 172.21.15.73:0/44934427 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f82100b40d0 0x7f82101625f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f820c009870 tx=0x7f820c02a1c0 comp rx=0 tx=0).stop 2024-04-26T20:52:22.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.527+0000 7f8217e45640 1 -- 172.21.15.73:0/44934427 shutdown_connections 2024-04-26T20:52:22.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.527+0000 7f8217e45640 1 --2- 172.21.15.73:0/44934427 >> v2:172.21.15.120:3300/0 conn(0x7f8210162b30 0x7f82100b4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:22.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.527+0000 7f8217e45640 1 --2- 172.21.15.73:0/44934427 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f82100b40d0 0x7f82101625f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:22.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.527+0000 7f8217e45640 1 -- 172.21.15.73:0/44934427 >> 172.21.15.73:0/44934427 conn(0x7f82100d4fd0 msgr2=0x7f82100d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:22.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.527+0000 7f8217e45640 1 -- 172.21.15.73:0/44934427 shutdown_connections 2024-04-26T20:52:22.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.527+0000 7f8217e45640 1 -- 172.21.15.73:0/44934427 wait complete. 2024-04-26T20:52:22.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.528+0000 7f8217e45640 1 Processor -- start 2024-04-26T20:52:22.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.528+0000 7f8217e45640 1 -- start start 2024-04-26T20:52:22.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.528+0000 7f8217e45640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f82100b40d0 0x7f8210154560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:22.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.528+0000 7f8217e45640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f82100d1a00 0x7f82100cc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:22.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.528+0000 7f8217e45640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8210162b30 0x7f82100b9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:22.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.528+0000 7f8217e45640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f82100d2670 con 0x7f82100d1a00 2024-04-26T20:52:22.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.528+0000 7f8217e45640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f82100d24f0 con 0x7f8210162b30 2024-04-26T20:52:22.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.528+0000 7f8217e45640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f82100b5040 con 0x7f82100b40d0 2024-04-26T20:52:22.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.529+0000 7f82153e0640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f82100d1a00 0x7f82100cc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:22.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.529+0000 7f82153e0640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f82100d1a00 0x7f82100cc0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38114/0 (socket says 172.21.15.73:38114) 2024-04-26T20:52:22.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.529+0000 7f82153e0640 1 -- 172.21.15.73:0/3607286257 learned_addr learned my addr 172.21.15.73:0/3607286257 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:22.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.529+0000 7f82163e2640 1 --2- 172.21.15.73:0/3607286257 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8210162b30 0x7f82100b9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:22.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.529+0000 7f8215be1640 1 --2- 172.21.15.73:0/3607286257 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f82100b40d0 0x7f8210154560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:22.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.529+0000 7f82153e0640 1 -- 172.21.15.73:0/3607286257 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f82100b40d0 msgr2=0x7f8210154560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:22.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.529+0000 7f82153e0640 1 --2- 172.21.15.73:0/3607286257 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f82100b40d0 0x7f8210154560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:22.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.529+0000 7f82153e0640 1 -- 172.21.15.73:0/3607286257 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8210162b30 msgr2=0x7f82100b9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:22.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.529+0000 7f82153e0640 1 --2- 172.21.15.73:0/3607286257 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8210162b30 0x7f82100b9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:22.533 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.529+0000 7f82153e0640 1 -- 172.21.15.73:0/3607286257 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82100cc9f0 con 0x7f82100d1a00 2024-04-26T20:52:22.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.529+0000 7f82153e0640 1 --2- 172.21.15.73:0/3607286257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f82100d1a00 0x7f82100cc0a0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f820c000c00 tx=0x7f820c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:22.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.529+0000 7f82067fc640 1 -- 172.21.15.73:0/3607286257 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820c037040 con 0x7f82100d1a00 2024-04-26T20:52:22.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.529+0000 7f82067fc640 1 -- 172.21.15.73:0/3607286257 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f820c042070 con 0x7f82100d1a00 2024-04-26T20:52:22.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.529+0000 7f8217e45640 1 -- 172.21.15.73:0/3607286257 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82100cccd0 con 0x7f82100d1a00 2024-04-26T20:52:22.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.529+0000 7f82067fc640 1 -- 172.21.15.73:0/3607286257 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820c030860 con 0x7f82100d1a00 2024-04-26T20:52:22.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.529+0000 7f8217e45640 1 -- 172.21.15.73:0/3607286257 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82100cd290 con 0x7f82100d1a00 2024-04-26T20:52:22.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.530+0000 7f8217e45640 1 -- 172.21.15.73:0/3607286257 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82100694f0 con 0x7f82100d1a00 2024-04-26T20:52:22.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.531+0000 7f82067fc640 1 -- 172.21.15.73:0/3607286257 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f820c02ad10 con 0x7f82100d1a00 2024-04-26T20:52:22.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.531+0000 7f82067fc640 1 --2- 172.21.15.73:0/3607286257 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f81f406c920 0x7f81f406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:22.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.531+0000 7f82067fc640 1 -- 172.21.15.73:0/3607286257 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f820c0ac250 con 0x7f82100d1a00 2024-04-26T20:52:22.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.531+0000 7f8215be1640 1 --2- 172.21.15.73:0/3607286257 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f81f406c920 0x7f81f406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:22.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.532+0000 7f8215be1640 1 --2- 172.21.15.73:0/3607286257 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f81f406c920 0x7f81f406ede0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8200001640 tx=0x7f8200009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:22.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.535+0000 7f82067fc640 1 -- 172.21.15.73:0/3607286257 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f820c07a4f0 con 0x7f82100d1a00 2024-04-26T20:52:22.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:22.750+0000 7f8217e45640 1 -- 172.21.15.73:0/3607286257 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f82100b3770 con 0x7f82100d1a00 2024-04-26T20:52:23.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.416+0000 7f82067fc640 1 -- 172.21.15.73:0/3607286257 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v26) ==== 122+0+0 (secure 0 0 0) 0x7f820c079df0 con 0x7f82100d1a00 2024-04-26T20:52:23.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.418+0000 7f8217e45640 1 -- 172.21.15.73:0/3607286257 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f81f406c920 msgr2=0x7f81f406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:23.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.418+0000 7f8217e45640 1 --2- 172.21.15.73:0/3607286257 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f81f406c920 0x7f81f406ede0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8200001640 tx=0x7f8200009040 comp rx=0 tx=0).stop 2024-04-26T20:52:23.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.418+0000 7f8217e45640 1 -- 172.21.15.73:0/3607286257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f82100d1a00 msgr2=0x7f82100cc0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:23.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.418+0000 7f8217e45640 1 --2- 172.21.15.73:0/3607286257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f82100d1a00 0x7f82100cc0a0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f820c000c00 tx=0x7f820c0046c0 comp rx=0 tx=0).stop 2024-04-26T20:52:23.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.418+0000 7f8217e45640 1 -- 172.21.15.73:0/3607286257 shutdown_connections 2024-04-26T20:52:23.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.418+0000 7f8217e45640 1 --2- 172.21.15.73:0/3607286257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f82100d1a00 0x7f82100cc0a0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:23.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.418+0000 7f8217e45640 1 --2- 172.21.15.73:0/3607286257 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f81f406c920 0x7f81f406ede0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:23.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.418+0000 7f8217e45640 1 --2- 172.21.15.73:0/3607286257 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8210162b30 0x7f82100b9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:23.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.418+0000 7f8217e45640 1 --2- 172.21.15.73:0/3607286257 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f82100b40d0 0x7f8210154560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:23.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.418+0000 7f8217e45640 1 -- 172.21.15.73:0/3607286257 >> 172.21.15.73:0/3607286257 conn(0x7f82100d4fd0 msgr2=0x7f82100d1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:23.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.418+0000 7f8217e45640 1 -- 172.21.15.73:0/3607286257 shutdown_connections 2024-04-26T20:52:23.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.418+0000 7f8217e45640 1 -- 172.21.15.73:0/3607286257 wait complete. 2024-04-26T20:52:23.432 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T20:52:23.432 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:52:23.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.528+0000 7f8bcf4c8640 1 Processor -- start 2024-04-26T20:52:23.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.529+0000 7f8bcf4c8640 1 -- start start 2024-04-26T20:52:23.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.529+0000 7f8bcf4c8640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8bc80b2340 0x7f8bc80cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:23.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.529+0000 7f8bcf4c8640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f8bc80cd930 0x7f8bc80d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:23.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.529+0000 7f8bcf4c8640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bc80d10d0 con 0x7f8bc8162810 2024-04-26T20:52:23.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.529+0000 7f8bcf4c8640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f8bc80d0750 con 0x7f8bc80cd930 2024-04-26T20:52:23.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.529+0000 7f8bcf4c8640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f8bc80d16a0 con 0x7f8bc80b2340 2024-04-26T20:52:23.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.530+0000 7f8bcd264640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f8bc8162810 0x7f8bc8070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:36610/0 (socket says 172.21.15.73:36610) 2024-04-26T20:52:23.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.530+0000 7f8bcd264640 1 -- 172.21.15.73:0/2413150735 learned_addr learned my addr 172.21.15.73:0/2413150735 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:52:23.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.530+0000 7f8bbffff640 1 -- 172.21.15.73:0/2413150735 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3817120055 0 0) 0x7f8bc80d10d0 con 0x7f8bc8162810 2024-04-26T20:52:23.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.530+0000 7f8bbffff640 1 -- 172.21.15.73:0/2413150735 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8bb4003520 con 0x7f8bc8162810 2024-04-26T20:52:23.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.530+0000 7f8bcca63640 1 --2- 172.21.15.73:0/2413150735 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8bc80b2340 0x7f8bc80cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:23.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.530+0000 7f8bcda65640 1 --2- 172.21.15.73:0/2413150735 >> v2:172.21.15.120:3300/0 conn(0x7f8bc80cd930 0x7f8bc80d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:23.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.530+0000 7f8bbffff640 1 -- 172.21.15.73:0/2413150735 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 532149430 0 0) 0x7f8bb4003520 con 0x7f8bc8162810 2024-04-26T20:52:23.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.530+0000 7f8bbffff640 1 -- 172.21.15.73:0/2413150735 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8bc80b2340 msgr2=0x7f8bc80cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:23.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.530+0000 7f8bbffff640 1 --2- 172.21.15.73:0/2413150735 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8bc80b2340 0x7f8bc80cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:23.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.530+0000 7f8bbffff640 1 -- 172.21.15.73:0/2413150735 >> v2:172.21.15.120:3300/0 conn(0x7f8bc80cd930 msgr2=0x7f8bc80d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:23.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.530+0000 7f8bbffff640 1 --2- 172.21.15.73:0/2413150735 >> v2:172.21.15.120:3300/0 conn(0x7f8bc80cd930 0x7f8bc80d1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:23.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.530+0000 7f8bbffff640 1 -- 172.21.15.73:0/2413150735 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bc80d1370 con 0x7f8bc8162810 2024-04-26T20:52:23.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.530+0000 7f8bbffff640 1 -- 172.21.15.73:0/2413150735 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7f8bc4001670 con 0x7f8bc8162810 2024-04-26T20:52:23.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.530+0000 7f8bbffff640 1 -- 172.21.15.73:0/2413150735 >> v1:172.21.15.73:6789/0 conn(0x7f8bc8162810 legacy=0x7f8bc8070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:23.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.530+0000 7f8bbffff640 1 --2- 172.21.15.73:0/2413150735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8bb40039b0 0x7f8bb4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:23.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.530+0000 7f8bbffff640 1 --2- 172.21.15.73:0/2413150735 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8bc80b2340 0x7f8bb4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:23.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.530+0000 7f8bbffff640 1 --2- 172.21.15.73:0/2413150735 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8bb4004d40 0x7f8bb40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:23.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.530+0000 7f8bbffff640 1 -- 172.21.15.73:0/2413150735 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc80d0750 con 0x7f8bb40039b0 2024-04-26T20:52:23.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.530+0000 7f8bbffff640 1 -- 172.21.15.73:0/2413150735 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc80d16a0 con 0x7f8bb4004d40 2024-04-26T20:52:23.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.530+0000 7f8bbffff640 1 -- 172.21.15.73:0/2413150735 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f8bb400b9c0 con 0x7f8bc80b2340 2024-04-26T20:52:23.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.530+0000 7f8bcd264640 1 --2- 172.21.15.73:0/2413150735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8bb40039b0 0x7f8bb4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:23.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.530+0000 7f8bcca63640 1 --2- 172.21.15.73:0/2413150735 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8bc80b2340 0x7f8bb4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:23.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.530+0000 7f8bcda65640 1 --2- 172.21.15.73:0/2413150735 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8bb4004d40 0x7f8bb40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:23.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.530+0000 7f8bcd264640 1 -- 172.21.15.73:0/2413150735 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8bc80b2340 msgr2=0x7f8bb4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:23.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.530+0000 7f8bcd264640 1 --2- 172.21.15.73:0/2413150735 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8bc80b2340 0x7f8bb4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:23.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.530+0000 7f8bcd264640 1 -- 172.21.15.73:0/2413150735 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8bb4004d40 msgr2=0x7f8bb40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:23.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.530+0000 7f8bcd264640 1 --2- 172.21.15.73:0/2413150735 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8bb4004d40 0x7f8bb40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:23.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.531+0000 7f8bcd264640 1 -- 172.21.15.73:0/2413150735 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bb4003520 con 0x7f8bb40039b0 2024-04-26T20:52:23.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.531+0000 7f8bcd264640 1 --2- 172.21.15.73:0/2413150735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8bb40039b0 0x7f8bb4003dd0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4007fd0 tx=0x7f8bb4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e957513aa1d22be5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:23.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.531+0000 7f8bbffff640 1 -- 172.21.15.73:0/2413150735 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4007d50 con 0x7f8bb40039b0 2024-04-26T20:52:23.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.531+0000 7f8bbffff640 1 -- 172.21.15.73:0/2413150735 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bc4005ad0 con 0x7f8bb40039b0 2024-04-26T20:52:23.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.531+0000 7f8bbffff640 1 -- 172.21.15.73:0/2413150735 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4007070 con 0x7f8bb40039b0 2024-04-26T20:52:23.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.532+0000 7f8bcf4c8640 1 -- 172.21.15.73:0/2413150735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8bb40039b0 msgr2=0x7f8bb4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:23.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.532+0000 7f8bcf4c8640 1 --2- 172.21.15.73:0/2413150735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8bb40039b0 0x7f8bb4003dd0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4007fd0 tx=0x7f8bb4005830 comp rx=0 tx=0).stop 2024-04-26T20:52:23.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.532+0000 7f8bcda65640 1 -- 172.21.15.73:0/2413150735 reap_dead start 2024-04-26T20:52:23.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.532+0000 7f8bcf4c8640 1 -- 172.21.15.73:0/2413150735 shutdown_connections 2024-04-26T20:52:23.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.532+0000 7f8bcf4c8640 1 -- 172.21.15.73:0/2413150735 >> 172.21.15.73:0/2413150735 conn(0x7f8bc80d2da0 msgr2=0x7f8bc80d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:23.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.532+0000 7f8bcf4c8640 1 -- 172.21.15.73:0/2413150735 shutdown_connections 2024-04-26T20:52:23.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.532+0000 7f8bcf4c8640 1 -- 172.21.15.73:0/2413150735 wait complete. 2024-04-26T20:52:23.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.533+0000 7f8bcf4c8640 1 Processor -- start 2024-04-26T20:52:23.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.533+0000 7f8bcf4c8640 1 -- start start 2024-04-26T20:52:23.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.534+0000 7f8bcf4c8640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8bb40039b0 0x7f8bc80c94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:23.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.534+0000 7f8bcf4c8640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8bc81f5e00 0x7f8bc81f4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:23.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.534+0000 7f8bcf4c8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8bc80b33c0 0x7f8bc80b3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:23.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.534+0000 7f8bcf4c8640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc80cd3f0 con 0x7f8bc81f5e00 2024-04-26T20:52:23.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.534+0000 7f8bcf4c8640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc80d0470 con 0x7f8bc80b33c0 2024-04-26T20:52:23.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.534+0000 7f8bcf4c8640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f8bc80cd570 con 0x7f8bb40039b0 2024-04-26T20:52:23.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.534+0000 7f8bcca63640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8bc81f5e00 0x7f8bc81f4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:23.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.534+0000 7f8bcca63640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8bc81f5e00 0x7f8bc81f4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38138/0 (socket says 172.21.15.73:38138) 2024-04-26T20:52:23.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.534+0000 7f8bcca63640 1 -- 172.21.15.73:0/3336302881 learned_addr learned my addr 172.21.15.73:0/3336302881 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:23.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.534+0000 7f8bcda65640 1 --2- 172.21.15.73:0/3336302881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8bc80b33c0 0x7f8bc80b3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:23.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.534+0000 7f8bcd264640 1 --2- 172.21.15.73:0/3336302881 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8bb40039b0 0x7f8bc80c94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:23.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.534+0000 7f8bcca63640 1 -- 172.21.15.73:0/3336302881 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8bb40039b0 msgr2=0x7f8bc80c94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:23.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.534+0000 7f8bcca63640 1 --2- 172.21.15.73:0/3336302881 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8bb40039b0 0x7f8bc80c94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:23.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.534+0000 7f8bcca63640 1 -- 172.21.15.73:0/3336302881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8bc80b33c0 msgr2=0x7f8bc80b3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:23.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.534+0000 7f8bcca63640 1 --2- 172.21.15.73:0/3336302881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8bc80b33c0 0x7f8bc80b3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:23.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.534+0000 7f8bcca63640 1 -- 172.21.15.73:0/3336302881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bc80c8450 con 0x7f8bc81f5e00 2024-04-26T20:52:23.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.534+0000 7f8bcca63640 1 --2- 172.21.15.73:0/3336302881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8bc81f5e00 0x7f8bc81f4ba0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8011540 tx=0x7f8bb8011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:23.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.534+0000 7f8bbdffb640 1 -- 172.21.15.73:0/3336302881 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb8015930 con 0x7f8bc81f5e00 2024-04-26T20:52:23.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.534+0000 7f8bbdffb640 1 -- 172.21.15.73:0/3336302881 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bb8015ad0 con 0x7f8bc81f5e00 2024-04-26T20:52:23.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.534+0000 7f8bcf4c8640 1 -- 172.21.15.73:0/3336302881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bc80c8730 con 0x7f8bc81f5e00 2024-04-26T20:52:23.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.534+0000 7f8bbdffb640 1 -- 172.21.15.73:0/3336302881 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb800f460 con 0x7f8bc81f5e00 2024-04-26T20:52:23.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.535+0000 7f8bcf4c8640 1 -- 172.21.15.73:0/3336302881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bc80c8cf0 con 0x7f8bc81f5e00 2024-04-26T20:52:23.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.536+0000 7f8bcf4c8640 1 -- 172.21.15.73:0/3336302881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bc80b8ab0 con 0x7f8bc81f5e00 2024-04-26T20:52:23.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.536+0000 7f8bbdffb640 1 -- 172.21.15.73:0/3336302881 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8bb800f600 con 0x7f8bc81f5e00 2024-04-26T20:52:23.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.537+0000 7f8bbdffb640 1 --2- 172.21.15.73:0/3336302881 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f8bac06c920 0x7f8bac06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:23.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.537+0000 7f8bcd264640 1 --2- 172.21.15.73:0/3336302881 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f8bac06c920 0x7f8bac06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:23.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.537+0000 7f8bbdffb640 1 -- 172.21.15.73:0/3336302881 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f8bb808c620 con 0x7f8bc81f5e00 2024-04-26T20:52:23.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.537+0000 7f8bcd264640 1 --2- 172.21.15.73:0/3336302881 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f8bac06c920 0x7f8bac06ede0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4033000 tx=0x7f8bc4009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:23.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.541+0000 7f8bbdffb640 1 -- 172.21.15.73:0/3336302881 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8bc80b8ab0 con 0x7f8bc81f5e00 2024-04-26T20:52:23.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.751+0000 7f8bcf4c8640 1 -- 172.21.15.73:0/3336302881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8bc80694f0 con 0x7f8bc81f5e00 2024-04-26T20:52:23.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.751+0000 7f8bbdffb640 1 -- 172.21.15.73:0/3336302881 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5181 (secure 0 0 0) 0x7f8bb8019030 con 0x7f8bc81f5e00 2024-04-26T20:52:23.754 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:52:23.754 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5096,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6834/528576665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":528576665},{"type":"v1","addr":"172.21.15.120:6834","nonce":528576665}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":5099,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/824086075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6833","nonce":824086075},{"type":"v1","addr":"172.21.15.120:6835","nonce":824086075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":5853,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/1515733826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1515733826},{"type":"v1","addr":"172.21.15.73:6835","nonce":1515733826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:52:20.754736+0000","modified":"2024-04-26T20:52:22.752295+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5856},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5856":{"gid":5856,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/2770303348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2770303348},{"type":"v1","addr":"172.21.15.73:6837","nonce":2770303348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5856,"qdb_cluster":[5856]},"id":3}]} 2024-04-26T20:52:23.754 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 26 2024-04-26T20:52:23.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.753+0000 7f8bcf4c8640 1 -- 172.21.15.73:0/3336302881 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f8bac06c920 msgr2=0x7f8bac06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:23.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.753+0000 7f8bcf4c8640 1 --2- 172.21.15.73:0/3336302881 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f8bac06c920 0x7f8bac06ede0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4033000 tx=0x7f8bc4009030 comp rx=0 tx=0).stop 2024-04-26T20:52:23.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.753+0000 7f8bcf4c8640 1 -- 172.21.15.73:0/3336302881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8bc81f5e00 msgr2=0x7f8bc81f4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:23.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.753+0000 7f8bcf4c8640 1 --2- 172.21.15.73:0/3336302881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8bc81f5e00 0x7f8bc81f4ba0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8011540 tx=0x7f8bb8011a00 comp rx=0 tx=0).stop 2024-04-26T20:52:23.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.753+0000 7f8bcf4c8640 1 -- 172.21.15.73:0/3336302881 shutdown_connections 2024-04-26T20:52:23.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.753+0000 7f8bcf4c8640 1 --2- 172.21.15.73:0/3336302881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8bc81f5e00 0x7f8bc81f4ba0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:23.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.753+0000 7f8bcf4c8640 1 --2- 172.21.15.73:0/3336302881 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f8bac06c920 0x7f8bac06ede0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:23.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.753+0000 7f8bcf4c8640 1 --2- 172.21.15.73:0/3336302881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8bc80b33c0 0x7f8bc80b3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:23.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.753+0000 7f8bcf4c8640 1 --2- 172.21.15.73:0/3336302881 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8bb40039b0 0x7f8bc80c94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:23.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.753+0000 7f8bcf4c8640 1 -- 172.21.15.73:0/3336302881 >> 172.21.15.73:0/3336302881 conn(0x7f8bc80d2da0 msgr2=0x7f8bc8070350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:23.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.754+0000 7f8bcf4c8640 1 -- 172.21.15.73:0/3336302881 shutdown_connections 2024-04-26T20:52:23.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.754+0000 7f8bcf4c8640 1 -- 172.21.15.73:0/3336302881 wait complete. 2024-04-26T20:52:23.767 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:52:23.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.861+0000 7f790bd87640 1 Processor -- start 2024-04-26T20:52:23.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.861+0000 7f790bd87640 1 -- start start 2024-04-26T20:52:23.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.862+0000 7f790bd87640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7904072950 0x7f7904167b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:23.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.862+0000 7f790bd87640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f790406e240 con 0x7f7904071e60 2024-04-26T20:52:23.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.862+0000 7f790bd87640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f790406fc30 con 0x7f79040cc5b0 2024-04-26T20:52:23.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.862+0000 7f790bd87640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f79040701f0 con 0x7f7904072950 2024-04-26T20:52:23.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.862+0000 7f7909b23640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f7904071e60 0x7f790415df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:36622/0 (socket says 172.21.15.73:36622) 2024-04-26T20:52:23.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.862+0000 7f7909b23640 1 -- 172.21.15.73:0/1029106547 learned_addr learned my addr 172.21.15.73:0/1029106547 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:52:23.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.862+0000 7f7908b21640 1 -- 172.21.15.73:0/1029106547 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4163066656 0 0) 0x7f790406e240 con 0x7f7904071e60 2024-04-26T20:52:23.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.862+0000 7f790a324640 1 --2- 172.21.15.73:0/1029106547 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7904072950 0x7f7904167b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:23.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.862+0000 7f7908b21640 1 -- 172.21.15.73:0/1029106547 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f78f0003520 con 0x7f7904071e60 2024-04-26T20:52:23.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.863+0000 7f7908b21640 1 -- 172.21.15.73:0/1029106547 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2764051851 0 0) 0x7f78f0003520 con 0x7f7904071e60 2024-04-26T20:52:23.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.863+0000 7f7908b21640 1 -- 172.21.15.73:0/1029106547 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7904072950 msgr2=0x7f7904167b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:23.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.863+0000 7f7908b21640 1 --2- 172.21.15.73:0/1029106547 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7904072950 0x7f7904167b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:23.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.863+0000 7f7908b21640 1 -- 172.21.15.73:0/1029106547 >> v1:172.21.15.120:6789/0 conn(0x7f79040cc5b0 legacy=0x7f7904072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:23.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.863+0000 7f7908b21640 1 -- 172.21.15.73:0/1029106547 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f790406ff10 con 0x7f7904071e60 2024-04-26T20:52:23.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.863+0000 7f7908b21640 1 -- 172.21.15.73:0/1029106547 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7f7900001670 con 0x7f7904071e60 2024-04-26T20:52:23.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.863+0000 7f7908b21640 1 -- 172.21.15.73:0/1029106547 >> v1:172.21.15.73:6789/0 conn(0x7f7904071e60 legacy=0x7f790415df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:23.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.863+0000 7f7908b21640 1 --2- 172.21.15.73:0/1029106547 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78f00039e0 0x7f78f0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:23.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.863+0000 7f7908b21640 1 --2- 172.21.15.73:0/1029106547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78f0004800 0x7f78f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:23.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.863+0000 7f7908b21640 1 --2- 172.21.15.73:0/1029106547 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7904072950 0x7f78f000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:23.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.863+0000 7f7908b21640 1 -- 172.21.15.73:0/1029106547 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f79040701f0 con 0x7f78f0004800 2024-04-26T20:52:23.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.863+0000 7f7908b21640 1 -- 172.21.15.73:0/1029106547 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f78f0005550 con 0x7f78f00039e0 2024-04-26T20:52:23.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.863+0000 7f7908b21640 1 -- 172.21.15.73:0/1029106547 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f78f000bf00 con 0x7f7904072950 2024-04-26T20:52:23.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.863+0000 7f7909322640 1 --2- 172.21.15.73:0/1029106547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78f0004800 0x7f78f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:23.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.863+0000 7f7909b23640 1 --2- 172.21.15.73:0/1029106547 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78f00039e0 0x7f78f0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:23.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.863+0000 7f790a324640 1 --2- 172.21.15.73:0/1029106547 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7904072950 0x7f78f000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:23.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.863+0000 7f7909322640 1 -- 172.21.15.73:0/1029106547 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7904072950 msgr2=0x7f78f000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:23.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.863+0000 7f7909322640 1 --2- 172.21.15.73:0/1029106547 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7904072950 0x7f78f000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:23.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.863+0000 7f7909322640 1 -- 172.21.15.73:0/1029106547 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78f00039e0 msgr2=0x7f78f0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:23.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.863+0000 7f7909322640 1 --2- 172.21.15.73:0/1029106547 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f78f00039e0 0x7f78f0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:23.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.863+0000 7f7909322640 1 -- 172.21.15.73:0/1029106547 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78f0003520 con 0x7f78f0004800 2024-04-26T20:52:23.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.864+0000 7f7909322640 1 --2- 172.21.15.73:0/1029106547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78f0004800 0x7f78f0004c80 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f78f4001750 tx=0x7f78f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb5d5a6b6e44f98c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:23.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.864+0000 7f7908b21640 1 -- 172.21.15.73:0/1029106547 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f4001370 con 0x7f78f0004800 2024-04-26T20:52:23.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.864+0000 7f7908b21640 1 -- 172.21.15.73:0/1029106547 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78f40052d0 con 0x7f78f0004800 2024-04-26T20:52:23.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.864+0000 7f7908b21640 1 -- 172.21.15.73:0/1029106547 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f402abe0 con 0x7f78f0004800 2024-04-26T20:52:23.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.864+0000 7f790bd87640 1 -- 172.21.15.73:0/1029106547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78f0004800 msgr2=0x7f78f0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:23.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.864+0000 7f790bd87640 1 --2- 172.21.15.73:0/1029106547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78f0004800 0x7f78f0004c80 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f78f4001750 tx=0x7f78f0005830 comp rx=0 tx=0).stop 2024-04-26T20:52:23.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.864+0000 7f790a324640 1 -- 172.21.15.73:0/1029106547 reap_dead start 2024-04-26T20:52:23.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.865+0000 7f790bd87640 1 -- 172.21.15.73:0/1029106547 shutdown_connections 2024-04-26T20:52:23.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.865+0000 7f790bd87640 1 -- 172.21.15.73:0/1029106547 >> 172.21.15.73:0/1029106547 conn(0x7f79040d0be0 msgr2=0x7f79040d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:23.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.865+0000 7f790bd87640 1 -- 172.21.15.73:0/1029106547 shutdown_connections 2024-04-26T20:52:23.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.865+0000 7f790bd87640 1 -- 172.21.15.73:0/1029106547 wait complete. 2024-04-26T20:52:23.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.866+0000 7f790bd87640 1 Processor -- start 2024-04-26T20:52:23.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.866+0000 7f790bd87640 1 -- start start 2024-04-26T20:52:23.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.866+0000 7f790bd87640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f78f0004800 0x7f790414d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:23.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.866+0000 7f790bd87640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7904156fc0 0x7f79041f2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:23.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.866+0000 7f790bd87640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7904156a70 0x7f790415e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:23.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.866+0000 7f790bd87640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f790406ddf0 con 0x7f7904156fc0 2024-04-26T20:52:23.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.866+0000 7f790bd87640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f790406f8e0 con 0x7f7904156a70 2024-04-26T20:52:23.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.866+0000 7f790bd87640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f790406df70 con 0x7f78f0004800 2024-04-26T20:52:23.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.867+0000 7f7909322640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7904156fc0 0x7f79041f2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:23.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.867+0000 7f7909322640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7904156fc0 0x7f79041f2bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38156/0 (socket says 172.21.15.73:38156) 2024-04-26T20:52:23.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.867+0000 7f7909322640 1 -- 172.21.15.73:0/517385708 learned_addr learned my addr 172.21.15.73:0/517385708 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:23.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.867+0000 7f790a324640 1 --2- 172.21.15.73:0/517385708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7904156a70 0x7f790415e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:23.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.867+0000 7f7909b23640 1 --2- 172.21.15.73:0/517385708 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f78f0004800 0x7f790414d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:23.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.867+0000 7f7909322640 1 -- 172.21.15.73:0/517385708 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f78f0004800 msgr2=0x7f790414d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:23.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.867+0000 7f7909322640 1 --2- 172.21.15.73:0/517385708 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f78f0004800 0x7f790414d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:23.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.867+0000 7f7909322640 1 -- 172.21.15.73:0/517385708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7904156a70 msgr2=0x7f790415e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:23.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.867+0000 7f7909322640 1 --2- 172.21.15.73:0/517385708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7904156a70 0x7f790415e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:23.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.867+0000 7f7909322640 1 -- 172.21.15.73:0/517385708 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79040cadc0 con 0x7f7904156fc0 2024-04-26T20:52:23.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.867+0000 7f790a324640 1 --2- 172.21.15.73:0/517385708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7904156a70 0x7f790415e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:52:23.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.867+0000 7f7909322640 1 --2- 172.21.15.73:0/517385708 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7904156fc0 0x7f79041f2bb0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f78f4008fd0 tx=0x7f78f4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:23.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.867+0000 7f78fa7fc640 1 -- 172.21.15.73:0/517385708 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f4040020 con 0x7f7904156fc0 2024-04-26T20:52:23.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.867+0000 7f78fa7fc640 1 -- 172.21.15.73:0/517385708 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78f4007680 con 0x7f7904156fc0 2024-04-26T20:52:23.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.867+0000 7f78fa7fc640 1 -- 172.21.15.73:0/517385708 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f4007a30 con 0x7f7904156fc0 2024-04-26T20:52:23.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.867+0000 7f790bd87640 1 -- 172.21.15.73:0/517385708 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79040b4e00 con 0x7f7904156fc0 2024-04-26T20:52:23.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.867+0000 7f790bd87640 1 -- 172.21.15.73:0/517385708 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79040b53c0 con 0x7f7904156fc0 2024-04-26T20:52:23.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.868+0000 7f790bd87640 1 -- 172.21.15.73:0/517385708 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79040694f0 con 0x7f7904156fc0 2024-04-26T20:52:23.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.869+0000 7f78fa7fc640 1 -- 172.21.15.73:0/517385708 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f78f4043020 con 0x7f7904156fc0 2024-04-26T20:52:23.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.869+0000 7f78fa7fc640 1 --2- 172.21.15.73:0/517385708 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f78e806c920 0x7f78e806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:23.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.869+0000 7f78fa7fc640 1 -- 172.21.15.73:0/517385708 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f78f40abc90 con 0x7f7904156fc0 2024-04-26T20:52:23.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.869+0000 7f7909b23640 1 --2- 172.21.15.73:0/517385708 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f78e806c920 0x7f78e806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:23.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.869+0000 7f7909b23640 1 --2- 172.21.15.73:0/517385708 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f78e806c920 0x7f78e806ede0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7900001750 tx=0x7f7900009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:23.875 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:23.873+0000 7f78fa7fc640 1 -- 172.21.15.73:0/517385708 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f78f404a030 con 0x7f7904156fc0 2024-04-26T20:52:24.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.051+0000 7f790bd87640 1 -- 172.21.15.73:0/517385708 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f790415b130 con 0x7f7904156fc0 2024-04-26T20:52:24.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.052+0000 7f78fa7fc640 1 -- 172.21.15.73:0/517385708 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) ==== 74+0+16942 (secure 0 0 0) 0x7f78f4079f30 con 0x7f7904156fc0 2024-04-26T20:52:24.054 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:52:24.055 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":52,"fsid":"fbd3f548-b547-4713-856b-09f98f5d1764","created":"2024-04-26T20:50:07.876786+0000","modified":"2024-04-26T20:52:22.406179+0000","last_up_change":"2024-04-26T20:50:19.082741+0000","last_in_change":"2024-04-26T20:50:10.018400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:50:19.993731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:52:18.960897+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-26T20:52:19.704261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d6f4f3-214a-4392-a02f-c33cbcab3c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6809","nonce":3461705124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6811","nonce":3461705124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6815","nonce":3461705124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6813","nonce":3461705124}]},"public_addr":"172.21.15.73:6809/3461705124","cluster_addr":"172.21.15.73:6811/3461705124","heartbeat_back_addr":"172.21.15.73:6815/3461705124","heartbeat_front_addr":"172.21.15.73:6813/3461705124","state":["exists","up"]},{"osd":1,"uuid":"786bf8ca-6ddc-4b3c-be6c-9152398bd5fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6817","nonce":2827619403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6819","nonce":2827619403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6825","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6827","nonce":2827619403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6821","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6823","nonce":2827619403}]},"public_addr":"172.21.15.73:6817/2827619403","cluster_addr":"172.21.15.73:6819/2827619403","heartbeat_back_addr":"172.21.15.73:6827/2827619403","heartbeat_front_addr":"172.21.15.73:6823/2827619403","state":["exists","up"]},{"osd":2,"uuid":"90963b1a-5437-4ef8-ad8b-6b74c534eeb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6801","nonce":2542681064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6803","nonce":2542681064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6808","nonce":2542681064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6805","nonce":2542681064}]},"public_addr":"172.21.15.73:6801/2542681064","cluster_addr":"172.21.15.73:6803/2542681064","heartbeat_back_addr":"172.21.15.73:6808/2542681064","heartbeat_front_addr":"172.21.15.73:6805/2542681064","state":["exists","up"]},{"osd":3,"uuid":"bd3d2148-6e38-41b4-a453-fe6e3b8b4ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6822","nonce":1604042718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6826","nonce":1604042718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6831","nonce":1604042718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6829","nonce":1604042718}]},"public_addr":"172.21.15.73:6822/1604042718","cluster_addr":"172.21.15.73:6826/1604042718","heartbeat_back_addr":"172.21.15.73:6831/1604042718","heartbeat_front_addr":"172.21.15.73:6829/1604042718","state":["exists","up"]},{"osd":4,"uuid":"e8668dc8-a23f-46d2-8311-cb882da08e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6817","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6819","nonce":3591055561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6824","nonce":3591055561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6831","nonce":3591055561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6828","nonce":3591055561}]},"public_addr":"172.21.15.120:6819/3591055561","cluster_addr":"172.21.15.120:6824/3591055561","heartbeat_back_addr":"172.21.15.120:6831/3591055561","heartbeat_front_addr":"172.21.15.120:6828/3591055561","state":["exists","up"]},{"osd":5,"uuid":"cbac0bdb-96bd-4852-bcce-e564495c3fcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6809","nonce":819450433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6811","nonce":819450433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6815","nonce":819450433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6813","nonce":819450433}]},"public_addr":"172.21.15.120:6809/819450433","cluster_addr":"172.21.15.120:6811/819450433","heartbeat_back_addr":"172.21.15.120:6815/819450433","heartbeat_front_addr":"172.21.15.120:6813/819450433","state":["exists","up"]},{"osd":6,"uuid":"1884b2a8-569f-4272-96a9-f25a09364dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6801","nonce":3946210727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6803","nonce":3946210727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6807","nonce":3946210727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6805","nonce":3946210727}]},"public_addr":"172.21.15.120:6801/3946210727","cluster_addr":"172.21.15.120:6803/3946210727","heartbeat_back_addr":"172.21.15.120:6807/3946210727","heartbeat_front_addr":"172.21.15.120:6805/3946210727","state":["exists","up"]},{"osd":7,"uuid":"c0c3bf6d-a4c5-4f1e-aa2d-330951830666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6818","nonce":2915574674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6821","nonce":2915574674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6827","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6829","nonce":2915574674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6823","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6825","nonce":2915574674}]},"public_addr":"172.21.15.120:6818/2915574674","cluster_addr":"172.21.15.120:6821/2915574674","heartbeat_back_addr":"172.21.15.120:6829/2915574674","heartbeat_front_addr":"172.21.15.120:6825/2915574674","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.218216+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.208710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.112705+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.182789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.164324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.174212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.225396+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.261559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:52:24.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.054+0000 7f790bd87640 1 -- 172.21.15.73:0/517385708 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f78e806c920 msgr2=0x7f78e806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:24.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.054+0000 7f790bd87640 1 --2- 172.21.15.73:0/517385708 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f78e806c920 0x7f78e806ede0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7900001750 tx=0x7f7900009040 comp rx=0 tx=0).stop 2024-04-26T20:52:24.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.055+0000 7f790bd87640 1 -- 172.21.15.73:0/517385708 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7904156fc0 msgr2=0x7f79041f2bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:24.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.055+0000 7f790bd87640 1 --2- 172.21.15.73:0/517385708 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7904156fc0 0x7f79041f2bb0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f78f4008fd0 tx=0x7f78f4001120 comp rx=0 tx=0).stop 2024-04-26T20:52:24.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.055+0000 7f790bd87640 1 -- 172.21.15.73:0/517385708 shutdown_connections 2024-04-26T20:52:24.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.055+0000 7f790bd87640 1 --2- 172.21.15.73:0/517385708 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7904156fc0 0x7f79041f2bb0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:24.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.055+0000 7f790bd87640 1 --2- 172.21.15.73:0/517385708 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f78e806c920 0x7f78e806ede0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:24.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.055+0000 7f790bd87640 1 --2- 172.21.15.73:0/517385708 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7904156a70 0x7f790415e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:24.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.055+0000 7f790bd87640 1 --2- 172.21.15.73:0/517385708 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f78f0004800 0x7f790414d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:24.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.055+0000 7f790bd87640 1 -- 172.21.15.73:0/517385708 >> 172.21.15.73:0/517385708 conn(0x7f79040d0be0 msgr2=0x7f790415b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:24.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.055+0000 7f790bd87640 1 -- 172.21.15.73:0/517385708 shutdown_connections 2024-04-26T20:52:24.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.055+0000 7f790bd87640 1 -- 172.21.15.73:0/517385708 wait complete. 2024-04-26T20:52:24.068 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T20:52:24.068 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:52:24.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.163+0000 7f07119f2640 1 Processor -- start 2024-04-26T20:52:24.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.163+0000 7f07119f2640 1 -- start start 2024-04-26T20:52:24.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.164+0000 7f07119f2640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f070c15f3a0 0x7f070c0d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:24.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.164+0000 7f07119f2640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f070c0d1770 0x7f070c0d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:24.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.164+0000 7f07119f2640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f070c0d2130 0x7f070c0d2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:24.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.164+0000 7f07119f2640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f070c0d25a0 con 0x7f070c0d2130 2024-04-26T20:52:24.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.164+0000 7f07119f2640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f070c0d0e20 con 0x7f070c0d1770 2024-04-26T20:52:24.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.164+0000 7f07119f2640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f070c0b1d70 con 0x7f070c15f3a0 2024-04-26T20:52:24.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.164+0000 7f070b7fe640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f070c0d2130 0x7f070c0d2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:24.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.164+0000 7f070b7fe640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f070c0d2130 0x7f070c0d2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38170/0 (socket says 172.21.15.73:38170) 2024-04-26T20:52:24.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.164+0000 7f070b7fe640 1 -- 172.21.15.73:0/1541300325 learned_addr learned my addr 172.21.15.73:0/1541300325 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:24.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.164+0000 7f070a7fc640 1 --2- 172.21.15.73:0/1541300325 >> v2:172.21.15.120:3300/0 conn(0x7f070c0d1770 0x7f070c0d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:24.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.164+0000 7f070affd640 1 --2- 172.21.15.73:0/1541300325 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f070c15f3a0 0x7f070c0d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:24.167 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.164+0000 7f070b7fe640 1 -- 172.21.15.73:0/1541300325 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f070c15f3a0 msgr2=0x7f070c0d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:24.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.164+0000 7f070b7fe640 1 --2- 172.21.15.73:0/1541300325 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f070c15f3a0 0x7f070c0d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:24.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.164+0000 7f070b7fe640 1 -- 172.21.15.73:0/1541300325 >> v2:172.21.15.120:3300/0 conn(0x7f070c0d1770 msgr2=0x7f070c0d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:24.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.164+0000 7f070b7fe640 1 --2- 172.21.15.73:0/1541300325 >> v2:172.21.15.120:3300/0 conn(0x7f070c0d1770 0x7f070c0d1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:24.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.164+0000 7f070b7fe640 1 -- 172.21.15.73:0/1541300325 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f070c0b2050 con 0x7f070c0d2130 2024-04-26T20:52:24.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.165+0000 7f070b7fe640 1 --2- 172.21.15.73:0/1541300325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f070c0d2130 0x7f070c0d2e50 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f070c073c50 tx=0x7f070402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=577662b9dcbf1814 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:24.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.165+0000 7f0709ffb640 1 -- 172.21.15.73:0/1541300325 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0704036070 con 0x7f070c0d2130 2024-04-26T20:52:24.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.165+0000 7f0709ffb640 1 -- 172.21.15.73:0/1541300325 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0704031600 con 0x7f070c0d2130 2024-04-26T20:52:24.168 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.165+0000 7f0709ffb640 1 -- 172.21.15.73:0/1541300325 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07040319d0 con 0x7f070c0d2130 2024-04-26T20:52:24.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.165+0000 7f07119f2640 1 -- 172.21.15.73:0/1541300325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f070c0d2130 msgr2=0x7f070c0d2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:24.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.165+0000 7f07119f2640 1 --2- 172.21.15.73:0/1541300325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f070c0d2130 0x7f070c0d2e50 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f070c073c50 tx=0x7f070402a6a0 comp rx=0 tx=0).stop 2024-04-26T20:52:24.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.166+0000 7f07119f2640 1 -- 172.21.15.73:0/1541300325 shutdown_connections 2024-04-26T20:52:24.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.166+0000 7f07119f2640 1 --2- 172.21.15.73:0/1541300325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f070c0d2130 0x7f070c0d2e50 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:24.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.166+0000 7f07119f2640 1 --2- 172.21.15.73:0/1541300325 >> v2:172.21.15.120:3300/0 conn(0x7f070c0d1770 0x7f070c0d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:24.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.166+0000 7f07119f2640 1 --2- 172.21.15.73:0/1541300325 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f070c15f3a0 0x7f070c0d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:24.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.166+0000 7f07119f2640 1 -- 172.21.15.73:0/1541300325 >> 172.21.15.73:0/1541300325 conn(0x7f070c0d4f90 msgr2=0x7f070c0d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:24.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.166+0000 7f07119f2640 1 -- 172.21.15.73:0/1541300325 shutdown_connections 2024-04-26T20:52:24.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.166+0000 7f07119f2640 1 -- 172.21.15.73:0/1541300325 wait complete. 2024-04-26T20:52:24.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.167+0000 7f07119f2640 1 Processor -- start 2024-04-26T20:52:24.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.167+0000 7f07119f2640 1 -- start start 2024-04-26T20:52:24.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.167+0000 7f07119f2640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f070c0d1770 0x7f070c153380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:24.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.167+0000 7f07119f2640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f070c0d2130 0x7f070c0ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:24.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.168+0000 7f07119f2640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f070c15f3a0 0x7f070c15fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:24.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.168+0000 7f07119f2640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f070c15f900 con 0x7f070c0d2130 2024-04-26T20:52:24.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.168+0000 7f07119f2640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f070c15f780 con 0x7f070c0d1770 2024-04-26T20:52:24.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.168+0000 7f07119f2640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f070c0d3390 con 0x7f070c15f3a0 2024-04-26T20:52:24.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.168+0000 7f070a7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f070c0d2130 0x7f070c0ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:24.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.168+0000 7f070a7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f070c0d2130 0x7f070c0ce8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38176/0 (socket says 172.21.15.73:38176) 2024-04-26T20:52:24.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.168+0000 7f070a7fc640 1 -- 172.21.15.73:0/4113137946 learned_addr learned my addr 172.21.15.73:0/4113137946 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:24.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.168+0000 7f070a7fc640 1 -- 172.21.15.73:0/4113137946 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f070c15f3a0 msgr2=0x7f070c15fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:24.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.168+0000 7f070a7fc640 1 --2- 172.21.15.73:0/4113137946 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f070c15f3a0 0x7f070c15fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:24.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.168+0000 7f070a7fc640 1 -- 172.21.15.73:0/4113137946 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f070c0d1770 msgr2=0x7f070c153380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:24.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.168+0000 7f070a7fc640 1 --2- 172.21.15.73:0/4113137946 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f070c0d1770 0x7f070c153380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:24.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.168+0000 7f070a7fc640 1 -- 172.21.15.73:0/4113137946 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f070c0b8020 con 0x7f070c0d2130 2024-04-26T20:52:24.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.168+0000 7f070a7fc640 1 --2- 172.21.15.73:0/4113137946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f070c0d2130 0x7f070c0ce8d0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f070000d830 tx=0x7f070000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:24.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.168+0000 7f06f3fff640 1 -- 172.21.15.73:0/4113137946 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0700018020 con 0x7f070c0d2130 2024-04-26T20:52:24.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.168+0000 7f06f3fff640 1 -- 172.21.15.73:0/4113137946 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07000096a0 con 0x7f070c0d2130 2024-04-26T20:52:24.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.168+0000 7f07119f2640 1 -- 172.21.15.73:0/4113137946 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f070c0b8300 con 0x7f070c0d2130 2024-04-26T20:52:24.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.168+0000 7f06f3fff640 1 -- 172.21.15.73:0/4113137946 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0700009a70 con 0x7f070c0d2130 2024-04-26T20:52:24.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.168+0000 7f07119f2640 1 -- 172.21.15.73:0/4113137946 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f070c0b88c0 con 0x7f070c0d2130 2024-04-26T20:52:24.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.169+0000 7f07119f2640 1 -- 172.21.15.73:0/4113137946 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f070c0694f0 con 0x7f070c0d2130 2024-04-26T20:52:24.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.170+0000 7f06f3fff640 1 -- 172.21.15.73:0/4113137946 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f070001b020 con 0x7f070c0d2130 2024-04-26T20:52:24.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.170+0000 7f06f3fff640 1 --2- 172.21.15.73:0/4113137946 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f06f806c8f0 0x7f06f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:24.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.170+0000 7f06f3fff640 1 -- 172.21.15.73:0/4113137946 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f070008c140 con 0x7f070c0d2130 2024-04-26T20:52:24.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.170+0000 7f070affd640 1 --2- 172.21.15.73:0/4113137946 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f06f806c8f0 0x7f06f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:24.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.170+0000 7f070affd640 1 --2- 172.21.15.73:0/4113137946 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f06f806c8f0 0x7f06f806edb0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f06fc006fd0 tx=0x7f06fc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:24.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.174+0000 7f06f3fff640 1 -- 172.21.15.73:0/4113137946 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0700022030 con 0x7f070c0d2130 2024-04-26T20:52:24.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.345+0000 7f07119f2640 1 -- 172.21.15.73:0/4113137946 --> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f070c06cca0 con 0x7f06f806c8f0 2024-04-26T20:52:24.353 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.351+0000 7f06f3fff640 1 -- 172.21.15.73:0/4113137946 <== mgr.4107 v2:172.21.15.73:6832/666905549 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240062 (secure 0 0 0) 0x7f070c06cca0 con 0x7f06f806c8f0 2024-04-26T20:52:24.355 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:52:24.356 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-26T20:52:24.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.354+0000 7f07119f2640 1 -- 172.21.15.73:0/4113137946 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f06f806c8f0 msgr2=0x7f06f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:24.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.354+0000 7f07119f2640 1 --2- 172.21.15.73:0/4113137946 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f06f806c8f0 0x7f06f806edb0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f06fc006fd0 tx=0x7f06fc008040 comp rx=0 tx=0).stop 2024-04-26T20:52:24.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.354+0000 7f07119f2640 1 -- 172.21.15.73:0/4113137946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f070c0d2130 msgr2=0x7f070c0ce8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:24.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.354+0000 7f07119f2640 1 --2- 172.21.15.73:0/4113137946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f070c0d2130 0x7f070c0ce8d0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f070000d830 tx=0x7f070000dcf0 comp rx=0 tx=0).stop 2024-04-26T20:52:24.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.354+0000 7f07119f2640 1 -- 172.21.15.73:0/4113137946 shutdown_connections 2024-04-26T20:52:24.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.354+0000 7f07119f2640 1 --2- 172.21.15.73:0/4113137946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f070c0d2130 0x7f070c0ce8d0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:24.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.354+0000 7f07119f2640 1 --2- 172.21.15.73:0/4113137946 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f06f806c8f0 0x7f06f806edb0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:24.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.354+0000 7f07119f2640 1 --2- 172.21.15.73:0/4113137946 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f070c15f3a0 0x7f070c15fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:24.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.354+0000 7f07119f2640 1 --2- 172.21.15.73:0/4113137946 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f070c0d1770 0x7f070c153380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:24.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.354+0000 7f07119f2640 1 -- 172.21.15.73:0/4113137946 >> 172.21.15.73:0/4113137946 conn(0x7f070c0d4f90 msgr2=0x7f070c06e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:24.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.354+0000 7f07119f2640 1 -- 172.21.15.73:0/4113137946 shutdown_connections 2024-04-26T20:52:24.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.354+0000 7f07119f2640 1 -- 172.21.15.73:0/4113137946 wait complete. 2024-04-26T20:52:24.366 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":107,"stamp":"2024-04-26T20:52:24.003511+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221860,"kb_used_data":5644,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752860,"statfs":{"total":773094113280,"available":772866928640,"internally_reserved":0,"allocated":5779456,"data_stored":3205651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.609830"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.395952+0000","last_change":"2024-04-26T20:52:19.395952+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.395952+0000","last_undegraded":"2024-04-26T20:52:19.395952+0000","last_fullsized":"2024-04-26T20:52:19.395952+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.397803+0000","last_change":"2024-04-26T20:52:19.397803+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.397803+0000","last_undegraded":"2024-04-26T20:52:19.397803+0000","last_fullsized":"2024-04-26T20:52:19.397803+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.397585+0000","last_change":"2024-04-26T20:52:19.397585+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.397585+0000","last_undegraded":"2024-04-26T20:52:19.397585+0000","last_fullsized":"2024-04-26T20:52:19.397585+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.396894+0000","last_change":"2024-04-26T20:52:19.396894+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.396894+0000","last_undegraded":"2024-04-26T20:52:19.396894+0000","last_fullsized":"2024-04-26T20:52:19.396894+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.397540+0000","last_change":"2024-04-26T20:52:19.397540+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.397540+0000","last_undegraded":"2024-04-26T20:52:19.397540+0000","last_fullsized":"2024-04-26T20:52:19.397540+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.393560+0000","last_change":"2024-04-26T20:52:19.393560+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.393560+0000","last_undegraded":"2024-04-26T20:52:19.393560+0000","last_fullsized":"2024-04-26T20:52:19.393560+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.395521+0000","last_change":"2024-04-26T20:52:19.395521+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.395521+0000","last_undegraded":"2024-04-26T20:52:19.395521+0000","last_fullsized":"2024-04-26T20:52:19.395521+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.397229+0000","last_change":"2024-04-26T20:52:19.397229+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.397229+0000","last_undegraded":"2024-04-26T20:52:19.397229+0000","last_fullsized":"2024-04-26T20:52:19.397229+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.397756+0000","last_change":"2024-04-26T20:52:19.397756+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.397756+0000","last_undegraded":"2024-04-26T20:52:19.397756+0000","last_fullsized":"2024-04-26T20:52:19.397756+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.393499+0000","last_change":"2024-04-26T20:52:19.393499+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.393499+0000","last_undegraded":"2024-04-26T20:52:19.393499+0000","last_fullsized":"2024-04-26T20:52:19.393499+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.398384+0000","last_change":"2024-04-26T20:52:19.398384+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.398384+0000","last_undegraded":"2024-04-26T20:52:19.398384+0000","last_fullsized":"2024-04-26T20:52:19.398384+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.398218+0000","last_change":"2024-04-26T20:52:19.398218+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.398218+0000","last_undegraded":"2024-04-26T20:52:19.398218+0000","last_fullsized":"2024-04-26T20:52:19.398218+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.396945+0000","last_change":"2024-04-26T20:52:19.396945+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.396945+0000","last_undegraded":"2024-04-26T20:52:19.396945+0000","last_fullsized":"2024-04-26T20:52:19.396945+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.395349+0000","last_change":"2024-04-26T20:52:19.395349+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.395349+0000","last_undegraded":"2024-04-26T20:52:19.395349+0000","last_fullsized":"2024-04-26T20:52:19.395349+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.397694+0000","last_change":"2024-04-26T20:52:19.397694+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.397694+0000","last_undegraded":"2024-04-26T20:52:19.397694+0000","last_fullsized":"2024-04-26T20:52:19.397694+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.397561+0000","last_change":"2024-04-26T20:52:19.397561+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.397561+0000","last_undegraded":"2024-04-26T20:52:19.397561+0000","last_fullsized":"2024-04-26T20:52:19.397561+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.396989+0000","last_change":"2024-04-26T20:52:19.396989+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.396989+0000","last_undegraded":"2024-04-26T20:52:19.396989+0000","last_fullsized":"2024-04-26T20:52:19.396989+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.400177+0000","last_change":"2024-04-26T20:52:19.400177+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.400177+0000","last_undegraded":"2024-04-26T20:52:19.400177+0000","last_fullsized":"2024-04-26T20:52:19.400177+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.395403+0000","last_change":"2024-04-26T20:52:19.395403+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.395403+0000","last_undegraded":"2024-04-26T20:52:19.395403+0000","last_fullsized":"2024-04-26T20:52:19.395403+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.395340+0000","last_change":"2024-04-26T20:52:19.395340+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.395340+0000","last_undegraded":"2024-04-26T20:52:19.395340+0000","last_fullsized":"2024-04-26T20:52:19.395340+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.397535+0000","last_change":"2024-04-26T20:52:19.397535+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.397535+0000","last_undegraded":"2024-04-26T20:52:19.397535+0000","last_fullsized":"2024-04-26T20:52:19.397535+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.397145+0000","last_change":"2024-04-26T20:52:19.397145+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.397145+0000","last_undegraded":"2024-04-26T20:52:19.397145+0000","last_fullsized":"2024-04-26T20:52:19.397145+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.396532+0000","last_change":"2024-04-26T20:52:19.396532+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.396532+0000","last_undegraded":"2024-04-26T20:52:19.396532+0000","last_fullsized":"2024-04-26T20:52:19.396532+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.400129+0000","last_change":"2024-04-26T20:52:19.400129+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.400129+0000","last_undegraded":"2024-04-26T20:52:19.400129+0000","last_fullsized":"2024-04-26T20:52:19.400129+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.395398+0000","last_change":"2024-04-26T20:52:19.395398+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.395398+0000","last_undegraded":"2024-04-26T20:52:19.395398+0000","last_fullsized":"2024-04-26T20:52:19.395398+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.397644+0000","last_change":"2024-04-26T20:52:19.397644+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.397644+0000","last_undegraded":"2024-04-26T20:52:19.397644+0000","last_fullsized":"2024-04-26T20:52:19.397644+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.396492+0000","last_change":"2024-04-26T20:52:19.396492+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.396492+0000","last_undegraded":"2024-04-26T20:52:19.396492+0000","last_fullsized":"2024-04-26T20:52:19.396492+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.397080+0000","last_change":"2024-04-26T20:52:19.397080+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.397080+0000","last_undegraded":"2024-04-26T20:52:19.397080+0000","last_fullsized":"2024-04-26T20:52:19.397080+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.396492+0000","last_change":"2024-04-26T20:52:19.396492+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.396492+0000","last_undegraded":"2024-04-26T20:52:19.396492+0000","last_fullsized":"2024-04-26T20:52:19.396492+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.393556+0000","last_change":"2024-04-26T20:52:19.393556+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.393556+0000","last_undegraded":"2024-04-26T20:52:19.393556+0000","last_fullsized":"2024-04-26T20:52:19.393556+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.396627+0000","last_change":"2024-04-26T20:52:19.396627+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.396627+0000","last_undegraded":"2024-04-26T20:52:19.396627+0000","last_fullsized":"2024-04-26T20:52:19.396627+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.396494+0000","last_change":"2024-04-26T20:52:19.396494+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.396494+0000","last_undegraded":"2024-04-26T20:52:19.396494+0000","last_fullsized":"2024-04-26T20:52:19.396494+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":221,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-26T20:52:19.397911+0000","last_change":"2024-04-26T20:50:21.150132+0000","last_active":"2024-04-26T20:52:19.397911+0000","last_peered":"2024-04-26T20:52:19.397911+0000","last_clean":"2024-04-26T20:52:19.397911+0000","last_became_active":"2024-04-26T20:50:21.147439+0000","last_became_peered":"2024-04-26T20:50:21.147439+0000","last_unstale":"2024-04-26T20:52:19.397911+0000","last_undegraded":"2024-04-26T20:52:19.397911+0000","last_fullsized":"2024-04-26T20:52:19.397911+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:20.084499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:20.084499+0000","last_clean_scrub_stamp":"2024-04-26T20:50:20.084499+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:19:29.395477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28160,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343680,"statfs":{"total":96636764160,"available":96607928320,"internally_reserved":0,"allocated":1163264,"data_stored":841154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27600,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344240,"statfs":{"total":96636764160,"available":96608501760,"internally_reserved":0,"allocated":581632,"data_stored":256995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":569344,"data_stored":250786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27584,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":569344,"data_stored":250786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27592,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":581632,"data_stored":256995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27588,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344252,"statfs":{"total":96636764160,"available":96608514048,"internally_reserved":0,"allocated":569344,"data_stored":250786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27584,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":569344,"data_stored":250786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28176,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343664,"statfs":{"total":96636764160,"available":96607911936,"internally_reserved":0,"allocated":1175552,"data_stored":847363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:52:24.370 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:52:24.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.502+0000 7fec23f6a640 1 Processor -- start 2024-04-26T20:52:24.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.502+0000 7fec23f6a640 1 -- start start 2024-04-26T20:52:24.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.503+0000 7fec23f6a640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fec1c0b2340 0x7fec1c0cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:24.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.503+0000 7fec23f6a640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fec1c0cd930 0x7fec1c0d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:24.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.503+0000 7fec23f6a640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec1c0d10d0 con 0x7fec1c162810 2024-04-26T20:52:24.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.503+0000 7fec23f6a640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7fec1c0d0750 con 0x7fec1c0b2340 2024-04-26T20:52:24.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.503+0000 7fec23f6a640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fec1c0d16a0 con 0x7fec1c0cd930 2024-04-26T20:52:24.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.503+0000 7fec21505640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fec1c0b2340 0x7fec1c0cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:24.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.503+0000 7fec21505640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fec1c0b2340 0x7fec1c0cd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38182/0 (socket says 172.21.15.73:38182) 2024-04-26T20:52:24.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.503+0000 7fec21505640 1 -- 172.21.15.73:0/56212737 learned_addr learned my addr 172.21.15.73:0/56212737 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:24.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.503+0000 7fec22507640 1 --2- 172.21.15.73:0/56212737 >> v2:172.21.15.120:3300/0 conn(0x7fec1c0cd930 0x7fec1c0d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:24.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.503+0000 7fec21505640 1 -- 172.21.15.73:0/56212737 >> v2:172.21.15.120:3300/0 conn(0x7fec1c0cd930 msgr2=0x7fec1c0d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:24.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.503+0000 7fec21505640 1 --2- 172.21.15.73:0/56212737 >> v2:172.21.15.120:3300/0 conn(0x7fec1c0cd930 0x7fec1c0d1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:24.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.503+0000 7fec21505640 1 -- 172.21.15.73:0/56212737 >> v1:172.21.15.120:6789/0 conn(0x7fec1c162810 legacy=0x7fec1c070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:24.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.503+0000 7fec21505640 1 -- 172.21.15.73:0/56212737 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec1c0d1370 con 0x7fec1c0b2340 2024-04-26T20:52:24.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.504+0000 7fec21505640 1 --2- 172.21.15.73:0/56212737 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec1c0b2340 0x7fec1c0cd3f0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fec1c0b2280 tx=0x7fec0c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=406a4c7b83de96df server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:24.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.504+0000 7fec20d04640 1 -- 172.21.15.73:0/56212737 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec0c037070 con 0x7fec1c0b2340 2024-04-26T20:52:24.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.504+0000 7fec20d04640 1 -- 172.21.15.73:0/56212737 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec0c032440 con 0x7fec1c0b2340 2024-04-26T20:52:24.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.504+0000 7fec20d04640 1 -- 172.21.15.73:0/56212737 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec0c032810 con 0x7fec1c0b2340 2024-04-26T20:52:24.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.505+0000 7fec23f6a640 1 -- 172.21.15.73:0/56212737 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec1c0b2340 msgr2=0x7fec1c0cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:24.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.505+0000 7fec23f6a640 1 --2- 172.21.15.73:0/56212737 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec1c0b2340 0x7fec1c0cd3f0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fec1c0b2280 tx=0x7fec0c02a270 comp rx=0 tx=0).stop 2024-04-26T20:52:24.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.505+0000 7fec23f6a640 1 -- 172.21.15.73:0/56212737 shutdown_connections 2024-04-26T20:52:24.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.505+0000 7fec23f6a640 1 --2- 172.21.15.73:0/56212737 >> v2:172.21.15.120:3300/0 conn(0x7fec1c0cd930 0x7fec1c0d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:24.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.505+0000 7fec23f6a640 1 --2- 172.21.15.73:0/56212737 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec1c0b2340 0x7fec1c0cd3f0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:24.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.505+0000 7fec23f6a640 1 -- 172.21.15.73:0/56212737 >> 172.21.15.73:0/56212737 conn(0x7fec1c0d2da0 msgr2=0x7fec1c0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:24.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.505+0000 7fec23f6a640 1 -- 172.21.15.73:0/56212737 shutdown_connections 2024-04-26T20:52:24.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.505+0000 7fec23f6a640 1 -- 172.21.15.73:0/56212737 wait complete. 2024-04-26T20:52:24.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.506+0000 7fec23f6a640 1 Processor -- start 2024-04-26T20:52:24.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.506+0000 7fec23f6a640 1 -- start start 2024-04-26T20:52:24.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.507+0000 7fec23f6a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec1c0b2340 0x7fec1c15fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:24.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.507+0000 7fec23f6a640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fec1c0cd930 0x7fec1c1614d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:24.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.507+0000 7fec23f6a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec1c162810 0x7fec1c0cb070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:24.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.507+0000 7fec23f6a640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fec1c0d2070 con 0x7fec1c162810 2024-04-26T20:52:24.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.507+0000 7fec23f6a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fec1c0d0470 con 0x7fec1c0b2340 2024-04-26T20:52:24.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.507+0000 7fec23f6a640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fec1c0d21f0 con 0x7fec1c0cd930 2024-04-26T20:52:24.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.507+0000 7fec22507640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec1c162810 0x7fec1c0cb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:24.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.507+0000 7fec22507640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec1c162810 0x7fec1c0cb070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38194/0 (socket says 172.21.15.73:38194) 2024-04-26T20:52:24.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.507+0000 7fec22507640 1 -- 172.21.15.73:0/3739067587 learned_addr learned my addr 172.21.15.73:0/3739067587 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:24.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.507+0000 7fec21505640 1 --2- 172.21.15.73:0/3739067587 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fec1c0cd930 0x7fec1c1614d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:24.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.507+0000 7fec21d06640 1 --2- 172.21.15.73:0/3739067587 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec1c0b2340 0x7fec1c15fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:24.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.507+0000 7fec22507640 1 -- 172.21.15.73:0/3739067587 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fec1c0cd930 msgr2=0x7fec1c1614d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:24.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.507+0000 7fec22507640 1 --2- 172.21.15.73:0/3739067587 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fec1c0cd930 0x7fec1c1614d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:24.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.507+0000 7fec22507640 1 -- 172.21.15.73:0/3739067587 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec1c0b2340 msgr2=0x7fec1c15fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:24.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.507+0000 7fec22507640 1 --2- 172.21.15.73:0/3739067587 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec1c0b2340 0x7fec1c15fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:24.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.507+0000 7fec22507640 1 -- 172.21.15.73:0/3739067587 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec1c1603c0 con 0x7fec1c162810 2024-04-26T20:52:24.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.507+0000 7fec21d06640 1 --2- 172.21.15.73:0/3739067587 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec1c0b2340 0x7fec1c15fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:52:24.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.507+0000 7fec22507640 1 --2- 172.21.15.73:0/3739067587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec1c162810 0x7fec1c0cb070 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fec14007de0 tx=0x7fec1400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:24.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.507+0000 7fec127fc640 1 -- 172.21.15.73:0/3739067587 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec14014620 con 0x7fec1c162810 2024-04-26T20:52:24.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.507+0000 7fec127fc640 1 -- 172.21.15.73:0/3739067587 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec14019070 con 0x7fec1c162810 2024-04-26T20:52:24.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.507+0000 7fec23f6a640 1 -- 172.21.15.73:0/3739067587 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec1c160710 con 0x7fec1c162810 2024-04-26T20:52:24.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.508+0000 7fec127fc640 1 -- 172.21.15.73:0/3739067587 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec14014a20 con 0x7fec1c162810 2024-04-26T20:52:24.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.508+0000 7fec23f6a640 1 -- 172.21.15.73:0/3739067587 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec1c160cd0 con 0x7fec1c162810 2024-04-26T20:52:24.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.508+0000 7fec23f6a640 1 -- 172.21.15.73:0/3739067587 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec1c0694f0 con 0x7fec1c162810 2024-04-26T20:52:24.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.509+0000 7fec127fc640 1 -- 172.21.15.73:0/3739067587 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fec14014bc0 con 0x7fec1c162810 2024-04-26T20:52:24.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.509+0000 7fec127fc640 1 --2- 172.21.15.73:0/3739067587 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fec0006c920 0x7fec0006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:24.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.509+0000 7fec127fc640 1 -- 172.21.15.73:0/3739067587 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7fec1408c510 con 0x7fec1c162810 2024-04-26T20:52:24.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.509+0000 7fec21d06640 1 --2- 172.21.15.73:0/3739067587 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fec0006c920 0x7fec0006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:24.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.510+0000 7fec21d06640 1 --2- 172.21.15.73:0/3739067587 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fec0006c920 0x7fec0006ede0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fec18001660 tx=0x7fec18009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:24.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.513+0000 7fec127fc640 1 -- 172.21.15.73:0/3739067587 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fec1405a840 con 0x7fec1c162810 2024-04-26T20:52:24.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.685+0000 7fec23f6a640 1 -- 172.21.15.73:0/3739067587 --> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fec1c0d37e0 con 0x7fec0006c920 2024-04-26T20:52:24.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.690+0000 7fec127fc640 1 -- 172.21.15.73:0/3739067587 <== mgr.4107 v2:172.21.15.73:6832/666905549 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240062 (secure 0 0 0) 0x7fec1c0d37e0 con 0x7fec0006c920 2024-04-26T20:52:24.694 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:52:24.696 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-26T20:52:24.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.693+0000 7fec23f6a640 1 -- 172.21.15.73:0/3739067587 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fec0006c920 msgr2=0x7fec0006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:24.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.693+0000 7fec23f6a640 1 --2- 172.21.15.73:0/3739067587 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fec0006c920 0x7fec0006ede0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fec18001660 tx=0x7fec18009040 comp rx=0 tx=0).stop 2024-04-26T20:52:24.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.693+0000 7fec23f6a640 1 -- 172.21.15.73:0/3739067587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec1c162810 msgr2=0x7fec1c0cb070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:24.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.693+0000 7fec23f6a640 1 --2- 172.21.15.73:0/3739067587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec1c162810 0x7fec1c0cb070 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fec14007de0 tx=0x7fec1400e4b0 comp rx=0 tx=0).stop 2024-04-26T20:52:24.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.693+0000 7fec23f6a640 1 -- 172.21.15.73:0/3739067587 shutdown_connections 2024-04-26T20:52:24.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.693+0000 7fec23f6a640 1 --2- 172.21.15.73:0/3739067587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec1c162810 0x7fec1c0cb070 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:24.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.693+0000 7fec23f6a640 1 --2- 172.21.15.73:0/3739067587 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fec0006c920 0x7fec0006ede0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:24.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.693+0000 7fec23f6a640 1 --2- 172.21.15.73:0/3739067587 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fec1c0cd930 0x7fec1c1614d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:24.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.693+0000 7fec23f6a640 1 --2- 172.21.15.73:0/3739067587 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fec1c0b2340 0x7fec1c15fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:24.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.693+0000 7fec23f6a640 1 -- 172.21.15.73:0/3739067587 >> 172.21.15.73:0/3739067587 conn(0x7fec1c0d2da0 msgr2=0x7fec1c06e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:24.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.693+0000 7fec23f6a640 1 -- 172.21.15.73:0/3739067587 shutdown_connections 2024-04-26T20:52:24.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.693+0000 7fec23f6a640 1 -- 172.21.15.73:0/3739067587 wait complete. 2024-04-26T20:52:24.706 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":107,"stamp":"2024-04-26T20:52:24.003511+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221860,"kb_used_data":5644,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752860,"statfs":{"total":773094113280,"available":772866928640,"internally_reserved":0,"allocated":5779456,"data_stored":3205651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.609830"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.395952+0000","last_change":"2024-04-26T20:52:19.395952+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.395952+0000","last_undegraded":"2024-04-26T20:52:19.395952+0000","last_fullsized":"2024-04-26T20:52:19.395952+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.397803+0000","last_change":"2024-04-26T20:52:19.397803+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.397803+0000","last_undegraded":"2024-04-26T20:52:19.397803+0000","last_fullsized":"2024-04-26T20:52:19.397803+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.397585+0000","last_change":"2024-04-26T20:52:19.397585+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.397585+0000","last_undegraded":"2024-04-26T20:52:19.397585+0000","last_fullsized":"2024-04-26T20:52:19.397585+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.396894+0000","last_change":"2024-04-26T20:52:19.396894+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.396894+0000","last_undegraded":"2024-04-26T20:52:19.396894+0000","last_fullsized":"2024-04-26T20:52:19.396894+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.397540+0000","last_change":"2024-04-26T20:52:19.397540+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.397540+0000","last_undegraded":"2024-04-26T20:52:19.397540+0000","last_fullsized":"2024-04-26T20:52:19.397540+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.393560+0000","last_change":"2024-04-26T20:52:19.393560+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.393560+0000","last_undegraded":"2024-04-26T20:52:19.393560+0000","last_fullsized":"2024-04-26T20:52:19.393560+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.395521+0000","last_change":"2024-04-26T20:52:19.395521+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.395521+0000","last_undegraded":"2024-04-26T20:52:19.395521+0000","last_fullsized":"2024-04-26T20:52:19.395521+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.397229+0000","last_change":"2024-04-26T20:52:19.397229+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.397229+0000","last_undegraded":"2024-04-26T20:52:19.397229+0000","last_fullsized":"2024-04-26T20:52:19.397229+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.397756+0000","last_change":"2024-04-26T20:52:19.397756+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.397756+0000","last_undegraded":"2024-04-26T20:52:19.397756+0000","last_fullsized":"2024-04-26T20:52:19.397756+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.393499+0000","last_change":"2024-04-26T20:52:19.393499+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.393499+0000","last_undegraded":"2024-04-26T20:52:19.393499+0000","last_fullsized":"2024-04-26T20:52:19.393499+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.398384+0000","last_change":"2024-04-26T20:52:19.398384+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.398384+0000","last_undegraded":"2024-04-26T20:52:19.398384+0000","last_fullsized":"2024-04-26T20:52:19.398384+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.398218+0000","last_change":"2024-04-26T20:52:19.398218+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.398218+0000","last_undegraded":"2024-04-26T20:52:19.398218+0000","last_fullsized":"2024-04-26T20:52:19.398218+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.396945+0000","last_change":"2024-04-26T20:52:19.396945+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.396945+0000","last_undegraded":"2024-04-26T20:52:19.396945+0000","last_fullsized":"2024-04-26T20:52:19.396945+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.395349+0000","last_change":"2024-04-26T20:52:19.395349+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.395349+0000","last_undegraded":"2024-04-26T20:52:19.395349+0000","last_fullsized":"2024-04-26T20:52:19.395349+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.397694+0000","last_change":"2024-04-26T20:52:19.397694+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.397694+0000","last_undegraded":"2024-04-26T20:52:19.397694+0000","last_fullsized":"2024-04-26T20:52:19.397694+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.397561+0000","last_change":"2024-04-26T20:52:19.397561+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.397561+0000","last_undegraded":"2024-04-26T20:52:19.397561+0000","last_fullsized":"2024-04-26T20:52:19.397561+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.396989+0000","last_change":"2024-04-26T20:52:19.396989+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.396989+0000","last_undegraded":"2024-04-26T20:52:19.396989+0000","last_fullsized":"2024-04-26T20:52:19.396989+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.400177+0000","last_change":"2024-04-26T20:52:19.400177+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.400177+0000","last_undegraded":"2024-04-26T20:52:19.400177+0000","last_fullsized":"2024-04-26T20:52:19.400177+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.395403+0000","last_change":"2024-04-26T20:52:19.395403+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.395403+0000","last_undegraded":"2024-04-26T20:52:19.395403+0000","last_fullsized":"2024-04-26T20:52:19.395403+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.395340+0000","last_change":"2024-04-26T20:52:19.395340+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.395340+0000","last_undegraded":"2024-04-26T20:52:19.395340+0000","last_fullsized":"2024-04-26T20:52:19.395340+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.397535+0000","last_change":"2024-04-26T20:52:19.397535+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.397535+0000","last_undegraded":"2024-04-26T20:52:19.397535+0000","last_fullsized":"2024-04-26T20:52:19.397535+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.397145+0000","last_change":"2024-04-26T20:52:19.397145+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.397145+0000","last_undegraded":"2024-04-26T20:52:19.397145+0000","last_fullsized":"2024-04-26T20:52:19.397145+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.396532+0000","last_change":"2024-04-26T20:52:19.396532+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.396532+0000","last_undegraded":"2024-04-26T20:52:19.396532+0000","last_fullsized":"2024-04-26T20:52:19.396532+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.400129+0000","last_change":"2024-04-26T20:52:19.400129+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.400129+0000","last_undegraded":"2024-04-26T20:52:19.400129+0000","last_fullsized":"2024-04-26T20:52:19.400129+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.395398+0000","last_change":"2024-04-26T20:52:19.395398+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.395398+0000","last_undegraded":"2024-04-26T20:52:19.395398+0000","last_fullsized":"2024-04-26T20:52:19.395398+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.397644+0000","last_change":"2024-04-26T20:52:19.397644+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.397644+0000","last_undegraded":"2024-04-26T20:52:19.397644+0000","last_fullsized":"2024-04-26T20:52:19.397644+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.396492+0000","last_change":"2024-04-26T20:52:19.396492+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.396492+0000","last_undegraded":"2024-04-26T20:52:19.396492+0000","last_fullsized":"2024-04-26T20:52:19.396492+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.397080+0000","last_change":"2024-04-26T20:52:19.397080+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.397080+0000","last_undegraded":"2024-04-26T20:52:19.397080+0000","last_fullsized":"2024-04-26T20:52:19.397080+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.396492+0000","last_change":"2024-04-26T20:52:19.396492+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.396492+0000","last_undegraded":"2024-04-26T20:52:19.396492+0000","last_fullsized":"2024-04-26T20:52:19.396492+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.393556+0000","last_change":"2024-04-26T20:52:19.393556+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.393556+0000","last_undegraded":"2024-04-26T20:52:19.393556+0000","last_fullsized":"2024-04-26T20:52:19.393556+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.396627+0000","last_change":"2024-04-26T20:52:19.396627+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.396627+0000","last_undegraded":"2024-04-26T20:52:19.396627+0000","last_fullsized":"2024-04-26T20:52:19.396627+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.396494+0000","last_change":"2024-04-26T20:52:19.396494+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.396494+0000","last_undegraded":"2024-04-26T20:52:19.396494+0000","last_fullsized":"2024-04-26T20:52:19.396494+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":221,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-26T20:52:19.397911+0000","last_change":"2024-04-26T20:50:21.150132+0000","last_active":"2024-04-26T20:52:19.397911+0000","last_peered":"2024-04-26T20:52:19.397911+0000","last_clean":"2024-04-26T20:52:19.397911+0000","last_became_active":"2024-04-26T20:50:21.147439+0000","last_became_peered":"2024-04-26T20:50:21.147439+0000","last_unstale":"2024-04-26T20:52:19.397911+0000","last_undegraded":"2024-04-26T20:52:19.397911+0000","last_fullsized":"2024-04-26T20:52:19.397911+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:20.084499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:20.084499+0000","last_clean_scrub_stamp":"2024-04-26T20:50:20.084499+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:19:29.395477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28160,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343680,"statfs":{"total":96636764160,"available":96607928320,"internally_reserved":0,"allocated":1163264,"data_stored":841154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27600,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344240,"statfs":{"total":96636764160,"available":96608501760,"internally_reserved":0,"allocated":581632,"data_stored":256995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":569344,"data_stored":250786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27584,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":569344,"data_stored":250786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27592,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":581632,"data_stored":256995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27588,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344252,"statfs":{"total":96636764160,"available":96608514048,"internally_reserved":0,"allocated":569344,"data_stored":250786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27584,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":569344,"data_stored":250786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28176,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343664,"statfs":{"total":96636764160,"available":96607911936,"internally_reserved":0,"allocated":1175552,"data_stored":847363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:52:24.709 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:52:24.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.842+0000 7f6f1dc1d640 1 Processor -- start 2024-04-26T20:52:24.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.842+0000 7f6f1dc1d640 1 -- start start 2024-04-26T20:52:24.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.843+0000 7f6f1dc1d640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6f18156460 0x7f6f180cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:24.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.843+0000 7f6f1dc1d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f6f1806fea0 0x7f6f18072c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:24.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.843+0000 7f6f1dc1d640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f18156840 con 0x7f6f180cba10 2024-04-26T20:52:24.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.843+0000 7f6f1dc1d640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f6f180cbf20 con 0x7f6f18156460 2024-04-26T20:52:24.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.843+0000 7f6f1dc1d640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f6f180cc200 con 0x7f6f1806fea0 2024-04-26T20:52:24.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.843+0000 7f6f177fe640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6f18156460 0x7f6f180cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:24.846 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.843+0000 7f6f177fe640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6f18156460 0x7f6f180cb4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38198/0 (socket says 172.21.15.73:38198) 2024-04-26T20:52:24.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.843+0000 7f6f177fe640 1 -- 172.21.15.73:0/364832361 learned_addr learned my addr 172.21.15.73:0/364832361 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:24.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.843+0000 7f6f177fe640 1 -- 172.21.15.73:0/364832361 >> v2:172.21.15.120:3300/0 conn(0x7f6f1806fea0 msgr2=0x7f6f18072c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:24.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.843+0000 7f6f177fe640 1 --2- 172.21.15.73:0/364832361 >> v2:172.21.15.120:3300/0 conn(0x7f6f1806fea0 0x7f6f18072c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:24.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.843+0000 7f6f177fe640 1 -- 172.21.15.73:0/364832361 >> v1:172.21.15.120:6789/0 conn(0x7f6f180cba10 legacy=0x7f6f1806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:24.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.843+0000 7f6f177fe640 1 -- 172.21.15.73:0/364832361 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f180cc4e0 con 0x7f6f18156460 2024-04-26T20:52:24.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.844+0000 7f6f177fe640 1 --2- 172.21.15.73:0/364832361 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f18156460 0x7f6f180cb4d0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f6f180c7c40 tx=0x7f6f0802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=769c5bba4c24b552 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:24.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.844+0000 7f6f167fc640 1 -- 172.21.15.73:0/364832361 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f08037070 con 0x7f6f18156460 2024-04-26T20:52:24.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.844+0000 7f6f167fc640 1 -- 172.21.15.73:0/364832361 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f08032440 con 0x7f6f18156460 2024-04-26T20:52:24.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.844+0000 7f6f167fc640 1 -- 172.21.15.73:0/364832361 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f08032810 con 0x7f6f18156460 2024-04-26T20:52:24.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.844+0000 7f6f1dc1d640 1 -- 172.21.15.73:0/364832361 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f18156460 msgr2=0x7f6f180cb4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:24.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.844+0000 7f6f1dc1d640 1 --2- 172.21.15.73:0/364832361 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f18156460 0x7f6f180cb4d0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f6f180c7c40 tx=0x7f6f0802a230 comp rx=0 tx=0).stop 2024-04-26T20:52:24.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.844+0000 7f6f1dc1d640 1 -- 172.21.15.73:0/364832361 shutdown_connections 2024-04-26T20:52:24.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.844+0000 7f6f1dc1d640 1 --2- 172.21.15.73:0/364832361 >> v2:172.21.15.120:3300/0 conn(0x7f6f1806fea0 0x7f6f18072c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:24.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.844+0000 7f6f1dc1d640 1 --2- 172.21.15.73:0/364832361 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f18156460 0x7f6f180cb4d0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:24.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.844+0000 7f6f1dc1d640 1 -- 172.21.15.73:0/364832361 >> 172.21.15.73:0/364832361 conn(0x7f6f180d0be0 msgr2=0x7f6f180d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:24.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.844+0000 7f6f1dc1d640 1 -- 172.21.15.73:0/364832361 shutdown_connections 2024-04-26T20:52:24.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.844+0000 7f6f1dc1d640 1 -- 172.21.15.73:0/364832361 wait complete. 2024-04-26T20:52:24.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.846+0000 7f6f1dc1d640 1 Processor -- start 2024-04-26T20:52:24.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.846+0000 7f6f1dc1d640 1 -- start start 2024-04-26T20:52:24.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.846+0000 7f6f1dc1d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f1806fea0 0x7f6f180b64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:24.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.846+0000 7f6f1dc1d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6f180cba10 0x7f6f181f7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:24.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.846+0000 7f6f1dc1d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f18156460 0x7f6f180c7390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:24.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.846+0000 7f6f1dc1d640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1806e260 con 0x7f6f1806fea0 2024-04-26T20:52:24.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.846+0000 7f6f1dc1d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6f18072700 con 0x7f6f18156460 2024-04-26T20:52:24.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.846+0000 7f6f1dc1d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f6f1806e3e0 con 0x7f6f180cba10 2024-04-26T20:52:24.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.847+0000 7f6f177fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f1806fea0 0x7f6f180b64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:24.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.847+0000 7f6f177fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f1806fea0 0x7f6f180b64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38214/0 (socket says 172.21.15.73:38214) 2024-04-26T20:52:24.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.847+0000 7f6f177fe640 1 -- 172.21.15.73:0/315567885 learned_addr learned my addr 172.21.15.73:0/315567885 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:24.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.847+0000 7f6f17fff640 1 --2- 172.21.15.73:0/315567885 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f18156460 0x7f6f180c7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:24.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.847+0000 7f6f16ffd640 1 --2- 172.21.15.73:0/315567885 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6f180cba10 0x7f6f181f7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:24.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.847+0000 7f6f177fe640 1 -- 172.21.15.73:0/315567885 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6f180cba10 msgr2=0x7f6f181f7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:24.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.847+0000 7f6f177fe640 1 --2- 172.21.15.73:0/315567885 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6f180cba10 0x7f6f181f7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:24.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.847+0000 7f6f177fe640 1 -- 172.21.15.73:0/315567885 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f18156460 msgr2=0x7f6f180c7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:24.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.847+0000 7f6f177fe640 1 --2- 172.21.15.73:0/315567885 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f18156460 0x7f6f180c7390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:24.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.847+0000 7f6f177fe640 1 -- 172.21.15.73:0/315567885 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f180c6070 con 0x7f6f1806fea0 2024-04-26T20:52:24.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.847+0000 7f6f17fff640 1 --2- 172.21.15.73:0/315567885 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f18156460 0x7f6f180c7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:52:24.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.847+0000 7f6f16ffd640 1 --2- 172.21.15.73:0/315567885 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6f180cba10 0x7f6f181f7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:52:24.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.847+0000 7f6f177fe640 1 --2- 172.21.15.73:0/315567885 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f1806fea0 0x7f6f180b64f0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f6f0802a270 tx=0x7f6f080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:24.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.847+0000 7f6efffff640 1 -- 172.21.15.73:0/315567885 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f08041070 con 0x7f6f1806fea0 2024-04-26T20:52:24.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.847+0000 7f6efffff640 1 -- 172.21.15.73:0/315567885 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f0803c870 con 0x7f6f1806fea0 2024-04-26T20:52:24.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.847+0000 7f6efffff640 1 -- 172.21.15.73:0/315567885 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f08037040 con 0x7f6f1806fea0 2024-04-26T20:52:24.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.847+0000 7f6f1dc1d640 1 -- 172.21.15.73:0/315567885 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f180c62f0 con 0x7f6f1806fea0 2024-04-26T20:52:24.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.847+0000 7f6f1dc1d640 1 -- 172.21.15.73:0/315567885 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f180c68b0 con 0x7f6f1806fea0 2024-04-26T20:52:24.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.848+0000 7f6f1dc1d640 1 -- 172.21.15.73:0/315567885 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f180694f0 con 0x7f6f1806fea0 2024-04-26T20:52:24.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.849+0000 7f6efffff640 1 -- 172.21.15.73:0/315567885 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6f08044070 con 0x7f6f1806fea0 2024-04-26T20:52:24.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.849+0000 7f6efffff640 1 --2- 172.21.15.73:0/315567885 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f6f0406c8f0 0x7f6f0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:24.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.849+0000 7f6efffff640 1 -- 172.21.15.73:0/315567885 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f6f080ac7e0 con 0x7f6f1806fea0 2024-04-26T20:52:24.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.849+0000 7f6f16ffd640 1 --2- 172.21.15.73:0/315567885 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f6f0406c8f0 0x7f6f0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:24.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.849+0000 7f6f16ffd640 1 --2- 172.21.15.73:0/315567885 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f6f0406c8f0 0x7f6f0406edb0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c007fd0 tx=0x7f6f0c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:24.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:24.853+0000 7f6efffff640 1 -- 172.21.15.73:0/315567885 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6f180c6070 con 0x7f6f1806fea0 2024-04-26T20:52:25.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.027+0000 7f6f1dc1d640 1 -- 172.21.15.73:0/315567885 --> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6f180736f0 con 0x7f6f0406c8f0 2024-04-26T20:52:25.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.032+0000 7f6efffff640 1 -- 172.21.15.73:0/315567885 <== mgr.4107 v2:172.21.15.73:6832/666905549 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240062 (secure 0 0 0) 0x7f6f180736f0 con 0x7f6f0406c8f0 2024-04-26T20:52:25.036 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:52:25.037 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-26T20:52:25.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.035+0000 7f6f1dc1d640 1 -- 172.21.15.73:0/315567885 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f6f0406c8f0 msgr2=0x7f6f0406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:25.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.035+0000 7f6f1dc1d640 1 --2- 172.21.15.73:0/315567885 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f6f0406c8f0 0x7f6f0406edb0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c007fd0 tx=0x7f6f0c009040 comp rx=0 tx=0).stop 2024-04-26T20:52:25.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.035+0000 7f6f1dc1d640 1 -- 172.21.15.73:0/315567885 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f1806fea0 msgr2=0x7f6f180b64f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:25.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.035+0000 7f6f1dc1d640 1 --2- 172.21.15.73:0/315567885 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f1806fea0 0x7f6f180b64f0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f6f0802a270 tx=0x7f6f080046c0 comp rx=0 tx=0).stop 2024-04-26T20:52:25.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.035+0000 7f6f1dc1d640 1 -- 172.21.15.73:0/315567885 shutdown_connections 2024-04-26T20:52:25.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.035+0000 7f6f1dc1d640 1 --2- 172.21.15.73:0/315567885 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f6f0406c8f0 0x7f6f0406edb0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:25.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.035+0000 7f6f1dc1d640 1 --2- 172.21.15.73:0/315567885 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f18156460 0x7f6f180c7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:25.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.035+0000 7f6f1dc1d640 1 --2- 172.21.15.73:0/315567885 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6f180cba10 0x7f6f181f7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:25.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.035+0000 7f6f1dc1d640 1 --2- 172.21.15.73:0/315567885 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f1806fea0 0x7f6f180b64f0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:25.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.035+0000 7f6f1dc1d640 1 -- 172.21.15.73:0/315567885 >> 172.21.15.73:0/315567885 conn(0x7f6f180d0be0 msgr2=0x7f6f1815d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:25.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.035+0000 7f6f1dc1d640 1 -- 172.21.15.73:0/315567885 shutdown_connections 2024-04-26T20:52:25.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.035+0000 7f6f1dc1d640 1 -- 172.21.15.73:0/315567885 wait complete. 2024-04-26T20:52:25.047 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":107,"stamp":"2024-04-26T20:52:24.003511+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221860,"kb_used_data":5644,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752860,"statfs":{"total":773094113280,"available":772866928640,"internally_reserved":0,"allocated":5779456,"data_stored":3205651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.609830"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.395952+0000","last_change":"2024-04-26T20:52:19.395952+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.395952+0000","last_undegraded":"2024-04-26T20:52:19.395952+0000","last_fullsized":"2024-04-26T20:52:19.395952+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.397803+0000","last_change":"2024-04-26T20:52:19.397803+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.397803+0000","last_undegraded":"2024-04-26T20:52:19.397803+0000","last_fullsized":"2024-04-26T20:52:19.397803+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.397585+0000","last_change":"2024-04-26T20:52:19.397585+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.397585+0000","last_undegraded":"2024-04-26T20:52:19.397585+0000","last_fullsized":"2024-04-26T20:52:19.397585+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.396894+0000","last_change":"2024-04-26T20:52:19.396894+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.396894+0000","last_undegraded":"2024-04-26T20:52:19.396894+0000","last_fullsized":"2024-04-26T20:52:19.396894+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.397540+0000","last_change":"2024-04-26T20:52:19.397540+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.397540+0000","last_undegraded":"2024-04-26T20:52:19.397540+0000","last_fullsized":"2024-04-26T20:52:19.397540+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.393560+0000","last_change":"2024-04-26T20:52:19.393560+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.393560+0000","last_undegraded":"2024-04-26T20:52:19.393560+0000","last_fullsized":"2024-04-26T20:52:19.393560+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.395521+0000","last_change":"2024-04-26T20:52:19.395521+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.395521+0000","last_undegraded":"2024-04-26T20:52:19.395521+0000","last_fullsized":"2024-04-26T20:52:19.395521+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.397229+0000","last_change":"2024-04-26T20:52:19.397229+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.397229+0000","last_undegraded":"2024-04-26T20:52:19.397229+0000","last_fullsized":"2024-04-26T20:52:19.397229+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.397756+0000","last_change":"2024-04-26T20:52:19.397756+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.397756+0000","last_undegraded":"2024-04-26T20:52:19.397756+0000","last_fullsized":"2024-04-26T20:52:19.397756+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.393499+0000","last_change":"2024-04-26T20:52:19.393499+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.393499+0000","last_undegraded":"2024-04-26T20:52:19.393499+0000","last_fullsized":"2024-04-26T20:52:19.393499+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.398384+0000","last_change":"2024-04-26T20:52:19.398384+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.398384+0000","last_undegraded":"2024-04-26T20:52:19.398384+0000","last_fullsized":"2024-04-26T20:52:19.398384+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.398218+0000","last_change":"2024-04-26T20:52:19.398218+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.398218+0000","last_undegraded":"2024-04-26T20:52:19.398218+0000","last_fullsized":"2024-04-26T20:52:19.398218+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.396945+0000","last_change":"2024-04-26T20:52:19.396945+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.396945+0000","last_undegraded":"2024-04-26T20:52:19.396945+0000","last_fullsized":"2024-04-26T20:52:19.396945+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.395349+0000","last_change":"2024-04-26T20:52:19.395349+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.395349+0000","last_undegraded":"2024-04-26T20:52:19.395349+0000","last_fullsized":"2024-04-26T20:52:19.395349+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.397694+0000","last_change":"2024-04-26T20:52:19.397694+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.397694+0000","last_undegraded":"2024-04-26T20:52:19.397694+0000","last_fullsized":"2024-04-26T20:52:19.397694+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.397561+0000","last_change":"2024-04-26T20:52:19.397561+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.397561+0000","last_undegraded":"2024-04-26T20:52:19.397561+0000","last_fullsized":"2024-04-26T20:52:19.397561+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.396989+0000","last_change":"2024-04-26T20:52:19.396989+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.396989+0000","last_undegraded":"2024-04-26T20:52:19.396989+0000","last_fullsized":"2024-04-26T20:52:19.396989+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.400177+0000","last_change":"2024-04-26T20:52:19.400177+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.400177+0000","last_undegraded":"2024-04-26T20:52:19.400177+0000","last_fullsized":"2024-04-26T20:52:19.400177+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.395403+0000","last_change":"2024-04-26T20:52:19.395403+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.395403+0000","last_undegraded":"2024-04-26T20:52:19.395403+0000","last_fullsized":"2024-04-26T20:52:19.395403+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.395340+0000","last_change":"2024-04-26T20:52:19.395340+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.395340+0000","last_undegraded":"2024-04-26T20:52:19.395340+0000","last_fullsized":"2024-04-26T20:52:19.395340+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.397535+0000","last_change":"2024-04-26T20:52:19.397535+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.397535+0000","last_undegraded":"2024-04-26T20:52:19.397535+0000","last_fullsized":"2024-04-26T20:52:19.397535+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.397145+0000","last_change":"2024-04-26T20:52:19.397145+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.397145+0000","last_undegraded":"2024-04-26T20:52:19.397145+0000","last_fullsized":"2024-04-26T20:52:19.397145+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.396532+0000","last_change":"2024-04-26T20:52:19.396532+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.396532+0000","last_undegraded":"2024-04-26T20:52:19.396532+0000","last_fullsized":"2024-04-26T20:52:19.396532+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.400129+0000","last_change":"2024-04-26T20:52:19.400129+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.400129+0000","last_undegraded":"2024-04-26T20:52:19.400129+0000","last_fullsized":"2024-04-26T20:52:19.400129+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.395398+0000","last_change":"2024-04-26T20:52:19.395398+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.395398+0000","last_undegraded":"2024-04-26T20:52:19.395398+0000","last_fullsized":"2024-04-26T20:52:19.395398+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.397644+0000","last_change":"2024-04-26T20:52:19.397644+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.397644+0000","last_undegraded":"2024-04-26T20:52:19.397644+0000","last_fullsized":"2024-04-26T20:52:19.397644+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.396492+0000","last_change":"2024-04-26T20:52:19.396492+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.396492+0000","last_undegraded":"2024-04-26T20:52:19.396492+0000","last_fullsized":"2024-04-26T20:52:19.396492+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.397080+0000","last_change":"2024-04-26T20:52:19.397080+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.397080+0000","last_undegraded":"2024-04-26T20:52:19.397080+0000","last_fullsized":"2024-04-26T20:52:19.397080+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.396492+0000","last_change":"2024-04-26T20:52:19.396492+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.396492+0000","last_undegraded":"2024-04-26T20:52:19.396492+0000","last_fullsized":"2024-04-26T20:52:19.396492+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.393556+0000","last_change":"2024-04-26T20:52:19.393556+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.393556+0000","last_undegraded":"2024-04-26T20:52:19.393556+0000","last_fullsized":"2024-04-26T20:52:19.393556+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.396627+0000","last_change":"2024-04-26T20:52:19.396627+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.396627+0000","last_undegraded":"2024-04-26T20:52:19.396627+0000","last_fullsized":"2024-04-26T20:52:19.396627+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.396494+0000","last_change":"2024-04-26T20:52:19.396494+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.396494+0000","last_undegraded":"2024-04-26T20:52:19.396494+0000","last_fullsized":"2024-04-26T20:52:19.396494+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":221,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-26T20:52:19.397911+0000","last_change":"2024-04-26T20:50:21.150132+0000","last_active":"2024-04-26T20:52:19.397911+0000","last_peered":"2024-04-26T20:52:19.397911+0000","last_clean":"2024-04-26T20:52:19.397911+0000","last_became_active":"2024-04-26T20:50:21.147439+0000","last_became_peered":"2024-04-26T20:50:21.147439+0000","last_unstale":"2024-04-26T20:52:19.397911+0000","last_undegraded":"2024-04-26T20:52:19.397911+0000","last_fullsized":"2024-04-26T20:52:19.397911+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:20.084499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:20.084499+0000","last_clean_scrub_stamp":"2024-04-26T20:50:20.084499+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:19:29.395477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28160,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343680,"statfs":{"total":96636764160,"available":96607928320,"internally_reserved":0,"allocated":1163264,"data_stored":841154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27600,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344240,"statfs":{"total":96636764160,"available":96608501760,"internally_reserved":0,"allocated":581632,"data_stored":256995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":569344,"data_stored":250786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27584,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":569344,"data_stored":250786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27592,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":581632,"data_stored":256995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27588,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344252,"statfs":{"total":96636764160,"available":96608514048,"internally_reserved":0,"allocated":569344,"data_stored":250786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27584,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":569344,"data_stored":250786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28176,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343664,"statfs":{"total":96636764160,"available":96607911936,"internally_reserved":0,"allocated":1175552,"data_stored":847363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:52:25.051 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3f is not active+clean 2024-04-26T20:52:25.051 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.052 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3e is not active+clean 2024-04-26T20:52:25.052 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.052 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3d is not active+clean 2024-04-26T20:52:25.052 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.052 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3c is not active+clean 2024-04-26T20:52:25.052 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.052 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3b is not active+clean 2024-04-26T20:52:25.052 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.052 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3a is not active+clean 2024-04-26T20:52:25.052 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.052 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.39 is not active+clean 2024-04-26T20:52:25.052 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.053 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.38 is not active+clean 2024-04-26T20:52:25.053 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.053 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.37 is not active+clean 2024-04-26T20:52:25.053 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.053 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.36 is not active+clean 2024-04-26T20:52:25.053 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.053 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.35 is not active+clean 2024-04-26T20:52:25.053 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.053 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.34 is not active+clean 2024-04-26T20:52:25.053 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.053 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.33 is not active+clean 2024-04-26T20:52:25.053 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.054 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.32 is not active+clean 2024-04-26T20:52:25.054 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.054 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.31 is not active+clean 2024-04-26T20:52:25.054 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.054 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.30 is not active+clean 2024-04-26T20:52:25.054 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.054 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2f is not active+clean 2024-04-26T20:52:25.054 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.054 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2e is not active+clean 2024-04-26T20:52:25.054 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.054 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2d is not active+clean 2024-04-26T20:52:25.055 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.055 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2c is not active+clean 2024-04-26T20:52:25.055 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.055 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2b is not active+clean 2024-04-26T20:52:25.055 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.055 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2a is not active+clean 2024-04-26T20:52:25.055 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.055 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.29 is not active+clean 2024-04-26T20:52:25.055 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.055 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.28 is not active+clean 2024-04-26T20:52:25.055 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.056 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.27 is not active+clean 2024-04-26T20:52:25.056 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.056 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.26 is not active+clean 2024-04-26T20:52:25.056 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.056 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.25 is not active+clean 2024-04-26T20:52:25.056 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.056 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.24 is not active+clean 2024-04-26T20:52:25.056 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.056 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.23 is not active+clean 2024-04-26T20:52:25.056 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.056 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.22 is not active+clean 2024-04-26T20:52:25.056 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.057 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.21 is not active+clean 2024-04-26T20:52:25.057 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.057 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.20 is not active+clean 2024-04-26T20:52:25.057 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.057 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.10 is not active+clean 2024-04-26T20:52:25.057 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.057 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1f is not active+clean 2024-04-26T20:52:25.057 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:52:19.395952+0000', 'last_change': '2024-04-26T20:52:19.395952+0000', 'last_active': '2024-04-26T20:52:19.382176+0000', 'last_peered': '2024-04-26T20:52:19.382176+0000', 'last_clean': '2024-04-26T20:52:19.382176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:19.395952+0000', 'last_undegraded': '2024-04-26T20:52:19.395952+0000', 'last_fullsized': '2024-04-26T20:52:19.395952+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T20:52:25.057 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.11 is not active+clean 2024-04-26T20:52:25.057 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.057 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1e is not active+clean 2024-04-26T20:52:25.057 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:52:19.397803+0000', 'last_change': '2024-04-26T20:52:19.397803+0000', 'last_active': '2024-04-26T20:52:19.382176+0000', 'last_peered': '2024-04-26T20:52:19.382176+0000', 'last_clean': '2024-04-26T20:52:19.382176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:19.397803+0000', 'last_undegraded': '2024-04-26T20:52:19.397803+0000', 'last_fullsized': '2024-04-26T20:52:19.397803+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T20:52:25.058 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.12 is not active+clean 2024-04-26T20:52:25.058 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.058 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1d is not active+clean 2024-04-26T20:52:25.058 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:52:19.397585+0000', 'last_change': '2024-04-26T20:52:19.397585+0000', 'last_active': '2024-04-26T20:52:19.382176+0000', 'last_peered': '2024-04-26T20:52:19.382176+0000', 'last_clean': '2024-04-26T20:52:19.382176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:19.397585+0000', 'last_undegraded': '2024-04-26T20:52:19.397585+0000', 'last_fullsized': '2024-04-26T20:52:19.397585+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T20:52:25.058 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.13 is not active+clean 2024-04-26T20:52:25.058 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.058 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1c is not active+clean 2024-04-26T20:52:25.058 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:52:19.396894+0000', 'last_change': '2024-04-26T20:52:19.396894+0000', 'last_active': '2024-04-26T20:52:19.382176+0000', 'last_peered': '2024-04-26T20:52:19.382176+0000', 'last_clean': '2024-04-26T20:52:19.382176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:19.396894+0000', 'last_undegraded': '2024-04-26T20:52:19.396894+0000', 'last_fullsized': '2024-04-26T20:52:19.396894+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T20:52:25.058 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.14 is not active+clean 2024-04-26T20:52:25.058 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.059 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1b is not active+clean 2024-04-26T20:52:25.059 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:52:19.397540+0000', 'last_change': '2024-04-26T20:52:19.397540+0000', 'last_active': '2024-04-26T20:52:19.382176+0000', 'last_peered': '2024-04-26T20:52:19.382176+0000', 'last_clean': '2024-04-26T20:52:19.382176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:19.397540+0000', 'last_undegraded': '2024-04-26T20:52:19.397540+0000', 'last_fullsized': '2024-04-26T20:52:19.397540+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T20:52:25.059 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.15 is not active+clean 2024-04-26T20:52:25.059 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.059 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1a is not active+clean 2024-04-26T20:52:25.059 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:52:19.393560+0000', 'last_change': '2024-04-26T20:52:19.393560+0000', 'last_active': '2024-04-26T20:52:19.382176+0000', 'last_peered': '2024-04-26T20:52:19.382176+0000', 'last_clean': '2024-04-26T20:52:19.382176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:19.393560+0000', 'last_undegraded': '2024-04-26T20:52:19.393560+0000', 'last_fullsized': '2024-04-26T20:52:19.393560+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T20:52:25.059 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.16 is not active+clean 2024-04-26T20:52:25.059 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.059 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.19 is not active+clean 2024-04-26T20:52:25.059 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:52:19.395521+0000', 'last_change': '2024-04-26T20:52:19.395521+0000', 'last_active': '2024-04-26T20:52:19.382176+0000', 'last_peered': '2024-04-26T20:52:19.382176+0000', 'last_clean': '2024-04-26T20:52:19.382176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:19.395521+0000', 'last_undegraded': '2024-04-26T20:52:19.395521+0000', 'last_fullsized': '2024-04-26T20:52:19.395521+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T20:52:25.059 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.17 is not active+clean 2024-04-26T20:52:25.059 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.060 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.18 is not active+clean 2024-04-26T20:52:25.060 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:52:19.397229+0000', 'last_change': '2024-04-26T20:52:19.397229+0000', 'last_active': '2024-04-26T20:52:19.382176+0000', 'last_peered': '2024-04-26T20:52:19.382176+0000', 'last_clean': '2024-04-26T20:52:19.382176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:19.397229+0000', 'last_undegraded': '2024-04-26T20:52:19.397229+0000', 'last_fullsized': '2024-04-26T20:52:19.397229+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T20:52:25.060 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.18 is not active+clean 2024-04-26T20:52:25.060 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.060 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.17 is not active+clean 2024-04-26T20:52:25.060 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:52:19.397756+0000', 'last_change': '2024-04-26T20:52:19.397756+0000', 'last_active': '2024-04-26T20:52:19.382176+0000', 'last_peered': '2024-04-26T20:52:19.382176+0000', 'last_clean': '2024-04-26T20:52:19.382176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:19.397756+0000', 'last_undegraded': '2024-04-26T20:52:19.397756+0000', 'last_fullsized': '2024-04-26T20:52:19.397756+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T20:52:25.060 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.19 is not active+clean 2024-04-26T20:52:25.060 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.060 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.16 is not active+clean 2024-04-26T20:52:25.060 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:52:19.393499+0000', 'last_change': '2024-04-26T20:52:19.393499+0000', 'last_active': '2024-04-26T20:52:19.382176+0000', 'last_peered': '2024-04-26T20:52:19.382176+0000', 'last_clean': '2024-04-26T20:52:19.382176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:19.393499+0000', 'last_undegraded': '2024-04-26T20:52:19.393499+0000', 'last_fullsized': '2024-04-26T20:52:19.393499+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T20:52:25.060 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1a is not active+clean 2024-04-26T20:52:25.061 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.061 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.15 is not active+clean 2024-04-26T20:52:25.061 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:52:19.398384+0000', 'last_change': '2024-04-26T20:52:19.398384+0000', 'last_active': '2024-04-26T20:52:19.382176+0000', 'last_peered': '2024-04-26T20:52:19.382176+0000', 'last_clean': '2024-04-26T20:52:19.382176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:19.398384+0000', 'last_undegraded': '2024-04-26T20:52:19.398384+0000', 'last_fullsized': '2024-04-26T20:52:19.398384+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T20:52:25.061 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1b is not active+clean 2024-04-26T20:52:25.061 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.061 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.14 is not active+clean 2024-04-26T20:52:25.061 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:52:19.398218+0000', 'last_change': '2024-04-26T20:52:19.398218+0000', 'last_active': '2024-04-26T20:52:19.382176+0000', 'last_peered': '2024-04-26T20:52:19.382176+0000', 'last_clean': '2024-04-26T20:52:19.382176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:19.398218+0000', 'last_undegraded': '2024-04-26T20:52:19.398218+0000', 'last_fullsized': '2024-04-26T20:52:19.398218+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T20:52:25.061 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1c is not active+clean 2024-04-26T20:52:25.061 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.061 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.13 is not active+clean 2024-04-26T20:52:25.061 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:52:19.396945+0000', 'last_change': '2024-04-26T20:52:19.396945+0000', 'last_active': '2024-04-26T20:52:19.382176+0000', 'last_peered': '2024-04-26T20:52:19.382176+0000', 'last_clean': '2024-04-26T20:52:19.382176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:19.396945+0000', 'last_undegraded': '2024-04-26T20:52:19.396945+0000', 'last_fullsized': '2024-04-26T20:52:19.396945+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T20:52:25.062 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1d is not active+clean 2024-04-26T20:52:25.062 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.062 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.12 is not active+clean 2024-04-26T20:52:25.062 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:52:19.395349+0000', 'last_change': '2024-04-26T20:52:19.395349+0000', 'last_active': '2024-04-26T20:52:19.382176+0000', 'last_peered': '2024-04-26T20:52:19.382176+0000', 'last_clean': '2024-04-26T20:52:19.382176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:19.395349+0000', 'last_undegraded': '2024-04-26T20:52:19.395349+0000', 'last_fullsized': '2024-04-26T20:52:19.395349+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T20:52:25.062 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1e is not active+clean 2024-04-26T20:52:25.062 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.062 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.11 is not active+clean 2024-04-26T20:52:25.062 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:52:19.397694+0000', 'last_change': '2024-04-26T20:52:19.397694+0000', 'last_active': '2024-04-26T20:52:19.382176+0000', 'last_peered': '2024-04-26T20:52:19.382176+0000', 'last_clean': '2024-04-26T20:52:19.382176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:19.397694+0000', 'last_undegraded': '2024-04-26T20:52:19.397694+0000', 'last_fullsized': '2024-04-26T20:52:19.397694+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T20:52:25.062 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1f is not active+clean 2024-04-26T20:52:25.062 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.062 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.10 is not active+clean 2024-04-26T20:52:25.062 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:52:19.397561+0000', 'last_change': '2024-04-26T20:52:19.397561+0000', 'last_active': '2024-04-26T20:52:19.382176+0000', 'last_peered': '2024-04-26T20:52:19.382176+0000', 'last_clean': '2024-04-26T20:52:19.382176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:19.397561+0000', 'last_undegraded': '2024-04-26T20:52:19.397561+0000', 'last_fullsized': '2024-04-26T20:52:19.397561+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T20:52:25.063 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.0 is not active+clean 2024-04-26T20:52:25.063 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.063 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.f is not active+clean 2024-04-26T20:52:25.063 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:52:19.396989+0000', 'last_change': '2024-04-26T20:52:19.396989+0000', 'last_active': '2024-04-26T20:52:19.382176+0000', 'last_peered': '2024-04-26T20:52:19.382176+0000', 'last_clean': '2024-04-26T20:52:19.382176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:19.396989+0000', 'last_undegraded': '2024-04-26T20:52:19.396989+0000', 'last_fullsized': '2024-04-26T20:52:19.396989+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T20:52:25.063 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1 is not active+clean 2024-04-26T20:52:25.063 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.063 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.e is not active+clean 2024-04-26T20:52:25.063 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:52:19.400177+0000', 'last_change': '2024-04-26T20:52:19.400177+0000', 'last_active': '2024-04-26T20:52:19.382176+0000', 'last_peered': '2024-04-26T20:52:19.382176+0000', 'last_clean': '2024-04-26T20:52:19.382176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:19.400177+0000', 'last_undegraded': '2024-04-26T20:52:19.400177+0000', 'last_fullsized': '2024-04-26T20:52:19.400177+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T20:52:25.063 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2 is not active+clean 2024-04-26T20:52:25.063 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.063 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.d is not active+clean 2024-04-26T20:52:25.064 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:52:19.395403+0000', 'last_change': '2024-04-26T20:52:19.395403+0000', 'last_active': '2024-04-26T20:52:19.382176+0000', 'last_peered': '2024-04-26T20:52:19.382176+0000', 'last_clean': '2024-04-26T20:52:19.382176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:19.395403+0000', 'last_undegraded': '2024-04-26T20:52:19.395403+0000', 'last_fullsized': '2024-04-26T20:52:19.395403+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T20:52:25.064 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3 is not active+clean 2024-04-26T20:52:25.064 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.064 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.c is not active+clean 2024-04-26T20:52:25.064 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:52:19.395340+0000', 'last_change': '2024-04-26T20:52:19.395340+0000', 'last_active': '2024-04-26T20:52:19.382176+0000', 'last_peered': '2024-04-26T20:52:19.382176+0000', 'last_clean': '2024-04-26T20:52:19.382176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:19.395340+0000', 'last_undegraded': '2024-04-26T20:52:19.395340+0000', 'last_fullsized': '2024-04-26T20:52:19.395340+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T20:52:25.064 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.4 is not active+clean 2024-04-26T20:52:25.064 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.064 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.b is not active+clean 2024-04-26T20:52:25.064 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:52:19.397535+0000', 'last_change': '2024-04-26T20:52:19.397535+0000', 'last_active': '2024-04-26T20:52:19.382176+0000', 'last_peered': '2024-04-26T20:52:19.382176+0000', 'last_clean': '2024-04-26T20:52:19.382176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:19.397535+0000', 'last_undegraded': '2024-04-26T20:52:19.397535+0000', 'last_fullsized': '2024-04-26T20:52:19.397535+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T20:52:25.064 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.5 is not active+clean 2024-04-26T20:52:25.064 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.064 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.a is not active+clean 2024-04-26T20:52:25.065 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:52:19.397145+0000', 'last_change': '2024-04-26T20:52:19.397145+0000', 'last_active': '2024-04-26T20:52:19.382176+0000', 'last_peered': '2024-04-26T20:52:19.382176+0000', 'last_clean': '2024-04-26T20:52:19.382176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:19.397145+0000', 'last_undegraded': '2024-04-26T20:52:19.397145+0000', 'last_fullsized': '2024-04-26T20:52:19.397145+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T20:52:25.065 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.6 is not active+clean 2024-04-26T20:52:25.065 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.065 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.9 is not active+clean 2024-04-26T20:52:25.065 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:52:19.396532+0000', 'last_change': '2024-04-26T20:52:19.396532+0000', 'last_active': '2024-04-26T20:52:19.382176+0000', 'last_peered': '2024-04-26T20:52:19.382176+0000', 'last_clean': '2024-04-26T20:52:19.382176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:19.396532+0000', 'last_undegraded': '2024-04-26T20:52:19.396532+0000', 'last_fullsized': '2024-04-26T20:52:19.396532+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T20:52:25.065 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.7 is not active+clean 2024-04-26T20:52:25.065 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.065 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.8 is not active+clean 2024-04-26T20:52:25.065 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:52:19.400129+0000', 'last_change': '2024-04-26T20:52:19.400129+0000', 'last_active': '2024-04-26T20:52:19.382176+0000', 'last_peered': '2024-04-26T20:52:19.382176+0000', 'last_clean': '2024-04-26T20:52:19.382176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:19.400129+0000', 'last_undegraded': '2024-04-26T20:52:19.400129+0000', 'last_fullsized': '2024-04-26T20:52:19.400129+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T20:52:25.065 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.8 is not active+clean 2024-04-26T20:52:25.065 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.065 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.7 is not active+clean 2024-04-26T20:52:25.066 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:52:19.395398+0000', 'last_change': '2024-04-26T20:52:19.395398+0000', 'last_active': '2024-04-26T20:52:19.382176+0000', 'last_peered': '2024-04-26T20:52:19.382176+0000', 'last_clean': '2024-04-26T20:52:19.382176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:19.395398+0000', 'last_undegraded': '2024-04-26T20:52:19.395398+0000', 'last_fullsized': '2024-04-26T20:52:19.395398+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T20:52:25.066 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.a is not active+clean 2024-04-26T20:52:25.066 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.066 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.5 is not active+clean 2024-04-26T20:52:25.066 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:52:19.397644+0000', 'last_change': '2024-04-26T20:52:19.397644+0000', 'last_active': '2024-04-26T20:52:19.382176+0000', 'last_peered': '2024-04-26T20:52:19.382176+0000', 'last_clean': '2024-04-26T20:52:19.382176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:19.397644+0000', 'last_undegraded': '2024-04-26T20:52:19.397644+0000', 'last_fullsized': '2024-04-26T20:52:19.397644+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T20:52:25.066 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.b is not active+clean 2024-04-26T20:52:25.066 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.066 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.4 is not active+clean 2024-04-26T20:52:25.066 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:52:19.396492+0000', 'last_change': '2024-04-26T20:52:19.396492+0000', 'last_active': '2024-04-26T20:52:19.382176+0000', 'last_peered': '2024-04-26T20:52:19.382176+0000', 'last_clean': '2024-04-26T20:52:19.382176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:19.396492+0000', 'last_undegraded': '2024-04-26T20:52:19.396492+0000', 'last_fullsized': '2024-04-26T20:52:19.396492+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T20:52:25.066 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.c is not active+clean 2024-04-26T20:52:25.066 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.067 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3 is not active+clean 2024-04-26T20:52:25.067 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:52:19.397080+0000', 'last_change': '2024-04-26T20:52:19.397080+0000', 'last_active': '2024-04-26T20:52:19.382176+0000', 'last_peered': '2024-04-26T20:52:19.382176+0000', 'last_clean': '2024-04-26T20:52:19.382176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:19.397080+0000', 'last_undegraded': '2024-04-26T20:52:19.397080+0000', 'last_fullsized': '2024-04-26T20:52:19.397080+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T20:52:25.067 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.d is not active+clean 2024-04-26T20:52:25.067 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.067 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2 is not active+clean 2024-04-26T20:52:25.067 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:52:19.396492+0000', 'last_change': '2024-04-26T20:52:19.396492+0000', 'last_active': '2024-04-26T20:52:19.382176+0000', 'last_peered': '2024-04-26T20:52:19.382176+0000', 'last_clean': '2024-04-26T20:52:19.382176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:19.396492+0000', 'last_undegraded': '2024-04-26T20:52:19.396492+0000', 'last_fullsized': '2024-04-26T20:52:19.396492+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T20:52:25.067 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.e is not active+clean 2024-04-26T20:52:25.067 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.067 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1 is not active+clean 2024-04-26T20:52:25.067 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:52:19.393556+0000', 'last_change': '2024-04-26T20:52:19.393556+0000', 'last_active': '2024-04-26T20:52:19.382176+0000', 'last_peered': '2024-04-26T20:52:19.382176+0000', 'last_clean': '2024-04-26T20:52:19.382176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:19.393556+0000', 'last_undegraded': '2024-04-26T20:52:19.393556+0000', 'last_fullsized': '2024-04-26T20:52:19.393556+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T20:52:25.067 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.f is not active+clean 2024-04-26T20:52:25.068 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.068 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.0 is not active+clean 2024-04-26T20:52:25.068 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:52:19.396627+0000', 'last_change': '2024-04-26T20:52:19.396627+0000', 'last_active': '2024-04-26T20:52:19.382176+0000', 'last_peered': '2024-04-26T20:52:19.382176+0000', 'last_clean': '2024-04-26T20:52:19.382176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:19.396627+0000', 'last_undegraded': '2024-04-26T20:52:19.396627+0000', 'last_fullsized': '2024-04-26T20:52:19.396627+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T20:52:25.068 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.9 is not active+clean 2024-04-26T20:52:25.068 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T20:52:20.391318+0000', 'last_change': '2024-04-26T20:52:20.391318+0000', 'last_active': '2024-04-26T20:52:20.391318+0000', 'last_peered': '2024-04-26T20:52:20.391318+0000', 'last_clean': '2024-04-26T20:52:20.391318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:20.391318+0000', 'last_undegraded': '2024-04-26T20:52:20.391318+0000', 'last_fullsized': '2024-04-26T20:52:20.391318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:20.391318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T20:52:25.068 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.6 is not active+clean 2024-04-26T20:52:25.068 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 49, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:52:19.396494+0000', 'last_change': '2024-04-26T20:52:19.396494+0000', 'last_active': '2024-04-26T20:52:19.382176+0000', 'last_peered': '2024-04-26T20:52:19.382176+0000', 'last_clean': '2024-04-26T20:52:19.382176+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:52:19.396494+0000', 'last_undegraded': '2024-04-26T20:52:19.396494+0000', 'last_fullsized': '2024-04-26T20:52:19.396494+0000', 'mapping_epoch': 49, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 49, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'last_clean_scrub_stamp': '2024-04-26T20:52:19.382176+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T20:52:25.069 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T20:52:25.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.181+0000 7f0543732640 1 Processor -- start 2024-04-26T20:52:25.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.181+0000 7f0543732640 1 -- start start 2024-04-26T20:52:25.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.182+0000 7f0543732640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f053c0b0380 0x7f053c0ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:25.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.182+0000 7f0543732640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f053c0b10b0 0x7f053c06c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:25.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.182+0000 7f0543732640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f053c0cfd80 con 0x7f053c06c780 2024-04-26T20:52:25.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.182+0000 7f0543732640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f053c06cbd0 con 0x7f053c0b10b0 2024-04-26T20:52:25.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.182+0000 7f0543732640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f053c0af8e0 con 0x7f053c0b0380 2024-04-26T20:52:25.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.182+0000 7f0540ccd640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f053c0b10b0 0x7f053c06c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:25.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.182+0000 7f0540ccd640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f053c0b10b0 0x7f053c06c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38222/0 (socket says 172.21.15.73:38222) 2024-04-26T20:52:25.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.182+0000 7f0540ccd640 1 -- 172.21.15.73:0/1935294761 learned_addr learned my addr 172.21.15.73:0/1935294761 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:25.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.182+0000 7f05414ce640 1 --2- 172.21.15.73:0/1935294761 >> v2:172.21.15.120:3300/0 conn(0x7f053c0b0380 0x7f053c0ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:25.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.183+0000 7f0540ccd640 1 -- 172.21.15.73:0/1935294761 >> v2:172.21.15.120:3300/0 conn(0x7f053c0b0380 msgr2=0x7f053c0ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:25.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.183+0000 7f0540ccd640 1 --2- 172.21.15.73:0/1935294761 >> v2:172.21.15.120:3300/0 conn(0x7f053c0b0380 0x7f053c0ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:25.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.183+0000 7f0540ccd640 1 -- 172.21.15.73:0/1935294761 >> v1:172.21.15.120:6789/0 conn(0x7f053c06c780 legacy=0x7f053c072f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:25.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.183+0000 7f0540ccd640 1 -- 172.21.15.73:0/1935294761 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f053c0d0020 con 0x7f053c0b10b0 2024-04-26T20:52:25.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.183+0000 7f05414ce640 1 --2- 172.21.15.73:0/1935294761 >> v2:172.21.15.120:3300/0 conn(0x7f053c0b0380 0x7f053c0ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:52:25.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.183+0000 7f0540ccd640 1 --2- 172.21.15.73:0/1935294761 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f053c0b10b0 0x7f053c06c240 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f053c06d290 tx=0x7f052c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5909fb09d88d5a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:25.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.183+0000 7f0533fff640 1 -- 172.21.15.73:0/1935294761 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052c037070 con 0x7f053c0b10b0 2024-04-26T20:52:25.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.183+0000 7f0533fff640 1 -- 172.21.15.73:0/1935294761 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f052c032440 con 0x7f053c0b10b0 2024-04-26T20:52:25.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.183+0000 7f0533fff640 1 -- 172.21.15.73:0/1935294761 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052c032810 con 0x7f053c0b10b0 2024-04-26T20:52:25.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.184+0000 7f0543732640 1 -- 172.21.15.73:0/1935294761 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f053c0b10b0 msgr2=0x7f053c06c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:25.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.184+0000 7f0543732640 1 --2- 172.21.15.73:0/1935294761 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f053c0b10b0 0x7f053c06c240 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f053c06d290 tx=0x7f052c02a1e0 comp rx=0 tx=0).stop 2024-04-26T20:52:25.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.184+0000 7f0543732640 1 -- 172.21.15.73:0/1935294761 shutdown_connections 2024-04-26T20:52:25.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.184+0000 7f0543732640 1 --2- 172.21.15.73:0/1935294761 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f053c0b10b0 0x7f053c06c240 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:25.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.184+0000 7f0543732640 1 --2- 172.21.15.73:0/1935294761 >> v2:172.21.15.120:3300/0 conn(0x7f053c0b0380 0x7f053c0ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:25.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.184+0000 7f0543732640 1 -- 172.21.15.73:0/1935294761 >> 172.21.15.73:0/1935294761 conn(0x7f053c0d1cf0 msgr2=0x7f053c0d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:25.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.184+0000 7f0543732640 1 -- 172.21.15.73:0/1935294761 shutdown_connections 2024-04-26T20:52:25.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.184+0000 7f0543732640 1 -- 172.21.15.73:0/1935294761 wait complete. 2024-04-26T20:52:25.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.185+0000 7f0543732640 1 Processor -- start 2024-04-26T20:52:25.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.185+0000 7f0543732640 1 -- start start 2024-04-26T20:52:25.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.186+0000 7f0543732640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f053c06c780 0x7f053c14fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:25.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.186+0000 7f0543732640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f053c0b0380 0x7f053c150570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:25.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.186+0000 7f0543732640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f053c0b10b0 0x7f053c0cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:25.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.186+0000 7f0543732640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f053c0d1980 con 0x7f053c0b0380 2024-04-26T20:52:25.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.186+0000 7f0543732640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f053c0d05e0 con 0x7f053c0b10b0 2024-04-26T20:52:25.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.186+0000 7f0543732640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f053c0b0760 con 0x7f053c06c780 2024-04-26T20:52:25.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.186+0000 7f0540ccd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f053c0b0380 0x7f053c150570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:25.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.186+0000 7f0540ccd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f053c0b0380 0x7f053c150570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38238/0 (socket says 172.21.15.73:38238) 2024-04-26T20:52:25.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.186+0000 7f0540ccd640 1 -- 172.21.15.73:0/2396819997 learned_addr learned my addr 172.21.15.73:0/2396819997 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:25.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.186+0000 7f0540ccd640 1 -- 172.21.15.73:0/2396819997 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f053c06c780 msgr2=0x7f053c14fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:25.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.186+0000 7f0541ccf640 1 --2- 172.21.15.73:0/2396819997 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f053c0b10b0 0x7f053c0cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:25.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.186+0000 7f05414ce640 1 --2- 172.21.15.73:0/2396819997 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f053c06c780 0x7f053c14fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:25.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.186+0000 7f0540ccd640 1 --2- 172.21.15.73:0/2396819997 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f053c06c780 0x7f053c14fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:25.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.186+0000 7f0540ccd640 1 -- 172.21.15.73:0/2396819997 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f053c0b10b0 msgr2=0x7f053c0cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:25.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.186+0000 7f0540ccd640 1 --2- 172.21.15.73:0/2396819997 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f053c0b10b0 0x7f053c0cbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:25.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.186+0000 7f0540ccd640 1 -- 172.21.15.73:0/2396819997 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f053c150260 con 0x7f053c0b0380 2024-04-26T20:52:25.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.186+0000 7f05414ce640 1 --2- 172.21.15.73:0/2396819997 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f053c06c780 0x7f053c14fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:52:25.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.186+0000 7f0541ccf640 1 --2- 172.21.15.73:0/2396819997 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f053c0b10b0 0x7f053c0cbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:52:25.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.186+0000 7f0540ccd640 1 --2- 172.21.15.73:0/2396819997 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f053c0b0380 0x7f053c150570 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f052c02a220 tx=0x7f052c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:25.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.186+0000 7f0531ffb640 1 -- 172.21.15.73:0/2396819997 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052c041020 con 0x7f053c0b0380 2024-04-26T20:52:25.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.186+0000 7f0531ffb640 1 -- 172.21.15.73:0/2396819997 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f052c0306c0 con 0x7f053c0b0380 2024-04-26T20:52:25.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.186+0000 7f0543732640 1 -- 172.21.15.73:0/2396819997 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f053c153450 con 0x7f053c0b0380 2024-04-26T20:52:25.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.187+0000 7f0531ffb640 1 -- 172.21.15.73:0/2396819997 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052c037040 con 0x7f053c0b0380 2024-04-26T20:52:25.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.187+0000 7f0543732640 1 -- 172.21.15.73:0/2396819997 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f053c0cb390 con 0x7f053c0b0380 2024-04-26T20:52:25.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.188+0000 7f0543732640 1 -- 172.21.15.73:0/2396819997 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f053c0694f0 con 0x7f053c0b0380 2024-04-26T20:52:25.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.188+0000 7f0531ffb640 1 -- 172.21.15.73:0/2396819997 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f052c044020 con 0x7f053c0b0380 2024-04-26T20:52:25.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.188+0000 7f0531ffb640 1 --2- 172.21.15.73:0/2396819997 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f052006c8f0 0x7f052006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:25.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.188+0000 7f0531ffb640 1 -- 172.21.15.73:0/2396819997 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f052c0ac230 con 0x7f053c0b0380 2024-04-26T20:52:25.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.189+0000 7f05414ce640 1 --2- 172.21.15.73:0/2396819997 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f052006c8f0 0x7f052006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:25.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.189+0000 7f05414ce640 1 --2- 172.21.15.73:0/2396819997 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f052006c8f0 0x7f052006edb0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0538009540 tx=0x7f053800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:25.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.193+0000 7f0531ffb640 1 -- 172.21.15.73:0/2396819997 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f052c04b070 con 0x7f053c0b0380 2024-04-26T20:52:25.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.437+0000 7f0543732640 1 -- 172.21.15.73:0/2396819997 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f053c0702c0 con 0x7f053c0b0380 2024-04-26T20:52:25.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.438+0000 7f0531ffb640 1 -- 172.21.15.73:0/2396819997 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1270 (secure 0 0 0) 0x7f052c07a4d0 con 0x7f053c0b0380 2024-04-26T20:52:25.444 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:52:25.445 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"fbd3f548-b547-4713-856b-09f98f5d1764","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":137,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":52,"num_osds":8,"num_up_osds":8,"osd_up_since":1714164619,"num_in_osds":8,"osd_in_since":1714164610,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":227184640,"bytes_avail":772866928640,"bytes_total":773094113280,"unknown_pgs_ratio":0.65979379415512085,"inactive_pgs_ratio":0.32989689707756042},"fsmap":{"epoch":26,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"c","status":"up:active","gid":5856}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-04-26T20:51:27.993985+0000","services":{"mds":{"daemons":{"summary":"","c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-26T20:52:25.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.440+0000 7f0543732640 1 -- 172.21.15.73:0/2396819997 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f052006c8f0 msgr2=0x7f052006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:25.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.440+0000 7f0543732640 1 --2- 172.21.15.73:0/2396819997 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f052006c8f0 0x7f052006edb0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0538009540 tx=0x7f053800b040 comp rx=0 tx=0).stop 2024-04-26T20:52:25.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.440+0000 7f0543732640 1 -- 172.21.15.73:0/2396819997 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f053c0b0380 msgr2=0x7f053c150570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:25.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.440+0000 7f0543732640 1 --2- 172.21.15.73:0/2396819997 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f053c0b0380 0x7f053c150570 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f052c02a220 tx=0x7f052c0046c0 comp rx=0 tx=0).stop 2024-04-26T20:52:25.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.440+0000 7f0543732640 1 -- 172.21.15.73:0/2396819997 shutdown_connections 2024-04-26T20:52:25.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.440+0000 7f0543732640 1 --2- 172.21.15.73:0/2396819997 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f053c0b0380 0x7f053c150570 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:25.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.440+0000 7f0543732640 1 --2- 172.21.15.73:0/2396819997 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f052006c8f0 0x7f052006edb0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:25.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.440+0000 7f0543732640 1 --2- 172.21.15.73:0/2396819997 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f053c0b10b0 0x7f053c0cbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:25.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.440+0000 7f0543732640 1 --2- 172.21.15.73:0/2396819997 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f053c06c780 0x7f053c14fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:25.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.440+0000 7f0543732640 1 -- 172.21.15.73:0/2396819997 >> 172.21.15.73:0/2396819997 conn(0x7f053c0d1cf0 msgr2=0x7f053c0b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:25.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.440+0000 7f0543732640 1 -- 172.21.15.73:0/2396819997 shutdown_connections 2024-04-26T20:52:25.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.440+0000 7f0543732640 1 -- 172.21.15.73:0/2396819997 wait complete. 2024-04-26T20:52:25.453 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-26T20:52:25.453 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:52:25.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.548+0000 7f9998c9c640 1 Processor -- start 2024-04-26T20:52:25.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.548+0000 7f9998c9c640 1 -- start start 2024-04-26T20:52:25.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.549+0000 7f9998c9c640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f99940b10b0 0x7f99940ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:25.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.549+0000 7f9998c9c640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99940cfd80 con 0x7f99940b0380 2024-04-26T20:52:25.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.549+0000 7f9998c9c640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99940d0020 con 0x7f999406c950 2024-04-26T20:52:25.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.549+0000 7f9998c9c640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f99940af8e0 con 0x7f99940b10b0 2024-04-26T20:52:25.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.549+0000 7f999259c640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f99940b0380 0x7f999406c240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:36624/0 (socket says 172.21.15.73:36624) 2024-04-26T20:52:25.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.549+0000 7f999259c640 1 -- 172.21.15.73:0/1826915902 learned_addr learned my addr 172.21.15.73:0/1826915902 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:52:25.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.549+0000 7f9991d9b640 1 -- 172.21.15.73:0/1826915902 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4187905124 0 0) 0x7f99940cfd80 con 0x7f99940b0380 2024-04-26T20:52:25.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.549+0000 7f9991d9b640 1 -- 172.21.15.73:0/1826915902 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9980003520 con 0x7f99940b0380 2024-04-26T20:52:25.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.549+0000 7f998bfff640 1 --2- 172.21.15.73:0/1826915902 >> v2:172.21.15.120:3300/0 conn(0x7f99940b10b0 0x7f99940ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:25.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.549+0000 7f9991d9b640 1 -- 172.21.15.73:0/1826915902 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 583074402 0 0) 0x7f9980003520 con 0x7f99940b0380 2024-04-26T20:52:25.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.549+0000 7f9991d9b640 1 -- 172.21.15.73:0/1826915902 >> v2:172.21.15.120:3300/0 conn(0x7f99940b10b0 msgr2=0x7f99940ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:25.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.549+0000 7f9991d9b640 1 --2- 172.21.15.73:0/1826915902 >> v2:172.21.15.120:3300/0 conn(0x7f99940b10b0 0x7f99940ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:25.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.549+0000 7f9991d9b640 1 -- 172.21.15.73:0/1826915902 >> v1:172.21.15.120:6789/0 conn(0x7f999406c950 legacy=0x7f999415e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:25.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.549+0000 7f9991d9b640 1 -- 172.21.15.73:0/1826915902 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99940d0300 con 0x7f99940b0380 2024-04-26T20:52:25.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.549+0000 7f9991d9b640 1 -- 172.21.15.73:0/1826915902 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7f9984001690 con 0x7f99940b0380 2024-04-26T20:52:25.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.549+0000 7f9991d9b640 1 -- 172.21.15.73:0/1826915902 >> v1:172.21.15.73:6789/0 conn(0x7f99940b0380 legacy=0x7f999406c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:25.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.549+0000 7f9991d9b640 1 --2- 172.21.15.73:0/1826915902 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f99800039b0 0x7f9980003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:25.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.549+0000 7f9991d9b640 1 --2- 172.21.15.73:0/1826915902 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9980004800 0x7f9980004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:25.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.549+0000 7f9991d9b640 1 --2- 172.21.15.73:0/1826915902 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99800051c0 0x7f998000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:25.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.549+0000 7f9991d9b640 1 -- 172.21.15.73:0/1826915902 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f99940af8e0 con 0x7f9980004800 2024-04-26T20:52:25.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.549+0000 7f9991d9b640 1 -- 172.21.15.73:0/1826915902 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f998000c160 con 0x7f99800051c0 2024-04-26T20:52:25.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.549+0000 7f9991d9b640 1 -- 172.21.15.73:0/1826915902 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f998000c360 con 0x7f99800039b0 2024-04-26T20:52:25.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.550+0000 7f998bfff640 1 --2- 172.21.15.73:0/1826915902 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9980004800 0x7f9980004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:25.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.550+0000 7f999259c640 1 --2- 172.21.15.73:0/1826915902 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f99800039b0 0x7f9980003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:25.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.550+0000 7f9992d9d640 1 --2- 172.21.15.73:0/1826915902 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99800051c0 0x7f998000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:25.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.550+0000 7f998bfff640 1 -- 172.21.15.73:0/1826915902 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f99800039b0 msgr2=0x7f9980003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:25.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.550+0000 7f998bfff640 1 --2- 172.21.15.73:0/1826915902 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f99800039b0 0x7f9980003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:25.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.550+0000 7f998bfff640 1 -- 172.21.15.73:0/1826915902 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99800051c0 msgr2=0x7f998000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:25.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.550+0000 7f998bfff640 1 --2- 172.21.15.73:0/1826915902 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99800051c0 0x7f998000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:25.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.550+0000 7f9992d9d640 1 -- 172.21.15.73:0/1826915902 reap_dead start 2024-04-26T20:52:25.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.550+0000 7f998bfff640 1 -- 172.21.15.73:0/1826915902 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9980003520 con 0x7f9980004800 2024-04-26T20:52:25.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.551+0000 7f998bfff640 1 --2- 172.21.15.73:0/1826915902 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9980004800 0x7f9980004c80 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f997c009ac0 tx=0x7f9980005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4de86018ee59e8bd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:25.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.551+0000 7f9991d9b640 1 -- 172.21.15.73:0/1826915902 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997c036070 con 0x7f9980004800 2024-04-26T20:52:25.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.551+0000 7f9991d9b640 1 -- 172.21.15.73:0/1826915902 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f997c031400 con 0x7f9980004800 2024-04-26T20:52:25.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.551+0000 7f9991d9b640 1 -- 172.21.15.73:0/1826915902 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997c009690 con 0x7f9980004800 2024-04-26T20:52:25.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.551+0000 7f9998c9c640 1 -- 172.21.15.73:0/1826915902 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9980004800 msgr2=0x7f9980004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:25.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.551+0000 7f9998c9c640 1 --2- 172.21.15.73:0/1826915902 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9980004800 0x7f9980004c80 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f997c009ac0 tx=0x7f9980005830 comp rx=0 tx=0).stop 2024-04-26T20:52:25.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.551+0000 7f9998c9c640 1 -- 172.21.15.73:0/1826915902 shutdown_connections 2024-04-26T20:52:25.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.551+0000 7f9998c9c640 1 --2- 172.21.15.73:0/1826915902 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9980004800 0x7f9980004c80 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:25.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.551+0000 7f9998c9c640 1 -- 172.21.15.73:0/1826915902 >> 172.21.15.73:0/1826915902 conn(0x7f99940d1cf0 msgr2=0x7f99940d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:25.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.551+0000 7f9998c9c640 1 -- 172.21.15.73:0/1826915902 shutdown_connections 2024-04-26T20:52:25.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.551+0000 7f9998c9c640 1 -- 172.21.15.73:0/1826915902 wait complete. 2024-04-26T20:52:25.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.553+0000 7f9998c9c640 1 Processor -- start 2024-04-26T20:52:25.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.553+0000 7f9998c9c640 1 -- start start 2024-04-26T20:52:25.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.553+0000 7f9998c9c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9980004800 0x7f9994157150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:25.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.553+0000 7f9998c9c640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f99940b4990 0x7f99940cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:25.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.553+0000 7f9998c9c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99940cc330 0x7f99940b60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:25.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.553+0000 7f9998c9c640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f99940b0760 con 0x7f9980004800 2024-04-26T20:52:25.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.553+0000 7f9998c9c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f99940d1980 con 0x7f99940cc330 2024-04-26T20:52:25.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.553+0000 7f9998c9c640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f99940d15c0 con 0x7f99940b4990 2024-04-26T20:52:25.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.553+0000 7f999259c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9980004800 0x7f9994157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:25.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.553+0000 7f999259c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9980004800 0x7f9994157150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38260/0 (socket says 172.21.15.73:38260) 2024-04-26T20:52:25.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.553+0000 7f999259c640 1 -- 172.21.15.73:0/3097275315 learned_addr learned my addr 172.21.15.73:0/3097275315 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:25.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.554+0000 7f9992d9d640 1 --2- 172.21.15.73:0/3097275315 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99940cc330 0x7f99940b60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:25.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.554+0000 7f998bfff640 1 --2- 172.21.15.73:0/3097275315 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f99940b4990 0x7f99940cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:25.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.554+0000 7f999259c640 1 -- 172.21.15.73:0/3097275315 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f99940b4990 msgr2=0x7f99940cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:25.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.554+0000 7f999259c640 1 --2- 172.21.15.73:0/3097275315 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f99940b4990 0x7f99940cbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:25.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.554+0000 7f999259c640 1 -- 172.21.15.73:0/3097275315 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99940cc330 msgr2=0x7f99940b60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:25.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.554+0000 7f999259c640 1 --2- 172.21.15.73:0/3097275315 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99940cc330 0x7f99940b60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:25.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.554+0000 7f999259c640 1 -- 172.21.15.73:0/3097275315 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99940b4e90 con 0x7f9980004800 2024-04-26T20:52:25.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.554+0000 7f999259c640 1 --2- 172.21.15.73:0/3097275315 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9980004800 0x7f9994157150 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f99840072c0 tx=0x7f9984007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:25.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.554+0000 7f998affd640 1 -- 172.21.15.73:0/3097275315 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9984001450 con 0x7f9980004800 2024-04-26T20:52:25.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.554+0000 7f998affd640 1 -- 172.21.15.73:0/3097275315 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9984018070 con 0x7f9980004800 2024-04-26T20:52:25.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.554+0000 7f998affd640 1 -- 172.21.15.73:0/3097275315 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998400fba0 con 0x7f9980004800 2024-04-26T20:52:25.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.554+0000 7f9998c9c640 1 -- 172.21.15.73:0/3097275315 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99940b5170 con 0x7f9980004800 2024-04-26T20:52:25.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.554+0000 7f9998c9c640 1 -- 172.21.15.73:0/3097275315 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99940b5730 con 0x7f9980004800 2024-04-26T20:52:25.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.555+0000 7f9998c9c640 1 -- 172.21.15.73:0/3097275315 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99940694f0 con 0x7f9980004800 2024-04-26T20:52:25.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.556+0000 7f998affd640 1 -- 172.21.15.73:0/3097275315 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f998400fd40 con 0x7f9980004800 2024-04-26T20:52:25.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.556+0000 7f998affd640 1 --2- 172.21.15.73:0/3097275315 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f997406c8f0 0x7f997406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:25.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.556+0000 7f998affd640 1 -- 172.21.15.73:0/3097275315 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f998408bda0 con 0x7f9980004800 2024-04-26T20:52:25.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.556+0000 7f998bfff640 1 --2- 172.21.15.73:0/3097275315 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f997406c8f0 0x7f997406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:25.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.556+0000 7f998bfff640 1 --2- 172.21.15.73:0/3097275315 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f997406c8f0 0x7f997406edb0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f997c033000 tx=0x7f997c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:25.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.560+0000 7f998affd640 1 -- 172.21.15.73:0/3097275315 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f998405a0d0 con 0x7f9980004800 2024-04-26T20:52:25.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.730+0000 7f9998c9c640 1 -- 172.21.15.73:0/3097275315 --> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f99940afba0 con 0x7f997406c8f0 2024-04-26T20:52:25.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.735+0000 7f998affd640 1 -- 172.21.15.73:0/3097275315 <== mgr.4107 v2:172.21.15.73:6832/666905549 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240062 (secure 0 0 0) 0x7f99940afba0 con 0x7f997406c8f0 2024-04-26T20:52:25.739 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:52:25.740 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-26T20:52:25.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.738+0000 7f9998c9c640 1 -- 172.21.15.73:0/3097275315 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f997406c8f0 msgr2=0x7f997406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:25.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.738+0000 7f9998c9c640 1 --2- 172.21.15.73:0/3097275315 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f997406c8f0 0x7f997406edb0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f997c033000 tx=0x7f997c008030 comp rx=0 tx=0).stop 2024-04-26T20:52:25.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.738+0000 7f9998c9c640 1 -- 172.21.15.73:0/3097275315 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9980004800 msgr2=0x7f9994157150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:25.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.738+0000 7f9998c9c640 1 --2- 172.21.15.73:0/3097275315 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9980004800 0x7f9994157150 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f99840072c0 tx=0x7f9984007780 comp rx=0 tx=0).stop 2024-04-26T20:52:25.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.738+0000 7f9998c9c640 1 -- 172.21.15.73:0/3097275315 shutdown_connections 2024-04-26T20:52:25.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.738+0000 7f9998c9c640 1 --2- 172.21.15.73:0/3097275315 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f997406c8f0 0x7f997406edb0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:25.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.738+0000 7f9998c9c640 1 --2- 172.21.15.73:0/3097275315 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f99940cc330 0x7f99940b60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:25.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.738+0000 7f9998c9c640 1 --2- 172.21.15.73:0/3097275315 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f99940b4990 0x7f99940cbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:25.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.738+0000 7f9998c9c640 1 --2- 172.21.15.73:0/3097275315 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9980004800 0x7f9994157150 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:25.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.738+0000 7f9998c9c640 1 -- 172.21.15.73:0/3097275315 >> 172.21.15.73:0/3097275315 conn(0x7f99940d1cf0 msgr2=0x7f99940b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:25.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.738+0000 7f9998c9c640 1 -- 172.21.15.73:0/3097275315 shutdown_connections 2024-04-26T20:52:25.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:25.738+0000 7f9998c9c640 1 -- 172.21.15.73:0/3097275315 wait complete. 2024-04-26T20:52:25.751 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":107,"stamp":"2024-04-26T20:52:24.003511+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221860,"kb_used_data":5644,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752860,"statfs":{"total":773094113280,"available":772866928640,"internally_reserved":0,"allocated":5779456,"data_stored":3205651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.609830"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.395952+0000","last_change":"2024-04-26T20:52:19.395952+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.395952+0000","last_undegraded":"2024-04-26T20:52:19.395952+0000","last_fullsized":"2024-04-26T20:52:19.395952+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.397803+0000","last_change":"2024-04-26T20:52:19.397803+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.397803+0000","last_undegraded":"2024-04-26T20:52:19.397803+0000","last_fullsized":"2024-04-26T20:52:19.397803+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.397585+0000","last_change":"2024-04-26T20:52:19.397585+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.397585+0000","last_undegraded":"2024-04-26T20:52:19.397585+0000","last_fullsized":"2024-04-26T20:52:19.397585+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.396894+0000","last_change":"2024-04-26T20:52:19.396894+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.396894+0000","last_undegraded":"2024-04-26T20:52:19.396894+0000","last_fullsized":"2024-04-26T20:52:19.396894+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.397540+0000","last_change":"2024-04-26T20:52:19.397540+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.397540+0000","last_undegraded":"2024-04-26T20:52:19.397540+0000","last_fullsized":"2024-04-26T20:52:19.397540+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.393560+0000","last_change":"2024-04-26T20:52:19.393560+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.393560+0000","last_undegraded":"2024-04-26T20:52:19.393560+0000","last_fullsized":"2024-04-26T20:52:19.393560+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.395521+0000","last_change":"2024-04-26T20:52:19.395521+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.395521+0000","last_undegraded":"2024-04-26T20:52:19.395521+0000","last_fullsized":"2024-04-26T20:52:19.395521+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.397229+0000","last_change":"2024-04-26T20:52:19.397229+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.397229+0000","last_undegraded":"2024-04-26T20:52:19.397229+0000","last_fullsized":"2024-04-26T20:52:19.397229+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.397756+0000","last_change":"2024-04-26T20:52:19.397756+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.397756+0000","last_undegraded":"2024-04-26T20:52:19.397756+0000","last_fullsized":"2024-04-26T20:52:19.397756+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.393499+0000","last_change":"2024-04-26T20:52:19.393499+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.393499+0000","last_undegraded":"2024-04-26T20:52:19.393499+0000","last_fullsized":"2024-04-26T20:52:19.393499+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.398384+0000","last_change":"2024-04-26T20:52:19.398384+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.398384+0000","last_undegraded":"2024-04-26T20:52:19.398384+0000","last_fullsized":"2024-04-26T20:52:19.398384+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.398218+0000","last_change":"2024-04-26T20:52:19.398218+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.398218+0000","last_undegraded":"2024-04-26T20:52:19.398218+0000","last_fullsized":"2024-04-26T20:52:19.398218+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.396945+0000","last_change":"2024-04-26T20:52:19.396945+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.396945+0000","last_undegraded":"2024-04-26T20:52:19.396945+0000","last_fullsized":"2024-04-26T20:52:19.396945+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.395349+0000","last_change":"2024-04-26T20:52:19.395349+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.395349+0000","last_undegraded":"2024-04-26T20:52:19.395349+0000","last_fullsized":"2024-04-26T20:52:19.395349+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.397694+0000","last_change":"2024-04-26T20:52:19.397694+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.397694+0000","last_undegraded":"2024-04-26T20:52:19.397694+0000","last_fullsized":"2024-04-26T20:52:19.397694+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.397561+0000","last_change":"2024-04-26T20:52:19.397561+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.397561+0000","last_undegraded":"2024-04-26T20:52:19.397561+0000","last_fullsized":"2024-04-26T20:52:19.397561+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.396989+0000","last_change":"2024-04-26T20:52:19.396989+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.396989+0000","last_undegraded":"2024-04-26T20:52:19.396989+0000","last_fullsized":"2024-04-26T20:52:19.396989+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.400177+0000","last_change":"2024-04-26T20:52:19.400177+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.400177+0000","last_undegraded":"2024-04-26T20:52:19.400177+0000","last_fullsized":"2024-04-26T20:52:19.400177+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.395403+0000","last_change":"2024-04-26T20:52:19.395403+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.395403+0000","last_undegraded":"2024-04-26T20:52:19.395403+0000","last_fullsized":"2024-04-26T20:52:19.395403+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.395340+0000","last_change":"2024-04-26T20:52:19.395340+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.395340+0000","last_undegraded":"2024-04-26T20:52:19.395340+0000","last_fullsized":"2024-04-26T20:52:19.395340+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.397535+0000","last_change":"2024-04-26T20:52:19.397535+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.397535+0000","last_undegraded":"2024-04-26T20:52:19.397535+0000","last_fullsized":"2024-04-26T20:52:19.397535+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.397145+0000","last_change":"2024-04-26T20:52:19.397145+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.397145+0000","last_undegraded":"2024-04-26T20:52:19.397145+0000","last_fullsized":"2024-04-26T20:52:19.397145+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.396532+0000","last_change":"2024-04-26T20:52:19.396532+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.396532+0000","last_undegraded":"2024-04-26T20:52:19.396532+0000","last_fullsized":"2024-04-26T20:52:19.396532+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.400129+0000","last_change":"2024-04-26T20:52:19.400129+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.400129+0000","last_undegraded":"2024-04-26T20:52:19.400129+0000","last_fullsized":"2024-04-26T20:52:19.400129+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.395398+0000","last_change":"2024-04-26T20:52:19.395398+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.395398+0000","last_undegraded":"2024-04-26T20:52:19.395398+0000","last_fullsized":"2024-04-26T20:52:19.395398+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.397644+0000","last_change":"2024-04-26T20:52:19.397644+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.397644+0000","last_undegraded":"2024-04-26T20:52:19.397644+0000","last_fullsized":"2024-04-26T20:52:19.397644+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.396492+0000","last_change":"2024-04-26T20:52:19.396492+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.396492+0000","last_undegraded":"2024-04-26T20:52:19.396492+0000","last_fullsized":"2024-04-26T20:52:19.396492+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.397080+0000","last_change":"2024-04-26T20:52:19.397080+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.397080+0000","last_undegraded":"2024-04-26T20:52:19.397080+0000","last_fullsized":"2024-04-26T20:52:19.397080+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.396492+0000","last_change":"2024-04-26T20:52:19.396492+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.396492+0000","last_undegraded":"2024-04-26T20:52:19.396492+0000","last_fullsized":"2024-04-26T20:52:19.396492+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.393556+0000","last_change":"2024-04-26T20:52:19.393556+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.393556+0000","last_undegraded":"2024-04-26T20:52:19.393556+0000","last_fullsized":"2024-04-26T20:52:19.393556+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.396627+0000","last_change":"2024-04-26T20:52:19.396627+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.396627+0000","last_undegraded":"2024-04-26T20:52:19.396627+0000","last_fullsized":"2024-04-26T20:52:19.396627+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T20:52:20.391318+0000","last_change":"2024-04-26T20:52:20.391318+0000","last_active":"2024-04-26T20:52:20.391318+0000","last_peered":"2024-04-26T20:52:20.391318+0000","last_clean":"2024-04-26T20:52:20.391318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:20.391318+0000","last_undegraded":"2024-04-26T20:52:20.391318+0000","last_fullsized":"2024-04-26T20:52:20.391318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-04-26T20:52:19.396494+0000","last_change":"2024-04-26T20:52:19.396494+0000","last_active":"2024-04-26T20:52:19.382176+0000","last_peered":"2024-04-26T20:52:19.382176+0000","last_clean":"2024-04-26T20:52:19.382176+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:52:19.396494+0000","last_undegraded":"2024-04-26T20:52:19.396494+0000","last_fullsized":"2024-04-26T20:52:19.396494+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":221,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-04-26T20:52:19.397911+0000","last_change":"2024-04-26T20:50:21.150132+0000","last_active":"2024-04-26T20:52:19.397911+0000","last_peered":"2024-04-26T20:52:19.397911+0000","last_clean":"2024-04-26T20:52:19.397911+0000","last_became_active":"2024-04-26T20:50:21.147439+0000","last_became_peered":"2024-04-26T20:50:21.147439+0000","last_unstale":"2024-04-26T20:52:19.397911+0000","last_undegraded":"2024-04-26T20:52:19.397911+0000","last_fullsized":"2024-04-26T20:52:19.397911+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:20.084499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:20.084499+0000","last_clean_scrub_stamp":"2024-04-26T20:50:20.084499+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:19:29.395477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28160,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343680,"statfs":{"total":96636764160,"available":96607928320,"internally_reserved":0,"allocated":1163264,"data_stored":841154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27600,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344240,"statfs":{"total":96636764160,"available":96608501760,"internally_reserved":0,"allocated":581632,"data_stored":256995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":569344,"data_stored":250786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27584,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":569344,"data_stored":250786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27592,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":581632,"data_stored":256995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27588,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344252,"statfs":{"total":96636764160,"available":96608514048,"internally_reserved":0,"allocated":569344,"data_stored":250786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27584,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":569344,"data_stored":250786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28176,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343664,"statfs":{"total":96636764160,"available":96607911936,"internally_reserved":0,"allocated":1175552,"data_stored":847363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:52:26.662 INFO:tasks.ceph.mds.c.smithi073.stderr:2024-04-26T20:52:26.660+0000 7fea69537640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T20:52:28.755 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:52:28.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.860+0000 7f164c104640 1 Processor -- start 2024-04-26T20:52:28.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.861+0000 7f164c104640 1 -- start start 2024-04-26T20:52:28.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.861+0000 7f164c104640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1644160a10 0x7f16440cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:28.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.861+0000 7f164c104640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f16440cf930 0x7f16440c6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:28.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.861+0000 7f164c104640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16440b41d0 con 0x7f1644160630 2024-04-26T20:52:28.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.861+0000 7f164c104640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f16440d0570 con 0x7f16440cf930 2024-04-26T20:52:28.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.861+0000 7f164c104640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f16440b44b0 con 0x7f1644160a10 2024-04-26T20:52:28.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.862+0000 7f164a6a1640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f16440cf930 0x7f16440c6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:28.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.862+0000 7f164a6a1640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f16440cf930 0x7f16440c6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38268/0 (socket says 172.21.15.73:38268) 2024-04-26T20:52:28.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.862+0000 7f164a6a1640 1 -- 172.21.15.73:0/4190487134 learned_addr learned my addr 172.21.15.73:0/4190487134 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:28.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.862+0000 7f164969f640 1 --2- 172.21.15.73:0/4190487134 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1644160a10 0x7f16440cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:28.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.862+0000 7f164a6a1640 1 -- 172.21.15.73:0/4190487134 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1644160a10 msgr2=0x7f16440cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:28.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.862+0000 7f164a6a1640 1 --2- 172.21.15.73:0/4190487134 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1644160a10 0x7f16440cf3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:28.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.862+0000 7f164a6a1640 1 -- 172.21.15.73:0/4190487134 >> v1:172.21.15.120:6789/0 conn(0x7f1644160630 legacy=0x7f16440d0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:28.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.862+0000 7f164a6a1640 1 -- 172.21.15.73:0/4190487134 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16440d0200 con 0x7f16440cf930 2024-04-26T20:52:28.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.863+0000 7f164a6a1640 1 --2- 172.21.15.73:0/4190487134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16440cf930 0x7f16440c6a80 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f163c009870 tx=0x7f163c02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36cea48d95f0b4eb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:28.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.863+0000 7f1648e9e640 1 -- 172.21.15.73:0/4190487134 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163c037070 con 0x7f16440cf930 2024-04-26T20:52:28.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.863+0000 7f1648e9e640 1 -- 172.21.15.73:0/4190487134 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f163c032440 con 0x7f16440cf930 2024-04-26T20:52:28.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.863+0000 7f1648e9e640 1 -- 172.21.15.73:0/4190487134 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163c032810 con 0x7f16440cf930 2024-04-26T20:52:28.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.863+0000 7f164c104640 1 -- 172.21.15.73:0/4190487134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16440cf930 msgr2=0x7f16440c6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:28.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.863+0000 7f164c104640 1 --2- 172.21.15.73:0/4190487134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16440cf930 0x7f16440c6a80 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f163c009870 tx=0x7f163c02a2a0 comp rx=0 tx=0).stop 2024-04-26T20:52:28.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.863+0000 7f164c104640 1 -- 172.21.15.73:0/4190487134 shutdown_connections 2024-04-26T20:52:28.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.863+0000 7f164c104640 1 --2- 172.21.15.73:0/4190487134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16440cf930 0x7f16440c6a80 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:28.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.863+0000 7f164c104640 1 --2- 172.21.15.73:0/4190487134 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1644160a10 0x7f16440cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:28.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.863+0000 7f164c104640 1 -- 172.21.15.73:0/4190487134 >> 172.21.15.73:0/4190487134 conn(0x7f16440d2dc0 msgr2=0x7f16440d31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:28.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.863+0000 7f164c104640 1 -- 172.21.15.73:0/4190487134 shutdown_connections 2024-04-26T20:52:28.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.863+0000 7f164c104640 1 -- 172.21.15.73:0/4190487134 wait complete. 2024-04-26T20:52:28.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.865+0000 7f164c104640 1 Processor -- start 2024-04-26T20:52:28.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.865+0000 7f164c104640 1 -- start start 2024-04-26T20:52:28.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.865+0000 7f164c104640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f16440cf930 0x7f16440cc410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:28.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.865+0000 7f164c104640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1644160630 0x7f16440b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:28.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.865+0000 7f164c104640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1644160a10 0x7f16440b1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:28.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.865+0000 7f164c104640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f16440cfdc0 con 0x7f1644160630 2024-04-26T20:52:28.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.865+0000 7f164c104640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f16440d3b10 con 0x7f1644160a10 2024-04-26T20:52:28.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.865+0000 7f164c104640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f16440cff40 con 0x7f16440cf930 2024-04-26T20:52:28.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.865+0000 7f164969f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1644160630 0x7f16440b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:28.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.865+0000 7f164969f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1644160630 0x7f16440b0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38272/0 (socket says 172.21.15.73:38272) 2024-04-26T20:52:28.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.865+0000 7f164969f640 1 -- 172.21.15.73:0/3712038946 learned_addr learned my addr 172.21.15.73:0/3712038946 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:28.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.865+0000 7f164a6a1640 1 --2- 172.21.15.73:0/3712038946 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1644160a10 0x7f16440b1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:28.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.866+0000 7f1649ea0640 1 --2- 172.21.15.73:0/3712038946 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f16440cf930 0x7f16440cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:28.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.866+0000 7f164969f640 1 -- 172.21.15.73:0/3712038946 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f16440cf930 msgr2=0x7f16440cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:28.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.866+0000 7f164969f640 1 --2- 172.21.15.73:0/3712038946 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f16440cf930 0x7f16440cc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:28.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.866+0000 7f164969f640 1 -- 172.21.15.73:0/3712038946 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1644160a10 msgr2=0x7f16440b1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:28.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.866+0000 7f164969f640 1 --2- 172.21.15.73:0/3712038946 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1644160a10 0x7f16440b1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:28.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.866+0000 7f164969f640 1 -- 172.21.15.73:0/3712038946 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16440cb370 con 0x7f1644160630 2024-04-26T20:52:28.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.866+0000 7f164969f640 1 --2- 172.21.15.73:0/3712038946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1644160630 0x7f16440b0ab0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f163400c820 tx=0x7f163400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:28.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.866+0000 7f163a7fc640 1 -- 172.21.15.73:0/3712038946 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1634015070 con 0x7f1644160630 2024-04-26T20:52:28.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.866+0000 7f163a7fc640 1 -- 172.21.15.73:0/3712038946 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1634004240 con 0x7f1644160630 2024-04-26T20:52:28.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.866+0000 7f164c104640 1 -- 172.21.15.73:0/3712038946 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16440cb650 con 0x7f1644160630 2024-04-26T20:52:28.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.866+0000 7f163a7fc640 1 -- 172.21.15.73:0/3712038946 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163400ce10 con 0x7f1644160630 2024-04-26T20:52:28.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.866+0000 7f164c104640 1 -- 172.21.15.73:0/3712038946 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16440cbc10 con 0x7f1644160630 2024-04-26T20:52:28.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.867+0000 7f164c104640 1 -- 172.21.15.73:0/3712038946 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16440694f0 con 0x7f1644160630 2024-04-26T20:52:28.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.868+0000 7f163a7fc640 1 -- 172.21.15.73:0/3712038946 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f16340077d0 con 0x7f1644160630 2024-04-26T20:52:28.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.868+0000 7f163a7fc640 1 --2- 172.21.15.73:0/3712038946 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f162806c920 0x7f162806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:28.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.868+0000 7f163a7fc640 1 -- 172.21.15.73:0/3712038946 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f163408ba40 con 0x7f1644160630 2024-04-26T20:52:28.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.868+0000 7f1649ea0640 1 --2- 172.21.15.73:0/3712038946 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f162806c920 0x7f162806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:28.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.869+0000 7f1649ea0640 1 --2- 172.21.15.73:0/3712038946 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f162806c920 0x7f162806ede0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1640005660 tx=0x7f1640003d90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:28.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:28.873+0000 7f163a7fc640 1 -- 172.21.15.73:0/3712038946 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1634059720 con 0x7f1644160630 2024-04-26T20:52:29.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.046+0000 7f164c104640 1 -- 172.21.15.73:0/3712038946 --> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f16440afd60 con 0x7f162806c920 2024-04-26T20:52:29.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.052+0000 7f163a7fc640 1 -- 172.21.15.73:0/3712038946 <== mgr.4107 v2:172.21.15.73:6832/666905549 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251957 (secure 0 0 0) 0x7f16440afd60 con 0x7f162806c920 2024-04-26T20:52:29.057 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:52:29.058 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-26T20:52:29.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.055+0000 7f164c104640 1 -- 172.21.15.73:0/3712038946 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f162806c920 msgr2=0x7f162806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:29.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.055+0000 7f164c104640 1 --2- 172.21.15.73:0/3712038946 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f162806c920 0x7f162806ede0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1640005660 tx=0x7f1640003d90 comp rx=0 tx=0).stop 2024-04-26T20:52:29.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.055+0000 7f164c104640 1 -- 172.21.15.73:0/3712038946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1644160630 msgr2=0x7f16440b0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:29.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.055+0000 7f164c104640 1 --2- 172.21.15.73:0/3712038946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1644160630 0x7f16440b0ab0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f163400c820 tx=0x7f163400cce0 comp rx=0 tx=0).stop 2024-04-26T20:52:29.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.055+0000 7f164c104640 1 -- 172.21.15.73:0/3712038946 shutdown_connections 2024-04-26T20:52:29.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.055+0000 7f164c104640 1 --2- 172.21.15.73:0/3712038946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1644160630 0x7f16440b0ab0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:29.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.055+0000 7f164c104640 1 --2- 172.21.15.73:0/3712038946 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f162806c920 0x7f162806ede0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:29.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.055+0000 7f164c104640 1 --2- 172.21.15.73:0/3712038946 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1644160a10 0x7f16440b1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:29.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.055+0000 7f164c104640 1 --2- 172.21.15.73:0/3712038946 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f16440cf930 0x7f16440cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:29.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.055+0000 7f164c104640 1 -- 172.21.15.73:0/3712038946 >> 172.21.15.73:0/3712038946 conn(0x7f16440d2dc0 msgr2=0x7f16440c8e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:29.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.055+0000 7f164c104640 1 -- 172.21.15.73:0/3712038946 shutdown_connections 2024-04-26T20:52:29.061 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.055+0000 7f164c104640 1 -- 172.21.15.73:0/3712038946 wait complete. 2024-04-26T20:52:29.068 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":109,"stamp":"2024-04-26T20:52:28.004472+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":173,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222240,"kb_used_data":6056,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752480,"statfs":{"total":773094113280,"available":772866539520,"internally_reserved":0,"allocated":6201344,"data_stored":3405724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.605960"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.631945+0000","last_change":"2024-04-26T20:52:21.426712+0000","last_active":"2024-04-26T20:52:22.631945+0000","last_peered":"2024-04-26T20:52:22.631945+0000","last_clean":"2024-04-26T20:52:22.631945+0000","last_became_active":"2024-04-26T20:52:21.426466+0000","last_became_peered":"2024-04-26T20:52:21.426466+0000","last_unstale":"2024-04-26T20:52:22.631945+0000","last_undegraded":"2024-04-26T20:52:22.631945+0000","last_fullsized":"2024-04-26T20:52:22.631945+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:30:02.928504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.632496+0000","last_change":"2024-04-26T20:52:21.417869+0000","last_active":"2024-04-26T20:52:22.632496+0000","last_peered":"2024-04-26T20:52:22.632496+0000","last_clean":"2024-04-26T20:52:22.632496+0000","last_became_active":"2024-04-26T20:52:21.417601+0000","last_became_peered":"2024-04-26T20:52:21.417601+0000","last_unstale":"2024-04-26T20:52:22.632496+0000","last_undegraded":"2024-04-26T20:52:22.632496+0000","last_fullsized":"2024-04-26T20:52:22.632496+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:18:19.419173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.633032+0000","last_change":"2024-04-26T20:52:21.416477+0000","last_active":"2024-04-26T20:52:22.633032+0000","last_peered":"2024-04-26T20:52:22.633032+0000","last_clean":"2024-04-26T20:52:22.633032+0000","last_became_active":"2024-04-26T20:52:21.416209+0000","last_became_peered":"2024-04-26T20:52:21.416209+0000","last_unstale":"2024-04-26T20:52:22.633032+0000","last_undegraded":"2024-04-26T20:52:22.633032+0000","last_fullsized":"2024-04-26T20:52:22.633032+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:05:49.897608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.632388+0000","last_change":"2024-04-26T20:52:21.417258+0000","last_active":"2024-04-26T20:52:22.632388+0000","last_peered":"2024-04-26T20:52:22.632388+0000","last_clean":"2024-04-26T20:52:22.632388+0000","last_became_active":"2024-04-26T20:52:21.416988+0000","last_became_peered":"2024-04-26T20:52:21.416988+0000","last_unstale":"2024-04-26T20:52:22.632388+0000","last_undegraded":"2024-04-26T20:52:22.632388+0000","last_fullsized":"2024-04-26T20:52:22.632388+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:55:48.694860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.632343+0000","last_change":"2024-04-26T20:52:21.415910+0000","last_active":"2024-04-26T20:52:22.632343+0000","last_peered":"2024-04-26T20:52:22.632343+0000","last_clean":"2024-04-26T20:52:22.632343+0000","last_became_active":"2024-04-26T20:52:21.415636+0000","last_became_peered":"2024-04-26T20:52:21.415636+0000","last_unstale":"2024-04-26T20:52:22.632343+0000","last_undegraded":"2024-04-26T20:52:22.632343+0000","last_fullsized":"2024-04-26T20:52:22.632343+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:30:02.928504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.632002+0000","last_change":"2024-04-26T20:52:21.426079+0000","last_active":"2024-04-26T20:52:22.632002+0000","last_peered":"2024-04-26T20:52:22.632002+0000","last_clean":"2024-04-26T20:52:22.632002+0000","last_became_active":"2024-04-26T20:52:21.425774+0000","last_became_peered":"2024-04-26T20:52:21.425774+0000","last_unstale":"2024-04-26T20:52:22.632002+0000","last_undegraded":"2024-04-26T20:52:22.632002+0000","last_fullsized":"2024-04-26T20:52:22.632002+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:02:40.232275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.633371+0000","last_change":"2024-04-26T20:52:21.425303+0000","last_active":"2024-04-26T20:52:22.633371+0000","last_peered":"2024-04-26T20:52:22.633371+0000","last_clean":"2024-04-26T20:52:22.633371+0000","last_became_active":"2024-04-26T20:52:21.424818+0000","last_became_peered":"2024-04-26T20:52:21.424818+0000","last_unstale":"2024-04-26T20:52:22.633371+0000","last_undegraded":"2024-04-26T20:52:22.633371+0000","last_fullsized":"2024-04-26T20:52:22.633371+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:34:47.196852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.414900+0000","last_change":"2024-04-26T20:52:21.435081+0000","last_active":"2024-04-26T20:52:22.414900+0000","last_peered":"2024-04-26T20:52:22.414900+0000","last_clean":"2024-04-26T20:52:22.414900+0000","last_became_active":"2024-04-26T20:52:21.434812+0000","last_became_peered":"2024-04-26T20:52:21.434812+0000","last_unstale":"2024-04-26T20:52:22.414900+0000","last_undegraded":"2024-04-26T20:52:22.414900+0000","last_fullsized":"2024-04-26T20:52:22.414900+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:17:14.587200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.415359+0000","last_change":"2024-04-26T20:52:21.432884+0000","last_active":"2024-04-26T20:52:22.415359+0000","last_peered":"2024-04-26T20:52:22.415359+0000","last_clean":"2024-04-26T20:52:22.415359+0000","last_became_active":"2024-04-26T20:52:21.432642+0000","last_became_peered":"2024-04-26T20:52:21.432642+0000","last_unstale":"2024-04-26T20:52:22.415359+0000","last_undegraded":"2024-04-26T20:52:22.415359+0000","last_fullsized":"2024-04-26T20:52:22.415359+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:50:29.798665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.632234+0000","last_change":"2024-04-26T20:52:21.423871+0000","last_active":"2024-04-26T20:52:22.632234+0000","last_peered":"2024-04-26T20:52:22.632234+0000","last_clean":"2024-04-26T20:52:22.632234+0000","last_became_active":"2024-04-26T20:52:21.423624+0000","last_became_peered":"2024-04-26T20:52:21.423624+0000","last_unstale":"2024-04-26T20:52:22.632234+0000","last_undegraded":"2024-04-26T20:52:22.632234+0000","last_fullsized":"2024-04-26T20:52:22.632234+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:55:48.694860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.632952+0000","last_change":"2024-04-26T20:52:21.420702+0000","last_active":"2024-04-26T20:52:22.632952+0000","last_peered":"2024-04-26T20:52:22.632952+0000","last_clean":"2024-04-26T20:52:22.632952+0000","last_became_active":"2024-04-26T20:52:21.420455+0000","last_became_peered":"2024-04-26T20:52:21.420455+0000","last_unstale":"2024-04-26T20:52:22.632952+0000","last_undegraded":"2024-04-26T20:52:22.632952+0000","last_fullsized":"2024-04-26T20:52:22.632952+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:26:49.176596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.415339+0000","last_change":"2024-04-26T20:52:21.422947+0000","last_active":"2024-04-26T20:52:22.415339+0000","last_peered":"2024-04-26T20:52:22.415339+0000","last_clean":"2024-04-26T20:52:22.415339+0000","last_became_active":"2024-04-26T20:52:21.422609+0000","last_became_peered":"2024-04-26T20:52:21.422609+0000","last_unstale":"2024-04-26T20:52:22.415339+0000","last_undegraded":"2024-04-26T20:52:22.415339+0000","last_fullsized":"2024-04-26T20:52:22.415339+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:48:10.639154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.632291+0000","last_change":"2024-04-26T20:52:21.420141+0000","last_active":"2024-04-26T20:52:22.632291+0000","last_peered":"2024-04-26T20:52:22.632291+0000","last_clean":"2024-04-26T20:52:22.632291+0000","last_became_active":"2024-04-26T20:52:21.419778+0000","last_became_peered":"2024-04-26T20:52:21.419778+0000","last_unstale":"2024-04-26T20:52:22.632291+0000","last_undegraded":"2024-04-26T20:52:22.632291+0000","last_fullsized":"2024-04-26T20:52:22.632291+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:18:19.419173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.633693+0000","last_change":"2024-04-26T20:52:21.415799+0000","last_active":"2024-04-26T20:52:22.633693+0000","last_peered":"2024-04-26T20:52:22.633693+0000","last_clean":"2024-04-26T20:52:22.633693+0000","last_became_active":"2024-04-26T20:52:21.415541+0000","last_became_peered":"2024-04-26T20:52:21.415541+0000","last_unstale":"2024-04-26T20:52:22.633693+0000","last_undegraded":"2024-04-26T20:52:22.633693+0000","last_fullsized":"2024-04-26T20:52:22.633693+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:11:03.316412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.632260+0000","last_change":"2024-04-26T20:52:21.421313+0000","last_active":"2024-04-26T20:52:22.632260+0000","last_peered":"2024-04-26T20:52:22.632260+0000","last_clean":"2024-04-26T20:52:22.632260+0000","last_became_active":"2024-04-26T20:52:21.420973+0000","last_became_peered":"2024-04-26T20:52:21.420973+0000","last_unstale":"2024-04-26T20:52:22.632260+0000","last_undegraded":"2024-04-26T20:52:22.632260+0000","last_fullsized":"2024-04-26T20:52:22.632260+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:57:36.490497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.631832+0000","last_change":"2024-04-26T20:52:21.419869+0000","last_active":"2024-04-26T20:52:22.631832+0000","last_peered":"2024-04-26T20:52:22.631832+0000","last_clean":"2024-04-26T20:52:22.631832+0000","last_became_active":"2024-04-26T20:52:21.419610+0000","last_became_peered":"2024-04-26T20:52:21.419610+0000","last_unstale":"2024-04-26T20:52:22.631832+0000","last_undegraded":"2024-04-26T20:52:22.631832+0000","last_fullsized":"2024-04-26T20:52:22.631832+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:30:50.792136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.632982+0000","last_change":"2024-04-26T20:52:21.428193+0000","last_active":"2024-04-26T20:52:22.632982+0000","last_peered":"2024-04-26T20:52:22.632982+0000","last_clean":"2024-04-26T20:52:22.632982+0000","last_became_active":"2024-04-26T20:52:21.427936+0000","last_became_peered":"2024-04-26T20:52:21.427936+0000","last_unstale":"2024-04-26T20:52:22.632982+0000","last_undegraded":"2024-04-26T20:52:22.632982+0000","last_fullsized":"2024-04-26T20:52:22.632982+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:10:31.357170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.632623+0000","last_change":"2024-04-26T20:52:21.421822+0000","last_active":"2024-04-26T20:52:22.632623+0000","last_peered":"2024-04-26T20:52:22.632623+0000","last_clean":"2024-04-26T20:52:22.632623+0000","last_became_active":"2024-04-26T20:52:21.421534+0000","last_became_peered":"2024-04-26T20:52:21.421534+0000","last_unstale":"2024-04-26T20:52:22.632623+0000","last_undegraded":"2024-04-26T20:52:22.632623+0000","last_fullsized":"2024-04-26T20:52:22.632623+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:04:51.913283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.633076+0000","last_change":"2024-04-26T20:52:21.432197+0000","last_active":"2024-04-26T20:52:22.633076+0000","last_peered":"2024-04-26T20:52:22.633076+0000","last_clean":"2024-04-26T20:52:22.633076+0000","last_became_active":"2024-04-26T20:52:21.431216+0000","last_became_peered":"2024-04-26T20:52:21.431216+0000","last_unstale":"2024-04-26T20:52:22.633076+0000","last_undegraded":"2024-04-26T20:52:22.633076+0000","last_fullsized":"2024-04-26T20:52:22.633076+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:57:13.271708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.415478+0000","last_change":"2024-04-26T20:52:21.414408+0000","last_active":"2024-04-26T20:52:22.415478+0000","last_peered":"2024-04-26T20:52:22.415478+0000","last_clean":"2024-04-26T20:52:22.415478+0000","last_became_active":"2024-04-26T20:52:21.413930+0000","last_became_peered":"2024-04-26T20:52:21.413930+0000","last_unstale":"2024-04-26T20:52:22.415478+0000","last_undegraded":"2024-04-26T20:52:22.415478+0000","last_fullsized":"2024-04-26T20:52:22.415478+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:48:18.863829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.415283+0000","last_change":"2024-04-26T20:52:21.419340+0000","last_active":"2024-04-26T20:52:22.415283+0000","last_peered":"2024-04-26T20:52:22.415283+0000","last_clean":"2024-04-26T20:52:22.415283+0000","last_became_active":"2024-04-26T20:52:21.419084+0000","last_became_peered":"2024-04-26T20:52:21.419084+0000","last_unstale":"2024-04-26T20:52:22.415283+0000","last_undegraded":"2024-04-26T20:52:22.415283+0000","last_fullsized":"2024-04-26T20:52:22.415283+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:43:25.478642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.414900+0000","last_change":"2024-04-26T20:52:21.429285+0000","last_active":"2024-04-26T20:52:22.414900+0000","last_peered":"2024-04-26T20:52:22.414900+0000","last_clean":"2024-04-26T20:52:22.414900+0000","last_became_active":"2024-04-26T20:52:21.429027+0000","last_became_peered":"2024-04-26T20:52:21.429027+0000","last_unstale":"2024-04-26T20:52:22.414900+0000","last_undegraded":"2024-04-26T20:52:22.414900+0000","last_fullsized":"2024-04-26T20:52:22.414900+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:56:54.984952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.632239+0000","last_change":"2024-04-26T20:52:21.420135+0000","last_active":"2024-04-26T20:52:22.632239+0000","last_peered":"2024-04-26T20:52:22.632239+0000","last_clean":"2024-04-26T20:52:22.632239+0000","last_became_active":"2024-04-26T20:52:21.419875+0000","last_became_peered":"2024-04-26T20:52:21.419875+0000","last_unstale":"2024-04-26T20:52:22.632239+0000","last_undegraded":"2024-04-26T20:52:22.632239+0000","last_fullsized":"2024-04-26T20:52:22.632239+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:07.515914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.632024+0000","last_change":"2024-04-26T20:52:21.427492+0000","last_active":"2024-04-26T20:52:22.632024+0000","last_peered":"2024-04-26T20:52:22.632024+0000","last_clean":"2024-04-26T20:52:22.632024+0000","last_became_active":"2024-04-26T20:52:21.425175+0000","last_became_peered":"2024-04-26T20:52:21.425175+0000","last_unstale":"2024-04-26T20:52:22.632024+0000","last_undegraded":"2024-04-26T20:52:22.632024+0000","last_fullsized":"2024-04-26T20:52:22.632024+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:43:10.360165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.633198+0000","last_change":"2024-04-26T20:52:21.427669+0000","last_active":"2024-04-26T20:52:22.633198+0000","last_peered":"2024-04-26T20:52:22.633198+0000","last_clean":"2024-04-26T20:52:22.633198+0000","last_became_active":"2024-04-26T20:52:21.426867+0000","last_became_peered":"2024-04-26T20:52:21.426867+0000","last_unstale":"2024-04-26T20:52:22.633198+0000","last_undegraded":"2024-04-26T20:52:22.633198+0000","last_fullsized":"2024-04-26T20:52:22.633198+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:13:13.059712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.415084+0000","last_change":"2024-04-26T20:52:21.425922+0000","last_active":"2024-04-26T20:52:22.415084+0000","last_peered":"2024-04-26T20:52:22.415084+0000","last_clean":"2024-04-26T20:52:22.415084+0000","last_became_active":"2024-04-26T20:52:21.425657+0000","last_became_peered":"2024-04-26T20:52:21.425657+0000","last_unstale":"2024-04-26T20:52:22.415084+0000","last_undegraded":"2024-04-26T20:52:22.415084+0000","last_fullsized":"2024-04-26T20:52:22.415084+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:09:40.238355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.632181+0000","last_change":"2024-04-26T20:52:21.432692+0000","last_active":"2024-04-26T20:52:22.632181+0000","last_peered":"2024-04-26T20:52:22.632181+0000","last_clean":"2024-04-26T20:52:22.632181+0000","last_became_active":"2024-04-26T20:52:21.432430+0000","last_became_peered":"2024-04-26T20:52:21.432430+0000","last_unstale":"2024-04-26T20:52:22.632181+0000","last_undegraded":"2024-04-26T20:52:22.632181+0000","last_fullsized":"2024-04-26T20:52:22.632181+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:11:07.874087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.415668+0000","last_change":"2024-04-26T20:52:21.415959+0000","last_active":"2024-04-26T20:52:22.415668+0000","last_peered":"2024-04-26T20:52:22.415668+0000","last_clean":"2024-04-26T20:52:22.415668+0000","last_became_active":"2024-04-26T20:52:21.415705+0000","last_became_peered":"2024-04-26T20:52:21.415705+0000","last_unstale":"2024-04-26T20:52:22.415668+0000","last_undegraded":"2024-04-26T20:52:22.415668+0000","last_fullsized":"2024-04-26T20:52:22.415668+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:15:48.267290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.632484+0000","last_change":"2024-04-26T20:52:21.428744+0000","last_active":"2024-04-26T20:52:22.632484+0000","last_peered":"2024-04-26T20:52:22.632484+0000","last_clean":"2024-04-26T20:52:22.632484+0000","last_became_active":"2024-04-26T20:52:21.428488+0000","last_became_peered":"2024-04-26T20:52:21.428488+0000","last_unstale":"2024-04-26T20:52:22.632484+0000","last_undegraded":"2024-04-26T20:52:22.632484+0000","last_fullsized":"2024-04-26T20:52:22.632484+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:22:50.853726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.632283+0000","last_change":"2024-04-26T20:52:21.418370+0000","last_active":"2024-04-26T20:52:22.632283+0000","last_peered":"2024-04-26T20:52:22.632283+0000","last_clean":"2024-04-26T20:52:22.632283+0000","last_became_active":"2024-04-26T20:52:21.418118+0000","last_became_peered":"2024-04-26T20:52:21.418118+0000","last_unstale":"2024-04-26T20:52:22.632283+0000","last_undegraded":"2024-04-26T20:52:22.632283+0000","last_fullsized":"2024-04-26T20:52:22.632283+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:11:07.874087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.633436+0000","last_change":"2024-04-26T20:52:21.419012+0000","last_active":"2024-04-26T20:52:22.633436+0000","last_peered":"2024-04-26T20:52:22.633436+0000","last_clean":"2024-04-26T20:52:22.633436+0000","last_became_active":"2024-04-26T20:52:21.418687+0000","last_became_peered":"2024-04-26T20:52:21.418687+0000","last_unstale":"2024-04-26T20:52:22.633436+0000","last_undegraded":"2024-04-26T20:52:22.633436+0000","last_fullsized":"2024-04-26T20:52:22.633436+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:09:40.238355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.632562+0000","last_change":"2024-04-26T20:52:21.415412+0000","last_active":"2024-04-26T20:52:22.632562+0000","last_peered":"2024-04-26T20:52:22.632562+0000","last_clean":"2024-04-26T20:52:22.632562+0000","last_became_active":"2024-04-26T20:52:21.415072+0000","last_became_peered":"2024-04-26T20:52:21.415072+0000","last_unstale":"2024-04-26T20:52:22.632562+0000","last_undegraded":"2024-04-26T20:52:22.632562+0000","last_fullsized":"2024-04-26T20:52:22.632562+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:02:07.235466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.632365+0000","last_change":"2024-04-26T20:52:21.417552+0000","last_active":"2024-04-26T20:52:22.632365+0000","last_peered":"2024-04-26T20:52:22.632365+0000","last_clean":"2024-04-26T20:52:22.632365+0000","last_became_active":"2024-04-26T20:52:21.417274+0000","last_became_peered":"2024-04-26T20:52:21.417274+0000","last_unstale":"2024-04-26T20:52:22.632365+0000","last_undegraded":"2024-04-26T20:52:22.632365+0000","last_fullsized":"2024-04-26T20:52:22.632365+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:31.714105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"51'4","reported_seq":18,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.415375+0000","last_change":"2024-04-26T20:52:20.418462+0000","last_active":"2024-04-26T20:52:22.415375+0000","last_peered":"2024-04-26T20:52:22.415375+0000","last_clean":"2024-04-26T20:52:22.415375+0000","last_became_active":"2024-04-26T20:52:20.418207+0000","last_became_peered":"2024-04-26T20:52:20.418207+0000","last_unstale":"2024-04-26T20:52:22.415375+0000","last_undegraded":"2024-04-26T20:52:22.415375+0000","last_fullsized":"2024-04-26T20:52:22.415375+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:34:54.441982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.632986+0000","last_change":"2024-04-26T20:52:21.416121+0000","last_active":"2024-04-26T20:52:22.632986+0000","last_peered":"2024-04-26T20:52:22.632986+0000","last_clean":"2024-04-26T20:52:22.632986+0000","last_became_active":"2024-04-26T20:52:21.415820+0000","last_became_peered":"2024-04-26T20:52:21.415820+0000","last_unstale":"2024-04-26T20:52:22.632986+0000","last_undegraded":"2024-04-26T20:52:22.632986+0000","last_fullsized":"2024-04-26T20:52:22.632986+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:16:53.389463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"51'1","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.632283+0000","last_change":"2024-04-26T20:52:20.413207+0000","last_active":"2024-04-26T20:52:22.632283+0000","last_peered":"2024-04-26T20:52:22.632283+0000","last_clean":"2024-04-26T20:52:22.632283+0000","last_became_active":"2024-04-26T20:52:20.412907+0000","last_became_peered":"2024-04-26T20:52:20.412907+0000","last_unstale":"2024-04-26T20:52:22.632283+0000","last_undegraded":"2024-04-26T20:52:22.632283+0000","last_fullsized":"2024-04-26T20:52:22.632283+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:30:23.377161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.415481+0000","last_change":"2024-04-26T20:52:21.418711+0000","last_active":"2024-04-26T20:52:22.415481+0000","last_peered":"2024-04-26T20:52:22.415481+0000","last_clean":"2024-04-26T20:52:22.415481+0000","last_became_active":"2024-04-26T20:52:21.418456+0000","last_became_peered":"2024-04-26T20:52:21.418456+0000","last_unstale":"2024-04-26T20:52:22.415481+0000","last_undegraded":"2024-04-26T20:52:22.415481+0000","last_fullsized":"2024-04-26T20:52:22.415481+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:30:12.603238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"51'2","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.632229+0000","last_change":"2024-04-26T20:52:20.419112+0000","last_active":"2024-04-26T20:52:22.632229+0000","last_peered":"2024-04-26T20:52:22.632229+0000","last_clean":"2024-04-26T20:52:22.632229+0000","last_became_active":"2024-04-26T20:52:20.418697+0000","last_became_peered":"2024-04-26T20:52:20.418697+0000","last_unstale":"2024-04-26T20:52:22.632229+0000","last_undegraded":"2024-04-26T20:52:22.632229+0000","last_fullsized":"2024-04-26T20:52:22.632229+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:39:59.885600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.414970+0000","last_change":"2024-04-26T20:52:21.415688+0000","last_active":"2024-04-26T20:52:22.414970+0000","last_peered":"2024-04-26T20:52:22.414970+0000","last_clean":"2024-04-26T20:52:22.414970+0000","last_became_active":"2024-04-26T20:52:21.415307+0000","last_became_peered":"2024-04-26T20:52:21.415307+0000","last_unstale":"2024-04-26T20:52:22.414970+0000","last_undegraded":"2024-04-26T20:52:22.414970+0000","last_fullsized":"2024-04-26T20:52:22.414970+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:13:13.059712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"51'2","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.632968+0000","last_change":"2024-04-26T20:52:20.412707+0000","last_active":"2024-04-26T20:52:22.632968+0000","last_peered":"2024-04-26T20:52:22.632968+0000","last_clean":"2024-04-26T20:52:22.632968+0000","last_became_active":"2024-04-26T20:52:20.411524+0000","last_became_peered":"2024-04-26T20:52:20.411524+0000","last_unstale":"2024-04-26T20:52:22.632968+0000","last_undegraded":"2024-04-26T20:52:22.632968+0000","last_fullsized":"2024-04-26T20:52:22.632968+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:02:39.223133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.632900+0000","last_change":"2024-04-26T20:52:21.423232+0000","last_active":"2024-04-26T20:52:22.632900+0000","last_peered":"2024-04-26T20:52:22.632900+0000","last_clean":"2024-04-26T20:52:22.632900+0000","last_became_active":"2024-04-26T20:52:21.422637+0000","last_became_peered":"2024-04-26T20:52:21.422637+0000","last_unstale":"2024-04-26T20:52:22.632900+0000","last_undegraded":"2024-04-26T20:52:22.632900+0000","last_fullsized":"2024-04-26T20:52:22.632900+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:32:47.374463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"51'1","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.632531+0000","last_change":"2024-04-26T20:52:20.416528+0000","last_active":"2024-04-26T20:52:22.632531+0000","last_peered":"2024-04-26T20:52:22.632531+0000","last_clean":"2024-04-26T20:52:22.632531+0000","last_became_active":"2024-04-26T20:52:20.416206+0000","last_became_peered":"2024-04-26T20:52:20.416206+0000","last_unstale":"2024-04-26T20:52:22.632531+0000","last_undegraded":"2024-04-26T20:52:22.632531+0000","last_fullsized":"2024-04-26T20:52:22.632531+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:50:09.638896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.632707+0000","last_change":"2024-04-26T20:52:21.419273+0000","last_active":"2024-04-26T20:52:22.632707+0000","last_peered":"2024-04-26T20:52:22.632707+0000","last_clean":"2024-04-26T20:52:22.632707+0000","last_became_active":"2024-04-26T20:52:21.419001+0000","last_became_peered":"2024-04-26T20:52:21.419001+0000","last_unstale":"2024-04-26T20:52:22.632707+0000","last_undegraded":"2024-04-26T20:52:22.632707+0000","last_fullsized":"2024-04-26T20:52:22.632707+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:02:07.235466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"51'1","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.632168+0000","last_change":"2024-04-26T20:52:20.415428+0000","last_active":"2024-04-26T20:52:22.632168+0000","last_peered":"2024-04-26T20:52:22.632168+0000","last_clean":"2024-04-26T20:52:22.632168+0000","last_became_active":"2024-04-26T20:52:20.415125+0000","last_became_peered":"2024-04-26T20:52:20.415125+0000","last_unstale":"2024-04-26T20:52:22.632168+0000","last_undegraded":"2024-04-26T20:52:22.632168+0000","last_fullsized":"2024-04-26T20:52:22.632168+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:59:02.612206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.415257+0000","last_change":"2024-04-26T20:52:21.416155+0000","last_active":"2024-04-26T20:52:22.415257+0000","last_peered":"2024-04-26T20:52:22.415257+0000","last_clean":"2024-04-26T20:52:22.415257+0000","last_became_active":"2024-04-26T20:52:21.415902+0000","last_became_peered":"2024-04-26T20:52:21.415902+0000","last_unstale":"2024-04-26T20:52:22.415257+0000","last_undegraded":"2024-04-26T20:52:22.415257+0000","last_fullsized":"2024-04-26T20:52:22.415257+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:40:09.957708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.415087+0000","last_change":"2024-04-26T20:52:20.418051+0000","last_active":"2024-04-26T20:52:22.415087+0000","last_peered":"2024-04-26T20:52:22.415087+0000","last_clean":"2024-04-26T20:52:22.415087+0000","last_became_active":"2024-04-26T20:52:20.417745+0000","last_became_peered":"2024-04-26T20:52:20.417745+0000","last_unstale":"2024-04-26T20:52:22.415087+0000","last_undegraded":"2024-04-26T20:52:22.415087+0000","last_fullsized":"2024-04-26T20:52:22.415087+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:14:31.202147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.632826+0000","last_change":"2024-04-26T20:52:21.419500+0000","last_active":"2024-04-26T20:52:22.632826+0000","last_peered":"2024-04-26T20:52:22.632826+0000","last_clean":"2024-04-26T20:52:22.632826+0000","last_became_active":"2024-04-26T20:52:21.418732+0000","last_became_peered":"2024-04-26T20:52:21.418732+0000","last_unstale":"2024-04-26T20:52:22.632826+0000","last_undegraded":"2024-04-26T20:52:22.632826+0000","last_fullsized":"2024-04-26T20:52:22.632826+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:02:08.862150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.631961+0000","last_change":"2024-04-26T20:52:20.420255+0000","last_active":"2024-04-26T20:52:22.631961+0000","last_peered":"2024-04-26T20:52:22.631961+0000","last_clean":"2024-04-26T20:52:22.631961+0000","last_became_active":"2024-04-26T20:52:20.419872+0000","last_became_peered":"2024-04-26T20:52:20.419872+0000","last_unstale":"2024-04-26T20:52:22.631961+0000","last_undegraded":"2024-04-26T20:52:22.631961+0000","last_fullsized":"2024-04-26T20:52:22.631961+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:30:49.782994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.632280+0000","last_change":"2024-04-26T20:52:21.416163+0000","last_active":"2024-04-26T20:52:22.632280+0000","last_peered":"2024-04-26T20:52:22.632280+0000","last_clean":"2024-04-26T20:52:22.632280+0000","last_became_active":"2024-04-26T20:52:21.415877+0000","last_became_peered":"2024-04-26T20:52:21.415877+0000","last_unstale":"2024-04-26T20:52:22.632280+0000","last_undegraded":"2024-04-26T20:52:22.632280+0000","last_fullsized":"2024-04-26T20:52:22.632280+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:30:50.792136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.632915+0000","last_change":"2024-04-26T20:52:20.415911+0000","last_active":"2024-04-26T20:52:22.632915+0000","last_peered":"2024-04-26T20:52:22.632915+0000","last_clean":"2024-04-26T20:52:22.632915+0000","last_became_active":"2024-04-26T20:52:20.415634+0000","last_became_peered":"2024-04-26T20:52:20.415634+0000","last_unstale":"2024-04-26T20:52:22.632915+0000","last_undegraded":"2024-04-26T20:52:22.632915+0000","last_fullsized":"2024-04-26T20:52:22.632915+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:41:18.472871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.632817+0000","last_change":"2024-04-26T20:52:21.412004+0000","last_active":"2024-04-26T20:52:22.632817+0000","last_peered":"2024-04-26T20:52:22.632817+0000","last_clean":"2024-04-26T20:52:22.632817+0000","last_became_active":"2024-04-26T20:52:21.411482+0000","last_became_peered":"2024-04-26T20:52:21.411482+0000","last_unstale":"2024-04-26T20:52:22.632817+0000","last_undegraded":"2024-04-26T20:52:22.632817+0000","last_fullsized":"2024-04-26T20:52:22.632817+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:47:26.973089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.632715+0000","last_change":"2024-04-26T20:52:20.420738+0000","last_active":"2024-04-26T20:52:22.632715+0000","last_peered":"2024-04-26T20:52:22.632715+0000","last_clean":"2024-04-26T20:52:22.632715+0000","last_became_active":"2024-04-26T20:52:20.420453+0000","last_became_peered":"2024-04-26T20:52:20.420453+0000","last_unstale":"2024-04-26T20:52:22.632715+0000","last_undegraded":"2024-04-26T20:52:22.632715+0000","last_fullsized":"2024-04-26T20:52:22.632715+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:43:24.469500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.633732+0000","last_change":"2024-04-26T20:52:21.421787+0000","last_active":"2024-04-26T20:52:22.633732+0000","last_peered":"2024-04-26T20:52:22.633732+0000","last_clean":"2024-04-26T20:52:22.633732+0000","last_became_active":"2024-04-26T20:52:21.421536+0000","last_became_peered":"2024-04-26T20:52:21.421536+0000","last_unstale":"2024-04-26T20:52:22.633732+0000","last_undegraded":"2024-04-26T20:52:22.633732+0000","last_fullsized":"2024-04-26T20:52:22.633732+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:40:09.957708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"51'3","reported_seq":17,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.632825+0000","last_change":"2024-04-26T20:52:20.422241+0000","last_active":"2024-04-26T20:52:22.632825+0000","last_peered":"2024-04-26T20:52:22.632825+0000","last_clean":"2024-04-26T20:52:22.632825+0000","last_became_active":"2024-04-26T20:52:20.422041+0000","last_became_peered":"2024-04-26T20:52:20.422041+0000","last_unstale":"2024-04-26T20:52:22.632825+0000","last_undegraded":"2024-04-26T20:52:22.632825+0000","last_fullsized":"2024-04-26T20:52:22.632825+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:43:24.469500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.632507+0000","last_change":"2024-04-26T20:52:21.417228+0000","last_active":"2024-04-26T20:52:22.632507+0000","last_peered":"2024-04-26T20:52:22.632507+0000","last_clean":"2024-04-26T20:52:22.632507+0000","last_became_active":"2024-04-26T20:52:21.416962+0000","last_became_peered":"2024-04-26T20:52:21.416962+0000","last_unstale":"2024-04-26T20:52:22.632507+0000","last_undegraded":"2024-04-26T20:52:22.632507+0000","last_fullsized":"2024-04-26T20:52:22.632507+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:02:40.232275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"51'6","reported_seq":20,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.415336+0000","last_change":"2024-04-26T20:52:20.416316+0000","last_active":"2024-04-26T20:52:22.415336+0000","last_peered":"2024-04-26T20:52:22.415336+0000","last_clean":"2024-04-26T20:52:22.415336+0000","last_became_active":"2024-04-26T20:52:20.416070+0000","last_became_peered":"2024-04-26T20:52:20.416070+0000","last_unstale":"2024-04-26T20:52:22.415336+0000","last_undegraded":"2024-04-26T20:52:22.415336+0000","last_fullsized":"2024-04-26T20:52:22.415336+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:02:06.226324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.632905+0000","last_change":"2024-04-26T20:52:21.417917+0000","last_active":"2024-04-26T20:52:22.632905+0000","last_peered":"2024-04-26T20:52:22.632905+0000","last_clean":"2024-04-26T20:52:22.632905+0000","last_became_active":"2024-04-26T20:52:21.416554+0000","last_became_peered":"2024-04-26T20:52:21.416554+0000","last_unstale":"2024-04-26T20:52:22.632905+0000","last_undegraded":"2024-04-26T20:52:22.632905+0000","last_fullsized":"2024-04-26T20:52:22.632905+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:24:19.055437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"51'2","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.632624+0000","last_change":"2024-04-26T20:52:20.413151+0000","last_active":"2024-04-26T20:52:22.632624+0000","last_peered":"2024-04-26T20:52:22.632624+0000","last_clean":"2024-04-26T20:52:22.632624+0000","last_became_active":"2024-04-26T20:52:20.412840+0000","last_became_peered":"2024-04-26T20:52:20.412840+0000","last_unstale":"2024-04-26T20:52:22.632624+0000","last_undegraded":"2024-04-26T20:52:22.632624+0000","last_fullsized":"2024-04-26T20:52:22.632624+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:57:42.143052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.633176+0000","last_change":"2024-04-26T20:52:21.420968+0000","last_active":"2024-04-26T20:52:22.633176+0000","last_peered":"2024-04-26T20:52:22.633176+0000","last_clean":"2024-04-26T20:52:22.633176+0000","last_became_active":"2024-04-26T20:52:21.420696+0000","last_became_peered":"2024-04-26T20:52:21.420696+0000","last_unstale":"2024-04-26T20:52:22.633176+0000","last_undegraded":"2024-04-26T20:52:22.633176+0000","last_fullsized":"2024-04-26T20:52:22.633176+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:46:12.362383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.415282+0000","last_change":"2024-04-26T20:52:20.424000+0000","last_active":"2024-04-26T20:52:22.415282+0000","last_peered":"2024-04-26T20:52:22.415282+0000","last_clean":"2024-04-26T20:52:22.415282+0000","last_became_active":"2024-04-26T20:52:20.423810+0000","last_became_peered":"2024-04-26T20:52:20.423810+0000","last_unstale":"2024-04-26T20:52:22.415282+0000","last_undegraded":"2024-04-26T20:52:22.415282+0000","last_fullsized":"2024-04-26T20:52:22.415282+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:11:10.686903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.415125+0000","last_change":"2024-04-26T20:52:21.426600+0000","last_active":"2024-04-26T20:52:22.415125+0000","last_peered":"2024-04-26T20:52:22.415125+0000","last_clean":"2024-04-26T20:52:22.415125+0000","last_became_active":"2024-04-26T20:52:21.426359+0000","last_became_peered":"2024-04-26T20:52:21.426359+0000","last_unstale":"2024-04-26T20:52:22.415125+0000","last_undegraded":"2024-04-26T20:52:22.415125+0000","last_fullsized":"2024-04-26T20:52:22.415125+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:11:03.316412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.633075+0000","last_change":"2024-04-26T20:52:20.418948+0000","last_active":"2024-04-26T20:52:22.633075+0000","last_peered":"2024-04-26T20:52:22.633075+0000","last_clean":"2024-04-26T20:52:22.633075+0000","last_became_active":"2024-04-26T20:52:20.418646+0000","last_became_peered":"2024-04-26T20:52:20.418646+0000","last_unstale":"2024-04-26T20:52:22.633075+0000","last_undegraded":"2024-04-26T20:52:22.633075+0000","last_fullsized":"2024-04-26T20:52:22.633075+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:59:02.612206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.633122+0000","last_change":"2024-04-26T20:52:21.426271+0000","last_active":"2024-04-26T20:52:22.633122+0000","last_peered":"2024-04-26T20:52:22.633122+0000","last_clean":"2024-04-26T20:52:22.633122+0000","last_became_active":"2024-04-26T20:52:21.425124+0000","last_became_peered":"2024-04-26T20:52:21.425124+0000","last_unstale":"2024-04-26T20:52:22.633122+0000","last_undegraded":"2024-04-26T20:52:22.633122+0000","last_fullsized":"2024-04-26T20:52:22.633122+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:56:54.984952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"51'8","reported_seq":22,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.414983+0000","last_change":"2024-04-26T20:52:20.417314+0000","last_active":"2024-04-26T20:52:22.414983+0000","last_peered":"2024-04-26T20:52:22.414983+0000","last_clean":"2024-04-26T20:52:22.414983+0000","last_became_active":"2024-04-26T20:52:20.417057+0000","last_became_peered":"2024-04-26T20:52:20.417057+0000","last_unstale":"2024-04-26T20:52:22.414983+0000","last_undegraded":"2024-04-26T20:52:22.414983+0000","last_fullsized":"2024-04-26T20:52:22.414983+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:04:50.904141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.414942+0000","last_change":"2024-04-26T20:52:21.414380+0000","last_active":"2024-04-26T20:52:22.414942+0000","last_peered":"2024-04-26T20:52:22.414942+0000","last_clean":"2024-04-26T20:52:22.414942+0000","last_became_active":"2024-04-26T20:52:21.413130+0000","last_became_peered":"2024-04-26T20:52:21.413130+0000","last_unstale":"2024-04-26T20:52:22.414942+0000","last_undegraded":"2024-04-26T20:52:22.414942+0000","last_fullsized":"2024-04-26T20:52:22.414942+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:46:12.362383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"51'1","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.633166+0000","last_change":"2024-04-26T20:52:20.415615+0000","last_active":"2024-04-26T20:52:22.633166+0000","last_peered":"2024-04-26T20:52:22.633166+0000","last_clean":"2024-04-26T20:52:22.633166+0000","last_became_active":"2024-04-26T20:52:20.415365+0000","last_became_peered":"2024-04-26T20:52:20.415365+0000","last_unstale":"2024-04-26T20:52:22.633166+0000","last_undegraded":"2024-04-26T20:52:22.633166+0000","last_fullsized":"2024-04-26T20:52:22.633166+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:48:17.854687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.632203+0000","last_change":"2024-04-26T20:52:21.427808+0000","last_active":"2024-04-26T20:52:22.632203+0000","last_peered":"2024-04-26T20:52:22.632203+0000","last_clean":"2024-04-26T20:52:22.632203+0000","last_became_active":"2024-04-26T20:52:21.427542+0000","last_became_peered":"2024-04-26T20:52:21.427542+0000","last_unstale":"2024-04-26T20:52:22.632203+0000","last_undegraded":"2024-04-26T20:52:22.632203+0000","last_fullsized":"2024-04-26T20:52:22.632203+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:31.714105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"51'2","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.633240+0000","last_change":"2024-04-26T20:52:20.416835+0000","last_active":"2024-04-26T20:52:22.633240+0000","last_peered":"2024-04-26T20:52:22.633240+0000","last_clean":"2024-04-26T20:52:22.633240+0000","last_became_active":"2024-04-26T20:52:20.416554+0000","last_became_peered":"2024-04-26T20:52:20.416554+0000","last_unstale":"2024-04-26T20:52:22.633240+0000","last_undegraded":"2024-04-26T20:52:22.633240+0000","last_fullsized":"2024-04-26T20:52:22.633240+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:47:28.125421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.632099+0000","last_change":"2024-04-26T20:52:21.422916+0000","last_active":"2024-04-26T20:52:22.632099+0000","last_peered":"2024-04-26T20:52:22.632099+0000","last_clean":"2024-04-26T20:52:22.632099+0000","last_became_active":"2024-04-26T20:52:21.422642+0000","last_became_peered":"2024-04-26T20:52:21.422642+0000","last_unstale":"2024-04-26T20:52:22.632099+0000","last_undegraded":"2024-04-26T20:52:22.632099+0000","last_fullsized":"2024-04-26T20:52:22.632099+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:57:43.152194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"51'1","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.415499+0000","last_change":"2024-04-26T20:52:20.417996+0000","last_active":"2024-04-26T20:52:22.415499+0000","last_peered":"2024-04-26T20:52:22.415499+0000","last_clean":"2024-04-26T20:52:22.415499+0000","last_became_active":"2024-04-26T20:52:20.417696+0000","last_became_peered":"2024-04-26T20:52:20.417696+0000","last_unstale":"2024-04-26T20:52:22.415499+0000","last_undegraded":"2024-04-26T20:52:22.415499+0000","last_fullsized":"2024-04-26T20:52:22.415499+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:21:45.798570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.632438+0000","last_change":"2024-04-26T20:52:21.416649+0000","last_active":"2024-04-26T20:52:22.632438+0000","last_peered":"2024-04-26T20:52:22.632438+0000","last_clean":"2024-04-26T20:52:22.632438+0000","last_became_active":"2024-04-26T20:52:21.416379+0000","last_became_peered":"2024-04-26T20:52:21.416379+0000","last_unstale":"2024-04-26T20:52:22.632438+0000","last_undegraded":"2024-04-26T20:52:22.632438+0000","last_fullsized":"2024-04-26T20:52:22.632438+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:04:51.913283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.415553+0000","last_change":"2024-04-26T20:52:20.421140+0000","last_active":"2024-04-26T20:52:22.415553+0000","last_peered":"2024-04-26T20:52:22.415553+0000","last_clean":"2024-04-26T20:52:22.415553+0000","last_became_active":"2024-04-26T20:52:20.420934+0000","last_became_peered":"2024-04-26T20:52:20.420934+0000","last_unstale":"2024-04-26T20:52:22.415553+0000","last_undegraded":"2024-04-26T20:52:22.415553+0000","last_fullsized":"2024-04-26T20:52:22.415553+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:21:20.076908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.632932+0000","last_change":"2024-04-26T20:52:21.413452+0000","last_active":"2024-04-26T20:52:22.632932+0000","last_peered":"2024-04-26T20:52:22.632932+0000","last_clean":"2024-04-26T20:52:22.632932+0000","last_became_active":"2024-04-26T20:52:21.413137+0000","last_became_peered":"2024-04-26T20:52:21.413137+0000","last_unstale":"2024-04-26T20:52:22.632932+0000","last_undegraded":"2024-04-26T20:52:22.632932+0000","last_fullsized":"2024-04-26T20:52:22.632932+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:31:09.556882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"51'2","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.632382+0000","last_change":"2024-04-26T20:52:20.409441+0000","last_active":"2024-04-26T20:52:22.632382+0000","last_peered":"2024-04-26T20:52:22.632382+0000","last_clean":"2024-04-26T20:52:22.632382+0000","last_became_active":"2024-04-26T20:52:20.409146+0000","last_became_peered":"2024-04-26T20:52:20.409146+0000","last_unstale":"2024-04-26T20:52:22.632382+0000","last_undegraded":"2024-04-26T20:52:22.632382+0000","last_fullsized":"2024-04-26T20:52:22.632382+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:45:19.546221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.415450+0000","last_change":"2024-04-26T20:52:21.413473+0000","last_active":"2024-04-26T20:52:22.415450+0000","last_peered":"2024-04-26T20:52:22.415450+0000","last_clean":"2024-04-26T20:52:22.415450+0000","last_became_active":"2024-04-26T20:52:21.413174+0000","last_became_peered":"2024-04-26T20:52:21.413174+0000","last_unstale":"2024-04-26T20:52:22.415450+0000","last_undegraded":"2024-04-26T20:52:22.415450+0000","last_fullsized":"2024-04-26T20:52:22.415450+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:41:19.482012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.414760+0000","last_change":"2024-04-26T20:52:20.420211+0000","last_active":"2024-04-26T20:52:22.414760+0000","last_peered":"2024-04-26T20:52:22.414760+0000","last_clean":"2024-04-26T20:52:22.414760+0000","last_became_active":"2024-04-26T20:52:20.419954+0000","last_became_peered":"2024-04-26T20:52:20.419954+0000","last_unstale":"2024-04-26T20:52:22.414760+0000","last_undegraded":"2024-04-26T20:52:22.414760+0000","last_fullsized":"2024-04-26T20:52:22.414760+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:18:18.410031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.633094+0000","last_change":"2024-04-26T20:52:21.428528+0000","last_active":"2024-04-26T20:52:22.633094+0000","last_peered":"2024-04-26T20:52:22.633094+0000","last_clean":"2024-04-26T20:52:22.633094+0000","last_became_active":"2024-04-26T20:52:21.427822+0000","last_became_peered":"2024-04-26T20:52:21.427822+0000","last_unstale":"2024-04-26T20:52:22.633094+0000","last_undegraded":"2024-04-26T20:52:22.633094+0000","last_fullsized":"2024-04-26T20:52:22.633094+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:17:14.587200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.633415+0000","last_change":"2024-04-26T20:52:20.417406+0000","last_active":"2024-04-26T20:52:22.633415+0000","last_peered":"2024-04-26T20:52:22.633415+0000","last_clean":"2024-04-26T20:52:22.633415+0000","last_became_active":"2024-04-26T20:52:20.417157+0000","last_became_peered":"2024-04-26T20:52:20.417157+0000","last_unstale":"2024-04-26T20:52:22.633415+0000","last_undegraded":"2024-04-26T20:52:22.633415+0000","last_fullsized":"2024-04-26T20:52:22.633415+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:15:47.258148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.632077+0000","last_change":"2024-04-26T20:52:21.424248+0000","last_active":"2024-04-26T20:52:22.632077+0000","last_peered":"2024-04-26T20:52:22.632077+0000","last_clean":"2024-04-26T20:52:22.632077+0000","last_became_active":"2024-04-26T20:52:21.423969+0000","last_became_peered":"2024-04-26T20:52:21.423969+0000","last_unstale":"2024-04-26T20:52:22.632077+0000","last_undegraded":"2024-04-26T20:52:22.632077+0000","last_fullsized":"2024-04-26T20:52:22.632077+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:32:47.374463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.633458+0000","last_change":"2024-04-26T20:52:20.415666+0000","last_active":"2024-04-26T20:52:22.633458+0000","last_peered":"2024-04-26T20:52:22.633458+0000","last_clean":"2024-04-26T20:52:22.633458+0000","last_became_active":"2024-04-26T20:52:20.415380+0000","last_became_peered":"2024-04-26T20:52:20.415380+0000","last_unstale":"2024-04-26T20:52:22.633458+0000","last_undegraded":"2024-04-26T20:52:22.633458+0000","last_fullsized":"2024-04-26T20:52:22.633458+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:11:22.108762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.632723+0000","last_change":"2024-04-26T20:52:21.415726+0000","last_active":"2024-04-26T20:52:22.632723+0000","last_peered":"2024-04-26T20:52:22.632723+0000","last_clean":"2024-04-26T20:52:22.632723+0000","last_became_active":"2024-04-26T20:52:21.415383+0000","last_became_peered":"2024-04-26T20:52:21.415383+0000","last_unstale":"2024-04-26T20:52:22.632723+0000","last_undegraded":"2024-04-26T20:52:22.632723+0000","last_fullsized":"2024-04-26T20:52:22.632723+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:02:08.862150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"51'2","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.415552+0000","last_change":"2024-04-26T20:52:20.424128+0000","last_active":"2024-04-26T20:52:22.415552+0000","last_peered":"2024-04-26T20:52:22.415552+0000","last_clean":"2024-04-26T20:52:22.415552+0000","last_became_active":"2024-04-26T20:52:20.423842+0000","last_became_peered":"2024-04-26T20:52:20.423842+0000","last_unstale":"2024-04-26T20:52:22.415552+0000","last_undegraded":"2024-04-26T20:52:22.415552+0000","last_fullsized":"2024-04-26T20:52:22.415552+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:36:08.825154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.632145+0000","last_change":"2024-04-26T20:52:21.420481+0000","last_active":"2024-04-26T20:52:22.632145+0000","last_peered":"2024-04-26T20:52:22.632145+0000","last_clean":"2024-04-26T20:52:22.632145+0000","last_became_active":"2024-04-26T20:52:21.420245+0000","last_became_peered":"2024-04-26T20:52:21.420245+0000","last_unstale":"2024-04-26T20:52:22.632145+0000","last_undegraded":"2024-04-26T20:52:22.632145+0000","last_fullsized":"2024-04-26T20:52:22.632145+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:57:36.490497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.632936+0000","last_change":"2024-04-26T20:52:20.419503+0000","last_active":"2024-04-26T20:52:22.632936+0000","last_peered":"2024-04-26T20:52:22.632936+0000","last_clean":"2024-04-26T20:52:22.632936+0000","last_became_active":"2024-04-26T20:52:20.419291+0000","last_became_peered":"2024-04-26T20:52:20.419291+0000","last_unstale":"2024-04-26T20:52:22.632936+0000","last_undegraded":"2024-04-26T20:52:22.632936+0000","last_fullsized":"2024-04-26T20:52:22.632936+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:48:17.854687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.415077+0000","last_change":"2024-04-26T20:52:21.413882+0000","last_active":"2024-04-26T20:52:22.415077+0000","last_peered":"2024-04-26T20:52:22.415077+0000","last_clean":"2024-04-26T20:52:22.415077+0000","last_became_active":"2024-04-26T20:52:21.413589+0000","last_became_peered":"2024-04-26T20:52:21.413589+0000","last_unstale":"2024-04-26T20:52:22.415077+0000","last_undegraded":"2024-04-26T20:52:22.415077+0000","last_fullsized":"2024-04-26T20:52:22.415077+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:34:47.196852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.632745+0000","last_change":"2024-04-26T20:52:20.411706+0000","last_active":"2024-04-26T20:52:22.632745+0000","last_peered":"2024-04-26T20:52:22.632745+0000","last_clean":"2024-04-26T20:52:22.632745+0000","last_became_active":"2024-04-26T20:52:20.411459+0000","last_became_peered":"2024-04-26T20:52:20.411459+0000","last_unstale":"2024-04-26T20:52:22.632745+0000","last_undegraded":"2024-04-26T20:52:22.632745+0000","last_fullsized":"2024-04-26T20:52:22.632745+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:41:18.472871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.632705+0000","last_change":"2024-04-26T20:52:21.429037+0000","last_active":"2024-04-26T20:52:22.632705+0000","last_peered":"2024-04-26T20:52:22.632705+0000","last_clean":"2024-04-26T20:52:22.632705+0000","last_became_active":"2024-04-26T20:52:21.428750+0000","last_became_peered":"2024-04-26T20:52:21.428750+0000","last_unstale":"2024-04-26T20:52:22.632705+0000","last_undegraded":"2024-04-26T20:52:22.632705+0000","last_fullsized":"2024-04-26T20:52:22.632705+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:09:53.553508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.415020+0000","last_change":"2024-04-26T20:52:20.416931+0000","last_active":"2024-04-26T20:52:22.415020+0000","last_peered":"2024-04-26T20:52:22.415020+0000","last_clean":"2024-04-26T20:52:22.415020+0000","last_became_active":"2024-04-26T20:52:20.416666+0000","last_became_peered":"2024-04-26T20:52:20.416666+0000","last_unstale":"2024-04-26T20:52:22.415020+0000","last_undegraded":"2024-04-26T20:52:22.415020+0000","last_fullsized":"2024-04-26T20:52:22.415020+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:02:07.853009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.633124+0000","last_change":"2024-04-26T20:52:21.416147+0000","last_active":"2024-04-26T20:52:22.633124+0000","last_peered":"2024-04-26T20:52:22.633124+0000","last_clean":"2024-04-26T20:52:22.633124+0000","last_became_active":"2024-04-26T20:52:21.415879+0000","last_became_peered":"2024-04-26T20:52:21.415879+0000","last_unstale":"2024-04-26T20:52:22.633124+0000","last_undegraded":"2024-04-26T20:52:22.633124+0000","last_fullsized":"2024-04-26T20:52:22.633124+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:43:10.360165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.414873+0000","last_change":"2024-04-26T20:52:20.414948+0000","last_active":"2024-04-26T20:52:22.414873+0000","last_peered":"2024-04-26T20:52:22.414873+0000","last_clean":"2024-04-26T20:52:22.414873+0000","last_became_active":"2024-04-26T20:52:20.414677+0000","last_became_peered":"2024-04-26T20:52:20.414677+0000","last_unstale":"2024-04-26T20:52:22.414873+0000","last_undegraded":"2024-04-26T20:52:22.414873+0000","last_fullsized":"2024-04-26T20:52:22.414873+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:55:47.685718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.632978+0000","last_change":"2024-04-26T20:52:21.418009+0000","last_active":"2024-04-26T20:52:22.632978+0000","last_peered":"2024-04-26T20:52:22.632978+0000","last_clean":"2024-04-26T20:52:22.632978+0000","last_became_active":"2024-04-26T20:52:21.417522+0000","last_became_peered":"2024-04-26T20:52:21.417522+0000","last_unstale":"2024-04-26T20:52:22.632978+0000","last_undegraded":"2024-04-26T20:52:22.632978+0000","last_fullsized":"2024-04-26T20:52:22.632978+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:21:14.355170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.632180+0000","last_change":"2024-04-26T20:52:20.414982+0000","last_active":"2024-04-26T20:52:22.632180+0000","last_peered":"2024-04-26T20:52:22.632180+0000","last_clean":"2024-04-26T20:52:22.632180+0000","last_became_active":"2024-04-26T20:52:20.414694+0000","last_became_peered":"2024-04-26T20:52:20.414694+0000","last_unstale":"2024-04-26T20:52:22.632180+0000","last_undegraded":"2024-04-26T20:52:22.632180+0000","last_fullsized":"2024-04-26T20:52:22.632180+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:18.298734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.415349+0000","last_change":"2024-04-26T20:52:21.414499+0000","last_active":"2024-04-26T20:52:22.415349+0000","last_peered":"2024-04-26T20:52:22.415349+0000","last_clean":"2024-04-26T20:52:22.415349+0000","last_became_active":"2024-04-26T20:52:21.414100+0000","last_became_peered":"2024-04-26T20:52:21.414100+0000","last_unstale":"2024-04-26T20:52:22.415349+0000","last_undegraded":"2024-04-26T20:52:22.415349+0000","last_fullsized":"2024-04-26T20:52:22.415349+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:48:45.853597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.632657+0000","last_change":"2024-04-26T20:52:20.414067+0000","last_active":"2024-04-26T20:52:22.632657+0000","last_peered":"2024-04-26T20:52:22.632657+0000","last_clean":"2024-04-26T20:52:22.632657+0000","last_became_active":"2024-04-26T20:52:20.413782+0000","last_became_peered":"2024-04-26T20:52:20.413782+0000","last_unstale":"2024-04-26T20:52:22.632657+0000","last_undegraded":"2024-04-26T20:52:22.632657+0000","last_fullsized":"2024-04-26T20:52:22.632657+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:30:01.919362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.415273+0000","last_change":"2024-04-26T20:52:21.407363+0000","last_active":"2024-04-26T20:52:22.415273+0000","last_peered":"2024-04-26T20:52:22.415273+0000","last_clean":"2024-04-26T20:52:22.415273+0000","last_became_active":"2024-04-26T20:52:21.407041+0000","last_became_peered":"2024-04-26T20:52:21.407041+0000","last_unstale":"2024-04-26T20:52:22.415273+0000","last_undegraded":"2024-04-26T20:52:22.415273+0000","last_fullsized":"2024-04-26T20:52:22.415273+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:20.391318+0000","last_clean_scrub_stamp":"2024-04-26T20:52:20.391318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:59:03.621348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"52'1","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.633058+0000","last_change":"2024-04-26T20:52:20.412380+0000","last_active":"2024-04-26T20:52:22.633058+0000","last_peered":"2024-04-26T20:52:22.633058+0000","last_clean":"2024-04-26T20:52:22.633058+0000","last_became_active":"2024-04-26T20:52:20.412124+0000","last_became_peered":"2024-04-26T20:52:20.412124+0000","last_unstale":"2024-04-26T20:52:22.633058+0000","last_undegraded":"2024-04-26T20:52:22.633058+0000","last_fullsized":"2024-04-26T20:52:22.633058+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:52:19.382176+0000","last_clean_scrub_stamp":"2024-04-26T20:52:19.382176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:30:11.594096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":228,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T20:52:22.631955+0000","last_change":"2024-04-26T20:50:21.150132+0000","last_active":"2024-04-26T20:52:22.631955+0000","last_peered":"2024-04-26T20:52:22.631955+0000","last_clean":"2024-04-26T20:52:22.631955+0000","last_became_active":"2024-04-26T20:50:21.147439+0000","last_became_peered":"2024-04-26T20:50:21.147439+0000","last_unstale":"2024-04-26T20:52:22.631955+0000","last_undegraded":"2024-04-26T20:52:22.631955+0000","last_fullsized":"2024-04-26T20:52:22.631955+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:20.084499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:20.084499+0000","last_clean_scrub_stamp":"2024-04-26T20:50:20.084499+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:19:29.395477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574876,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28212,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343628,"statfs":{"total":96636764160,"available":96607875072,"internally_reserved":0,"allocated":1220608,"data_stored":868013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574877,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27636,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344204,"statfs":{"total":96636764160,"available":96608464896,"internally_reserved":0,"allocated":626688,"data_stored":278173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574877,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27628,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344212,"statfs":{"total":96636764160,"available":96608473088,"internally_reserved":0,"allocated":618496,"data_stored":277521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574877,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":622592,"data_stored":278103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574877,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":634880,"data_stored":279155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":630784,"data_stored":279133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574876,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27636,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344204,"statfs":{"total":96636764160,"available":96608464896,"internally_reserved":0,"allocated":626688,"data_stored":277681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28212,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343628,"statfs":{"total":96636764160,"available":96607875072,"internally_reserved":0,"allocated":1220608,"data_stored":867945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:52:29.072 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T20:52:29.073 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-26T20:52:29.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.211+0000 7fb2b6739640 1 Processor -- start 2024-04-26T20:52:29.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.212+0000 7fb2b6739640 1 -- start start 2024-04-26T20:52:29.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.212+0000 7fb2b6739640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fb2b00d1a00 0x7fb2b00d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:29.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.212+0000 7fb2b6739640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb2b00d2320 0x7fb2b01625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:29.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.212+0000 7fb2b6739640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2b00d0a10 con 0x7fb2b00b40d0 2024-04-26T20:52:29.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.212+0000 7fb2b6739640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7fb2b00d0cf0 con 0x7fb2b00d2320 2024-04-26T20:52:29.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.212+0000 7fb2b6739640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fb2b00d0fd0 con 0x7fb2b00d1a00 2024-04-26T20:52:29.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.212+0000 7fb2af7fe640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb2b00d2320 0x7fb2b01625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:29.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.212+0000 7fb2af7fe640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb2b00d2320 0x7fb2b01625f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38284/0 (socket says 172.21.15.73:38284) 2024-04-26T20:52:29.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.212+0000 7fb2af7fe640 1 -- 172.21.15.73:0/1657984149 learned_addr learned my addr 172.21.15.73:0/1657984149 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:29.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.212+0000 7fb2b4cd6640 1 --1- 172.21.15.73:0/1657984149 >> v1:172.21.15.73:6789/0 conn(0x7fb2b00b40d0 0x7fb2b00d2ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:36626/0 (socket says 172.21.15.73:36626) 2024-04-26T20:52:29.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.213+0000 7fb2affff640 1 --2- 172.21.15.73:0/1657984149 >> v2:172.21.15.120:3300/0 conn(0x7fb2b00d1a00 0x7fb2b00d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:29.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.213+0000 7fb2af7fe640 1 -- 172.21.15.73:0/1657984149 >> v2:172.21.15.120:3300/0 conn(0x7fb2b00d1a00 msgr2=0x7fb2b00d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:29.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.213+0000 7fb2af7fe640 1 --2- 172.21.15.73:0/1657984149 >> v2:172.21.15.120:3300/0 conn(0x7fb2b00d1a00 0x7fb2b00d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:29.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.213+0000 7fb2af7fe640 1 -- 172.21.15.73:0/1657984149 >> v1:172.21.15.73:6789/0 conn(0x7fb2b00b40d0 legacy=0x7fb2b00d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:29.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.213+0000 7fb2af7fe640 1 -- 172.21.15.73:0/1657984149 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2b00d40a0 con 0x7fb2b00d2320 2024-04-26T20:52:29.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.213+0000 7fb2aeffd640 1 -- 172.21.15.73:0/1657984149 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3524223652 0 0) 0x7fb2b00d0a10 con 0x7fb2b00b40d0 2024-04-26T20:52:29.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.213+0000 7fb2affff640 1 --2- 172.21.15.73:0/1657984149 >> v2:172.21.15.120:3300/0 conn(0x7fb2b00d1a00 0x7fb2b00d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:52:29.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.213+0000 7fb2af7fe640 1 --2- 172.21.15.73:0/1657984149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2b00d2320 0x7fb2b01625f0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fb2b006c650 tx=0x7fb2a402a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9cf490c684fe29c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:29.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.214+0000 7fb2aeffd640 1 -- 172.21.15.73:0/1657984149 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a4037070 con 0x7fb2b00d2320 2024-04-26T20:52:29.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.214+0000 7fb2aeffd640 1 -- 172.21.15.73:0/1657984149 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2a4032440 con 0x7fb2b00d2320 2024-04-26T20:52:29.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.214+0000 7fb2aeffd640 1 -- 172.21.15.73:0/1657984149 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a4032810 con 0x7fb2b00d2320 2024-04-26T20:52:29.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.214+0000 7fb2b6739640 1 -- 172.21.15.73:0/1657984149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2b00d2320 msgr2=0x7fb2b01625f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:29.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.214+0000 7fb2b6739640 1 --2- 172.21.15.73:0/1657984149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2b00d2320 0x7fb2b01625f0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fb2b006c650 tx=0x7fb2a402a150 comp rx=0 tx=0).stop 2024-04-26T20:52:29.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.214+0000 7fb2b6739640 1 -- 172.21.15.73:0/1657984149 shutdown_connections 2024-04-26T20:52:29.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.214+0000 7fb2b6739640 1 --2- 172.21.15.73:0/1657984149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2b00d2320 0x7fb2b01625f0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:29.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.214+0000 7fb2b6739640 1 --2- 172.21.15.73:0/1657984149 >> v2:172.21.15.120:3300/0 conn(0x7fb2b00d1a00 0x7fb2b00d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:29.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.214+0000 7fb2b6739640 1 -- 172.21.15.73:0/1657984149 >> 172.21.15.73:0/1657984149 conn(0x7fb2b00d4fd0 msgr2=0x7fb2b00d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:29.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.214+0000 7fb2b6739640 1 -- 172.21.15.73:0/1657984149 shutdown_connections 2024-04-26T20:52:29.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.214+0000 7fb2b6739640 1 -- 172.21.15.73:0/1657984149 wait complete. 2024-04-26T20:52:29.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.215+0000 7fb2b6739640 1 Processor -- start 2024-04-26T20:52:29.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.216+0000 7fb2b6739640 1 -- start start 2024-04-26T20:52:29.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.216+0000 7fb2b6739640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2b00b40d0 0x7fb2b01520f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:29.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.216+0000 7fb2b6739640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb2b00d1a00 0x7fb2b00c86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:29.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.216+0000 7fb2b6739640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb2b00d2320 0x7fb2b014f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:29.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.216+0000 7fb2b6739640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b0162b30 con 0x7fb2b00b40d0 2024-04-26T20:52:29.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.216+0000 7fb2b6739640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b00b5d60 con 0x7fb2b00d2320 2024-04-26T20:52:29.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.216+0000 7fb2b6739640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fb2b0162cb0 con 0x7fb2b00d1a00 2024-04-26T20:52:29.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.216+0000 7fb2affff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2b00b40d0 0x7fb2b01520f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:29.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.216+0000 7fb2affff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2b00b40d0 0x7fb2b01520f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38298/0 (socket says 172.21.15.73:38298) 2024-04-26T20:52:29.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.216+0000 7fb2affff640 1 -- 172.21.15.73:0/1643518950 learned_addr learned my addr 172.21.15.73:0/1643518950 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:29.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.216+0000 7fb2b4cd6640 1 --2- 172.21.15.73:0/1643518950 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb2b00d2320 0x7fb2b014f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:29.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.217+0000 7fb2af7fe640 1 --2- 172.21.15.73:0/1643518950 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb2b00d1a00 0x7fb2b00c86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:29.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.217+0000 7fb2affff640 1 -- 172.21.15.73:0/1643518950 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb2b00d1a00 msgr2=0x7fb2b00c86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:29.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.217+0000 7fb2affff640 1 --2- 172.21.15.73:0/1643518950 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb2b00d1a00 0x7fb2b00c86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:29.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.217+0000 7fb2affff640 1 -- 172.21.15.73:0/1643518950 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb2b00d2320 msgr2=0x7fb2b014f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:29.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.217+0000 7fb2affff640 1 --2- 172.21.15.73:0/1643518950 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb2b00d2320 0x7fb2b014f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:29.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.217+0000 7fb2affff640 1 -- 172.21.15.73:0/1643518950 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2b00b8c40 con 0x7fb2b00b40d0 2024-04-26T20:52:29.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.217+0000 7fb2b4cd6640 1 --2- 172.21.15.73:0/1643518950 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb2b00d2320 0x7fb2b014f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:52:29.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.217+0000 7fb2af7fe640 1 --2- 172.21.15.73:0/1643518950 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb2b00d1a00 0x7fb2b00c86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:52:29.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.217+0000 7fb2affff640 1 --2- 172.21.15.73:0/1643518950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2b00b40d0 0x7fb2b01520f0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fb2a000e7e0 tx=0x7fb2a000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:29.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.217+0000 7fb2acff9640 1 -- 172.21.15.73:0/1643518950 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a000f040 con 0x7fb2b00b40d0 2024-04-26T20:52:29.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.217+0000 7fb2acff9640 1 -- 172.21.15.73:0/1643518950 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2a0004030 con 0x7fb2b00b40d0 2024-04-26T20:52:29.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.217+0000 7fb2b6739640 1 -- 172.21.15.73:0/1643518950 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2b00b8f20 con 0x7fb2b00b40d0 2024-04-26T20:52:29.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.217+0000 7fb2acff9640 1 -- 172.21.15.73:0/1643518950 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a0004400 con 0x7fb2b00b40d0 2024-04-26T20:52:29.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.217+0000 7fb2b6739640 1 -- 172.21.15.73:0/1643518950 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2b00b94e0 con 0x7fb2b00b40d0 2024-04-26T20:52:29.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.218+0000 7fb2b6739640 1 -- 172.21.15.73:0/1643518950 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2b00694f0 con 0x7fb2b00b40d0 2024-04-26T20:52:29.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.218+0000 7fb2acff9640 1 -- 172.21.15.73:0/1643518950 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb2a000c6d0 con 0x7fb2b00b40d0 2024-04-26T20:52:29.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.219+0000 7fb2acff9640 1 --2- 172.21.15.73:0/1643518950 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb29c06c8f0 0x7fb29c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:29.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.219+0000 7fb2af7fe640 1 --2- 172.21.15.73:0/1643518950 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb29c06c8f0 0x7fb29c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:29.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.219+0000 7fb2acff9640 1 -- 172.21.15.73:0/1643518950 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7fb2a008bb80 con 0x7fb2b00b40d0 2024-04-26T20:52:29.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.219+0000 7fb2af7fe640 1 --2- 172.21.15.73:0/1643518950 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb29c06c8f0 0x7fb29c06edb0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb2a4034000 tx=0x7fb2a402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:29.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.223+0000 7fb2acff9640 1 -- 172.21.15.73:0/1643518950 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb2a0017020 con 0x7fb2b00b40d0 2024-04-26T20:52:29.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.468+0000 7fb2b6739640 1 -- 172.21.15.73:0/1643518950 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fb2b006d4b0 con 0x7fb2b00b40d0 2024-04-26T20:52:29.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.470+0000 7fb2acff9640 1 -- 172.21.15.73:0/1643518950 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) ==== 190+0+147 (secure 0 0 0) 0x7fb2a0059d90 con 0x7fb2b00b40d0 2024-04-26T20:52:29.472 INFO:teuthology.orchestra.run.smithi073.stdout:[client.0] 2024-04-26T20:52:29.472 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQBUEyxmch+BMBAAap3kvW3+/FlCuAkhRtbWxQ== 2024-04-26T20:52:29.472 INFO:teuthology.orchestra.run.smithi073.stdout: caps mds = "allow" 2024-04-26T20:52:29.472 INFO:teuthology.orchestra.run.smithi073.stdout: caps mon = "allow r" 2024-04-26T20:52:29.472 INFO:teuthology.orchestra.run.smithi073.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-26T20:52:29.473 INFO:teuthology.orchestra.run.smithi073.stderr:updated caps for client.0 2024-04-26T20:52:29.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.472+0000 7fb2b6739640 1 -- 172.21.15.73:0/1643518950 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb29c06c8f0 msgr2=0x7fb29c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:29.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.472+0000 7fb2b6739640 1 --2- 172.21.15.73:0/1643518950 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb29c06c8f0 0x7fb29c06edb0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb2a4034000 tx=0x7fb2a402e000 comp rx=0 tx=0).stop 2024-04-26T20:52:29.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.472+0000 7fb2b6739640 1 -- 172.21.15.73:0/1643518950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2b00b40d0 msgr2=0x7fb2b01520f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:29.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.472+0000 7fb2b6739640 1 --2- 172.21.15.73:0/1643518950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2b00b40d0 0x7fb2b01520f0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fb2a000e7e0 tx=0x7fb2a000eca0 comp rx=0 tx=0).stop 2024-04-26T20:52:29.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.472+0000 7fb2b6739640 1 -- 172.21.15.73:0/1643518950 shutdown_connections 2024-04-26T20:52:29.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.472+0000 7fb2b6739640 1 --2- 172.21.15.73:0/1643518950 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb29c06c8f0 0x7fb29c06edb0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:29.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.472+0000 7fb2b6739640 1 --2- 172.21.15.73:0/1643518950 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb2b00d2320 0x7fb2b014f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:29.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.472+0000 7fb2b6739640 1 --2- 172.21.15.73:0/1643518950 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb2b00d1a00 0x7fb2b00c86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:29.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.472+0000 7fb2b6739640 1 --2- 172.21.15.73:0/1643518950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2b00b40d0 0x7fb2b01520f0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:29.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.472+0000 7fb2b6739640 1 -- 172.21.15.73:0/1643518950 >> 172.21.15.73:0/1643518950 conn(0x7fb2b00d4fd0 msgr2=0x7fb2b0161410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:29.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.473+0000 7fb2b6739640 1 -- 172.21.15.73:0/1643518950 shutdown_connections 2024-04-26T20:52:29.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.473+0000 7fb2b6739640 1 -- 172.21.15.73:0/1643518950 wait complete. 2024-04-26T20:52:29.485 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:52:29.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.579+0000 7f311eb40640 1 Processor -- start 2024-04-26T20:52:29.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.580+0000 7f311eb40640 1 -- start start 2024-04-26T20:52:29.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.580+0000 7f311eb40640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f311815f2d0 0x7f31181604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:29.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.580+0000 7f311eb40640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f311806fea0 0x7f31180ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:29.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.580+0000 7f311eb40640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3118072f70 con 0x7f3118072700 2024-04-26T20:52:29.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.580+0000 7f311eb40640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f3118074010 con 0x7f311806fea0 2024-04-26T20:52:29.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.580+0000 7f311eb40640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f3118073250 con 0x7f311815f2d0 2024-04-26T20:52:29.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.580+0000 7f311d0dd640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f311806fea0 0x7f31180ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:29.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.581+0000 7f311d0dd640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f311806fea0 0x7f31180ce7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48434/0 (socket says 172.21.15.73:48434) 2024-04-26T20:52:29.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.581+0000 7f311d0dd640 1 -- 172.21.15.73:0/2562348776 learned_addr learned my addr 172.21.15.73:0/2562348776 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:29.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.581+0000 7f311d0dd640 1 -- 172.21.15.73:0/2562348776 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f311815f2d0 msgr2=0x7f31181604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:29.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.581+0000 7f311d0dd640 1 --2- 172.21.15.73:0/2562348776 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f311815f2d0 0x7f31181604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:29.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.581+0000 7f311d0dd640 1 -- 172.21.15.73:0/2562348776 >> v1:172.21.15.120:6789/0 conn(0x7f3118072700 legacy=0x7f311806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:29.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.581+0000 7f311d0dd640 1 -- 172.21.15.73:0/2562348776 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3118073530 con 0x7f311806fea0 2024-04-26T20:52:29.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.581+0000 7f311d0dd640 1 --2- 172.21.15.73:0/2562348776 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f311806fea0 0x7f31180ce7b0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f3110009e50 tx=0x7f311002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3217857a844f8d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:29.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.582+0000 7f31177fe640 1 -- 172.21.15.73:0/2562348776 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3110036070 con 0x7f311806fea0 2024-04-26T20:52:29.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.582+0000 7f31177fe640 1 -- 172.21.15.73:0/2562348776 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3110031600 con 0x7f311806fea0 2024-04-26T20:52:29.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.582+0000 7f31177fe640 1 -- 172.21.15.73:0/2562348776 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31100319d0 con 0x7f311806fea0 2024-04-26T20:52:29.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.582+0000 7f311eb40640 1 -- 172.21.15.73:0/2562348776 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f311806fea0 msgr2=0x7f31180ce7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:29.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.582+0000 7f311eb40640 1 --2- 172.21.15.73:0/2562348776 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f311806fea0 0x7f31180ce7b0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f3110009e50 tx=0x7f311002a6c0 comp rx=0 tx=0).stop 2024-04-26T20:52:29.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.582+0000 7f311eb40640 1 -- 172.21.15.73:0/2562348776 shutdown_connections 2024-04-26T20:52:29.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.582+0000 7f311eb40640 1 --2- 172.21.15.73:0/2562348776 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f311806fea0 0x7f31180ce7b0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:29.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.582+0000 7f311eb40640 1 --2- 172.21.15.73:0/2562348776 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f311815f2d0 0x7f31181604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:29.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.582+0000 7f311eb40640 1 -- 172.21.15.73:0/2562348776 >> 172.21.15.73:0/2562348776 conn(0x7f31180d0be0 msgr2=0x7f31180d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:29.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.582+0000 7f311eb40640 1 -- 172.21.15.73:0/2562348776 shutdown_connections 2024-04-26T20:52:29.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.582+0000 7f311eb40640 1 -- 172.21.15.73:0/2562348776 wait complete. 2024-04-26T20:52:29.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.583+0000 7f311eb40640 1 Processor -- start 2024-04-26T20:52:29.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.583+0000 7f311eb40640 1 -- start start 2024-04-26T20:52:29.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.584+0000 7f311eb40640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f311806fea0 0x7f31181f9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:29.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.584+0000 7f311eb40640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3118072700 0x7f31181f8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:29.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.584+0000 7f311eb40640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f311815f2d0 0x7f311806e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:29.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.584+0000 7f311eb40640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f311815f6b0 con 0x7f3118072700 2024-04-26T20:52:29.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.584+0000 7f311eb40640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3118073af0 con 0x7f311806fea0 2024-04-26T20:52:29.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.584+0000 7f311eb40640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f3118072b60 con 0x7f311815f2d0 2024-04-26T20:52:29.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.584+0000 7f3117fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3118072700 0x7f31181f8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:29.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.584+0000 7f3117fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3118072700 0x7f31181f8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48438/0 (socket says 172.21.15.73:48438) 2024-04-26T20:52:29.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.584+0000 7f3117fff640 1 -- 172.21.15.73:0/2526067346 learned_addr learned my addr 172.21.15.73:0/2526067346 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:29.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.584+0000 7f311d0dd640 1 --2- 172.21.15.73:0/2526067346 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f311815f2d0 0x7f311806e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:29.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.584+0000 7f311c8dc640 1 --2- 172.21.15.73:0/2526067346 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f311806fea0 0x7f31181f9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:29.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.585+0000 7f3117fff640 1 -- 172.21.15.73:0/2526067346 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f311815f2d0 msgr2=0x7f311806e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:29.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.585+0000 7f3117fff640 1 --2- 172.21.15.73:0/2526067346 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f311815f2d0 0x7f311806e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:29.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.585+0000 7f3117fff640 1 -- 172.21.15.73:0/2526067346 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f311806fea0 msgr2=0x7f31181f9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:29.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.585+0000 7f3117fff640 1 --2- 172.21.15.73:0/2526067346 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f311806fea0 0x7f31181f9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:29.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.585+0000 7f3117fff640 1 -- 172.21.15.73:0/2526067346 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31180c9e70 con 0x7f3118072700 2024-04-26T20:52:29.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.585+0000 7f3117fff640 1 --2- 172.21.15.73:0/2526067346 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3118072700 0x7f31181f8450 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f310c00c410 tx=0x7f310c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:29.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.585+0000 7f31157fa640 1 -- 172.21.15.73:0/2526067346 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310c017070 con 0x7f3118072700 2024-04-26T20:52:29.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.585+0000 7f31157fa640 1 -- 172.21.15.73:0/2526067346 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f310c0035f0 con 0x7f3118072700 2024-04-26T20:52:29.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.585+0000 7f311eb40640 1 -- 172.21.15.73:0/2526067346 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31180ca150 con 0x7f3118072700 2024-04-26T20:52:29.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.585+0000 7f31157fa640 1 -- 172.21.15.73:0/2526067346 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310c003a20 con 0x7f3118072700 2024-04-26T20:52:29.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.585+0000 7f311eb40640 1 -- 172.21.15.73:0/2526067346 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31180ca710 con 0x7f3118072700 2024-04-26T20:52:29.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.586+0000 7f311eb40640 1 -- 172.21.15.73:0/2526067346 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31180764d0 con 0x7f3118072700 2024-04-26T20:52:29.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.587+0000 7f31157fa640 1 -- 172.21.15.73:0/2526067346 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f310c01a070 con 0x7f3118072700 2024-04-26T20:52:29.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.587+0000 7f31157fa640 1 --2- 172.21.15.73:0/2526067346 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f310406c8f0 0x7f310406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:29.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.587+0000 7f31157fa640 1 -- 172.21.15.73:0/2526067346 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f310c08b0b0 con 0x7f3118072700 2024-04-26T20:52:29.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.587+0000 7f311c8dc640 1 --2- 172.21.15.73:0/2526067346 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f310406c8f0 0x7f310406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:29.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.587+0000 7f311c8dc640 1 --2- 172.21.15.73:0/2526067346 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f310406c8f0 0x7f310406edb0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f31080098e0 tx=0x7f3108009060 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:29.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.591+0000 7f31157fa640 1 -- 172.21.15.73:0/2526067346 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f31180764d0 con 0x7f3118072700 2024-04-26T20:52:29.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.807+0000 7f311eb40640 1 -- 172.21.15.73:0/2526067346 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f31180694f0 con 0x7f3118072700 2024-04-26T20:52:29.809 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.807+0000 7f31157fa640 1 -- 172.21.15.73:0/2526067346 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5181 (secure 0 0 0) 0x7f310c058d00 con 0x7f3118072700 2024-04-26T20:52:29.810 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:52:29.810 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5096,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6834/528576665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":528576665},{"type":"v1","addr":"172.21.15.120:6834","nonce":528576665}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":5099,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/824086075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6833","nonce":824086075},{"type":"v1","addr":"172.21.15.120:6835","nonce":824086075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":5853,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/1515733826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1515733826},{"type":"v1","addr":"172.21.15.73:6835","nonce":1515733826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:52:20.754736+0000","modified":"2024-04-26T20:52:22.752295+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5856},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5856":{"gid":5856,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/2770303348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2770303348},{"type":"v1","addr":"172.21.15.73:6837","nonce":2770303348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5856,"qdb_cluster":[5856]},"id":3}]} 2024-04-26T20:52:29.810 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 26 2024-04-26T20:52:29.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.809+0000 7f311eb40640 1 -- 172.21.15.73:0/2526067346 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f310406c8f0 msgr2=0x7f310406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:29.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.809+0000 7f311eb40640 1 --2- 172.21.15.73:0/2526067346 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f310406c8f0 0x7f310406edb0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f31080098e0 tx=0x7f3108009060 comp rx=0 tx=0).stop 2024-04-26T20:52:29.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.809+0000 7f311eb40640 1 -- 172.21.15.73:0/2526067346 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3118072700 msgr2=0x7f31181f8450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:29.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.809+0000 7f311eb40640 1 --2- 172.21.15.73:0/2526067346 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3118072700 0x7f31181f8450 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f310c00c410 tx=0x7f310c00c8d0 comp rx=0 tx=0).stop 2024-04-26T20:52:29.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.809+0000 7f311eb40640 1 -- 172.21.15.73:0/2526067346 shutdown_connections 2024-04-26T20:52:29.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.810+0000 7f311eb40640 1 --2- 172.21.15.73:0/2526067346 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3118072700 0x7f31181f8450 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:29.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.810+0000 7f311eb40640 1 --2- 172.21.15.73:0/2526067346 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f310406c8f0 0x7f310406edb0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:29.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.810+0000 7f311eb40640 1 --2- 172.21.15.73:0/2526067346 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f311815f2d0 0x7f311806e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:29.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.810+0000 7f311eb40640 1 --2- 172.21.15.73:0/2526067346 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f311806fea0 0x7f31181f9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:29.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.810+0000 7f311eb40640 1 -- 172.21.15.73:0/2526067346 >> 172.21.15.73:0/2526067346 conn(0x7f31180d0be0 msgr2=0x7f311815df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:29.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.810+0000 7f311eb40640 1 -- 172.21.15.73:0/2526067346 shutdown_connections 2024-04-26T20:52:29.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:29.810+0000 7f311eb40640 1 -- 172.21.15.73:0/2526067346 wait complete. 2024-04-26T20:52:30.227 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T20:52:20.754736+0000', 'modified': '2024-04-26T20:52:22.752295+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5856}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5856': {'gid': 5856, 'name': 'c', 'rank': 0, 'incarnation': 24, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.73:6837/2770303348', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6836', 'nonce': 2770303348}, {'type': 'v1', 'addr': '172.21.15.73:6837', 'nonce': 2770303348}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5856, 'qdb_cluster': [5856]} 2024-04-26T20:52:30.227 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T20:52:30.227 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-26T20:52:30.326 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.324+0000 7f1d769c6640 1 Processor -- start 2024-04-26T20:52:30.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.324+0000 7f1d769c6640 1 -- start start 2024-04-26T20:52:30.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.325+0000 7f1d769c6640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f1d701606f0 0x7f1d700ce890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:30.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.325+0000 7f1d769c6640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1d700d38f0 0x7f1d700d09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:30.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.325+0000 7f1d769c6640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1d700d10f0 0x7f1d700d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:30.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.325+0000 7f1d769c6640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f1d700cdb10 con 0x7f1d700d38f0 2024-04-26T20:52:30.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.325+0000 7f1d769c6640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f1d700cddb0 con 0x7f1d701606f0 2024-04-26T20:52:30.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.325+0000 7f1d769c6640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f1d700ce090 con 0x7f1d700d10f0 2024-04-26T20:52:30.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.325+0000 7f1d6f7fe640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1d700d38f0 0x7f1d700d09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:30.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.325+0000 7f1d6f7fe640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1d700d38f0 0x7f1d700d09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48440/0 (socket says 172.21.15.73:48440) 2024-04-26T20:52:30.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.325+0000 7f1d6f7fe640 1 -- 172.21.15.73:0/4123880441 learned_addr learned my addr 172.21.15.73:0/4123880441 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:30.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.325+0000 7f1d74f63640 1 --2- 172.21.15.73:0/4123880441 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1d700d10f0 0x7f1d700d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:30.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.325+0000 7f1d6ffff640 1 --2- 172.21.15.73:0/4123880441 >> v2:172.21.15.120:3300/0 conn(0x7f1d701606f0 0x7f1d700ce890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:30.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.326+0000 7f1d6f7fe640 1 -- 172.21.15.73:0/4123880441 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1d700d10f0 msgr2=0x7f1d700d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:30.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.326+0000 7f1d6f7fe640 1 --2- 172.21.15.73:0/4123880441 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1d700d10f0 0x7f1d700d1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:30.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.326+0000 7f1d6f7fe640 1 -- 172.21.15.73:0/4123880441 >> v2:172.21.15.120:3300/0 conn(0x7f1d701606f0 msgr2=0x7f1d700ce890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:30.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.326+0000 7f1d6f7fe640 1 --2- 172.21.15.73:0/4123880441 >> v2:172.21.15.120:3300/0 conn(0x7f1d701606f0 0x7f1d700ce890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:30.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.326+0000 7f1d6f7fe640 1 -- 172.21.15.73:0/4123880441 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d700ce370 con 0x7f1d700d38f0 2024-04-26T20:52:30.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.326+0000 7f1d6ffff640 1 --2- 172.21.15.73:0/4123880441 >> v2:172.21.15.120:3300/0 conn(0x7f1d701606f0 0x7f1d700ce890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:52:30.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.326+0000 7f1d74f63640 1 --2- 172.21.15.73:0/4123880441 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1d700d10f0 0x7f1d700d1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:52:30.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.326+0000 7f1d6f7fe640 1 --2- 172.21.15.73:0/4123880441 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d700d38f0 0x7f1d700d09a0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f1d70073380 tx=0x7f1d6402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53d86c03fbd6a1c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:30.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.326+0000 7f1d6effd640 1 -- 172.21.15.73:0/4123880441 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d64037070 con 0x7f1d700d38f0 2024-04-26T20:52:30.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.326+0000 7f1d6effd640 1 -- 172.21.15.73:0/4123880441 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d64032440 con 0x7f1d700d38f0 2024-04-26T20:52:30.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.326+0000 7f1d6effd640 1 -- 172.21.15.73:0/4123880441 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d64032810 con 0x7f1d700d38f0 2024-04-26T20:52:30.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.327+0000 7f1d769c6640 1 -- 172.21.15.73:0/4123880441 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d700d38f0 msgr2=0x7f1d700d09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:30.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.327+0000 7f1d769c6640 1 --2- 172.21.15.73:0/4123880441 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d700d38f0 0x7f1d700d09a0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f1d70073380 tx=0x7f1d6402a140 comp rx=0 tx=0).stop 2024-04-26T20:52:30.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.327+0000 7f1d769c6640 1 -- 172.21.15.73:0/4123880441 shutdown_connections 2024-04-26T20:52:30.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.327+0000 7f1d769c6640 1 --2- 172.21.15.73:0/4123880441 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1d700d10f0 0x7f1d700d1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:30.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.327+0000 7f1d769c6640 1 --2- 172.21.15.73:0/4123880441 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d700d38f0 0x7f1d700d09a0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:30.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.327+0000 7f1d769c6640 1 --2- 172.21.15.73:0/4123880441 >> v2:172.21.15.120:3300/0 conn(0x7f1d701606f0 0x7f1d700ce890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:30.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.327+0000 7f1d769c6640 1 -- 172.21.15.73:0/4123880441 >> 172.21.15.73:0/4123880441 conn(0x7f1d700d2dd0 msgr2=0x7f1d700d31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:30.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.327+0000 7f1d769c6640 1 -- 172.21.15.73:0/4123880441 shutdown_connections 2024-04-26T20:52:30.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.327+0000 7f1d769c6640 1 -- 172.21.15.73:0/4123880441 wait complete. 2024-04-26T20:52:30.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.328+0000 7f1d769c6640 1 Processor -- start 2024-04-26T20:52:30.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.328+0000 7f1d769c6640 1 -- start start 2024-04-26T20:52:30.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.329+0000 7f1d769c6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d700d10f0 0x7f1d70158450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:30.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.329+0000 7f1d769c6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d700d38f0 0x7f1d700cba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:30.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.329+0000 7f1d769c6640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1d701606f0 0x7f1d700c8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:30.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.329+0000 7f1d769c6640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1d700b5c20 con 0x7f1d700d38f0 2024-04-26T20:52:30.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.329+0000 7f1d769c6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1d700d1cd0 con 0x7f1d700d10f0 2024-04-26T20:52:30.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.329+0000 7f1d769c6640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f1d700b5da0 con 0x7f1d701606f0 2024-04-26T20:52:30.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.329+0000 7f1d6f7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d700d38f0 0x7f1d700cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:30.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.329+0000 7f1d6f7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d700d38f0 0x7f1d700cba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48446/0 (socket says 172.21.15.73:48446) 2024-04-26T20:52:30.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.329+0000 7f1d6f7fe640 1 -- 172.21.15.73:0/2252248098 learned_addr learned my addr 172.21.15.73:0/2252248098 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:30.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.329+0000 7f1d6ffff640 1 --2- 172.21.15.73:0/2252248098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d700d10f0 0x7f1d70158450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:30.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.329+0000 7f1d74f63640 1 --2- 172.21.15.73:0/2252248098 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1d701606f0 0x7f1d700c8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:30.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.329+0000 7f1d6f7fe640 1 -- 172.21.15.73:0/2252248098 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1d701606f0 msgr2=0x7f1d700c8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:30.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.329+0000 7f1d6f7fe640 1 --2- 172.21.15.73:0/2252248098 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1d701606f0 0x7f1d700c8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:30.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.329+0000 7f1d6f7fe640 1 -- 172.21.15.73:0/2252248098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d700d10f0 msgr2=0x7f1d70158450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:30.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.329+0000 7f1d6f7fe640 1 --2- 172.21.15.73:0/2252248098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d700d10f0 0x7f1d70158450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:30.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.329+0000 7f1d6f7fe640 1 -- 172.21.15.73:0/2252248098 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d70158990 con 0x7f1d700d38f0 2024-04-26T20:52:30.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.329+0000 7f1d6ffff640 1 --2- 172.21.15.73:0/2252248098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d700d10f0 0x7f1d70158450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:52:30.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.330+0000 7f1d6f7fe640 1 --2- 172.21.15.73:0/2252248098 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d700d38f0 0x7f1d700cba60 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f1d640097c0 tx=0x7f1d640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:30.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.330+0000 7f1d6cff9640 1 -- 172.21.15.73:0/2252248098 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d64041020 con 0x7f1d700d38f0 2024-04-26T20:52:30.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.330+0000 7f1d6cff9640 1 -- 172.21.15.73:0/2252248098 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d640306c0 con 0x7f1d700d38f0 2024-04-26T20:52:30.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.330+0000 7f1d769c6640 1 -- 172.21.15.73:0/2252248098 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d700cbfa0 con 0x7f1d700d38f0 2024-04-26T20:52:30.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.330+0000 7f1d6cff9640 1 -- 172.21.15.73:0/2252248098 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d64037040 con 0x7f1d700d38f0 2024-04-26T20:52:30.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.330+0000 7f1d769c6640 1 -- 172.21.15.73:0/2252248098 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d700caf80 con 0x7f1d700d38f0 2024-04-26T20:52:30.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.332+0000 7f1d6cff9640 1 -- 172.21.15.73:0/2252248098 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1d64044020 con 0x7f1d700d38f0 2024-04-26T20:52:30.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.332+0000 7f1d6cff9640 1 --2- 172.21.15.73:0/2252248098 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1d5c06c8f0 0x7f1d5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:30.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.332+0000 7f1d6cff9640 1 -- 172.21.15.73:0/2252248098 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f1d640ac4e0 con 0x7f1d700d38f0 2024-04-26T20:52:30.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.332+0000 7f1d6ffff640 1 --2- 172.21.15.73:0/2252248098 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1d5c06c8f0 0x7f1d5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:30.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.333+0000 7f1d6ffff640 1 --2- 172.21.15.73:0/2252248098 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1d5c06c8f0 0x7f1d5c06edb0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1d700d19f0 tx=0x7f1d60008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:30.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.333+0000 7f1d797c1740 1 Processor -- start 2024-04-26T20:52:30.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.333+0000 7f1d797c1740 1 -- start start 2024-04-26T20:52:30.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.333+0000 7f1d797c1740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6b6caf440 0x55e6b6caf820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:30.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.333+0000 7f1d797c1740 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55e6b6cccce0 0x55e6b6ccd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:30.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.333+0000 7f1d797c1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6b6c85120 0x55e6b6ccb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:30.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.333+0000 7f1d797c1740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55e6b6be9740 con 0x55e6b6caf440 2024-04-26T20:52:30.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.333+0000 7f1d797c1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e6b6b853a0 con 0x55e6b6c85120 2024-04-26T20:52:30.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.333+0000 7f1d797c1740 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x55e6b6be5700 con 0x55e6b6cccce0 2024-04-26T20:52:30.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.333+0000 7f1d6ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6b6caf440 0x55e6b6caf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:30.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.333+0000 7f1d6ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6b6caf440 0x55e6b6caf820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48458/0 (socket says 172.21.15.73:48458) 2024-04-26T20:52:30.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.333+0000 7f1d6ffff640 1 -- 172.21.15.73:0/1980845798 learned_addr learned my addr 172.21.15.73:0/1980845798 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:30.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.334+0000 7f1d74f63640 1 --2- 172.21.15.73:0/1980845798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6b6c85120 0x55e6b6ccb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:30.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.334+0000 7f1d6f7fe640 1 --2- 172.21.15.73:0/1980845798 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55e6b6cccce0 0x55e6b6ccd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:30.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.334+0000 7f1d6ffff640 1 -- 172.21.15.73:0/1980845798 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55e6b6cccce0 msgr2=0x55e6b6ccd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:30.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.334+0000 7f1d6ffff640 1 --2- 172.21.15.73:0/1980845798 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55e6b6cccce0 0x55e6b6ccd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:30.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.334+0000 7f1d6ffff640 1 -- 172.21.15.73:0/1980845798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6b6c85120 msgr2=0x55e6b6ccb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:30.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.334+0000 7f1d6ffff640 1 --2- 172.21.15.73:0/1980845798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6b6c85120 0x55e6b6ccb3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:30.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.334+0000 7f1d6ffff640 1 -- 172.21.15.73:0/1980845798 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6b6ba1750 con 0x55e6b6caf440 2024-04-26T20:52:30.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.334+0000 7f1d6ffff640 1 --2- 172.21.15.73:0/1980845798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6b6caf440 0x55e6b6caf820 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f1d6000fa50 tx=0x7f1d60012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb11d77b546fdcfd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:30.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.334+0000 7f1d55ffb640 1 -- 172.21.15.73:0/1980845798 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d6001b050 con 0x55e6b6caf440 2024-04-26T20:52:30.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.334+0000 7f1d55ffb640 1 -- 172.21.15.73:0/1980845798 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d60025070 con 0x55e6b6caf440 2024-04-26T20:52:30.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.334+0000 7f1d55ffb640 1 -- 172.21.15.73:0/1980845798 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d60014de0 con 0x55e6b6caf440 2024-04-26T20:52:30.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.335+0000 7f1d797c1740 1 -- 172.21.15.73:0/1980845798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6b6caf440 msgr2=0x55e6b6caf820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:30.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.335+0000 7f1d797c1740 1 --2- 172.21.15.73:0/1980845798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6b6caf440 0x55e6b6caf820 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f1d6000fa50 tx=0x7f1d60012000 comp rx=0 tx=0).stop 2024-04-26T20:52:30.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.335+0000 7f1d797c1740 1 -- 172.21.15.73:0/1980845798 shutdown_connections 2024-04-26T20:52:30.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.335+0000 7f1d797c1740 1 --2- 172.21.15.73:0/1980845798 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6b6c85120 0x55e6b6ccb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:30.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.335+0000 7f1d797c1740 1 --2- 172.21.15.73:0/1980845798 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55e6b6cccce0 0x55e6b6ccd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:30.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.335+0000 7f1d797c1740 1 --2- 172.21.15.73:0/1980845798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6b6caf440 0x55e6b6caf820 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:30.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.335+0000 7f1d797c1740 1 -- 172.21.15.73:0/1980845798 >> 172.21.15.73:0/1980845798 conn(0x55e6b6c81a50 msgr2=0x55e6b6cca250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:30.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.335+0000 7f1d797c1740 1 -- 172.21.15.73:0/1980845798 shutdown_connections 2024-04-26T20:52:30.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.335+0000 7f1d797c1740 1 -- 172.21.15.73:0/1980845798 wait complete. 2024-04-26T20:52:30.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.335+0000 7f1d797c1740 1 -- start start 2024-04-26T20:52:30.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.335+0000 7f1d797c1740 1 Processor -- start 2024-04-26T20:52:30.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.336+0000 7f1d797c1740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6b6caf440 0x55e6b6cdad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:30.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.336+0000 7f1d797c1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6b6cccce0 0x55e6b6c7ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:30.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.336+0000 7f1d797c1740 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55e6b6caaf50 0x55e6b6cb0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:30.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.336+0000 7f1d797c1740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55e6b6c85ec0 con 0x55e6b6caf440 2024-04-26T20:52:30.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.336+0000 7f1d797c1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e6b6b70fe0 con 0x55e6b6cccce0 2024-04-26T20:52:30.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.336+0000 7f1d797c1740 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x55e6b6c7f980 con 0x55e6b6caaf50 2024-04-26T20:52:30.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.336+0000 7f1d797c1740 10 client.0 _ll_register_callbacks cb 0x55e6b6b87150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:52:30.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.336+0000 7f1d6ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6b6caf440 0x55e6b6cdad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:30.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.336+0000 7f1d6ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6b6caf440 0x55e6b6cdad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48472/0 (socket says 172.21.15.73:48472) 2024-04-26T20:52:30.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.336+0000 7f1d6ffff640 1 -- 172.21.15.73:0/225075967 learned_addr learned my addr 172.21.15.73:0/225075967 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:30.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.336+0000 7f1d74f63640 1 --2- 172.21.15.73:0/225075967 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55e6b6caaf50 0x55e6b6cb0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:30.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.336+0000 7f1d6f7fe640 1 --2- 172.21.15.73:0/225075967 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6b6cccce0 0x55e6b6c7ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:30.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.337+0000 7f1d6ffff640 1 -- 172.21.15.73:0/225075967 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55e6b6caaf50 msgr2=0x55e6b6cb0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:30.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.337+0000 7f1d6ffff640 1 --2- 172.21.15.73:0/225075967 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55e6b6caaf50 0x55e6b6cb0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:30.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.337+0000 7f1d6ffff640 1 -- 172.21.15.73:0/225075967 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6b6cccce0 msgr2=0x55e6b6c7ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:30.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.337+0000 7f1d6ffff640 1 --2- 172.21.15.73:0/225075967 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6b6cccce0 0x55e6b6c7ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:30.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.337+0000 7f1d6ffff640 1 -- 172.21.15.73:0/225075967 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6b68fb2d0 con 0x55e6b6caf440 2024-04-26T20:52:30.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.337+0000 7f1d6ffff640 1 --2- 172.21.15.73:0/225075967 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6b6caf440 0x55e6b6cdad40 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f1d60007390 tx=0x7f1d60004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e35c9fa91e7d5d34 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:30.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.337+0000 7f1d4effd640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-04-26T20:52:30.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.337+0000 7f1d4effd640 1 -- 172.21.15.73:0/225075967 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d60020c80 con 0x55e6b6caf440 2024-04-26T20:52:30.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.337+0000 7f1d4effd640 1 -- 172.21.15.73:0/225075967 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f1d60034470 con 0x55e6b6caf440 2024-04-26T20:52:30.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.337+0000 7f1d797c1740 1 -- 172.21.15.73:0/225075967 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d60025070 con 0x55e6b6caf440 2024-04-26T20:52:30.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.337+0000 7f1d4effd640 1 -- 172.21.15.73:0/225075967 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1d6001b020 con 0x55e6b6caf440 2024-04-26T20:52:30.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.337+0000 7f1d797c1740 10 client.6024 fetch_fsmap learned FSMap version 26 2024-04-26T20:52:30.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.337+0000 7f1d797c1740 1 -- 172.21.15.73:0/225075967 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x55e6b69c56b0 con 0x55e6b6caf440 2024-04-26T20:52:30.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.337+0000 7f1d4effd640 1 -- 172.21.15.73:0/225075967 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d60025070 con 0x55e6b6caf440 2024-04-26T20:52:30.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.337+0000 7f1d4effd640 1 -- 172.21.15.73:0/225075967 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7f1d60034670 con 0x55e6b6caf440 2024-04-26T20:52:30.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.337+0000 7f1d797c1740 10 client.6024 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T20:52:30.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.337+0000 7f1d797c1740 10 client.6024 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T20:52:30.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.337+0000 7f1d797c1740 20 client.6024 populate_metadata read hostname 'smithi073' 2024-04-26T20:52:30.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.337+0000 7f1d797c1740 1 --2- 172.21.15.73:0/225075967 >> [v2:172.21.15.73:6836/2770303348,v1:172.21.15.73:6837/2770303348] conn(0x55e6b6cadc90 0x55e6b6cae070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:30.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.337+0000 7f1d797c1740 4 client.6024 mds_command: new command op to 5856 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:52:30.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.337+0000 7f1d797c1740 1 -- 172.21.15.73:0/225075967 --> [v2:172.21.15.73:6836/2770303348,v1:172.21.15.73:6837/2770303348] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6b6bf7350 con 0x55e6b6cadc90 2024-04-26T20:52:30.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.337+0000 7f1d6f7fe640 1 --2- 172.21.15.73:0/225075967 >> [v2:172.21.15.73:6836/2770303348,v1:172.21.15.73:6837/2770303348] conn(0x55e6b6cadc90 0x55e6b6cae070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:30.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.338+0000 7f1d4effd640 10 client.6024 ms_handle_connect on v2:172.21.15.73:6836/2770303348 2024-04-26T20:52:30.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.338+0000 7f1d6f7fe640 1 --2- 172.21.15.73:0/225075967 >> [v2:172.21.15.73:6836/2770303348,v1:172.21.15.73:6837/2770303348] conn(0x55e6b6cadc90 0x55e6b6cae070 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=974f5a0cd8071582 server_cookie=394fc1ec614ad1f3 in_seq=0 out_seq=0 2024-04-26T20:52:30.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.338+0000 7f1d4effd640 1 -- 172.21.15.73:0/225075967 <== mds.0 v2:172.21.15.73:6836/2770303348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e6b6bf7350 con 0x55e6b6cadc90 2024-04-26T20:52:30.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.338+0000 7f1d4effd640 10 client.6024 handle_command_reply: tid=0 2024-04-26T20:52:30.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.338+0000 7f1d797c1740 1 client.6024 shutdown 2024-04-26T20:52:30.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.339+0000 7f1d797c1740 1 -- 172.21.15.73:0/225075967 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6b6caf440 msgr2=0x55e6b6cdad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:30.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.339+0000 7f1d797c1740 1 --2- 172.21.15.73:0/225075967 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6b6caf440 0x55e6b6cdad40 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f1d60007390 tx=0x7f1d60004730 comp rx=0 tx=0).stop 2024-04-26T20:52:30.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.339+0000 7f1d797c1740 1 -- 172.21.15.73:0/225075967 shutdown_connections 2024-04-26T20:52:30.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.339+0000 7f1d797c1740 1 --2- 172.21.15.73:0/225075967 >> [v2:172.21.15.73:6836/2770303348,v1:172.21.15.73:6837/2770303348] conn(0x55e6b6cadc90 0x55e6b6cae070 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:30.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.339+0000 7f1d797c1740 1 --2- 172.21.15.73:0/225075967 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55e6b6caaf50 0x55e6b6cb0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:30.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.339+0000 7f1d797c1740 1 --2- 172.21.15.73:0/225075967 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6b6cccce0 0x55e6b6c7ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:30.348 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.339+0000 7f1d4effd640 0 client.6024 ms_handle_reset on v2:172.21.15.73:6836/2770303348 2024-04-26T20:52:30.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.339+0000 7f1d797c1740 1 --2- 172.21.15.73:0/225075967 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6b6caf440 0x55e6b6cdad40 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:30.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.339+0000 7f1d797c1740 1 -- 172.21.15.73:0/225075967 >> 172.21.15.73:0/225075967 conn(0x55e6b6c85120 msgr2=0x55e6b6ccd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:30.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.339+0000 7f1d797c1740 1 -- 172.21.15.73:0/225075967 shutdown_connections 2024-04-26T20:52:30.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.339+0000 7f1d797c1740 1 -- 172.21.15.73:0/225075967 wait complete. 2024-04-26T20:52:30.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.339+0000 7f1d797c1740 20 client.6024 trim_cache size 0 max 16384 2024-04-26T20:52:30.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.356+0000 7f1d797c1740 1 Processor -- start 2024-04-26T20:52:30.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.357+0000 7f1d797c1740 1 -- start start 2024-04-26T20:52:30.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.357+0000 7f1d797c1740 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55e6b6caaf50 0x55e6b6cae490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:30.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.357+0000 7f1d797c1740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6b6caf440 0x55e6b6cb2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:30.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.357+0000 7f1d797c1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6b6cccce0 0x55e6b6cb3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:30.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.357+0000 7f1d797c1740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1d701fed00 con 0x55e6b6caf440 2024-04-26T20:52:30.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.357+0000 7f1d797c1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1d700c7440 con 0x55e6b6cccce0 2024-04-26T20:52:30.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.357+0000 7f1d797c1740 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f1d7015e9f0 con 0x55e6b6caaf50 2024-04-26T20:52:30.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.357+0000 7f1d6f7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6b6caf440 0x55e6b6cb2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:30.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.357+0000 7f1d6f7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6b6caf440 0x55e6b6cb2c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48476/0 (socket says 172.21.15.73:48476) 2024-04-26T20:52:30.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.357+0000 7f1d6f7fe640 1 -- 172.21.15.73:0/3785897216 learned_addr learned my addr 172.21.15.73:0/3785897216 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:30.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.358+0000 7f1d6f7fe640 1 -- 172.21.15.73:0/3785897216 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55e6b6caaf50 msgr2=0x55e6b6cae490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:30.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.358+0000 7f1d6f7fe640 1 --2- 172.21.15.73:0/3785897216 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55e6b6caaf50 0x55e6b6cae490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:30.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.358+0000 7f1d74f63640 1 --2- 172.21.15.73:0/3785897216 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6b6cccce0 0x55e6b6cb3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:30.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.358+0000 7f1d6f7fe640 1 -- 172.21.15.73:0/3785897216 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6b6cccce0 msgr2=0x55e6b6cb3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:30.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.358+0000 7f1d6f7fe640 1 --2- 172.21.15.73:0/3785897216 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6b6cccce0 0x55e6b6cb3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:30.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.358+0000 7f1d6f7fe640 1 -- 172.21.15.73:0/3785897216 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6b6cca050 con 0x55e6b6caf440 2024-04-26T20:52:30.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.358+0000 7f1d74f63640 1 --2- 172.21.15.73:0/3785897216 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6b6cccce0 0x55e6b6cb3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:52:30.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.358+0000 7f1d6f7fe640 1 --2- 172.21.15.73:0/3785897216 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6b6caf440 0x55e6b6cb2c00 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f1d64079d80 tx=0x7f1d640aa750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad6081247ffdc115 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:30.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.358+0000 7f1d557fa640 1 -- 172.21.15.73:0/3785897216 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d6407a080 con 0x55e6b6caf440 2024-04-26T20:52:30.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.358+0000 7f1d557fa640 1 -- 172.21.15.73:0/3785897216 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d6407e430 con 0x55e6b6caf440 2024-04-26T20:52:30.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.358+0000 7f1d557fa640 1 -- 172.21.15.73:0/3785897216 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d64082c60 con 0x55e6b6caf440 2024-04-26T20:52:30.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.358+0000 7f1d797c1740 1 -- 172.21.15.73:0/3785897216 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6b6caf440 msgr2=0x55e6b6cb2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:30.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.358+0000 7f1d797c1740 1 --2- 172.21.15.73:0/3785897216 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6b6caf440 0x55e6b6cb2c00 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f1d64079d80 tx=0x7f1d640aa750 comp rx=0 tx=0).stop 2024-04-26T20:52:30.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.358+0000 7f1d797c1740 1 -- 172.21.15.73:0/3785897216 shutdown_connections 2024-04-26T20:52:30.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.358+0000 7f1d797c1740 1 --2- 172.21.15.73:0/3785897216 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6b6caf440 0x55e6b6cb2c00 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:30.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.358+0000 7f1d797c1740 1 --2- 172.21.15.73:0/3785897216 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6b6cccce0 0x55e6b6cb3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:30.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.358+0000 7f1d797c1740 1 --2- 172.21.15.73:0/3785897216 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55e6b6caaf50 0x55e6b6cae490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:30.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.358+0000 7f1d797c1740 1 -- 172.21.15.73:0/3785897216 >> 172.21.15.73:0/3785897216 conn(0x55e6b6c85120 msgr2=0x55e6b6c859c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:30.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.358+0000 7f1d797c1740 1 -- 172.21.15.73:0/3785897216 shutdown_connections 2024-04-26T20:52:30.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.358+0000 7f1d797c1740 1 -- 172.21.15.73:0/3785897216 wait complete. 2024-04-26T20:52:30.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.359+0000 7f1d797c1740 1 -- start start 2024-04-26T20:52:30.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.359+0000 7f1d797c1740 1 Processor -- start 2024-04-26T20:52:30.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.360+0000 7f1d797c1740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6b6caaf50 0x55e6b6ca92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:30.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.360+0000 7f1d797c1740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6b6caf440 0x55e6b6ccd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:30.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.360+0000 7f1d797c1740 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55e6b6cccce0 0x55e6b6ccdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:30.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.360+0000 7f1d797c1740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55e6b6cb1d50 con 0x55e6b6caaf50 2024-04-26T20:52:30.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.360+0000 7f1d797c1740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x55e6b6cb03e0 con 0x55e6b6caf440 2024-04-26T20:52:30.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.360+0000 7f1d797c1740 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x55e6b6ca9e10 con 0x55e6b6cccce0 2024-04-26T20:52:30.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.360+0000 7f1d797c1740 10 client.0 _ll_register_callbacks cb 0x7f1d6000f920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:52:30.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.360+0000 7f1d6ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6b6caaf50 0x55e6b6ca92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:30.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.360+0000 7f1d6ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6b6caaf50 0x55e6b6ca92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48486/0 (socket says 172.21.15.73:48486) 2024-04-26T20:52:30.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.360+0000 7f1d6ffff640 1 -- 172.21.15.73:0/2125313226 learned_addr learned my addr 172.21.15.73:0/2125313226 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:30.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.360+0000 7f1d6f7fe640 1 --2- 172.21.15.73:0/2125313226 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6b6caf440 0x55e6b6ccd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:30.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.360+0000 7f1d74f63640 1 --2- 172.21.15.73:0/2125313226 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55e6b6cccce0 0x55e6b6ccdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:30.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.360+0000 7f1d6ffff640 1 -- 172.21.15.73:0/2125313226 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55e6b6cccce0 msgr2=0x55e6b6ccdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:30.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.360+0000 7f1d6ffff640 1 --2- 172.21.15.73:0/2125313226 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55e6b6cccce0 0x55e6b6ccdc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:30.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.360+0000 7f1d6ffff640 1 -- 172.21.15.73:0/2125313226 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6b6caf440 msgr2=0x55e6b6ccd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:30.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.360+0000 7f1d6ffff640 1 --2- 172.21.15.73:0/2125313226 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6b6caf440 0x55e6b6ccd740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:30.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.360+0000 7f1d6ffff640 1 -- 172.21.15.73:0/2125313226 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6b6c84140 con 0x55e6b6caaf50 2024-04-26T20:52:30.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.360+0000 7f1d6ffff640 1 --2- 172.21.15.73:0/2125313226 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6b6caaf50 0x55e6b6ca92c0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f1d6000f620 tx=0x7f1d600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4636c934d4cd554b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:30.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.360+0000 7f1d4effd640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-04-26T20:52:30.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.361+0000 7f1d4effd640 1 -- 172.21.15.73:0/2125313226 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d60034470 con 0x55e6b6caaf50 2024-04-26T20:52:30.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.361+0000 7f1d4effd640 1 -- 172.21.15.73:0/2125313226 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f1d60034610 con 0x55e6b6caaf50 2024-04-26T20:52:30.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.361+0000 7f1d797c1740 1 -- 172.21.15.73:0/2125313226 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d6407e430 con 0x55e6b6caaf50 2024-04-26T20:52:30.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.361+0000 7f1d4effd640 1 -- 172.21.15.73:0/2125313226 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1d60048050 con 0x55e6b6caaf50 2024-04-26T20:52:30.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.361+0000 7f1d797c1740 10 client.6030 fetch_fsmap learned FSMap version 26 2024-04-26T20:52:30.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.361+0000 7f1d797c1740 1 -- 172.21.15.73:0/2125313226 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x55e6b6d44ac0 con 0x55e6b6caaf50 2024-04-26T20:52:30.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.361+0000 7f1d4effd640 1 -- 172.21.15.73:0/2125313226 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d6407e430 con 0x55e6b6caaf50 2024-04-26T20:52:30.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.361+0000 7f1d4effd640 1 -- 172.21.15.73:0/2125313226 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7f1d6004b020 con 0x55e6b6caaf50 2024-04-26T20:52:30.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.361+0000 7f1d797c1740 10 client.6030 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T20:52:30.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.361+0000 7f1d797c1740 10 client.6030 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T20:52:30.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.361+0000 7f1d797c1740 20 client.6030 populate_metadata read hostname 'smithi073' 2024-04-26T20:52:30.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.361+0000 7f1d797c1740 1 --2- 172.21.15.73:0/2125313226 >> [v2:172.21.15.73:6836/2770303348,v1:172.21.15.73:6837/2770303348] conn(0x55e6b6ca7b50 0x55e6b6d437d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:30.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.361+0000 7f1d797c1740 4 client.6030 mds_command: new command op to 5856 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T20:52:30.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.361+0000 7f1d797c1740 1 -- 172.21.15.73:0/2125313226 --> [v2:172.21.15.73:6836/2770303348,v1:172.21.15.73:6837/2770303348] -- command(tid 0: {"prefix": "status"}) -- 0x55e6b6bf11c0 con 0x55e6b6ca7b50 2024-04-26T20:52:30.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.361+0000 7f1d6f7fe640 1 --2- 172.21.15.73:0/2125313226 >> [v2:172.21.15.73:6836/2770303348,v1:172.21.15.73:6837/2770303348] conn(0x55e6b6ca7b50 0x55e6b6d437d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:30.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.361+0000 7f1d4effd640 10 client.6030 ms_handle_connect on v2:172.21.15.73:6836/2770303348 2024-04-26T20:52:30.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.361+0000 7f1d6f7fe640 1 --2- 172.21.15.73:0/2125313226 >> [v2:172.21.15.73:6836/2770303348,v1:172.21.15.73:6837/2770303348] conn(0x55e6b6ca7b50 0x55e6b6d437d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc4ac9bcb2dcd5f7 server_cookie=6fb832f38d54166 in_seq=0 out_seq=0 2024-04-26T20:52:30.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.362+0000 7f1d4effd640 1 -- 172.21.15.73:0/2125313226 <== mds.0 v2:172.21.15.73:6836/2770303348 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55e6b6bf11c0 con 0x55e6b6ca7b50 2024-04-26T20:52:30.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.362+0000 7f1d4effd640 10 client.6030 handle_command_reply: tid=0 2024-04-26T20:52:30.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.362+0000 7f1d797c1740 1 client.6030 shutdown 2024-04-26T20:52:30.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.362+0000 7f1d797c1740 1 -- 172.21.15.73:0/2125313226 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6b6caaf50 msgr2=0x55e6b6ca92c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:30.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.362+0000 7f1d797c1740 1 --2- 172.21.15.73:0/2125313226 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6b6caaf50 0x55e6b6ca92c0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f1d6000f620 tx=0x7f1d600046c0 comp rx=0 tx=0).stop 2024-04-26T20:52:30.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.362+0000 7f1d797c1740 1 -- 172.21.15.73:0/2125313226 shutdown_connections 2024-04-26T20:52:30.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.362+0000 7f1d797c1740 1 --2- 172.21.15.73:0/2125313226 >> [v2:172.21.15.73:6836/2770303348,v1:172.21.15.73:6837/2770303348] conn(0x55e6b6ca7b50 0x55e6b6d437d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:30.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.362+0000 7f1d797c1740 1 --2- 172.21.15.73:0/2125313226 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x55e6b6cccce0 0x55e6b6ccdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:30.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.362+0000 7f1d4effd640 0 client.6030 ms_handle_reset on v2:172.21.15.73:6836/2770303348 2024-04-26T20:52:30.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.362+0000 7f1d797c1740 1 --2- 172.21.15.73:0/2125313226 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x55e6b6caf440 0x55e6b6ccd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:30.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.362+0000 7f1d797c1740 1 --2- 172.21.15.73:0/2125313226 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e6b6caaf50 0x55e6b6ca92c0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:30.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.362+0000 7f1d797c1740 1 -- 172.21.15.73:0/2125313226 >> 172.21.15.73:0/2125313226 conn(0x55e6b6c85120 msgr2=0x55e6b6cae490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:30.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.362+0000 7f1d797c1740 1 -- 172.21.15.73:0/2125313226 shutdown_connections 2024-04-26T20:52:30.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.362+0000 7f1d797c1740 1 -- 172.21.15.73:0/2125313226 wait complete. 2024-04-26T20:52:30.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.362+0000 7f1d797c1740 20 client.6030 trim_cache size 0 max 16384 2024-04-26T20:52:30.371 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-26T20:52:30.371 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_fsid": "fbd3f548-b547-4713-856b-09f98f5d1764", 2024-04-26T20:52:30.371 INFO:teuthology.orchestra.run.smithi073.stdout: "whoami": 0, 2024-04-26T20:52:30.371 INFO:teuthology.orchestra.run.smithi073.stdout: "id": 5856, 2024-04-26T20:52:30.371 INFO:teuthology.orchestra.run.smithi073.stdout: "want_state": "up:active", 2024-04-26T20:52:30.371 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "up:active", 2024-04-26T20:52:30.371 INFO:teuthology.orchestra.run.smithi073.stdout: "fs_name": "cephfs", 2024-04-26T20:52:30.371 INFO:teuthology.orchestra.run.smithi073.stdout: "rank_uptime": 8.9638969799999995, 2024-04-26T20:52:30.371 INFO:teuthology.orchestra.run.smithi073.stdout: "mdsmap_epoch": 26, 2024-04-26T20:52:30.371 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch": 52, 2024-04-26T20:52:30.371 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch_barrier": 52, 2024-04-26T20:52:30.371 INFO:teuthology.orchestra.run.smithi073.stdout: "uptime": 14.031953596999999 2024-04-26T20:52:30.371 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-26T20:52:30.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.362+0000 7f1d769c6640 1 -- 172.21.15.73:0/2252248098 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1d5c06c8f0 msgr2=0x7f1d5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:30.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.362+0000 7f1d769c6640 1 --2- 172.21.15.73:0/2252248098 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1d5c06c8f0 0x7f1d5c06edb0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1d700d19f0 tx=0x7f1d60008040 comp rx=0 tx=0).stop 2024-04-26T20:52:30.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.362+0000 7f1d769c6640 1 -- 172.21.15.73:0/2252248098 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d700d38f0 msgr2=0x7f1d700cba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:30.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.362+0000 7f1d769c6640 1 --2- 172.21.15.73:0/2252248098 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d700d38f0 0x7f1d700cba60 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f1d640097c0 tx=0x7f1d640046c0 comp rx=0 tx=0).stop 2024-04-26T20:52:30.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.363+0000 7f1d769c6640 1 -- 172.21.15.73:0/2252248098 shutdown_connections 2024-04-26T20:52:30.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.363+0000 7f1d769c6640 1 --2- 172.21.15.73:0/2252248098 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d700d38f0 0x7f1d700cba60 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:30.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.363+0000 7f1d769c6640 1 --2- 172.21.15.73:0/2252248098 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1d5c06c8f0 0x7f1d5c06edb0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:30.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.363+0000 7f1d769c6640 1 --2- 172.21.15.73:0/2252248098 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1d701606f0 0x7f1d700c8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:30.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.363+0000 7f1d769c6640 1 --2- 172.21.15.73:0/2252248098 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d700d10f0 0x7f1d70158450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:30.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.363+0000 7f1d769c6640 1 -- 172.21.15.73:0/2252248098 >> 172.21.15.73:0/2252248098 conn(0x7f1d700d2dd0 msgr2=0x7f1d7015b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:30.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.363+0000 7f1d769c6640 1 -- 172.21.15.73:0/2252248098 shutdown_connections 2024-04-26T20:52:30.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.363+0000 7f1d769c6640 1 -- 172.21.15.73:0/2252248098 wait complete. 2024-04-26T20:52:30.375 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T20:52:30.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.472+0000 7f08593f7640 1 Processor -- start 2024-04-26T20:52:30.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.472+0000 7f08593f7640 1 -- start start 2024-04-26T20:52:30.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.472+0000 7f08593f7640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f0854071af0 0x7f0854167bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:30.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.472+0000 7f08593f7640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08540ce840 con 0x7f08540d1380 2024-04-26T20:52:30.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.472+0000 7f08593f7640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08540ceb00 con 0x7f085415d1e0 2024-04-26T20:52:30.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.472+0000 7f08593f7640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f08540cede0 con 0x7f0854071af0 2024-04-26T20:52:30.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.472+0000 7f08527fc640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f08540d1380 0x7f08540d17a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:54776/0 (socket says 172.21.15.73:54776) 2024-04-26T20:52:30.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.472+0000 7f08527fc640 1 -- 172.21.15.73:0/2955721274 learned_addr learned my addr 172.21.15.73:0/2955721274 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:52:30.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.473+0000 7f08537fe640 1 --2- 172.21.15.73:0/2955721274 >> v2:172.21.15.120:3300/0 conn(0x7f0854071af0 0x7f0854167bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:30.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.473+0000 7f08537fe640 1 -- 172.21.15.73:0/2955721274 >> v1:172.21.15.120:6789/0 conn(0x7f085415d1e0 legacy=0x7f08540d09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:30.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.473+0000 7f08537fe640 1 -- 172.21.15.73:0/2955721274 >> v1:172.21.15.73:6789/0 conn(0x7f08540d1380 legacy=0x7f08540d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:30.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.473+0000 7f08537fe640 1 -- 172.21.15.73:0/2955721274 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08540702e0 con 0x7f0854071af0 2024-04-26T20:52:30.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.474+0000 7f08537fe640 1 --2- 172.21.15.73:0/2955721274 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0854071af0 0x7f0854167bc0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f084c009d90 tx=0x7f084c02a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f4dd9c682f23ab3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:30.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.474+0000 7f0851ffb640 1 -- 172.21.15.73:0/2955721274 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c007740 con 0x7f0854071af0 2024-04-26T20:52:30.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.474+0000 7f0851ffb640 1 -- 172.21.15.73:0/2955721274 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f084c0078e0 con 0x7f0854071af0 2024-04-26T20:52:30.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.474+0000 7f0851ffb640 1 -- 172.21.15.73:0/2955721274 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c007cb0 con 0x7f0854071af0 2024-04-26T20:52:30.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.475+0000 7f08593f7640 1 -- 172.21.15.73:0/2955721274 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0854071af0 msgr2=0x7f0854167bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:30.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.475+0000 7f08593f7640 1 --2- 172.21.15.73:0/2955721274 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0854071af0 0x7f0854167bc0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f084c009d90 tx=0x7f084c02a6f0 comp rx=0 tx=0).stop 2024-04-26T20:52:30.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.475+0000 7f08593f7640 1 -- 172.21.15.73:0/2955721274 shutdown_connections 2024-04-26T20:52:30.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.475+0000 7f08593f7640 1 --2- 172.21.15.73:0/2955721274 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0854071af0 0x7f0854167bc0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:30.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.475+0000 7f08593f7640 1 -- 172.21.15.73:0/2955721274 >> 172.21.15.73:0/2955721274 conn(0x7f08540d2da0 msgr2=0x7f08540d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:30.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.475+0000 7f08593f7640 1 -- 172.21.15.73:0/2955721274 shutdown_connections 2024-04-26T20:52:30.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.475+0000 7f08593f7640 1 -- 172.21.15.73:0/2955721274 wait complete. 2024-04-26T20:52:30.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.476+0000 7f08593f7640 1 Processor -- start 2024-04-26T20:52:30.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.476+0000 7f08593f7640 1 -- start start 2024-04-26T20:52:30.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.477+0000 7f08593f7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0854071af0 0x7f085406e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:30.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.477+0000 7f08593f7640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f08540d1380 0x7f0854159630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:30.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.477+0000 7f08593f7640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f085415d1e0 0x7f0854159b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:30.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.477+0000 7f08593f7640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f08540ce4c0 con 0x7f085415d1e0 2024-04-26T20:52:30.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.477+0000 7f08593f7640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0854070650 con 0x7f0854071af0 2024-04-26T20:52:30.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.477+0000 7f08593f7640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f0854168100 con 0x7f08540d1380 2024-04-26T20:52:30.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.477+0000 7f08537fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f085415d1e0 0x7f0854159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:30.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.477+0000 7f08537fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f085415d1e0 0x7f0854159b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48502/0 (socket says 172.21.15.73:48502) 2024-04-26T20:52:30.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.477+0000 7f08537fe640 1 -- 172.21.15.73:0/3300817231 learned_addr learned my addr 172.21.15.73:0/3300817231 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:30.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.477+0000 7f08527fc640 1 --2- 172.21.15.73:0/3300817231 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f08540d1380 0x7f0854159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:30.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.477+0000 7f0852ffd640 1 --2- 172.21.15.73:0/3300817231 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0854071af0 0x7f085406e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:30.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.477+0000 7f08537fe640 1 -- 172.21.15.73:0/3300817231 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f08540d1380 msgr2=0x7f0854159630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:30.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.477+0000 7f08537fe640 1 --2- 172.21.15.73:0/3300817231 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f08540d1380 0x7f0854159630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:30.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.477+0000 7f08537fe640 1 -- 172.21.15.73:0/3300817231 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0854071af0 msgr2=0x7f085406e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:30.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.477+0000 7f08537fe640 1 --2- 172.21.15.73:0/3300817231 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0854071af0 0x7f085406e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:30.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.477+0000 7f08537fe640 1 -- 172.21.15.73:0/3300817231 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f085406d350 con 0x7f085415d1e0 2024-04-26T20:52:30.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.477+0000 7f08537fe640 1 --2- 172.21.15.73:0/3300817231 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f085415d1e0 0x7f0854159b70 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f084c009eb0 tx=0x7f084c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:30.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.477+0000 7f083b7fe640 1 -- 172.21.15.73:0/3300817231 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c040070 con 0x7f085415d1e0 2024-04-26T20:52:30.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.477+0000 7f083b7fe640 1 -- 172.21.15.73:0/3300817231 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f084c031c90 con 0x7f085415d1e0 2024-04-26T20:52:30.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.477+0000 7f083b7fe640 1 -- 172.21.15.73:0/3300817231 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084c045680 con 0x7f085415d1e0 2024-04-26T20:52:30.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.477+0000 7f08593f7640 1 -- 172.21.15.73:0/3300817231 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f085406d610 con 0x7f085415d1e0 2024-04-26T20:52:30.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.478+0000 7f08593f7640 1 -- 172.21.15.73:0/3300817231 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f085406dbd0 con 0x7f085415d1e0 2024-04-26T20:52:30.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.478+0000 7f08593f7640 1 -- 172.21.15.73:0/3300817231 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08540694f0 con 0x7f085415d1e0 2024-04-26T20:52:30.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.479+0000 7f083b7fe640 1 -- 172.21.15.73:0/3300817231 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f084c039040 con 0x7f085415d1e0 2024-04-26T20:52:30.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.479+0000 7f083b7fe640 1 --2- 172.21.15.73:0/3300817231 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f084006c8f0 0x7f084006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:30.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.479+0000 7f083b7fe640 1 -- 172.21.15.73:0/3300817231 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f084c0ad560 con 0x7f085415d1e0 2024-04-26T20:52:30.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.479+0000 7f0852ffd640 1 --2- 172.21.15.73:0/3300817231 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f084006c8f0 0x7f084006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:30.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.480+0000 7f0852ffd640 1 --2- 172.21.15.73:0/3300817231 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f084006c8f0 0x7f084006edb0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f08440016e0 tx=0x7f0844009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:30.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.484+0000 7f083b7fe640 1 -- 172.21.15.73:0/3300817231 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f084c07b300 con 0x7f085415d1e0 2024-04-26T20:52:30.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.697+0000 7f08593f7640 1 -- 172.21.15.73:0/3300817231 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f085406ce60 con 0x7f085415d1e0 2024-04-26T20:52:30.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.697+0000 7f083b7fe640 1 -- 172.21.15.73:0/3300817231 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+73 (secure 0 0 0) 0x7f084c07f600 con 0x7f085415d1e0 2024-04-26T20:52:30.699 INFO:teuthology.orchestra.run.smithi073.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T20:52:30.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.699+0000 7f08593f7640 1 -- 172.21.15.73:0/3300817231 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f084006c8f0 msgr2=0x7f084006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:30.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.699+0000 7f08593f7640 1 --2- 172.21.15.73:0/3300817231 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f084006c8f0 0x7f084006edb0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f08440016e0 tx=0x7f0844009040 comp rx=0 tx=0).stop 2024-04-26T20:52:30.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.699+0000 7f08593f7640 1 -- 172.21.15.73:0/3300817231 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f085415d1e0 msgr2=0x7f0854159b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:30.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.699+0000 7f08593f7640 1 --2- 172.21.15.73:0/3300817231 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f085415d1e0 0x7f0854159b70 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f084c009eb0 tx=0x7f084c004a40 comp rx=0 tx=0).stop 2024-04-26T20:52:30.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.699+0000 7f08593f7640 1 -- 172.21.15.73:0/3300817231 shutdown_connections 2024-04-26T20:52:30.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.699+0000 7f08593f7640 1 --2- 172.21.15.73:0/3300817231 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f085415d1e0 0x7f0854159b70 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:30.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.699+0000 7f08593f7640 1 --2- 172.21.15.73:0/3300817231 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f084006c8f0 0x7f084006edb0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:30.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.699+0000 7f08593f7640 1 --2- 172.21.15.73:0/3300817231 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f08540d1380 0x7f0854159630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:30.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.699+0000 7f08593f7640 1 --2- 172.21.15.73:0/3300817231 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0854071af0 0x7f085406e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:30.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.699+0000 7f08593f7640 1 -- 172.21.15.73:0/3300817231 >> 172.21.15.73:0/3300817231 conn(0x7f08540d2da0 msgr2=0x7f08540b07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:30.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.699+0000 7f08593f7640 1 -- 172.21.15.73:0/3300817231 shutdown_connections 2024-04-26T20:52:30.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30.699+0000 7f08593f7640 1 -- 172.21.15.73:0/3300817231 wait complete. 2024-04-26T20:52:30.712 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T20:52:30.712 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T20:52:30.712 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi073.front.sepia.ceph.com 2024-04-26T20:52:30.712 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T20:52:30.712 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T20:52:30.712 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T20:52:30.712 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T20:52:30.712 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T20:52:30.712 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T20:52:30.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:52:30.713 DEBUG:teuthology.orchestra.run.smithi073:> ip addr 2024-04-26T20:52:30.728 INFO:teuthology.orchestra.run.smithi073.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:52:30.728 INFO:teuthology.orchestra.run.smithi073.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:52:30.728 INFO:teuthology.orchestra.run.smithi073.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:52:30.728 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:52:30.728 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 ::1/128 scope host 2024-04-26T20:52:30.728 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:52:30.728 INFO:teuthology.orchestra.run.smithi073.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:52:30.728 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:ea brd ff:ff:ff:ff:ff:ff 2024-04-26T20:52:30.729 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f0 2024-04-26T20:52:30.729 INFO:teuthology.orchestra.run.smithi073.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:52:30.729 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:eb brd ff:ff:ff:ff:ff:ff 2024-04-26T20:52:30.729 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f1 2024-04-26T20:52:30.729 INFO:teuthology.orchestra.run.smithi073.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:52:30.729 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7c brd ff:ff:ff:ff:ff:ff 2024-04-26T20:52:30.729 INFO:teuthology.orchestra.run.smithi073.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T20:52:30.729 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7d brd ff:ff:ff:ff:ff:ff 2024-04-26T20:52:30.729 INFO:teuthology.orchestra.run.smithi073.stdout: inet 172.21.15.73/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-26T20:52:30.729 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft 42636sec preferred_lft 42636sec 2024-04-26T20:52:30.729 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::b99b:8e91:ffc2:71f/64 scope link noprefixroute 2024-04-26T20:52:30.729 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:52:30.729 INFO:teuthology.orchestra.run.smithi073.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T20:52:30.729 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 3e:22:aa:11:2d:0e brd ff:ff:ff:ff:ff:ff 2024-04-26T20:52:30.730 INFO:teuthology.orchestra.run.smithi073.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T20:52:30.730 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:52:30.730 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::1cd0:bfff:fec2:3f98/64 scope link 2024-04-26T20:52:30.730 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:52:30.730 INFO:teuthology.orchestra.run.smithi073.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T20:52:30.730 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 66:7f:fa:7c:30:18 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:52:30.730 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::647f:faff:fe7c:3018/64 scope link 2024-04-26T20:52:30.730 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:52:30.730 INFO:teuthology.orchestra.run.smithi073.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T20:52:30.730 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 3e:22:aa:11:2d:0e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:52:30.730 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::3c22:aaff:fe11:2d0e/64 scope link 2024-04-26T20:52:30.731 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:52:30.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:52:30.731 DEBUG:teuthology.orchestra.run.smithi073:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T20:52:30.791 INFO:teuthology.orchestra.run.smithi073.stdout:1 2024-04-26T20:52:30.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:52:30.792 DEBUG:teuthology.orchestra.run.smithi073:> route 2024-04-26T20:52:30.848 INFO:teuthology.orchestra.run.smithi073.stdout:Kernel IP routing table 2024-04-26T20:52:30.848 INFO:teuthology.orchestra.run.smithi073.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T20:52:30.848 INFO:teuthology.orchestra.run.smithi073.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T20:52:30.848 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T20:52:30.848 INFO:teuthology.orchestra.run.smithi073.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T20:52:30.849 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:52:30.849 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2024-04-26T20:52:30.849 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T20:52:30.849 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T20:52:30.849 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T20:52:30.849 DEBUG:teuthology.orchestra.run.smithi073:> ') 2024-04-26T20:52:30.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:52:30.966 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:52:30.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:52:30.972 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list 2024-04-26T20:52:30.987 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T20:52:30.987 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T20:52:30.987 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T20:52:30.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:52:30.987 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link set brx.0 up 2024-04-26T20:52:31.050 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:52:31.050 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T20:52:31.105 INFO:teuthology.orchestra.run.smithi073.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T20:52:31.106 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:52:31.106 DEBUG:teuthology.orchestra.run.smithi073:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T20:52:31.160 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T20:52:31.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:52:31.160 DEBUG:teuthology.orchestra.run.smithi073:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T20:52:31.241 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T20:52:31.241 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T20:52:31.241 INFO:teuthology.orchestra.run.smithi073.stdout:invalid new device string format 2024-04-26T20:52:31.250 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T20:52:31.250 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T20:52:31.250 INFO:teuthology.orchestra.run.smithi073.stdout:invalid new device string format 2024-04-26T20:52:31.250 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: resolved to: "172.21.15.73:3300,172.21.15.120:3300,172.21.15.120:3301" 2024-04-26T20:52:31.251 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T20:52:31.255 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T20:52:31.255 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T20:52:31.255 INFO:teuthology.orchestra.run.smithi073.stdout:invalid new device string format 2024-04-26T20:52:31.255 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: resolved to: "172.21.15.73:3300,172.21.15.120:3300,172.21.15.120:3301" 2024-04-26T20:52:31.256 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T20:52:31.256 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: resolved to: "172.21.15.73:6789,172.21.15.120:6789,172.21.15.120:6790" 2024-04-26T20:52:31.256 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.73:6789,172.21.15.120:6789,172.21.15.120:6790:/ 2024-04-26T20:52:31.256 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=fbd3f548-b547-4713-856b-09f98f5d1764" will pass to kernel 2024-04-26T20:52:31.257 INFO:teuthology.orchestra.run.smithi073.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T20:52:31.257 INFO:teuthology.orchestra.run.smithi073.stdout: You just mounted a file system that supports labels which does not 2024-04-26T20:52:31.257 INFO:teuthology.orchestra.run.smithi073.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T20:52:31.257 INFO:teuthology.orchestra.run.smithi073.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T20:52:31.257 INFO:teuthology.orchestra.run.smithi073.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T20:52:31.258 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T20:52:31.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:52:31.258 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T20:52:31.323 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T20:52:31.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T20:52:31.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:52:31.418 INFO:teuthology.orchestra.run.smithi073.stdout:client5587 2024-04-26T20:52:31.419 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-26T20:52:31.419 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:52:31.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:52:31.434 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch test_file1) 2024-04-26T20:52:31.491 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-26T20:52:31.491 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:52:31.546 DEBUG:teuthology.orchestra.run.smithi073:> stdin-killer -- python3 -c ' 2024-04-26T20:52:31.546 DEBUG:teuthology.orchestra.run.smithi073:> import fcntl 2024-04-26T20:52:31.546 DEBUG:teuthology.orchestra.run.smithi073:> import os 2024-04-26T20:52:31.546 DEBUG:teuthology.orchestra.run.smithi073:> import sys 2024-04-26T20:52:31.547 DEBUG:teuthology.orchestra.run.smithi073:> import time 2024-04-26T20:52:31.547 DEBUG:teuthology.orchestra.run.smithi073:> 2024-04-26T20:52:31.547 DEBUG:teuthology.orchestra.run.smithi073:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-26T20:52:31.547 DEBUG:teuthology.orchestra.run.smithi073:> 2024-04-26T20:52:31.547 DEBUG:teuthology.orchestra.run.smithi073:> fd = os.open("/home/ubuntu/cephtest/mnt.0/background_file", os.O_RDWR | os.O_CREAT, 0o644) 2024-04-26T20:52:31.547 DEBUG:teuthology.orchestra.run.smithi073:> try: 2024-04-26T20:52:31.547 DEBUG:teuthology.orchestra.run.smithi073:> while True: 2024-04-26T20:52:31.547 DEBUG:teuthology.orchestra.run.smithi073:> print("write_background: writing", file=sys.stderr) 2024-04-26T20:52:31.547 DEBUG:teuthology.orchestra.run.smithi073:> os.write(fd, b'"'"'content'"'"') 2024-04-26T20:52:31.547 DEBUG:teuthology.orchestra.run.smithi073:> if not True: 2024-04-26T20:52:31.547 DEBUG:teuthology.orchestra.run.smithi073:> break 2024-04-26T20:52:31.547 DEBUG:teuthology.orchestra.run.smithi073:> try: 2024-04-26T20:52:31.547 DEBUG:teuthology.orchestra.run.smithi073:> if os.read(0, 4096) == b"": 2024-04-26T20:52:31.547 DEBUG:teuthology.orchestra.run.smithi073:> break 2024-04-26T20:52:31.547 DEBUG:teuthology.orchestra.run.smithi073:> except BlockingIOError: 2024-04-26T20:52:31.547 DEBUG:teuthology.orchestra.run.smithi073:> pass 2024-04-26T20:52:31.548 DEBUG:teuthology.orchestra.run.smithi073:> time.sleep(2) 2024-04-26T20:52:31.548 DEBUG:teuthology.orchestra.run.smithi073:> except IOError as e: 2024-04-26T20:52:31.548 DEBUG:teuthology.orchestra.run.smithi073:> pass 2024-04-26T20:52:31.548 DEBUG:teuthology.orchestra.run.smithi073:> os.close(fd) 2024-04-26T20:52:31.548 DEBUG:teuthology.orchestra.run.smithi073:> ' 2024-04-26T20:52:31.651 INFO:teuthology.orchestra.run.smithi073.stderr:write_background: writing 2024-04-26T20:52:33.655 INFO:teuthology.orchestra.run.smithi073.stderr:write_background: writing 2024-04-26T20:52:35.658 INFO:teuthology.orchestra.run.smithi073.stderr:write_background: writing 2024-04-26T20:52:37.660 INFO:teuthology.orchestra.run.smithi073.stderr:write_background: writing 2024-04-26T20:52:39.662 INFO:teuthology.orchestra.run.smithi073.stderr:write_background: writing 2024-04-26T20:52:41.664 INFO:teuthology.orchestra.run.smithi073.stderr:write_background: writing 2024-04-26T20:52:41.689 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:52:41.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:41.802+0000 7f50935f7640 1 Processor -- start 2024-04-26T20:52:41.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:41.802+0000 7f50935f7640 1 -- start start 2024-04-26T20:52:41.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:41.803+0000 7f50935f7640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f508c071e60 0x7f508c06f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:41.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:41.803+0000 7f50935f7640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f508c15d7c0 con 0x7f508c06fe20 2024-04-26T20:52:41.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:41.803+0000 7f50935f7640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f508c15daa0 con 0x7f508c0cc480 2024-04-26T20:52:41.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:41.803+0000 7f50935f7640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f508c15dd80 con 0x7f508c071e60 2024-04-26T20:52:41.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:41.803+0000 7f5091393640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f508c071e60 0x7f508c06f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:41.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:41.803+0000 7f5090b92640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f508c06fe20 0x7f508c072240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:43116/0 (socket says 172.21.15.73:43116) 2024-04-26T20:52:41.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:41.803+0000 7f5090b92640 1 -- 172.21.15.73:0/1933709450 learned_addr learned my addr 172.21.15.73:0/1933709450 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:52:41.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:41.804+0000 7f5091393640 1 -- 172.21.15.73:0/1933709450 >> v1:172.21.15.120:6789/0 conn(0x7f508c0cc480 legacy=0x7f508c166ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:41.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:41.804+0000 7f5091393640 1 -- 172.21.15.73:0/1933709450 >> v1:172.21.15.73:6789/0 conn(0x7f508c06fe20 legacy=0x7f508c072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:41.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:41.804+0000 7f5083fff640 1 -- 172.21.15.73:0/1933709450 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3812259451 0 0) 0x7f508c15d7c0 con 0x7f508c06fe20 2024-04-26T20:52:41.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:41.804+0000 7f5091393640 1 -- 172.21.15.73:0/1933709450 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f508c15e060 con 0x7f508c071e60 2024-04-26T20:52:41.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:41.804+0000 7f5091393640 1 --2- 172.21.15.73:0/1933709450 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f508c071e60 0x7f508c06f8e0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f50880097a0 tx=0x7f508802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50ddb2e8d2a44be6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:41.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:41.804+0000 7f5083fff640 1 -- 172.21.15.73:0/1933709450 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5088037070 con 0x7f508c071e60 2024-04-26T20:52:41.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:41.804+0000 7f5083fff640 1 -- 172.21.15.73:0/1933709450 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5088032440 con 0x7f508c071e60 2024-04-26T20:52:41.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:41.804+0000 7f5083fff640 1 -- 172.21.15.73:0/1933709450 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5088032810 con 0x7f508c071e60 2024-04-26T20:52:41.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:41.805+0000 7f50935f7640 1 -- 172.21.15.73:0/1933709450 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f508c071e60 msgr2=0x7f508c06f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:41.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:41.805+0000 7f50935f7640 1 --2- 172.21.15.73:0/1933709450 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f508c071e60 0x7f508c06f8e0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f50880097a0 tx=0x7f508802a1e0 comp rx=0 tx=0).stop 2024-04-26T20:52:41.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:41.805+0000 7f50935f7640 1 -- 172.21.15.73:0/1933709450 shutdown_connections 2024-04-26T20:52:41.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:41.805+0000 7f50935f7640 1 --2- 172.21.15.73:0/1933709450 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f508c071e60 0x7f508c06f8e0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:41.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:41.805+0000 7f50935f7640 1 -- 172.21.15.73:0/1933709450 >> 172.21.15.73:0/1933709450 conn(0x7f508c0d0be0 msgr2=0x7f508c0d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:41.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:41.805+0000 7f50935f7640 1 -- 172.21.15.73:0/1933709450 shutdown_connections 2024-04-26T20:52:41.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:41.805+0000 7f50935f7640 1 -- 172.21.15.73:0/1933709450 wait complete. 2024-04-26T20:52:41.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:41.806+0000 7f50935f7640 1 Processor -- start 2024-04-26T20:52:41.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:41.806+0000 7f50935f7640 1 -- start start 2024-04-26T20:52:41.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:41.807+0000 7f50935f7640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f508c06fe20 0x7f508c0c9750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:41.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:41.807+0000 7f50935f7640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f508c071e60 0x7f508c1554e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:41.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:41.807+0000 7f50935f7640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f508c0cc480 0x7f508c156de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:41.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:41.807+0000 7f50935f7640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f508c15d570 con 0x7f508c0cc480 2024-04-26T20:52:41.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:41.807+0000 7f50935f7640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f508c15d3f0 con 0x7f508c071e60 2024-04-26T20:52:41.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:41.807+0000 7f50935f7640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f508c072950 con 0x7f508c06fe20 2024-04-26T20:52:41.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:41.807+0000 7f5091b94640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f508c0cc480 0x7f508c156de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:41.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:41.807+0000 7f5091b94640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f508c0cc480 0x7f508c156de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53448/0 (socket says 172.21.15.73:53448) 2024-04-26T20:52:41.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:41.807+0000 7f5091b94640 1 -- 172.21.15.73:0/3671323536 learned_addr learned my addr 172.21.15.73:0/3671323536 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:41.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:41.807+0000 7f5091393640 1 --2- 172.21.15.73:0/3671323536 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f508c06fe20 0x7f508c0c9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:41.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:41.807+0000 7f5090b92640 1 --2- 172.21.15.73:0/3671323536 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f508c071e60 0x7f508c1554e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:41.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:41.808+0000 7f5091b94640 1 -- 172.21.15.73:0/3671323536 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f508c06fe20 msgr2=0x7f508c0c9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:41.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:41.808+0000 7f5091b94640 1 --2- 172.21.15.73:0/3671323536 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f508c06fe20 0x7f508c0c9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:41.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:41.808+0000 7f5091b94640 1 -- 172.21.15.73:0/3671323536 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f508c071e60 msgr2=0x7f508c1554e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:41.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:41.808+0000 7f5091b94640 1 --2- 172.21.15.73:0/3671323536 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f508c071e60 0x7f508c1554e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:41.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:41.808+0000 7f5091b94640 1 -- 172.21.15.73:0/3671323536 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f508c159e60 con 0x7f508c0cc480 2024-04-26T20:52:41.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:41.808+0000 7f5090b92640 1 --2- 172.21.15.73:0/3671323536 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f508c071e60 0x7f508c1554e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:52:41.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:41.808+0000 7f5091b94640 1 --2- 172.21.15.73:0/3671323536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f508c0cc480 0x7f508c156de0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f508400b970 tx=0x7f508400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:41.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:41.808+0000 7f5081ffb640 1 -- 172.21.15.73:0/3671323536 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5084019070 con 0x7f508c0cc480 2024-04-26T20:52:41.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:41.808+0000 7f5081ffb640 1 -- 172.21.15.73:0/3671323536 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f508400cb40 con 0x7f508c0cc480 2024-04-26T20:52:41.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:41.808+0000 7f5081ffb640 1 -- 172.21.15.73:0/3671323536 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5084014440 con 0x7f508c0cc480 2024-04-26T20:52:41.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:41.808+0000 7f50935f7640 1 -- 172.21.15.73:0/3671323536 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f508c15a0d0 con 0x7f508c0cc480 2024-04-26T20:52:41.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:41.808+0000 7f50935f7640 1 -- 172.21.15.73:0/3671323536 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f508c15a690 con 0x7f508c0cc480 2024-04-26T20:52:41.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:41.809+0000 7f50935f7640 1 -- 172.21.15.73:0/3671323536 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f508c0694f0 con 0x7f508c0cc480 2024-04-26T20:52:41.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:41.810+0000 7f5081ffb640 1 -- 172.21.15.73:0/3671323536 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f508401c070 con 0x7f508c0cc480 2024-04-26T20:52:41.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:41.810+0000 7f5081ffb640 1 --2- 172.21.15.73:0/3671323536 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f507006c920 0x7f507006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:41.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:41.810+0000 7f5081ffb640 1 -- 172.21.15.73:0/3671323536 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f508408b7f0 con 0x7f508c0cc480 2024-04-26T20:52:41.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:41.810+0000 7f5091393640 1 --2- 172.21.15.73:0/3671323536 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f507006c920 0x7f507006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:41.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:41.810+0000 7f5091393640 1 --2- 172.21.15.73:0/3671323536 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f507006c920 0x7f507006ede0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f5088034000 tx=0x7f508802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:41.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:41.814+0000 7f5081ffb640 1 -- 172.21.15.73:0/3671323536 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5084059440 con 0x7f508c0cc480 2024-04-26T20:52:42.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:42.027+0000 7f50935f7640 1 -- 172.21.15.73:0/3671323536 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f508c0b11b0 con 0x7f508c0cc480 2024-04-26T20:52:42.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:42.027+0000 7f5081ffb640 1 -- 172.21.15.73:0/3671323536 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5181 (secure 0 0 0) 0x7f508405d740 con 0x7f508c0cc480 2024-04-26T20:52:42.028 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:52:42.029 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5096,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6834/528576665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":528576665},{"type":"v1","addr":"172.21.15.120:6834","nonce":528576665}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":5099,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/824086075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6833","nonce":824086075},{"type":"v1","addr":"172.21.15.120:6835","nonce":824086075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":5853,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/1515733826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1515733826},{"type":"v1","addr":"172.21.15.73:6835","nonce":1515733826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:52:20.754736+0000","modified":"2024-04-26T20:52:22.752295+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5856},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5856":{"gid":5856,"name":"c","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/2770303348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2770303348},{"type":"v1","addr":"172.21.15.73:6837","nonce":2770303348}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5856,"qdb_cluster":[5856]},"id":3}]} 2024-04-26T20:52:42.029 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 26 2024-04-26T20:52:42.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:42.029+0000 7f50935f7640 1 -- 172.21.15.73:0/3671323536 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f507006c920 msgr2=0x7f507006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:42.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:42.029+0000 7f50935f7640 1 --2- 172.21.15.73:0/3671323536 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f507006c920 0x7f507006ede0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f5088034000 tx=0x7f508802e000 comp rx=0 tx=0).stop 2024-04-26T20:52:42.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:42.029+0000 7f50935f7640 1 -- 172.21.15.73:0/3671323536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f508c0cc480 msgr2=0x7f508c156de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:42.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:42.029+0000 7f50935f7640 1 --2- 172.21.15.73:0/3671323536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f508c0cc480 0x7f508c156de0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f508400b970 tx=0x7f508400be30 comp rx=0 tx=0).stop 2024-04-26T20:52:42.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:42.029+0000 7f50935f7640 1 -- 172.21.15.73:0/3671323536 shutdown_connections 2024-04-26T20:52:42.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:42.029+0000 7f50935f7640 1 --2- 172.21.15.73:0/3671323536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f508c0cc480 0x7f508c156de0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:42.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:42.029+0000 7f50935f7640 1 --2- 172.21.15.73:0/3671323536 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f507006c920 0x7f507006ede0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:42.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:42.029+0000 7f50935f7640 1 --2- 172.21.15.73:0/3671323536 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f508c071e60 0x7f508c1554e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:42.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:42.029+0000 7f50935f7640 1 --2- 172.21.15.73:0/3671323536 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f508c06fe20 0x7f508c0c9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:42.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:42.029+0000 7f50935f7640 1 -- 172.21.15.73:0/3671323536 >> 172.21.15.73:0/3671323536 conn(0x7f508c0d0be0 msgr2=0x7f508c158360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:42.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:42.030+0000 7f50935f7640 1 -- 172.21.15.73:0/3671323536 shutdown_connections 2024-04-26T20:52:42.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:42.030+0000 7f50935f7640 1 -- 172.21.15.73:0/3671323536 wait complete. 2024-04-26T20:52:42.041 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json force_readonly 2024-04-26T20:52:42.155 DEBUG:tasks.cephfs.filesystem:_json_asok output empty 2024-04-26T20:52:43.666 INFO:teuthology.orchestra.run.smithi073.stderr:write_background: writing 2024-04-26T20:52:43.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:52:52.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:52:52.156 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch test_file1) 2024-04-26T20:52:52.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:52:52.176 INFO:teuthology.orchestra.run.smithi073.stderr:touch: setting times of 'test_file1': Read-only file system 2024-04-26T20:52:52.177 INFO:tasks.ceph_test_case:waiting 30s for health warning matching MDS_READ_ONLY 2024-04-26T20:52:52.177 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-26T20:52:52.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.320+0000 7f6f81122640 1 Processor -- start 2024-04-26T20:52:52.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.320+0000 7f6f81122640 1 -- start start 2024-04-26T20:52:52.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.320+0000 7f6f81122640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6f7c0d0070 0x7f6f7c0d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:52.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.320+0000 7f6f81122640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f6f7c06d560 0x7f6f7c06db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:52.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.320+0000 7f6f81122640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6f7c06e0a0 0x7f6f7c162f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:52.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.320+0000 7f6f81122640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f6f7c162d00 con 0x7f6f7c06e0a0 2024-04-26T20:52:52.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.320+0000 7f6f81122640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f6f7c0d2d10 con 0x7f6f7c06d560 2024-04-26T20:52:52.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.320+0000 7f6f81122640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f6f7c0d2420 con 0x7f6f7c0d0070 2024-04-26T20:52:52.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.321+0000 7f6f7b59e640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6f7c06e0a0 0x7f6f7c162f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:52.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.321+0000 7f6f7b59e640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6f7c06e0a0 0x7f6f7c162f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47756/0 (socket says 172.21.15.73:47756) 2024-04-26T20:52:52.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.321+0000 7f6f7b59e640 1 -- 172.21.15.73:0/1387644250 learned_addr learned my addr 172.21.15.73:0/1387644250 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:52.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.321+0000 7f6f73fff640 1 --2- 172.21.15.73:0/1387644250 >> v2:172.21.15.120:3300/0 conn(0x7f6f7c06d560 0x7f6f7c06db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:52.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.321+0000 7f6f7b59e640 1 -- 172.21.15.73:0/1387644250 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6f7c0d0070 msgr2=0x7f6f7c0d1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:52.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.321+0000 7f6f7ad9d640 1 --2- 172.21.15.73:0/1387644250 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6f7c0d0070 0x7f6f7c0d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:52.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.321+0000 7f6f7b59e640 1 --2- 172.21.15.73:0/1387644250 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6f7c0d0070 0x7f6f7c0d1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:52.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.321+0000 7f6f7b59e640 1 -- 172.21.15.73:0/1387644250 >> v2:172.21.15.120:3300/0 conn(0x7f6f7c06d560 msgr2=0x7f6f7c06db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:52.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.321+0000 7f6f7b59e640 1 --2- 172.21.15.73:0/1387644250 >> v2:172.21.15.120:3300/0 conn(0x7f6f7c06d560 0x7f6f7c06db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:52.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.321+0000 7f6f7b59e640 1 -- 172.21.15.73:0/1387644250 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f7c0cd6b0 con 0x7f6f7c06e0a0 2024-04-26T20:52:52.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.321+0000 7f6f73fff640 1 --2- 172.21.15.73:0/1387644250 >> v2:172.21.15.120:3300/0 conn(0x7f6f7c06d560 0x7f6f7c06db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:52:52.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.321+0000 7f6f7ad9d640 1 --2- 172.21.15.73:0/1387644250 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6f7c0d0070 0x7f6f7c0d1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:52:52.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.322+0000 7f6f7b59e640 1 --2- 172.21.15.73:0/1387644250 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f7c06e0a0 0x7f6f7c162f10 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c073b40 tx=0x7f6f7402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10bb2db49e7ee67d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:52.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.322+0000 7f6f7a59c640 1 -- 172.21.15.73:0/1387644250 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f74036070 con 0x7f6f7c06e0a0 2024-04-26T20:52:52.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.322+0000 7f6f7a59c640 1 -- 172.21.15.73:0/1387644250 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f74031600 con 0x7f6f7c06e0a0 2024-04-26T20:52:52.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.322+0000 7f6f7a59c640 1 -- 172.21.15.73:0/1387644250 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f740319d0 con 0x7f6f7c06e0a0 2024-04-26T20:52:52.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.322+0000 7f6f81122640 1 -- 172.21.15.73:0/1387644250 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f7c06e0a0 msgr2=0x7f6f7c162f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:52.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.322+0000 7f6f81122640 1 --2- 172.21.15.73:0/1387644250 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f7c06e0a0 0x7f6f7c162f10 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c073b40 tx=0x7f6f7402a730 comp rx=0 tx=0).stop 2024-04-26T20:52:52.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.322+0000 7f6f81122640 1 -- 172.21.15.73:0/1387644250 shutdown_connections 2024-04-26T20:52:52.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.322+0000 7f6f81122640 1 --2- 172.21.15.73:0/1387644250 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f7c06e0a0 0x7f6f7c162f10 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:52.326 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.322+0000 7f6f81122640 1 --2- 172.21.15.73:0/1387644250 >> v2:172.21.15.120:3300/0 conn(0x7f6f7c06d560 0x7f6f7c06db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:52.326 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.322+0000 7f6f81122640 1 --2- 172.21.15.73:0/1387644250 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6f7c0d0070 0x7f6f7c0d1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:52.326 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.322+0000 7f6f81122640 1 -- 172.21.15.73:0/1387644250 >> 172.21.15.73:0/1387644250 conn(0x7f6f7c0d2ec0 msgr2=0x7f6f7c0d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:52.326 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.322+0000 7f6f81122640 1 -- 172.21.15.73:0/1387644250 shutdown_connections 2024-04-26T20:52:52.326 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.322+0000 7f6f81122640 1 -- 172.21.15.73:0/1387644250 wait complete. 2024-04-26T20:52:52.326 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.324+0000 7f6f81122640 1 Processor -- start 2024-04-26T20:52:52.326 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.324+0000 7f6f81122640 1 -- start start 2024-04-26T20:52:52.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.324+0000 7f6f81122640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f7c06d560 0x7f6f7c0a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:52.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.324+0000 7f6f81122640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f7c06e0a0 0x7f6f7c1f54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:52.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.324+0000 7f6f81122640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6f7c0d0070 0x7f6f7c0c9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:52.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.325+0000 7f6f81122640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6f7c163450 con 0x7f6f7c06d560 2024-04-26T20:52:52.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.325+0000 7f6f81122640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6f7c0cdc70 con 0x7f6f7c06e0a0 2024-04-26T20:52:52.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.325+0000 7f6f81122640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f6f7c1635d0 con 0x7f6f7c0d0070 2024-04-26T20:52:52.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.325+0000 7f6f7ad9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f7c06d560 0x7f6f7c0a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:52.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.325+0000 7f6f7ad9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f7c06d560 0x7f6f7c0a0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47760/0 (socket says 172.21.15.73:47760) 2024-04-26T20:52:52.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.325+0000 7f6f7ad9d640 1 -- 172.21.15.73:0/2366794552 learned_addr learned my addr 172.21.15.73:0/2366794552 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:52.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.325+0000 7f6f73fff640 1 --2- 172.21.15.73:0/2366794552 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f7c06e0a0 0x7f6f7c1f54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:52.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.325+0000 7f6f7b59e640 1 --2- 172.21.15.73:0/2366794552 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6f7c0d0070 0x7f6f7c0c9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:52.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.325+0000 7f6f7ad9d640 1 -- 172.21.15.73:0/2366794552 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6f7c0d0070 msgr2=0x7f6f7c0c9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:52.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.325+0000 7f6f7ad9d640 1 --2- 172.21.15.73:0/2366794552 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6f7c0d0070 0x7f6f7c0c9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:52.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.325+0000 7f6f7ad9d640 1 -- 172.21.15.73:0/2366794552 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f7c06e0a0 msgr2=0x7f6f7c1f54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:52.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.325+0000 7f6f7ad9d640 1 --2- 172.21.15.73:0/2366794552 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f7c06e0a0 0x7f6f7c1f54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:52.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.325+0000 7f6f7ad9d640 1 -- 172.21.15.73:0/2366794552 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f7c15a930 con 0x7f6f7c06d560 2024-04-26T20:52:52.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.325+0000 7f6f7ad9d640 1 --2- 172.21.15.73:0/2366794552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f7c06d560 0x7f6f7c0a0630 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c00c820 tx=0x7f6f6c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:52.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.325+0000 7f6f737fe640 1 -- 172.21.15.73:0/2366794552 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c015070 con 0x7f6f7c06d560 2024-04-26T20:52:52.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.325+0000 7f6f737fe640 1 -- 172.21.15.73:0/2366794552 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f6c0079d0 con 0x7f6f7c06d560 2024-04-26T20:52:52.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.325+0000 7f6f81122640 1 -- 172.21.15.73:0/2366794552 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f7c15c5d0 con 0x7f6f7c06d560 2024-04-26T20:52:52.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.325+0000 7f6f737fe640 1 -- 172.21.15.73:0/2366794552 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c007d80 con 0x7f6f7c06d560 2024-04-26T20:52:52.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.325+0000 7f6f81122640 1 -- 172.21.15.73:0/2366794552 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f7c15cb90 con 0x7f6f7c06d560 2024-04-26T20:52:52.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.326+0000 7f6f81122640 1 -- 172.21.15.73:0/2366794552 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f7c0d3e00 con 0x7f6f7c06d560 2024-04-26T20:52:52.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.327+0000 7f6f737fe640 1 -- 172.21.15.73:0/2366794552 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6f6c018070 con 0x7f6f7c06d560 2024-04-26T20:52:52.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.327+0000 7f6f737fe640 1 --2- 172.21.15.73:0/2366794552 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f6f5c06c8f0 0x7f6f5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:52.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.327+0000 7f6f737fe640 1 -- 172.21.15.73:0/2366794552 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f6f6c08ba80 con 0x7f6f7c06d560 2024-04-26T20:52:52.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.327+0000 7f6f73fff640 1 --2- 172.21.15.73:0/2366794552 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f6f5c06c8f0 0x7f6f5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:52.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.328+0000 7f6f73fff640 1 --2- 172.21.15.73:0/2366794552 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f6f5c06c8f0 0x7f6f5c06edb0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c0cf150 tx=0x7f6f64008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:52.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.332+0000 7f6f737fe640 1 -- 172.21.15.73:0/2366794552 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6f7c0d3e00 con 0x7f6f7c06d560 2024-04-26T20:52:52.368 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:52:52.371 DEBUG:teuthology.orchestra.run.smithi120:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:52:52.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.549+0000 7f6f81122640 1 -- 172.21.15.73:0/2366794552 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f6f7c0694f0 con 0x7f6f7c06d560 2024-04-26T20:52:52.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.549+0000 7f6f737fe640 1 -- 172.21.15.73:0/2366794552 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+159 (secure 0 0 0) 0x7f6f7c15a930 con 0x7f6f7c06d560 2024-04-26T20:52:52.550 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:52:52.550 INFO:teuthology.orchestra.run.smithi073.stdout:{"status":"HEALTH_WARN","checks":{"MDS_READ_ONLY":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs are read only","count":1},"muted":false}},"mutes":[]} 2024-04-26T20:52:52.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.551+0000 7f6f81122640 1 -- 172.21.15.73:0/2366794552 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f6f5c06c8f0 msgr2=0x7f6f5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:52.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.551+0000 7f6f81122640 1 --2- 172.21.15.73:0/2366794552 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f6f5c06c8f0 0x7f6f5c06edb0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c0cf150 tx=0x7f6f64008040 comp rx=0 tx=0).stop 2024-04-26T20:52:52.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.551+0000 7f6f81122640 1 -- 172.21.15.73:0/2366794552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f7c06d560 msgr2=0x7f6f7c0a0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:52.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.551+0000 7f6f81122640 1 --2- 172.21.15.73:0/2366794552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f7c06d560 0x7f6f7c0a0630 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c00c820 tx=0x7f6f6c00cce0 comp rx=0 tx=0).stop 2024-04-26T20:52:52.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.551+0000 7f6f81122640 1 -- 172.21.15.73:0/2366794552 shutdown_connections 2024-04-26T20:52:52.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.551+0000 7f6f81122640 1 --2- 172.21.15.73:0/2366794552 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f6f5c06c8f0 0x7f6f5c06edb0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:52.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.551+0000 7f6f81122640 1 --2- 172.21.15.73:0/2366794552 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6f7c0d0070 0x7f6f7c0c9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:52.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.551+0000 7f6f81122640 1 --2- 172.21.15.73:0/2366794552 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6f7c06e0a0 0x7f6f7c1f54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:52.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.551+0000 7f6f81122640 1 --2- 172.21.15.73:0/2366794552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f7c06d560 0x7f6f7c0a0630 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:52.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.551+0000 7f6f81122640 1 -- 172.21.15.73:0/2366794552 >> 172.21.15.73:0/2366794552 conn(0x7f6f7c0d2ec0 msgr2=0x7f6f7c0c9f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:52.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.551+0000 7f6f81122640 1 -- 172.21.15.73:0/2366794552 shutdown_connections 2024-04-26T20:52:52.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.551+0000 7f6f81122640 1 -- 172.21.15.73:0/2366794552 wait complete. 2024-04-26T20:52:52.563 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T20:52:52.563 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T20:52:52.563 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:52:52.564 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T20:52:52.564 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:52:52.564 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T20:52:52.564 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:52:52.565 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T20:52:52.565 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:52:52.573 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T20:52:52.573 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail a 2024-04-26T20:52:52.574 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T20:52:52.574 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail c 2024-04-26T20:52:52.576 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T20:52:52.577 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail b 2024-04-26T20:52:52.577 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T20:52:52.577 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail d 2024-04-26T20:52:52.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.715+0000 7fb4d0d04640 1 Processor -- start 2024-04-26T20:52:52.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.715+0000 7f62dcc64640 1 Processor -- start 2024-04-26T20:52:52.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.715+0000 7f6cdfd04640 1 Processor -- start 2024-04-26T20:52:52.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.715+0000 7fb4d0d04640 1 -- start start 2024-04-26T20:52:52.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.715+0000 7f62dcc64640 1 -- start start 2024-04-26T20:52:52.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.715+0000 7fdb26bda640 1 Processor -- start 2024-04-26T20:52:52.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.716+0000 7fdb26bda640 1 -- start start 2024-04-26T20:52:52.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.716+0000 7f6cdfd04640 1 -- start start 2024-04-26T20:52:52.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.716+0000 7fb4d0d04640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb4cc06f790 0x7fb4cc06fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:52.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.716+0000 7f6cdfd04640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f6cd80cf5e0 0x7f6cd80ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:52.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.716+0000 7f6cdfd04640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6cd80cbbd0 0x7f6cd80cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:52.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.716+0000 7f62dcc64640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f62d80b3860 0x7f62d816a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:52.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.716+0000 7f6cdfd04640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cd815b9e0 con 0x7f6cd815b600 2024-04-26T20:52:52.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.716+0000 7f6cdfd04640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f6cd80cb9b0 con 0x7f6cd80cf5e0 2024-04-26T20:52:52.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.716+0000 7f6cdfd04640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f6cd80d0740 con 0x7f6cd80cbbd0 2024-04-26T20:52:52.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.716+0000 7fb4d0d04640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4cc156840 con 0x7fb4cc156460 2024-04-26T20:52:52.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.716+0000 7fdb26bda640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fdb2015b600 0x7fdb20071f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:52.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.716+0000 7fb4d0d04640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4cc0cbf20 con 0x7fb4cc072c50 2024-04-26T20:52:52.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.716+0000 7f62dcc64640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62d80d0ba0 con 0x7f62d815f3a0 2024-04-26T20:52:52.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.716+0000 7f62dcc64640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62d80d2590 con 0x7f62d80d06f0 2024-04-26T20:52:52.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.716+0000 7f62dcc64640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f62d80d0e20 con 0x7f62d80b3860 2024-04-26T20:52:52.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.717+0000 7fb4d0d04640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7fb4cc0cc200 con 0x7fb4cc06f790 2024-04-26T20:52:52.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.717+0000 7f6cddaa0640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f6cd815b600 0x7f6cd8071f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:45670/0 (socket says 172.21.15.73:45670) 2024-04-26T20:52:52.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.717+0000 7f6cddaa0640 1 -- 172.21.15.73:0/1427188051 learned_addr learned my addr 172.21.15.73:0/1427188051 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:52:52.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.717+0000 7fdb26bda640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb20072450 con 0x7fdb200cf0e0 2024-04-26T20:52:52.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.717+0000 7fdb26bda640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb2015b9e0 con 0x7fdb200cf5e0 2024-04-26T20:52:52.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.717+0000 7fdb26bda640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fdb200cb9b0 con 0x7fdb2015b600 2024-04-26T20:52:52.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.717+0000 7f62d659c640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f62d815f3a0 0x7f62d80d1770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:45684/0 (socket says 172.21.15.73:45684) 2024-04-26T20:52:52.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.717+0000 7f62d659c640 1 -- 172.21.15.73:0/4092337470 learned_addr learned my addr 172.21.15.73:0/4092337470 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:52:52.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.717+0000 7f6cdd29f640 1 --2- 172.21.15.73:0/1427188051 >> v2:172.21.15.120:3300/0 conn(0x7f6cd80cf5e0 0x7f6cd80ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:52.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.717+0000 7fb4ca59c640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fb4cc156460 0x7fb4cc0cb4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:45664/0 (socket says 172.21.15.73:45664) 2024-04-26T20:52:52.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.717+0000 7f6cde2a1640 1 --2- 172.21.15.73:0/1427188051 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6cd80cbbd0 0x7f6cd80cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:52.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.717+0000 7fb4ca59c640 1 -- 172.21.15.73:0/433703294 learned_addr learned my addr 172.21.15.73:0/433703294 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:52:52.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.717+0000 7fdb24976640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fdb2015b600 0x7fdb20071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:52.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.717+0000 7fb4c9d9b640 1 --2- 172.21.15.73:0/433703294 >> v2:172.21.15.73:3300/0 conn(0x7fb4cc06f790 0x7fb4cc06fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:52.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.717+0000 7fdb1ffff640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7fdb200cf5e0 0x7fdb200ce9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.73:40416/0 (socket says 172.21.15.73:40416) 2024-04-26T20:52:52.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.717+0000 7fdb1ffff640 1 -- 172.21.15.73:0/3900421816 learned_addr learned my addr 172.21.15.73:0/3900421816 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:52:52.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.717+0000 7f62d559a640 1 -- 172.21.15.73:0/4092337470 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2056281042 0 0) 0x7f62d80d2590 con 0x7f62d80d06f0 2024-04-26T20:52:52.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.717+0000 7f62d559a640 1 -- 172.21.15.73:0/4092337470 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f62c0003520 con 0x7f62d80d06f0 2024-04-26T20:52:52.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.717+0000 7f6cde2a1640 1 -- 172.21.15.73:0/1427188051 >> v2:172.21.15.120:3300/0 conn(0x7f6cd80cf5e0 msgr2=0x7f6cd80ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:52.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.717+0000 7f6cde2a1640 1 --2- 172.21.15.73:0/1427188051 >> v2:172.21.15.120:3300/0 conn(0x7f6cd80cf5e0 0x7f6cd80ce9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:52.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.717+0000 7fb4c959a640 1 -- 172.21.15.73:0/433703294 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 182536922 0 0) 0x7fb4cc0cbf20 con 0x7fb4cc072c50 2024-04-26T20:52:52.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.717+0000 7f6cde2a1640 1 -- 172.21.15.73:0/1427188051 >> v1:172.21.15.73:6789/0 conn(0x7f6cd815b600 legacy=0x7f6cd8071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:52.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.717+0000 7f6cdca9e640 1 -- 172.21.15.73:0/1427188051 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3818021739 0 0) 0x7f6cd815b9e0 con 0x7f6cd815b600 2024-04-26T20:52:52.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.717+0000 7f6cde2a1640 1 -- 172.21.15.73:0/1427188051 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cd80cb360 con 0x7f6cd80cbbd0 2024-04-26T20:52:52.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.717+0000 7fb4c959a640 1 -- 172.21.15.73:0/433703294 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb4b4003520 con 0x7fb4cc072c50 2024-04-26T20:52:52.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.717+0000 7f62d559a640 1 -- 172.21.15.73:0/4092337470 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2839562242 0 0) 0x7f62c8001040 con 0x7f62d815f3a0 2024-04-26T20:52:52.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.717+0000 7f62d559a640 1 -- 172.21.15.73:0/4092337470 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f62d80d2590 con 0x7f62d815f3a0 2024-04-26T20:52:52.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.717+0000 7f62d559a640 1 -- 172.21.15.73:0/4092337470 <== mon.1 v1:172.21.15.120:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 995337644 0 0) 0x7f62c0003520 con 0x7f62d80d06f0 2024-04-26T20:52:52.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.717+0000 7fdb1f7fe640 1 -- 172.21.15.73:0/3900421816 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4182135175 0 0) 0x7fdb2015b9e0 con 0x7fdb200cf5e0 2024-04-26T20:52:52.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.717+0000 7fdb1f7fe640 1 -- 172.21.15.73:0/3900421816 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdb10003520 con 0x7fdb200cf5e0 2024-04-26T20:52:52.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.717+0000 7fdb1f7fe640 1 -- 172.21.15.73:0/3900421816 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 931224095 0 0) 0x7fdb180013c0 con 0x7fdb200cf0e0 2024-04-26T20:52:52.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.717+0000 7fdb1f7fe640 1 -- 172.21.15.73:0/3900421816 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdb100048e0 con 0x7fdb200cf0e0 2024-04-26T20:52:52.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.717+0000 7f62d559a640 1 -- 172.21.15.73:0/4092337470 >> v2:172.21.15.73:3300/0 conn(0x7f62d80b3860 msgr2=0x7f62d816a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:52.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.717+0000 7fb4c9d9b640 1 -- 172.21.15.73:0/433703294 >> v1:172.21.15.120:6789/0 conn(0x7fb4cc072c50 legacy=0x7fb4cc073060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:52.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.717+0000 7fb4c9d9b640 1 -- 172.21.15.73:0/433703294 >> v1:172.21.15.73:6789/0 conn(0x7fb4cc156460 legacy=0x7fb4cc0cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:52.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.717+0000 7fb4c9d9b640 1 -- 172.21.15.73:0/433703294 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4cc0cc4e0 con 0x7fb4cc06f790 2024-04-26T20:52:52.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.717+0000 7fb4c959a640 1 -- 172.21.15.73:0/433703294 <== mon.1 v1:172.21.15.120:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1274581318 0 0) 0x7fb4b4003520 con 0x7fb4cc072c50 2024-04-26T20:52:52.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.717+0000 7f62d6d9d640 1 --2- 172.21.15.73:0/4092337470 >> v2:172.21.15.73:3300/0 conn(0x7f62d80b3860 0x7f62d816a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:52.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.717+0000 7f62d559a640 1 --2- 172.21.15.73:0/4092337470 >> v2:172.21.15.73:3300/0 conn(0x7f62d80b3860 0x7f62d816a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:52.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.717+0000 7f62d559a640 1 -- 172.21.15.73:0/4092337470 >> v1:172.21.15.73:6789/0 conn(0x7f62d815f3a0 legacy=0x7f62d80d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:52.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.717+0000 7f62d559a640 1 -- 172.21.15.73:0/4092337470 --> v1:172.21.15.120:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62d80b1d70 con 0x7f62d80d06f0 2024-04-26T20:52:52.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.717+0000 7f62d559a640 1 -- 172.21.15.73:0/4092337470 <== mon.1 v1:172.21.15.120:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7f62cc001670 con 0x7f62d80d06f0 2024-04-26T20:52:52.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.718+0000 7f62d559a640 1 -- 172.21.15.73:0/4092337470 >> v1:172.21.15.120:6789/0 conn(0x7f62d80d06f0 legacy=0x7f62d80d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:52.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.718+0000 7f62d559a640 1 --2- 172.21.15.73:0/4092337470 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f62c0003be0 0x7f62c0003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:52.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.718+0000 7f62d559a640 1 --2- 172.21.15.73:0/4092337470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62c0004800 0x7f62c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:52.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.718+0000 7fdb1f7fe640 1 -- 172.21.15.73:0/3900421816 <== mon.1 v1:172.21.15.120:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1687770546 0 0) 0x7fdb10003520 con 0x7fdb200cf5e0 2024-04-26T20:52:52.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.718+0000 7fdb1f7fe640 1 -- 172.21.15.73:0/3900421816 >> v2:172.21.15.120:3300/0 conn(0x7fdb2015b600 msgr2=0x7fdb20071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:52.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.718+0000 7fdb1f7fe640 1 --2- 172.21.15.73:0/3900421816 >> v2:172.21.15.120:3300/0 conn(0x7fdb2015b600 0x7fdb20071f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:52.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.718+0000 7fdb1f7fe640 1 -- 172.21.15.73:0/3900421816 >> v1:172.21.15.73:6789/0 conn(0x7fdb200cf0e0 legacy=0x7fdb2006f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:52.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.718+0000 7fdb1f7fe640 1 -- 172.21.15.73:0/3900421816 --> v1:172.21.15.120:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb200d0740 con 0x7fdb200cf5e0 2024-04-26T20:52:52.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.718+0000 7fdb1f7fe640 1 -- 172.21.15.73:0/3900421816 <== mon.1 v1:172.21.15.120:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7fdb0c001650 con 0x7fdb200cf5e0 2024-04-26T20:52:52.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.718+0000 7f62d559a640 1 --2- 172.21.15.73:0/4092337470 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f62c00053c0 0x7f62c000ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:52.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.718+0000 7f62d559a640 1 -- 172.21.15.73:0/4092337470 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f62d80d0e20 con 0x7f62c0004800 2024-04-26T20:52:52.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.718+0000 7f62d559a640 1 -- 172.21.15.73:0/4092337470 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f62c000c300 con 0x7f62c00053c0 2024-04-26T20:52:52.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.718+0000 7f62d559a640 1 -- 172.21.15.73:0/4092337470 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f62c000c500 con 0x7f62c0003be0 2024-04-26T20:52:52.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.718+0000 7f62d6d9d640 1 --2- 172.21.15.73:0/4092337470 >> v2:172.21.15.73:3300/0 conn(0x7f62d80b3860 0x7f62d816a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:52:52.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.718+0000 7fdb1f7fe640 1 -- 172.21.15.73:0/3900421816 >> v1:172.21.15.120:6789/0 conn(0x7fdb200cf5e0 legacy=0x7fdb200ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:52.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.718+0000 7fdb1f7fe640 1 --2- 172.21.15.73:0/3900421816 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb10004d90 0x7fdb10005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:52.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.718+0000 7fdb1f7fe640 1 --2- 172.21.15.73:0/3900421816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb100037a0 0x7fdb10003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:52.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.718+0000 7fdb1f7fe640 1 --2- 172.21.15.73:0/3900421816 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdb10004270 0x7fdb1000fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:52.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.718+0000 7fdb1f7fe640 1 -- 172.21.15.73:0/3900421816 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdb200cb9b0 con 0x7fdb100037a0 2024-04-26T20:52:52.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.718+0000 7fdb1f7fe640 1 -- 172.21.15.73:0/3900421816 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdb10010640 con 0x7fdb10004d90 2024-04-26T20:52:52.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.718+0000 7fdb1f7fe640 1 -- 172.21.15.73:0/3900421816 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fdb10010840 con 0x7fdb10004270 2024-04-26T20:52:52.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.718+0000 7f62d5d9b640 1 --2- 172.21.15.73:0/4092337470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62c0004800 0x7f62c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:52.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.718+0000 7f62d659c640 1 --2- 172.21.15.73:0/4092337470 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f62c0003be0 0x7f62c0003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:52.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.718+0000 7f62d6d9d640 1 --2- 172.21.15.73:0/4092337470 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f62c00053c0 0x7f62c000ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:52.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.718+0000 7fdb24976640 1 --2- 172.21.15.73:0/3900421816 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb10004d90 0x7fdb10005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:52.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.718+0000 7fdb1ffff640 1 --2- 172.21.15.73:0/3900421816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb100037a0 0x7fdb10003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:52.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.718+0000 7fdb25177640 1 --2- 172.21.15.73:0/3900421816 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdb10004270 0x7fdb1000fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:52.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.718+0000 7f62d5d9b640 1 -- 172.21.15.73:0/4092337470 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f62c0003be0 msgr2=0x7f62c0003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:52.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.718+0000 7f62d5d9b640 1 --2- 172.21.15.73:0/4092337470 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f62c0003be0 0x7f62c0003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:52.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.718+0000 7f62d5d9b640 1 -- 172.21.15.73:0/4092337470 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f62c00053c0 msgr2=0x7f62c000ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:52.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.718+0000 7f62d5d9b640 1 --2- 172.21.15.73:0/4092337470 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f62c00053c0 0x7f62c000ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:52.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.718+0000 7f62d5d9b640 1 -- 172.21.15.73:0/4092337470 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62c0003520 con 0x7f62c0004800 2024-04-26T20:52:52.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.718+0000 7f62d6d9d640 1 -- 172.21.15.73:0/4092337470 reap_dead start 2024-04-26T20:52:52.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.718+0000 7f6cde2a1640 1 --2- 172.21.15.73:0/1427188051 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6cd80cbbd0 0x7f6cd80cef10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6cd40097a0 tx=0x7f6cd402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bce91d02e53dbad5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:52.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.718+0000 7fdb24976640 1 -- 172.21.15.73:0/3900421816 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdb10004270 msgr2=0x7fdb1000fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:52.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.718+0000 7fdb24976640 1 --2- 172.21.15.73:0/3900421816 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdb10004270 0x7fdb1000fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:52.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.718+0000 7fdb24976640 1 -- 172.21.15.73:0/3900421816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb100037a0 msgr2=0x7fdb10003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:52.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.718+0000 7f6cdca9e640 1 -- 172.21.15.73:0/1427188051 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd4037070 con 0x7f6cd80cbbd0 2024-04-26T20:52:52.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.719+0000 7f6cdca9e640 1 -- 172.21.15.73:0/1427188051 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cd4032440 con 0x7f6cd80cbbd0 2024-04-26T20:52:52.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.719+0000 7f6cdca9e640 1 -- 172.21.15.73:0/1427188051 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd4032810 con 0x7f6cd80cbbd0 2024-04-26T20:52:52.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.719+0000 7fb4c9d9b640 1 --2- 172.21.15.73:0/433703294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4cc06f790 0x7fb4cc06fc40 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0009720 tx=0x7fb4c002a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e8d6dbf340c1f71 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:52.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.719+0000 7fb4c959a640 1 -- 172.21.15.73:0/433703294 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c0037070 con 0x7fb4cc06f790 2024-04-26T20:52:52.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.719+0000 7fb4c959a640 1 -- 172.21.15.73:0/433703294 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4c002ac60 con 0x7fb4cc06f790 2024-04-26T20:52:52.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.719+0000 7fb4c959a640 1 -- 172.21.15.73:0/433703294 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c40019f0 con 0x7fb4cc06f790 2024-04-26T20:52:52.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.718+0000 7fdb24976640 1 --2- 172.21.15.73:0/3900421816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb100037a0 0x7fdb10003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:52.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.718+0000 7fdb25177640 1 -- 172.21.15.73:0/3900421816 reap_dead start 2024-04-26T20:52:52.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.718+0000 7fdb24976640 1 -- 172.21.15.73:0/3900421816 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb100048e0 con 0x7fdb10004d90 2024-04-26T20:52:52.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.718+0000 7fdb1ffff640 1 --2- 172.21.15.73:0/3900421816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb100037a0 0x7fdb10003c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:52:52.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.719+0000 7f62d5d9b640 1 --2- 172.21.15.73:0/4092337470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62c0004800 0x7f62c0004c80 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f62cc007f10 tx=0x7f62c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e2a20ada6e7916d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:52.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.719+0000 7f6cdfd04640 1 -- 172.21.15.73:0/1427188051 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6cd80cbbd0 msgr2=0x7f6cd80cef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:52.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.719+0000 7f6cdfd04640 1 --2- 172.21.15.73:0/1427188051 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6cd80cbbd0 0x7f6cd80cef10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6cd40097a0 tx=0x7f6cd402a180 comp rx=0 tx=0).stop 2024-04-26T20:52:52.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.719+0000 7f6cdfd04640 1 -- 172.21.15.73:0/1427188051 shutdown_connections 2024-04-26T20:52:52.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.719+0000 7f6cdfd04640 1 --2- 172.21.15.73:0/1427188051 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6cd80cbbd0 0x7f6cd80cef10 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:52.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.719+0000 7f6cdfd04640 1 --2- 172.21.15.73:0/1427188051 >> v2:172.21.15.120:3300/0 conn(0x7f6cd80cf5e0 0x7f6cd80ce9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:52.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.719+0000 7f6cdfd04640 1 -- 172.21.15.73:0/1427188051 >> 172.21.15.73:0/1427188051 conn(0x7f6cd80d0be0 msgr2=0x7f6cd80d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:52.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.719+0000 7f6cdfd04640 1 -- 172.21.15.73:0/1427188051 shutdown_connections 2024-04-26T20:52:52.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.720+0000 7f6cdfd04640 1 -- 172.21.15.73:0/1427188051 wait complete. 2024-04-26T20:52:52.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.720+0000 7f62d559a640 1 -- 172.21.15.73:0/4092337470 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62cc007b30 con 0x7f62c0004800 2024-04-26T20:52:52.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.720+0000 7f62d559a640 1 -- 172.21.15.73:0/4092337470 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62cc031620 con 0x7f62c0004800 2024-04-26T20:52:52.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.720+0000 7f62d559a640 1 -- 172.21.15.73:0/4092337470 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62cc031a10 con 0x7f62c0004800 2024-04-26T20:52:52.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.719+0000 7fdb24976640 1 --2- 172.21.15.73:0/3900421816 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb10004d90 0x7fdb10005170 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdb1400bc30 tx=0x7fdb10005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b4455f979c0364d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:52.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.720+0000 7fdb1f7fe640 1 -- 172.21.15.73:0/3900421816 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb14038070 con 0x7fdb10004d90 2024-04-26T20:52:52.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.720+0000 7fb4d0d04640 1 -- 172.21.15.73:0/433703294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4cc06f790 msgr2=0x7fb4cc06fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:52.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.720+0000 7fb4d0d04640 1 --2- 172.21.15.73:0/433703294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4cc06f790 0x7fb4cc06fc40 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0009720 tx=0x7fb4c002a150 comp rx=0 tx=0).stop 2024-04-26T20:52:52.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.720+0000 7fdb1f7fe640 1 -- 172.21.15.73:0/3900421816 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb14033410 con 0x7fdb10004d90 2024-04-26T20:52:52.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.720+0000 7fdb1f7fe640 1 -- 172.21.15.73:0/3900421816 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1400b800 con 0x7fdb10004d90 2024-04-26T20:52:52.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.720+0000 7fb4d0d04640 1 -- 172.21.15.73:0/433703294 shutdown_connections 2024-04-26T20:52:52.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.720+0000 7fb4d0d04640 1 --2- 172.21.15.73:0/433703294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4cc06f790 0x7fb4cc06fc40 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:52.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.720+0000 7fb4d0d04640 1 -- 172.21.15.73:0/433703294 >> 172.21.15.73:0/433703294 conn(0x7fb4cc0d0be0 msgr2=0x7fb4cc0d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:52.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.720+0000 7fb4d0d04640 1 -- 172.21.15.73:0/433703294 shutdown_connections 2024-04-26T20:52:52.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.720+0000 7fb4d0d04640 1 -- 172.21.15.73:0/433703294 wait complete. 2024-04-26T20:52:52.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.720+0000 7f62dcc64640 1 -- 172.21.15.73:0/4092337470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62c0004800 msgr2=0x7f62c0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:52.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.720+0000 7f62dcc64640 1 --2- 172.21.15.73:0/4092337470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62c0004800 0x7f62c0004c80 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f62cc007f10 tx=0x7f62c0005830 comp rx=0 tx=0).stop 2024-04-26T20:52:52.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.721+0000 7fdb26bda640 1 -- 172.21.15.73:0/3900421816 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb10004d90 msgr2=0x7fdb10005170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:52.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.721+0000 7fdb26bda640 1 --2- 172.21.15.73:0/3900421816 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb10004d90 0x7fdb10005170 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdb1400bc30 tx=0x7fdb10005830 comp rx=0 tx=0).stop 2024-04-26T20:52:52.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.721+0000 7f62dcc64640 1 -- 172.21.15.73:0/4092337470 shutdown_connections 2024-04-26T20:52:52.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.721+0000 7f62dcc64640 1 --2- 172.21.15.73:0/4092337470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62c0004800 0x7f62c0004c80 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:52.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.721+0000 7f62dcc64640 1 -- 172.21.15.73:0/4092337470 >> 172.21.15.73:0/4092337470 conn(0x7f62d80d4f90 msgr2=0x7f62d80d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:52.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.721+0000 7f62dcc64640 1 -- 172.21.15.73:0/4092337470 shutdown_connections 2024-04-26T20:52:52.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.721+0000 7fdb26bda640 1 -- 172.21.15.73:0/3900421816 shutdown_connections 2024-04-26T20:52:52.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.721+0000 7fdb26bda640 1 --2- 172.21.15.73:0/3900421816 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb10004d90 0x7fdb10005170 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:52.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.721+0000 7fdb26bda640 1 -- 172.21.15.73:0/3900421816 >> 172.21.15.73:0/3900421816 conn(0x7fdb200d0be0 msgr2=0x7fdb200d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:52.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.721+0000 7f62dcc64640 1 -- 172.21.15.73:0/4092337470 wait complete. 2024-04-26T20:52:52.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.721+0000 7f6cdfd04640 1 Processor -- start 2024-04-26T20:52:52.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.721+0000 7fdb26bda640 1 -- 172.21.15.73:0/3900421816 shutdown_connections 2024-04-26T20:52:52.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.721+0000 7f6cdfd04640 1 -- start start 2024-04-26T20:52:52.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.721+0000 7fdb26bda640 1 -- 172.21.15.73:0/3900421816 wait complete. 2024-04-26T20:52:52.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.722+0000 7f6cdfd04640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6cd80cbbd0 0x7f6cd81571e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:52.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.722+0000 7f6cdfd04640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cd80cf5e0 0x7f6cd80c3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:52.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.722+0000 7fb4d0d04640 1 Processor -- start 2024-04-26T20:52:52.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.722+0000 7f6cdfd04640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6cd815b600 0x7f6cd81e7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:52.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.722+0000 7f6cdfd04640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd80d0470 con 0x7f6cd80cf5e0 2024-04-26T20:52:52.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.722+0000 7f6cdfd04640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd80d02f0 con 0x7f6cd80cbbd0 2024-04-26T20:52:52.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.722+0000 7f6cdfd04640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f6cd806ddf0 con 0x7f6cd815b600 2024-04-26T20:52:52.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.722+0000 7fb4d0d04640 1 -- start start 2024-04-26T20:52:52.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.722+0000 7f6cdd29f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cd80cf5e0 0x7f6cd80c3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:52.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.722+0000 7f6cdd29f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cd80cf5e0 0x7f6cd80c3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47792/0 (socket says 172.21.15.73:47792) 2024-04-26T20:52:52.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.722+0000 7f6cdd29f640 1 -- 172.21.15.73:0/2592913668 learned_addr learned my addr 172.21.15.73:0/2592913668 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:52.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.722+0000 7f6cddaa0640 1 --2- 172.21.15.73:0/2592913668 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6cd80cbbd0 0x7f6cd81571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:52.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.722+0000 7f6cde2a1640 1 --2- 172.21.15.73:0/2592913668 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6cd815b600 0x7f6cd81e7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:52.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.722+0000 7f6cdd29f640 1 -- 172.21.15.73:0/2592913668 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6cd815b600 msgr2=0x7f6cd81e7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:52.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.722+0000 7f6cdd29f640 1 --2- 172.21.15.73:0/2592913668 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6cd815b600 0x7f6cd81e7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:52.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.722+0000 7f6cdd29f640 1 -- 172.21.15.73:0/2592913668 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6cd80cbbd0 msgr2=0x7f6cd81571e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:52.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.722+0000 7f6cdd29f640 1 --2- 172.21.15.73:0/2592913668 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6cd80cbbd0 0x7f6cd81571e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:52.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.722+0000 7f6cdd29f640 1 -- 172.21.15.73:0/2592913668 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cd80c5650 con 0x7f6cd80cf5e0 2024-04-26T20:52:52.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.722+0000 7f6cde2a1640 1 --2- 172.21.15.73:0/2592913668 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6cd815b600 0x7f6cd81e7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:52:52.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.722+0000 7f6cdd29f640 1 --2- 172.21.15.73:0/2592913668 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cd80cf5e0 0x7f6cd80c3dc0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc00f9f0 tx=0x7f6ccc00feb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:52.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.722+0000 7f6cca7fc640 1 -- 172.21.15.73:0/2592913668 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc017070 con 0x7f6cd80cf5e0 2024-04-26T20:52:52.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.722+0000 7f6cca7fc640 1 -- 172.21.15.73:0/2592913668 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ccc00dd70 con 0x7f6cd80cf5e0 2024-04-26T20:52:52.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.723+0000 7f6cdfd04640 1 -- 172.21.15.73:0/2592913668 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cd80c5930 con 0x7f6cd80cf5e0 2024-04-26T20:52:52.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.723+0000 7fb4d0d04640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4cc06f790 0x7fb4cc15ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:52.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.723+0000 7f6cca7fc640 1 -- 172.21.15.73:0/2592913668 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc0126a0 con 0x7f6cd80cf5e0 2024-04-26T20:52:52.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.723+0000 7fdb26bda640 1 Processor -- start 2024-04-26T20:52:52.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.723+0000 7fb4d0d04640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4cc072c50 0x7fb4cc15de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:52.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.723+0000 7f6cdfd04640 1 -- 172.21.15.73:0/2592913668 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cd80c5ef0 con 0x7f6cd80cf5e0 2024-04-26T20:52:52.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.723+0000 7fb4d0d04640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb4cc156460 0x7fb4cc1e88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:52.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.723+0000 7fdb26bda640 1 -- start start 2024-04-26T20:52:52.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.723+0000 7fb4d0d04640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb4cc0cbbe0 con 0x7fb4cc072c50 2024-04-26T20:52:52.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.723+0000 7fb4c9d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4cc072c50 0x7fb4cc15de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:52.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.723+0000 7fb4d0d04640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb4cc072700 con 0x7fb4cc06f790 2024-04-26T20:52:52.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.723+0000 7fb4d0d04640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fb4cc06e260 con 0x7fb4cc156460 2024-04-26T20:52:52.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.723+0000 7f62dcc64640 1 Processor -- start 2024-04-26T20:52:52.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.723+0000 7fb4c9d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4cc072c50 0x7fb4cc15de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47804/0 (socket says 172.21.15.73:47804) 2024-04-26T20:52:52.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.723+0000 7fb4c9d9b640 1 -- 172.21.15.73:0/2441907190 learned_addr learned my addr 172.21.15.73:0/2441907190 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:52.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.723+0000 7fb4cad9d640 1 --2- 172.21.15.73:0/2441907190 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb4cc156460 0x7fb4cc1e88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:52.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.723+0000 7f62dcc64640 1 -- start start 2024-04-26T20:52:52.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.723+0000 7fb4c9d9b640 1 -- 172.21.15.73:0/2441907190 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb4cc156460 msgr2=0x7fb4cc1e88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:52.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.723+0000 7fb4c9d9b640 1 --2- 172.21.15.73:0/2441907190 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb4cc156460 0x7fb4cc1e88a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:52.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.723+0000 7fb4c9d9b640 1 -- 172.21.15.73:0/2441907190 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4cc06f790 msgr2=0x7fb4cc15ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:52.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.723+0000 7fb4c9d9b640 1 --2- 172.21.15.73:0/2441907190 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4cc06f790 0x7fb4cc15ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:52.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.723+0000 7fb4c9d9b640 1 -- 172.21.15.73:0/2441907190 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4cc0ca940 con 0x7fb4cc072c50 2024-04-26T20:52:52.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.723+0000 7fb4c9d9b640 1 --2- 172.21.15.73:0/2441907190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4cc072c50 0x7fb4cc15de50 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fb4c002a190 tx=0x7fb4c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:52.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.724+0000 7fb4baffd640 1 -- 172.21.15.73:0/2441907190 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c0041070 con 0x7fb4cc072c50 2024-04-26T20:52:52.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.724+0000 7fb4baffd640 1 -- 172.21.15.73:0/2441907190 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4c003c670 con 0x7fb4cc072c50 2024-04-26T20:52:52.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.724+0000 7fb4baffd640 1 -- 172.21.15.73:0/2441907190 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c0037040 con 0x7fb4cc072c50 2024-04-26T20:52:52.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.724+0000 7f6cdfd04640 1 -- 172.21.15.73:0/2592913668 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cd80694f0 con 0x7f6cd80cf5e0 2024-04-26T20:52:52.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.724+0000 7f62dcc64640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f62c0004800 0x7f62d815dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:52.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.724+0000 7f62dcc64640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f62d80cca40 0x7f62d80ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:52.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.724+0000 7fdb26bda640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb10004d90 0x7fdb201f4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:52.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.724+0000 7fb4d0d04640 1 -- 172.21.15.73:0/2441907190 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4cc0c9b80 con 0x7fb4cc072c50 2024-04-26T20:52:52.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.724+0000 7fb4d0d04640 1 -- 172.21.15.73:0/2441907190 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4cc0ca140 con 0x7fb4cc072c50 2024-04-26T20:52:52.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.724+0000 7f62dcc64640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62d81f97f0 0x7f62d81f9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:52.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.724+0000 7f62dcc64640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f62d815f780 con 0x7f62d81f97f0 2024-04-26T20:52:52.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.724+0000 7fdb26bda640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdb200b5a50 0x7fdb200b5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:52.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.724+0000 7f62d5d9b640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f62d80cca40 0x7f62d80ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:52.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.724+0000 7fdb26bda640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb201f1050 0x7fdb201f14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:52.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.724+0000 7f62d5d9b640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f62d80cca40 0x7f62d80ccec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.73:60670/0 (socket says 172.21.15.73:60670) 2024-04-26T20:52:52.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.725+0000 7fdb24976640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb10004d90 0x7fdb201f4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:52.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.725+0000 7fdb24976640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb10004d90 0x7fdb201f4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47806/0 (socket says 172.21.15.73:47806) 2024-04-26T20:52:52.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.725+0000 7fdb26bda640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdb200d02f0 con 0x7fdb10004d90 2024-04-26T20:52:52.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.725+0000 7fdb26bda640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdb200cb640 con 0x7fdb201f1050 2024-04-26T20:52:52.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.725+0000 7fdb26bda640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fdb200d0470 con 0x7fdb200b5a50 2024-04-26T20:52:52.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.724+0000 7f62d5d9b640 1 -- 172.21.15.73:0/985045113 learned_addr learned my addr 172.21.15.73:0/985045113 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:52.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.725+0000 7fdb24976640 1 -- 172.21.15.73:0/4193596021 learned_addr learned my addr 172.21.15.73:0/4193596021 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:52.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.725+0000 7f62d659c640 1 --2- 172.21.15.73:0/985045113 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f62c0004800 0x7f62d815dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:52.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.725+0000 7f62dcc64640 1 -- 172.21.15.73:0/985045113 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f62d80b2330 con 0x7f62c0004800 2024-04-26T20:52:52.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.725+0000 7f62dcc64640 1 -- 172.21.15.73:0/985045113 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f62d815f900 con 0x7f62d80cca40 2024-04-26T20:52:52.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.725+0000 7f62d6d9d640 1 --2- 172.21.15.73:0/985045113 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62d81f97f0 0x7f62d81f9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:52.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.725+0000 7fdb25177640 1 --2- 172.21.15.73:0/4193596021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb201f1050 0x7fdb201f14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:52.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.725+0000 7f6cca7fc640 1 -- 172.21.15.73:0/2592913668 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6ccc01a070 con 0x7f6cd80cf5e0 2024-04-26T20:52:52.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.725+0000 7f62d5d9b640 1 -- 172.21.15.73:0/985045113 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f62c0004800 msgr2=0x7f62d815dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:52.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.725+0000 7fb4d0d04640 1 -- 172.21.15.73:0/2441907190 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4cc0694f0 con 0x7fb4cc072c50 2024-04-26T20:52:52.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.725+0000 7fdb25177640 1 -- 172.21.15.73:0/4193596021 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdb200b5a50 msgr2=0x7fdb200b5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:52.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.725+0000 7fdb25177640 1 --2- 172.21.15.73:0/4193596021 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdb200b5a50 0x7fdb200b5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:52.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.725+0000 7fdb25177640 1 -- 172.21.15.73:0/4193596021 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb10004d90 msgr2=0x7fdb201f4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:52.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.725+0000 7fdb25177640 1 --2- 172.21.15.73:0/4193596021 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb10004d90 0x7fdb201f4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:52.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.725+0000 7fdb25177640 1 -- 172.21.15.73:0/4193596021 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb201f1a10 con 0x7fdb201f1050 2024-04-26T20:52:52.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.725+0000 7f62d5d9b640 1 --2- 172.21.15.73:0/985045113 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f62c0004800 0x7f62d815dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:52.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.725+0000 7fdb24976640 1 --2- 172.21.15.73:0/4193596021 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb10004d90 0x7fdb201f4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:52:52.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.725+0000 7f62d5d9b640 1 -- 172.21.15.73:0/985045113 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62d81f97f0 msgr2=0x7f62d81f9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:52.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.725+0000 7f62d5d9b640 1 --2- 172.21.15.73:0/985045113 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62d81f97f0 0x7f62d81f9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:52.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.725+0000 7f62d5d9b640 1 -- 172.21.15.73:0/985045113 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62d81fa1b0 con 0x7f62d80cca40 2024-04-26T20:52:52.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.726+0000 7fdb25177640 1 --2- 172.21.15.73:0/4193596021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb201f1050 0x7fdb201f14d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fdb20160180 tx=0x7fdb1800d520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:52.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.726+0000 7fb4baffd640 1 -- 172.21.15.73:0/2441907190 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb4c0044070 con 0x7fb4cc072c50 2024-04-26T20:52:52.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.726+0000 7f6cca7fc640 1 --2- 172.21.15.73:0/2592913668 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f6cbc06c8f0 0x7f6cbc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:52.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.726+0000 7f62d5d9b640 1 --2- 172.21.15.73:0/985045113 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f62d80cca40 0x7f62d80ccec0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f62cc00c0e0 tx=0x7f62cc001970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:52.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.726+0000 7fdb1d7fa640 1 -- 172.21.15.73:0/4193596021 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1800db80 con 0x7fdb201f1050 2024-04-26T20:52:52.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.726+0000 7f6cddaa0640 1 --2- 172.21.15.73:0/2592913668 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f6cbc06c8f0 0x7f6cbc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:52.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.726+0000 7f6cca7fc640 1 -- 172.21.15.73:0/2592913668 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f6ccc08c0d0 con 0x7f6cd80cf5e0 2024-04-26T20:52:52.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.726+0000 7fdb1d7fa640 1 -- 172.21.15.73:0/4193596021 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb18018070 con 0x7fdb201f1050 2024-04-26T20:52:52.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.726+0000 7fdb1d7fa640 1 -- 172.21.15.73:0/4193596021 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb18003970 con 0x7fdb201f1050 2024-04-26T20:52:52.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.726+0000 7f62c6ffd640 1 -- 172.21.15.73:0/985045113 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62cc03f070 con 0x7f62d80cca40 2024-04-26T20:52:52.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.726+0000 7fdb26bda640 1 -- 172.21.15.73:0/4193596021 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb201f2540 con 0x7fdb201f1050 2024-04-26T20:52:52.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.726+0000 7f62c6ffd640 1 -- 172.21.15.73:0/985045113 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62cc0052b0 con 0x7f62d80cca40 2024-04-26T20:52:52.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.726+0000 7fdb26bda640 1 -- 172.21.15.73:0/4193596021 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb201f1f60 con 0x7fdb201f1050 2024-04-26T20:52:52.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.726+0000 7f62c6ffd640 1 -- 172.21.15.73:0/985045113 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62cc003830 con 0x7f62d80cca40 2024-04-26T20:52:52.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.726+0000 7f62dcc64640 1 -- 172.21.15.73:0/985045113 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62d81fa640 con 0x7f62d80cca40 2024-04-26T20:52:52.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.726+0000 7f62dcc64640 1 -- 172.21.15.73:0/985045113 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f62d80cd8c0 con 0x7f62d80cca40 2024-04-26T20:52:52.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.727+0000 7f6cddaa0640 1 --2- 172.21.15.73:0/2592913668 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f6cbc06c8f0 0x7f6cbc06edb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6cd00012e0 tx=0x7f6cd0001630 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:52.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.726+0000 7fb4baffd640 1 --2- 172.21.15.73:0/2441907190 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb4ac06c8f0 0x7fb4ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:52.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.726+0000 7fb4ca59c640 1 --2- 172.21.15.73:0/2441907190 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb4ac06c8f0 0x7fb4ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:52.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.727+0000 7fb4baffd640 1 -- 172.21.15.73:0/2441907190 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7fb4c00ac5a0 con 0x7fb4cc072c50 2024-04-26T20:52:52.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.727+0000 7fb4ca59c640 1 --2- 172.21.15.73:0/2441907190 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb4ac06c8f0 0x7fb4ac06edb0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc000f60 tx=0x7fb4bc007a80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:52.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.728+0000 7f62dcc64640 1 -- 172.21.15.73:0/985045113 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62d80694f0 con 0x7f62d80cca40 2024-04-26T20:52:52.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.728+0000 7fdb26bda640 1 -- 172.21.15.73:0/4193596021 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb200694f0 con 0x7fdb201f1050 2024-04-26T20:52:52.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.728+0000 7fdb1d7fa640 1 -- 172.21.15.73:0/4193596021 <== mon.1 v2:172.21.15.120:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdb18003b10 con 0x7fdb201f1050 2024-04-26T20:52:52.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.728+0000 7f62c6ffd640 1 -- 172.21.15.73:0/985045113 <== mon.2 v2:172.21.15.120:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f62cc042070 con 0x7f62d80cca40 2024-04-26T20:52:52.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.728+0000 7fdb1d7fa640 1 --2- 172.21.15.73:0/4193596021 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fdb0406c8f0 0x7fdb0406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:52.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.728+0000 7f62c6ffd640 1 --2- 172.21.15.73:0/985045113 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f62b806c970 0x7f62b806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:52.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.728+0000 7fdb1d7fa640 1 -- 172.21.15.73:0/4193596021 <== mon.1 v2:172.21.15.120:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7fdb1808bbf0 con 0x7fdb201f1050 2024-04-26T20:52:52.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.728+0000 7fdb24976640 1 --2- 172.21.15.73:0/4193596021 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fdb0406c8f0 0x7fdb0406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:52.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.728+0000 7f62c6ffd640 1 -- 172.21.15.73:0/985045113 <== mon.2 v2:172.21.15.120:3301/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f62cc0ac420 con 0x7f62d80cca40 2024-04-26T20:52:52.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.728+0000 7f62d659c640 1 --2- 172.21.15.73:0/985045113 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f62b806c970 0x7f62b806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:52.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.729+0000 7fdb24976640 1 --2- 172.21.15.73:0/4193596021 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fdb0406c8f0 0x7fdb0406edb0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fdb14033590 tx=0x7fdb1402c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:52.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.729+0000 7f62d659c640 1 --2- 172.21.15.73:0/985045113 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f62b806c970 0x7f62b806ee30 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f62c8007fd0 tx=0x7f62c8007dc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:52.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.731+0000 7f6cca7fc640 1 -- 172.21.15.73:0/2592913668 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6ccc059db0 con 0x7f6cd80cf5e0 2024-04-26T20:52:52.771 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.733+0000 7f62c6ffd640 1 -- 172.21.15.73:0/985045113 <== mon.2 v2:172.21.15.120:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f62d81fa1b0 con 0x7f62d80cca40 2024-04-26T20:52:52.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.733+0000 7fdb1d7fa640 1 -- 172.21.15.73:0/4193596021 <== mon.1 v2:172.21.15.120:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdb20072450 con 0x7fdb201f1050 2024-04-26T20:52:52.772 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.737+0000 7fb4baffd640 1 -- 172.21.15.73:0/2441907190 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb4c007a840 con 0x7fb4cc072c50 2024-04-26T20:52:52.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.987+0000 7f6cdfd04640 1 -- 172.21.15.73:0/2592913668 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "d"} v 0) -- 0x7f6cd80b1fd0 con 0x7f6cd80cf5e0 2024-04-26T20:52:52.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.990+0000 7f62dcc64640 1 -- 172.21.15.73:0/985045113 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "b"} v 0) -- 0x7f62d80cbe20 con 0x7f62d80cca40 2024-04-26T20:52:52.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.993+0000 7fdb26bda640 1 -- 172.21.15.73:0/4193596021 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "a"} v 0) -- 0x7fdb200ce7b0 con 0x7fdb201f1050 2024-04-26T20:52:52.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:52.997+0000 7fb4d0d04640 1 -- 172.21.15.73:0/2441907190 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "c"} v 0) -- 0x7fb4cc06de80 con 0x7fb4cc072c50 2024-04-26T20:52:53.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.527+0000 7f6cca7fc640 1 -- 172.21.15.73:0/2592913668 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "d"}]=0 failed mds gid 5099 v28) ==== 95+0+0 (secure 0 0 0) 0x7f6ccc05e0b0 con 0x7f6cd80cf5e0 2024-04-26T20:52:53.528 INFO:teuthology.orchestra.run.smithi073.stderr:failed mds gid 5099 2024-04-26T20:52:53.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.529+0000 7f6cdfd04640 1 -- 172.21.15.73:0/2592913668 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f6cbc06c8f0 msgr2=0x7f6cbc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:53.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.529+0000 7f6cdfd04640 1 --2- 172.21.15.73:0/2592913668 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f6cbc06c8f0 0x7f6cbc06edb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6cd00012e0 tx=0x7f6cd0001630 comp rx=0 tx=0).stop 2024-04-26T20:52:53.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.529+0000 7f6cdfd04640 1 -- 172.21.15.73:0/2592913668 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cd80cf5e0 msgr2=0x7f6cd80c3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:53.530 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.529+0000 7f6cdfd04640 1 --2- 172.21.15.73:0/2592913668 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cd80cf5e0 0x7f6cd80c3dc0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc00f9f0 tx=0x7f6ccc00feb0 comp rx=0 tx=0).stop 2024-04-26T20:52:53.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.529+0000 7f6cdfd04640 1 -- 172.21.15.73:0/2592913668 shutdown_connections 2024-04-26T20:52:53.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.529+0000 7f6cdfd04640 1 --2- 172.21.15.73:0/2592913668 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cd80cf5e0 0x7f6cd80c3dc0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:53.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.529+0000 7f6cdfd04640 1 --2- 172.21.15.73:0/2592913668 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f6cbc06c8f0 0x7f6cbc06edb0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:53.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.529+0000 7f6cdfd04640 1 --2- 172.21.15.73:0/2592913668 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6cd815b600 0x7f6cd81e7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:53.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.529+0000 7f6cdfd04640 1 --2- 172.21.15.73:0/2592913668 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6cd80cbbd0 0x7f6cd81571e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:53.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.529+0000 7f6cdfd04640 1 -- 172.21.15.73:0/2592913668 >> 172.21.15.73:0/2592913668 conn(0x7f6cd80d0be0 msgr2=0x7f6cd80b17e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:53.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.529+0000 7f6cdfd04640 1 -- 172.21.15.73:0/2592913668 shutdown_connections 2024-04-26T20:52:53.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.529+0000 7f6cdfd04640 1 -- 172.21.15.73:0/2592913668 wait complete. 2024-04-26T20:52:53.540 INFO:tasks.ceph.mon.a.smithi073.stderr:2024-04-26T20:52:53.539+0000 7f712829d640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T20:52:53.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.541+0000 7fb4baffd640 1 -- 172.21.15.73:0/2441907190 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "c"}]=0 failed mds gid 5856 v29) ==== 95+0+0 (secure 0 0 0) 0x7fb4c007a140 con 0x7fb4cc072c50 2024-04-26T20:52:53.543 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T20:52:53.543 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T20:52:53.543 INFO:teuthology.orchestra.run.smithi073.stderr:failed mds gid 5856 2024-04-26T20:52:53.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.543+0000 7fb4d0d04640 1 -- 172.21.15.73:0/2441907190 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb4ac06c8f0 msgr2=0x7fb4ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:53.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.543+0000 7fb4d0d04640 1 --2- 172.21.15.73:0/2441907190 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb4ac06c8f0 0x7fb4ac06edb0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc000f60 tx=0x7fb4bc007a80 comp rx=0 tx=0).stop 2024-04-26T20:52:53.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.543+0000 7fb4d0d04640 1 -- 172.21.15.73:0/2441907190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4cc072c50 msgr2=0x7fb4cc15de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:53.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.543+0000 7fb4d0d04640 1 --2- 172.21.15.73:0/2441907190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4cc072c50 0x7fb4cc15de50 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fb4c002a190 tx=0x7fb4c00046c0 comp rx=0 tx=0).stop 2024-04-26T20:52:53.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.543+0000 7fb4d0d04640 1 -- 172.21.15.73:0/2441907190 shutdown_connections 2024-04-26T20:52:53.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.543+0000 7fb4d0d04640 1 --2- 172.21.15.73:0/2441907190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4cc072c50 0x7fb4cc15de50 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:53.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.543+0000 7fb4d0d04640 1 --2- 172.21.15.73:0/2441907190 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb4ac06c8f0 0x7fb4ac06edb0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:53.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.543+0000 7fb4d0d04640 1 --2- 172.21.15.73:0/2441907190 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb4cc156460 0x7fb4cc1e88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:53.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.543+0000 7fb4d0d04640 1 --2- 172.21.15.73:0/2441907190 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb4cc06f790 0x7fb4cc15ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:53.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.543+0000 7fb4d0d04640 1 -- 172.21.15.73:0/2441907190 >> 172.21.15.73:0/2441907190 conn(0x7fb4cc0d0be0 msgr2=0x7fb4cc0b1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:53.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.543+0000 7fb4d0d04640 1 -- 172.21.15.73:0/2441907190 shutdown_connections 2024-04-26T20:52:53.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.543+0000 7fb4d0d04640 1 -- 172.21.15.73:0/2441907190 wait complete. 2024-04-26T20:52:53.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.544+0000 7fdb1d7fa640 1 -- 172.21.15.73:0/4193596021 <== mon.1 v2:172.21.15.120:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "a"}]=0 failed mds gid 5853 v29) ==== 95+0+0 (secure 0 0 0) 0x7fdb18059ec0 con 0x7fdb201f1050 2024-04-26T20:52:53.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.544+0000 7f62c6ffd640 1 -- 172.21.15.73:0/985045113 <== mon.2 v2:172.21.15.120:3301/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "b"}]=0 failed mds gid 5096 v29) ==== 95+0+0 (secure 0 0 0) 0x7f62cc02ae20 con 0x7f62d80cca40 2024-04-26T20:52:53.547 INFO:teuthology.orchestra.run.smithi073.stderr:failed mds gid 5853 2024-04-26T20:52:53.547 INFO:teuthology.orchestra.run.smithi073.stderr:failed mds gid 5096 2024-04-26T20:52:53.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.547+0000 7f62dcc64640 1 -- 172.21.15.73:0/985045113 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f62b806c970 msgr2=0x7f62b806ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:53.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.547+0000 7f62dcc64640 1 --2- 172.21.15.73:0/985045113 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f62b806c970 0x7f62b806ee30 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f62c8007fd0 tx=0x7f62c8007dc0 comp rx=0 tx=0).stop 2024-04-26T20:52:53.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.547+0000 7f62dcc64640 1 -- 172.21.15.73:0/985045113 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f62d80cca40 msgr2=0x7f62d80ccec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:53.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.547+0000 7f62dcc64640 1 --2- 172.21.15.73:0/985045113 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f62d80cca40 0x7f62d80ccec0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f62cc00c0e0 tx=0x7f62cc001970 comp rx=0 tx=0).stop 2024-04-26T20:52:53.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.547+0000 7f62dcc64640 1 -- 172.21.15.73:0/985045113 shutdown_connections 2024-04-26T20:52:53.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.547+0000 7fdb26bda640 1 -- 172.21.15.73:0/4193596021 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fdb0406c8f0 msgr2=0x7fdb0406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:53.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.547+0000 7fdb26bda640 1 --2- 172.21.15.73:0/4193596021 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fdb0406c8f0 0x7fdb0406edb0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fdb14033590 tx=0x7fdb1402c000 comp rx=0 tx=0).stop 2024-04-26T20:52:53.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.547+0000 7fdb26bda640 1 -- 172.21.15.73:0/4193596021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb201f1050 msgr2=0x7fdb201f14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:53.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.547+0000 7fdb26bda640 1 --2- 172.21.15.73:0/4193596021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb201f1050 0x7fdb201f14d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fdb20160180 tx=0x7fdb1800d520 comp rx=0 tx=0).stop 2024-04-26T20:52:53.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.547+0000 7f62dcc64640 1 --2- 172.21.15.73:0/985045113 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62d81f97f0 0x7f62d81f9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:53.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.547+0000 7f62dcc64640 1 --2- 172.21.15.73:0/985045113 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f62b806c970 0x7f62b806ee30 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:53.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.547+0000 7f62dcc64640 1 --2- 172.21.15.73:0/985045113 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f62d80cca40 0x7f62d80ccec0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:53.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.547+0000 7f62dcc64640 1 --2- 172.21.15.73:0/985045113 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f62c0004800 0x7f62d815dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:53.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.547+0000 7f62dcc64640 1 -- 172.21.15.73:0/985045113 >> 172.21.15.73:0/985045113 conn(0x7f62d80d4f90 msgr2=0x7f62d80d3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:53.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.547+0000 7f62dcc64640 1 -- 172.21.15.73:0/985045113 shutdown_connections 2024-04-26T20:52:53.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.547+0000 7f62dcc64640 1 -- 172.21.15.73:0/985045113 wait complete. 2024-04-26T20:52:53.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.547+0000 7fdb26bda640 1 -- 172.21.15.73:0/4193596021 shutdown_connections 2024-04-26T20:52:53.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.547+0000 7fdb26bda640 1 --2- 172.21.15.73:0/4193596021 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fdb0406c8f0 0x7fdb0406edb0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:53.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.547+0000 7fdb26bda640 1 --2- 172.21.15.73:0/4193596021 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdb201f1050 0x7fdb201f14d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:53.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.547+0000 7fdb26bda640 1 --2- 172.21.15.73:0/4193596021 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdb200b5a50 0x7fdb200b5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:53.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.547+0000 7fdb26bda640 1 --2- 172.21.15.73:0/4193596021 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb10004d90 0x7fdb201f4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:53.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.547+0000 7fdb26bda640 1 -- 172.21.15.73:0/4193596021 >> 172.21.15.73:0/4193596021 conn(0x7fdb200d0be0 msgr2=0x7fdb200728c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:53.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.547+0000 7fdb26bda640 1 -- 172.21.15.73:0/4193596021 shutdown_connections 2024-04-26T20:52:53.551 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.547+0000 7fdb26bda640 1 -- 172.21.15.73:0/4193596021 wait complete. 2024-04-26T20:52:53.551 INFO:tasks.ceph.mds.d:Started 2024-04-26T20:52:53.552 DEBUG:teuthology.parallel:result is None 2024-04-26T20:52:53.560 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T20:52:53.560 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T20:52:53.562 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T20:52:53.562 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T20:52:53.564 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T20:52:53.564 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T20:52:53.566 INFO:tasks.ceph.mds.b:Started 2024-04-26T20:52:53.566 DEBUG:teuthology.parallel:result is None 2024-04-26T20:52:53.605 INFO:tasks.ceph.mds.c:Started 2024-04-26T20:52:53.605 DEBUG:teuthology.parallel:result is None 2024-04-26T20:52:53.606 INFO:tasks.ceph.mds.a:Started 2024-04-26T20:52:53.606 DEBUG:teuthology.parallel:result is None 2024-04-26T20:52:53.607 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:52:53.628 INFO:tasks.ceph.mds.d.smithi120.stdout:starting mds.d at 2024-04-26T20:52:53.644 INFO:tasks.ceph.mds.b.smithi120.stdout:starting mds.b at 2024-04-26T20:52:53.680 INFO:tasks.ceph.mds.c.smithi073.stdout:starting mds.c at 2024-04-26T20:52:53.680 INFO:tasks.ceph.mds.a.smithi073.stdout:starting mds.a at 2024-04-26T20:52:53.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.709+0000 7f0b941a4640 1 Processor -- start 2024-04-26T20:52:53.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.709+0000 7f0b941a4640 1 -- start start 2024-04-26T20:52:53.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.710+0000 7f0b941a4640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0b8c15cb40 0x7f0b8c06c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:53.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.710+0000 7f0b941a4640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f0b8c06c780 0x7f0b8c15f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:53.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.710+0000 7f0b941a4640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0b8c15fd50 0x7f0b8c072f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:53.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.710+0000 7f0b941a4640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f0b8c162d30 con 0x7f0b8c15cb40 2024-04-26T20:52:53.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.710+0000 7f0b941a4640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f0b8c162fa0 con 0x7f0b8c06c780 2024-04-26T20:52:53.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.710+0000 7f0b941a4640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f0b8c163280 con 0x7f0b8c15fd50 2024-04-26T20:52:53.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.710+0000 7f0b91f40640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0b8c15cb40 0x7f0b8c06c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:53.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.711+0000 7f0b91f40640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0b8c15cb40 0x7f0b8c06c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47854/0 (socket says 172.21.15.73:47854) 2024-04-26T20:52:53.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.711+0000 7f0b91f40640 1 -- 172.21.15.73:0/2884873772 learned_addr learned my addr 172.21.15.73:0/2884873772 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:53.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.711+0000 7f0b92741640 1 --2- 172.21.15.73:0/2884873772 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0b8c15fd50 0x7f0b8c072f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:53.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.711+0000 7f0b9173f640 1 --2- 172.21.15.73:0/2884873772 >> v2:172.21.15.120:3300/0 conn(0x7f0b8c06c780 0x7f0b8c15f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:53.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.711+0000 7f0b91f40640 1 -- 172.21.15.73:0/2884873772 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0b8c15fd50 msgr2=0x7f0b8c072f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:53.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.711+0000 7f0b91f40640 1 --2- 172.21.15.73:0/2884873772 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0b8c15fd50 0x7f0b8c072f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:53.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.711+0000 7f0b91f40640 1 -- 172.21.15.73:0/2884873772 >> v2:172.21.15.120:3300/0 conn(0x7f0b8c06c780 msgr2=0x7f0b8c15f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:53.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.711+0000 7f0b91f40640 1 --2- 172.21.15.73:0/2884873772 >> v2:172.21.15.120:3300/0 conn(0x7f0b8c06c780 0x7f0b8c15f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:53.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.711+0000 7f0b91f40640 1 -- 172.21.15.73:0/2884873772 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b8c163560 con 0x7f0b8c15cb40 2024-04-26T20:52:53.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.711+0000 7f0b91f40640 1 --2- 172.21.15.73:0/2884873772 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b8c15cb40 0x7f0b8c06c240 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c072ee0 tx=0x7f0b8802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9d71f1ced1e34e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:53.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.712+0000 7f0b90f3e640 1 -- 172.21.15.73:0/2884873772 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b88037070 con 0x7f0b8c15cb40 2024-04-26T20:52:53.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.712+0000 7f0b90f3e640 1 -- 172.21.15.73:0/2884873772 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b88032440 con 0x7f0b8c15cb40 2024-04-26T20:52:53.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.712+0000 7f0b90f3e640 1 -- 172.21.15.73:0/2884873772 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b88032810 con 0x7f0b8c15cb40 2024-04-26T20:52:53.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.712+0000 7f0b941a4640 1 -- 172.21.15.73:0/2884873772 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b8c15cb40 msgr2=0x7f0b8c06c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:53.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.712+0000 7f0b941a4640 1 --2- 172.21.15.73:0/2884873772 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b8c15cb40 0x7f0b8c06c240 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c072ee0 tx=0x7f0b8802a140 comp rx=0 tx=0).stop 2024-04-26T20:52:53.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.712+0000 7f0b941a4640 1 -- 172.21.15.73:0/2884873772 shutdown_connections 2024-04-26T20:52:53.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.712+0000 7f0b941a4640 1 --2- 172.21.15.73:0/2884873772 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0b8c15fd50 0x7f0b8c072f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:53.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.712+0000 7f0b941a4640 1 --2- 172.21.15.73:0/2884873772 >> v2:172.21.15.120:3300/0 conn(0x7f0b8c06c780 0x7f0b8c15f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:53.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.712+0000 7f0b941a4640 1 --2- 172.21.15.73:0/2884873772 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b8c15cb40 0x7f0b8c06c240 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:53.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.712+0000 7f0b941a4640 1 -- 172.21.15.73:0/2884873772 >> 172.21.15.73:0/2884873772 conn(0x7f0b8c0d1cb0 msgr2=0x7f0b8c0d2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:53.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.712+0000 7f0b941a4640 1 -- 172.21.15.73:0/2884873772 shutdown_connections 2024-04-26T20:52:53.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.712+0000 7f0b941a4640 1 -- 172.21.15.73:0/2884873772 wait complete. 2024-04-26T20:52:53.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.714+0000 7f0b941a4640 1 Processor -- start 2024-04-26T20:52:53.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.714+0000 7f0b941a4640 1 -- start start 2024-04-26T20:52:53.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.714+0000 7f0b941a4640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0b8c06c780 0x7f0b8c15bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:53.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.714+0000 7f0b941a4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b8c15cb40 0x7f0b8c15a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:53.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.714+0000 7f0b941a4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b8c15fd50 0x7f0b8c14be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:53.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.714+0000 7f0b941a4640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8c163a80 con 0x7f0b8c15cb40 2024-04-26T20:52:53.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.714+0000 7f0b941a4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8c160180 con 0x7f0b8c15fd50 2024-04-26T20:52:53.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.714+0000 7f0b941a4640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f0b8c15ac40 con 0x7f0b8c06c780 2024-04-26T20:52:53.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.714+0000 7f0b9173f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b8c15cb40 0x7f0b8c15a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:53.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.714+0000 7f0b9173f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b8c15cb40 0x7f0b8c15a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47866/0 (socket says 172.21.15.73:47866) 2024-04-26T20:52:53.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.714+0000 7f0b9173f640 1 -- 172.21.15.73:0/2065105195 learned_addr learned my addr 172.21.15.73:0/2065105195 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:53.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.715+0000 7f0b91f40640 1 --2- 172.21.15.73:0/2065105195 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0b8c06c780 0x7f0b8c15bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:53.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.715+0000 7f0b92741640 1 --2- 172.21.15.73:0/2065105195 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b8c15fd50 0x7f0b8c14be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:53.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.715+0000 7f0b9173f640 1 -- 172.21.15.73:0/2065105195 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0b8c06c780 msgr2=0x7f0b8c15bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:53.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.715+0000 7f0b9173f640 1 --2- 172.21.15.73:0/2065105195 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0b8c06c780 0x7f0b8c15bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:53.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.715+0000 7f0b9173f640 1 -- 172.21.15.73:0/2065105195 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b8c15fd50 msgr2=0x7f0b8c14be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:53.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.715+0000 7f0b9173f640 1 --2- 172.21.15.73:0/2065105195 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b8c15fd50 0x7f0b8c14be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:53.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.715+0000 7f0b9173f640 1 -- 172.21.15.73:0/2065105195 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b8c15af20 con 0x7f0b8c15cb40 2024-04-26T20:52:53.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.715+0000 7f0b92741640 1 --2- 172.21.15.73:0/2065105195 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b8c15fd50 0x7f0b8c14be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:52:53.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.715+0000 7f0b9173f640 1 --2- 172.21.15.73:0/2065105195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b8c15cb40 0x7f0b8c15a340 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c00e8a0 tx=0x7f0b7c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:53.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.715+0000 7f0b827fc640 1 -- 172.21.15.73:0/2065105195 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c017070 con 0x7f0b8c15cb40 2024-04-26T20:52:53.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.715+0000 7f0b827fc640 1 -- 172.21.15.73:0/2065105195 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b7c00c940 con 0x7f0b8c15cb40 2024-04-26T20:52:53.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.715+0000 7f0b827fc640 1 -- 172.21.15.73:0/2065105195 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c00cd30 con 0x7f0b8c15cb40 2024-04-26T20:52:53.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.715+0000 7f0b941a4640 1 -- 172.21.15.73:0/2065105195 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b8c15b200 con 0x7f0b8c15cb40 2024-04-26T20:52:53.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.715+0000 7f0b941a4640 1 -- 172.21.15.73:0/2065105195 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b8c15b7c0 con 0x7f0b8c15cb40 2024-04-26T20:52:53.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.716+0000 7f0b941a4640 1 -- 172.21.15.73:0/2065105195 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b8c0694f0 con 0x7f0b8c15cb40 2024-04-26T20:52:53.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.716+0000 7f0b827fc640 1 -- 172.21.15.73:0/2065105195 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0b7c01a070 con 0x7f0b8c15cb40 2024-04-26T20:52:53.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.717+0000 7f0b827fc640 1 --2- 172.21.15.73:0/2065105195 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f0b7006c8f0 0x7f0b7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:53.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.717+0000 7f0b91f40640 1 --2- 172.21.15.73:0/2065105195 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f0b7006c8f0 0x7f0b7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:53.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.717+0000 7f0b827fc640 1 -- 172.21.15.73:0/2065105195 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6187+0+0 (secure 0 0 0) 0x7f0b7c08c000 con 0x7f0b8c15cb40 2024-04-26T20:52:53.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.718+0000 7f0b91f40640 1 --2- 172.21.15.73:0/2065105195 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f0b7006c8f0 0x7f0b7006edb0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0b88034000 tx=0x7f0b8802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:53.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.722+0000 7f0b827fc640 1 -- 172.21.15.73:0/2065105195 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0b7c05a1c0 con 0x7f0b8c15cb40 2024-04-26T20:52:53.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.933+0000 7f0b941a4640 1 -- 172.21.15.73:0/2065105195 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b8c0d2b20 con 0x7f0b8c15cb40 2024-04-26T20:52:53.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.933+0000 7f0b827fc640 1 -- 172.21.15.73:0/2065105195 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+1867 (secure 0 0 0) 0x7f0b7c059ac0 con 0x7f0b8c15cb40 2024-04-26T20:52:53.934 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:52:53.934 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":29,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:52:20.754736+0000","modified":"2024-04-26T20:52:53.540441+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":3}]} 2024-04-26T20:52:53.935 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 29 2024-04-26T20:52:53.936 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.935+0000 7f0b941a4640 1 -- 172.21.15.73:0/2065105195 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f0b7006c8f0 msgr2=0x7f0b7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:53.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.935+0000 7f0b941a4640 1 --2- 172.21.15.73:0/2065105195 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f0b7006c8f0 0x7f0b7006edb0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0b88034000 tx=0x7f0b8802e000 comp rx=0 tx=0).stop 2024-04-26T20:52:53.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.935+0000 7f0b941a4640 1 -- 172.21.15.73:0/2065105195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b8c15cb40 msgr2=0x7f0b8c15a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:53.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.935+0000 7f0b941a4640 1 --2- 172.21.15.73:0/2065105195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b8c15cb40 0x7f0b8c15a340 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c00e8a0 tx=0x7f0b7c00ed60 comp rx=0 tx=0).stop 2024-04-26T20:52:53.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.935+0000 7f0b941a4640 1 -- 172.21.15.73:0/2065105195 shutdown_connections 2024-04-26T20:52:53.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.935+0000 7f0b941a4640 1 --2- 172.21.15.73:0/2065105195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b8c15cb40 0x7f0b8c15a340 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:53.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.935+0000 7f0b941a4640 1 --2- 172.21.15.73:0/2065105195 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f0b7006c8f0 0x7f0b7006edb0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:53.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.935+0000 7f0b941a4640 1 --2- 172.21.15.73:0/2065105195 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b8c15fd50 0x7f0b8c14be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:53.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.935+0000 7f0b941a4640 1 --2- 172.21.15.73:0/2065105195 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0b8c06c780 0x7f0b8c15bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:53.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.935+0000 7f0b941a4640 1 -- 172.21.15.73:0/2065105195 >> 172.21.15.73:0/2065105195 conn(0x7f0b8c0d1cb0 msgr2=0x7f0b8c162040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:53.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.935+0000 7f0b941a4640 1 -- 172.21.15.73:0/2065105195 shutdown_connections 2024-04-26T20:52:53.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:53.935+0000 7f0b941a4640 1 -- 172.21.15.73:0/2065105195 wait complete. 2024-04-26T20:52:54.349 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T20:52:20.754736+0000', 'modified': '2024-04-26T20:52:53.540441+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 56, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {}, 'failed': [0], 'damaged': [], 'stopped': [], 'info': {}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-26T20:52:54.350 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 0/1 2024-04-26T20:52:55.351 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:52:55.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.458+0000 7f6b55146640 1 Processor -- start 2024-04-26T20:52:55.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.458+0000 7f6b55146640 1 -- start start 2024-04-26T20:52:55.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.458+0000 7f6b55146640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f6b501604e0 0x7f6b5006f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:55.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.458+0000 7f6b55146640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6b5006fcd0 0x7f6b500ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:55.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.458+0000 7f6b55146640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b50160970 con 0x7f6b5015f2d0 2024-04-26T20:52:55.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.458+0000 7f6b55146640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f6b50074010 con 0x7f6b501604e0 2024-04-26T20:52:55.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.458+0000 7f6b55146640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f6b50070100 con 0x7f6b5006fcd0 2024-04-26T20:52:55.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.459+0000 7f6b4f59e640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6b5006fcd0 0x7f6b500ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:55.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.459+0000 7f6b4e59c640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f6b501604e0 0x7f6b5006f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:55.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.459+0000 7f6b4ed9d640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f6b5015f2d0 0x7f6b50072700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.73:40424/0 (socket says 172.21.15.73:40424) 2024-04-26T20:52:55.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.459+0000 7f6b4ed9d640 1 -- 172.21.15.73:0/2401602796 learned_addr learned my addr 172.21.15.73:0/2401602796 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:52:55.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.459+0000 7f6b4f59e640 1 -- 172.21.15.73:0/2401602796 >> v2:172.21.15.120:3300/0 conn(0x7f6b501604e0 msgr2=0x7f6b5006f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:55.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.459+0000 7f6b4f59e640 1 --2- 172.21.15.73:0/2401602796 >> v2:172.21.15.120:3300/0 conn(0x7f6b501604e0 0x7f6b5006f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:55.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.459+0000 7f6b4f59e640 1 -- 172.21.15.73:0/2401602796 >> v1:172.21.15.120:6789/0 conn(0x7f6b5015f2d0 legacy=0x7f6b50072700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:55.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.459+0000 7f6b4f59e640 1 -- 172.21.15.73:0/2401602796 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b50072f70 con 0x7f6b5006fcd0 2024-04-26T20:52:55.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.459+0000 7f6b4dd9b640 1 -- 172.21.15.73:0/2401602796 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3396309632 0 0) 0x7f6b50160970 con 0x7f6b5015f2d0 2024-04-26T20:52:55.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.459+0000 7f6b4e59c640 1 --2- 172.21.15.73:0/2401602796 >> v2:172.21.15.120:3300/0 conn(0x7f6b501604e0 0x7f6b5006f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:52:55.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.460+0000 7f6b4f59e640 1 --2- 172.21.15.73:0/2401602796 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6b5006fcd0 0x7f6b500ce7b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6b480097a0 tx=0x7f6b4802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec43b60bf1be87f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:55.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.460+0000 7f6b4dd9b640 1 -- 172.21.15.73:0/2401602796 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b48037070 con 0x7f6b5006fcd0 2024-04-26T20:52:55.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.460+0000 7f6b4dd9b640 1 -- 172.21.15.73:0/2401602796 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b48032440 con 0x7f6b5006fcd0 2024-04-26T20:52:55.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.460+0000 7f6b4dd9b640 1 -- 172.21.15.73:0/2401602796 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b48032810 con 0x7f6b5006fcd0 2024-04-26T20:52:55.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.461+0000 7f6b55146640 1 -- 172.21.15.73:0/2401602796 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6b5006fcd0 msgr2=0x7f6b500ce7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:55.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.461+0000 7f6b55146640 1 --2- 172.21.15.73:0/2401602796 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6b5006fcd0 0x7f6b500ce7b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6b480097a0 tx=0x7f6b4802a180 comp rx=0 tx=0).stop 2024-04-26T20:52:55.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.461+0000 7f6b55146640 1 -- 172.21.15.73:0/2401602796 shutdown_connections 2024-04-26T20:52:55.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.461+0000 7f6b55146640 1 --2- 172.21.15.73:0/2401602796 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6b5006fcd0 0x7f6b500ce7b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:55.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.461+0000 7f6b55146640 1 --2- 172.21.15.73:0/2401602796 >> v2:172.21.15.120:3300/0 conn(0x7f6b501604e0 0x7f6b5006f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:55.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.461+0000 7f6b55146640 1 -- 172.21.15.73:0/2401602796 >> 172.21.15.73:0/2401602796 conn(0x7f6b500d0be0 msgr2=0x7f6b500d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:55.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.461+0000 7f6b55146640 1 -- 172.21.15.73:0/2401602796 shutdown_connections 2024-04-26T20:52:55.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.461+0000 7f6b55146640 1 -- 172.21.15.73:0/2401602796 wait complete. 2024-04-26T20:52:55.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.462+0000 7f6b55146640 1 Processor -- start 2024-04-26T20:52:55.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.462+0000 7f6b55146640 1 -- start start 2024-04-26T20:52:55.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.463+0000 7f6b55146640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b5006fcd0 0x7f6b500cf910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:55.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.463+0000 7f6b55146640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b5015f2d0 0x7f6b500cfe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:55.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.463+0000 7f6b55146640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6b501604e0 0x7f6b501477e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:55.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.463+0000 7f6b55146640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6b50073810 con 0x7f6b5015f2d0 2024-04-26T20:52:55.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.463+0000 7f6b55146640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f6b50073530 con 0x7f6b5006fcd0 2024-04-26T20:52:55.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.463+0000 7f6b55146640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f6b50073af0 con 0x7f6b501604e0 2024-04-26T20:52:55.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.463+0000 7f6b4e59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b5015f2d0 0x7f6b500cfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:55.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.463+0000 7f6b4e59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b5015f2d0 0x7f6b500cfe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47872/0 (socket says 172.21.15.73:47872) 2024-04-26T20:52:55.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.463+0000 7f6b4e59c640 1 -- 172.21.15.73:0/2409461246 learned_addr learned my addr 172.21.15.73:0/2409461246 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:55.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.463+0000 7f6b4ed9d640 1 --2- 172.21.15.73:0/2409461246 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b5006fcd0 0x7f6b500cf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:55.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.463+0000 7f6b4f59e640 1 --2- 172.21.15.73:0/2409461246 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6b501604e0 0x7f6b501477e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:55.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.464+0000 7f6b4e59c640 1 -- 172.21.15.73:0/2409461246 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6b501604e0 msgr2=0x7f6b501477e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:55.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.464+0000 7f6b4e59c640 1 --2- 172.21.15.73:0/2409461246 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6b501604e0 0x7f6b501477e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:55.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.464+0000 7f6b4e59c640 1 -- 172.21.15.73:0/2409461246 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b5006fcd0 msgr2=0x7f6b500cf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:55.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.464+0000 7f6b4e59c640 1 --2- 172.21.15.73:0/2409461246 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b5006fcd0 0x7f6b500cf910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:55.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.464+0000 7f6b4e59c640 1 -- 172.21.15.73:0/2409461246 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b50159d10 con 0x7f6b5015f2d0 2024-04-26T20:52:55.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.464+0000 7f6b4e59c640 1 --2- 172.21.15.73:0/2409461246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b5015f2d0 0x7f6b500cfe50 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f6b4400e960 tx=0x7f6b4400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:55.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.464+0000 7f6b377fe640 1 -- 172.21.15.73:0/2409461246 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b44017070 con 0x7f6b5015f2d0 2024-04-26T20:52:55.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.464+0000 7f6b377fe640 1 -- 172.21.15.73:0/2409461246 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b4400ca00 con 0x7f6b5015f2d0 2024-04-26T20:52:55.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.464+0000 7f6b377fe640 1 -- 172.21.15.73:0/2409461246 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b4400cdd0 con 0x7f6b5015f2d0 2024-04-26T20:52:55.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.464+0000 7f6b55146640 1 -- 172.21.15.73:0/2409461246 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b50159ff0 con 0x7f6b5015f2d0 2024-04-26T20:52:55.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.464+0000 7f6b55146640 1 -- 172.21.15.73:0/2409461246 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b5015a5b0 con 0x7f6b5015f2d0 2024-04-26T20:52:55.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.465+0000 7f6b55146640 1 -- 172.21.15.73:0/2409461246 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b500694f0 con 0x7f6b5015f2d0 2024-04-26T20:52:55.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.465+0000 7f6b377fe640 1 -- 172.21.15.73:0/2409461246 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6b4401a070 con 0x7f6b5015f2d0 2024-04-26T20:52:55.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.466+0000 7f6b377fe640 1 --2- 172.21.15.73:0/2409461246 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f6b3c06c8f0 0x7f6b3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:55.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.466+0000 7f6b4ed9d640 1 --2- 172.21.15.73:0/2409461246 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f6b3c06c8f0 0x7f6b3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:55.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.466+0000 7f6b377fe640 1 -- 172.21.15.73:0/2409461246 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6187+0+0 (secure 0 0 0) 0x7f6b4408be60 con 0x7f6b5015f2d0 2024-04-26T20:52:55.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.466+0000 7f6b4ed9d640 1 --2- 172.21.15.73:0/2409461246 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f6b3c06c8f0 0x7f6b3c06edb0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6b400012e0 tx=0x7f6b40001700 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:55.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.470+0000 7f6b377fe640 1 -- 172.21.15.73:0/2409461246 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6b44059af0 con 0x7f6b5015f2d0 2024-04-26T20:52:55.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.687+0000 7f6b55146640 1 -- 172.21.15.73:0/2409461246 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b50167b70 con 0x7f6b5015f2d0 2024-04-26T20:52:55.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.688+0000 7f6b377fe640 1 -- 172.21.15.73:0/2409461246 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+5178 (secure 0 0 0) 0x7f6b4405ddf0 con 0x7f6b5015f2d0 2024-04-26T20:52:55.689 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:52:55.689 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":32,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5207,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/256948100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":256948100},{"type":"v1","addr":"172.21.15.120:6833","nonce":256948100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5632,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/4104096099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":4104096099},{"type":"v1","addr":"172.21.15.120:6835","nonce":4104096099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5641,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/272782804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":272782804},{"type":"v1","addr":"172.21.15.73:6837","nonce":272782804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:52:20.754736+0000","modified":"2024-04-26T20:52:54.563328+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6099":{"gid":6099,"name":"c","rank":0,"incarnation":31,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.73:6836/1169682578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1169682578},{"type":"v1","addr":"172.21.15.73:6836","nonce":1169682578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":3}]} 2024-04-26T20:52:55.689 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 32 2024-04-26T20:52:55.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.690+0000 7f6b55146640 1 -- 172.21.15.73:0/2409461246 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f6b3c06c8f0 msgr2=0x7f6b3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:55.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.690+0000 7f6b55146640 1 --2- 172.21.15.73:0/2409461246 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f6b3c06c8f0 0x7f6b3c06edb0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6b400012e0 tx=0x7f6b40001700 comp rx=0 tx=0).stop 2024-04-26T20:52:55.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.690+0000 7f6b55146640 1 -- 172.21.15.73:0/2409461246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b5015f2d0 msgr2=0x7f6b500cfe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:55.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.690+0000 7f6b55146640 1 --2- 172.21.15.73:0/2409461246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b5015f2d0 0x7f6b500cfe50 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f6b4400e960 tx=0x7f6b4400ee20 comp rx=0 tx=0).stop 2024-04-26T20:52:55.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.690+0000 7f6b55146640 1 -- 172.21.15.73:0/2409461246 shutdown_connections 2024-04-26T20:52:55.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.690+0000 7f6b55146640 1 --2- 172.21.15.73:0/2409461246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b5015f2d0 0x7f6b500cfe50 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:55.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.690+0000 7f6b55146640 1 --2- 172.21.15.73:0/2409461246 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f6b3c06c8f0 0x7f6b3c06edb0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:55.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.690+0000 7f6b55146640 1 --2- 172.21.15.73:0/2409461246 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f6b501604e0 0x7f6b501477e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:55.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.690+0000 7f6b55146640 1 --2- 172.21.15.73:0/2409461246 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f6b5006fcd0 0x7f6b500cf910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:55.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.690+0000 7f6b55146640 1 -- 172.21.15.73:0/2409461246 >> 172.21.15.73:0/2409461246 conn(0x7f6b500d0be0 msgr2=0x7f6b5015dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:55.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.690+0000 7f6b55146640 1 -- 172.21.15.73:0/2409461246 shutdown_connections 2024-04-26T20:52:55.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:55.690+0000 7f6b55146640 1 -- 172.21.15.73:0/2409461246 wait complete. 2024-04-26T20:52:56.105 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 32, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T20:52:20.754736+0000', 'modified': '2024-04-26T20:52:54.563328+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 56, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6099}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6099': {'gid': 6099, 'name': 'c', 'rank': 0, 'incarnation': 31, 'state': 'up:reconnect', 'state_seq': 2, 'addr': '172.21.15.73:6836/1169682578', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6834', 'nonce': 1169682578}, {'type': 'v1', 'addr': '172.21.15.73:6836', 'nonce': 1169682578}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-26T20:52:56.105 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6099:up:reconnect 2024-04-26T20:52:57.106 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:52:57.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.212+0000 7f93dfd04640 1 Processor -- start 2024-04-26T20:52:57.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.212+0000 7f93dfd04640 1 -- start start 2024-04-26T20:52:57.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.213+0000 7f93dfd04640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f93d806c240 0x7f93d806c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:57.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.213+0000 7f93dfd04640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93d80b4480 con 0x7f93d80aed50 2024-04-26T20:52:57.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.213+0000 7f93dfd04640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93d80b4720 con 0x7f93d815f1d0 2024-04-26T20:52:57.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.213+0000 7f93dfd04640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f93d806cbe0 con 0x7f93d806c240 2024-04-26T20:52:57.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.213+0000 7f93dd29f640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f93d806c240 0x7f93d806c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:57.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.213+0000 7f93dd29f640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f93d806c240 0x7f93d806c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47874/0 (socket says 172.21.15.73:47874) 2024-04-26T20:52:57.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.213+0000 7f93de2a1640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f93d80aed50 0x7f93d80af1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:45724/0 (socket says 172.21.15.73:45724) 2024-04-26T20:52:57.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.213+0000 7f93dd29f640 1 -- 172.21.15.73:0/2094337103 learned_addr learned my addr 172.21.15.73:0/2094337103 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:57.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.214+0000 7f93dd29f640 1 -- 172.21.15.73:0/2094337103 >> v1:172.21.15.120:6789/0 conn(0x7f93d815f1d0 legacy=0x7f93d806f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:57.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.214+0000 7f93dd29f640 1 -- 172.21.15.73:0/2094337103 >> v1:172.21.15.73:6789/0 conn(0x7f93d80aed50 legacy=0x7f93d80af1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:57.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.214+0000 7f93dd29f640 1 -- 172.21.15.73:0/2094337103 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93d80b4a00 con 0x7f93d806c240 2024-04-26T20:52:57.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.214+0000 7f93dca9e640 1 -- 172.21.15.73:0/2094337103 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 424140331 0 0) 0x7f93d80b4480 con 0x7f93d80aed50 2024-04-26T20:52:57.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.214+0000 7f93dd29f640 1 --2- 172.21.15.73:0/2094337103 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93d806c240 0x7f93d806c6a0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f93d4009870 tx=0x7f93d402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39217bdd2610e46e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:57.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.215+0000 7f93dca9e640 1 -- 172.21.15.73:0/2094337103 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d4037070 con 0x7f93d806c240 2024-04-26T20:52:57.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.215+0000 7f93dca9e640 1 -- 172.21.15.73:0/2094337103 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93d402ae50 con 0x7f93d806c240 2024-04-26T20:52:57.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.215+0000 7f93dca9e640 1 -- 172.21.15.73:0/2094337103 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d4032860 con 0x7f93d806c240 2024-04-26T20:52:57.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.215+0000 7f93dfd04640 1 -- 172.21.15.73:0/2094337103 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93d806c240 msgr2=0x7f93d806c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:57.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.215+0000 7f93dfd04640 1 --2- 172.21.15.73:0/2094337103 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93d806c240 0x7f93d806c6a0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f93d4009870 tx=0x7f93d402a300 comp rx=0 tx=0).stop 2024-04-26T20:52:57.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.215+0000 7f93dfd04640 1 -- 172.21.15.73:0/2094337103 shutdown_connections 2024-04-26T20:52:57.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.215+0000 7f93dfd04640 1 --2- 172.21.15.73:0/2094337103 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93d806c240 0x7f93d806c6a0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:57.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.215+0000 7f93dfd04640 1 -- 172.21.15.73:0/2094337103 >> 172.21.15.73:0/2094337103 conn(0x7f93d80b3180 msgr2=0x7f93d80b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:57.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.215+0000 7f93dfd04640 1 -- 172.21.15.73:0/2094337103 shutdown_connections 2024-04-26T20:52:57.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.215+0000 7f93dfd04640 1 -- 172.21.15.73:0/2094337103 wait complete. 2024-04-26T20:52:57.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.216+0000 7f93dfd04640 1 Processor -- start 2024-04-26T20:52:57.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.216+0000 7f93dfd04640 1 -- start start 2024-04-26T20:52:57.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.217+0000 7f93dfd04640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f93d806c240 0x7f93d814b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:57.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.217+0000 7f93dfd04640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93d80aed50 0x7f93d814bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:57.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.217+0000 7f93dfd04640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93d815f1d0 0x7f93d814e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:57.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.217+0000 7f93dfd04640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f93d80b4fc0 con 0x7f93d80aed50 2024-04-26T20:52:57.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.217+0000 7f93dfd04640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f93d80ae8c0 con 0x7f93d815f1d0 2024-04-26T20:52:57.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.217+0000 7f93dfd04640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f93d8075ea0 con 0x7f93d806c240 2024-04-26T20:52:57.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.217+0000 7f93dd29f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93d80aed50 0x7f93d814bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:57.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.217+0000 7f93dd29f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93d80aed50 0x7f93d814bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47884/0 (socket says 172.21.15.73:47884) 2024-04-26T20:52:57.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.217+0000 7f93dd29f640 1 -- 172.21.15.73:0/1534755224 learned_addr learned my addr 172.21.15.73:0/1534755224 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:57.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.217+0000 7f93ddaa0640 1 --2- 172.21.15.73:0/1534755224 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f93d806c240 0x7f93d814b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:57.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.217+0000 7f93de2a1640 1 --2- 172.21.15.73:0/1534755224 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93d815f1d0 0x7f93d814e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:57.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.218+0000 7f93dd29f640 1 -- 172.21.15.73:0/1534755224 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f93d806c240 msgr2=0x7f93d814b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:57.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.218+0000 7f93dd29f640 1 --2- 172.21.15.73:0/1534755224 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f93d806c240 0x7f93d814b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:57.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.218+0000 7f93dd29f640 1 -- 172.21.15.73:0/1534755224 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93d815f1d0 msgr2=0x7f93d814e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:57.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.218+0000 7f93dd29f640 1 --2- 172.21.15.73:0/1534755224 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93d815f1d0 0x7f93d814e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:57.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.218+0000 7f93dd29f640 1 -- 172.21.15.73:0/1534755224 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93d814c330 con 0x7f93d80aed50 2024-04-26T20:52:57.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.218+0000 7f93de2a1640 1 --2- 172.21.15.73:0/1534755224 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93d815f1d0 0x7f93d814e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:52:57.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.218+0000 7f93dd29f640 1 --2- 172.21.15.73:0/1534755224 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93d80aed50 0x7f93d814bc00 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f93d4000c00 tx=0x7f93d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:57.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.218+0000 7f93ca7fc640 1 -- 172.21.15.73:0/1534755224 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d4042070 con 0x7f93d80aed50 2024-04-26T20:52:57.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.218+0000 7f93ca7fc640 1 -- 172.21.15.73:0/1534755224 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93d403e870 con 0x7f93d80aed50 2024-04-26T20:52:57.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.218+0000 7f93dfd04640 1 -- 172.21.15.73:0/1534755224 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93d814c610 con 0x7f93d80aed50 2024-04-26T20:52:57.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.218+0000 7f93ca7fc640 1 -- 172.21.15.73:0/1534755224 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d4037040 con 0x7f93d80aed50 2024-04-26T20:52:57.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.218+0000 7f93dfd04640 1 -- 172.21.15.73:0/1534755224 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93d814cbd0 con 0x7f93d80aed50 2024-04-26T20:52:57.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.219+0000 7f93dfd04640 1 -- 172.21.15.73:0/1534755224 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93d8068fc0 con 0x7f93d80aed50 2024-04-26T20:52:57.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.219+0000 7f93ca7fc640 1 -- 172.21.15.73:0/1534755224 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f93d4045070 con 0x7f93d80aed50 2024-04-26T20:52:57.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.220+0000 7f93ca7fc640 1 --2- 172.21.15.73:0/1534755224 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f93bc06c8f0 0x7f93bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:57.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.220+0000 7f93ca7fc640 1 -- 172.21.15.73:0/1534755224 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6187+0+0 (secure 0 0 0) 0x7f93d40ac610 con 0x7f93d80aed50 2024-04-26T20:52:57.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.220+0000 7f93ddaa0640 1 --2- 172.21.15.73:0/1534755224 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f93bc06c8f0 0x7f93bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:57.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.220+0000 7f93ddaa0640 1 --2- 172.21.15.73:0/1534755224 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f93bc06c8f0 0x7f93bc06edb0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f93cc000f60 tx=0x7f93cc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:57.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.224+0000 7f93ca7fc640 1 -- 172.21.15.73:0/1534755224 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f93d4032e20 con 0x7f93d80aed50 2024-04-26T20:52:57.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.434+0000 7f93dfd04640 1 -- 172.21.15.73:0/1534755224 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f93d80694f0 con 0x7f93d80aed50 2024-04-26T20:52:57.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.435+0000 7f93ca7fc640 1 -- 172.21.15.73:0/1534755224 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+5175 (secure 0 0 0) 0x7f93d407a210 con 0x7f93d80aed50 2024-04-26T20:52:57.436 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:52:57.436 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":33,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5207,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/256948100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":256948100},{"type":"v1","addr":"172.21.15.120:6833","nonce":256948100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5632,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/4104096099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":4104096099},{"type":"v1","addr":"172.21.15.120:6835","nonce":4104096099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5641,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/272782804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":272782804},{"type":"v1","addr":"172.21.15.73:6837","nonce":272782804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:52:20.754736+0000","modified":"2024-04-26T20:52:55.559171+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6099":{"gid":6099,"name":"c","rank":0,"incarnation":31,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.73:6836/1169682578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1169682578},{"type":"v1","addr":"172.21.15.73:6836","nonce":1169682578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":3}]} 2024-04-26T20:52:57.436 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 33 2024-04-26T20:52:57.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.437+0000 7f93dfd04640 1 -- 172.21.15.73:0/1534755224 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f93bc06c8f0 msgr2=0x7f93bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:57.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.437+0000 7f93dfd04640 1 --2- 172.21.15.73:0/1534755224 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f93bc06c8f0 0x7f93bc06edb0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f93cc000f60 tx=0x7f93cc009040 comp rx=0 tx=0).stop 2024-04-26T20:52:57.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.437+0000 7f93dfd04640 1 -- 172.21.15.73:0/1534755224 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93d80aed50 msgr2=0x7f93d814bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:57.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.437+0000 7f93dfd04640 1 --2- 172.21.15.73:0/1534755224 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93d80aed50 0x7f93d814bc00 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f93d4000c00 tx=0x7f93d40046c0 comp rx=0 tx=0).stop 2024-04-26T20:52:57.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.437+0000 7f93dfd04640 1 -- 172.21.15.73:0/1534755224 shutdown_connections 2024-04-26T20:52:57.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.437+0000 7f93dfd04640 1 --2- 172.21.15.73:0/1534755224 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93d80aed50 0x7f93d814bc00 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:57.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.437+0000 7f93dfd04640 1 --2- 172.21.15.73:0/1534755224 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f93bc06c8f0 0x7f93bc06edb0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:57.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.437+0000 7f93dfd04640 1 --2- 172.21.15.73:0/1534755224 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f93d815f1d0 0x7f93d814e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:57.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.437+0000 7f93dfd04640 1 --2- 172.21.15.73:0/1534755224 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f93d806c240 0x7f93d814b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:57.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.437+0000 7f93dfd04640 1 -- 172.21.15.73:0/1534755224 >> 172.21.15.73:0/1534755224 conn(0x7f93d80b3180 msgr2=0x7f93d80b1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:57.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.437+0000 7f93dfd04640 1 -- 172.21.15.73:0/1534755224 shutdown_connections 2024-04-26T20:52:57.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:57.437+0000 7f93dfd04640 1 -- 172.21.15.73:0/1534755224 wait complete. 2024-04-26T20:52:57.852 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 33, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T20:52:20.754736+0000', 'modified': '2024-04-26T20:52:55.559171+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 56, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6099}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6099': {'gid': 6099, 'name': 'c', 'rank': 0, 'incarnation': 31, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.73:6836/1169682578', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6834', 'nonce': 1169682578}, {'type': 'v1', 'addr': '172.21.15.73:6836', 'nonce': 1169682578}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-26T20:52:57.852 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6099:up:rejoin 2024-04-26T20:52:58.853 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:52:58.967 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.966+0000 7f97f636a640 1 Processor -- start 2024-04-26T20:52:58.967 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.966+0000 7f97f636a640 1 -- start start 2024-04-26T20:52:58.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.967+0000 7f97f636a640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f97f015f3a0 0x7f97f00d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:58.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.967+0000 7f97f636a640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f97f00d1770 0x7f97f00d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:58.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.967+0000 7f97f636a640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97f00d25a0 con 0x7f97f00d2130 2024-04-26T20:52:58.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.967+0000 7f97f636a640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f97f00d0e20 con 0x7f97f00d1770 2024-04-26T20:52:58.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.967+0000 7f97f636a640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f97f00b1d70 con 0x7f97f015f3a0 2024-04-26T20:52:58.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.967+0000 7f97effff640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f97f015f3a0 0x7f97f00d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:58.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.967+0000 7f97ef7fe640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f97f00d1770 0x7f97f00d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:58.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.967+0000 7f97effff640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f97f015f3a0 0x7f97f00d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.73:60746/0 (socket says 172.21.15.73:60746) 2024-04-26T20:52:58.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.967+0000 7f97effff640 1 -- 172.21.15.73:0/2023744783 learned_addr learned my addr 172.21.15.73:0/2023744783 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:58.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.968+0000 7f97effff640 1 -- 172.21.15.73:0/2023744783 >> v2:172.21.15.120:3300/0 conn(0x7f97f00d1770 msgr2=0x7f97f00d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:58.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.968+0000 7f97effff640 1 --2- 172.21.15.73:0/2023744783 >> v2:172.21.15.120:3300/0 conn(0x7f97f00d1770 0x7f97f00d1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:58.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.968+0000 7f97effff640 1 -- 172.21.15.73:0/2023744783 >> v1:172.21.15.120:6789/0 conn(0x7f97f00d2130 legacy=0x7f97f00d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:58.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.968+0000 7f97eeffd640 1 -- 172.21.15.73:0/2023744783 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2168315889 0 0) 0x7f97e8001490 con 0x7f97f00d2130 2024-04-26T20:52:58.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.968+0000 7f97effff640 1 -- 172.21.15.73:0/2023744783 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97f00b2050 con 0x7f97f015f3a0 2024-04-26T20:52:58.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.968+0000 7f97effff640 1 --2- 172.21.15.73:0/2023744783 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f97f015f3a0 0x7f97f00d06f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f97e4009870 tx=0x7f97e402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c73bd2bab423b643 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:58.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.969+0000 7f97eeffd640 1 -- 172.21.15.73:0/2023744783 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e4037070 con 0x7f97f015f3a0 2024-04-26T20:52:58.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.969+0000 7f97eeffd640 1 -- 172.21.15.73:0/2023744783 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97e4032440 con 0x7f97f015f3a0 2024-04-26T20:52:58.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.969+0000 7f97eeffd640 1 -- 172.21.15.73:0/2023744783 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e4032810 con 0x7f97f015f3a0 2024-04-26T20:52:58.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.969+0000 7f97f636a640 1 -- 172.21.15.73:0/2023744783 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f97f015f3a0 msgr2=0x7f97f00d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:58.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.969+0000 7f97f636a640 1 --2- 172.21.15.73:0/2023744783 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f97f015f3a0 0x7f97f00d06f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f97e4009870 tx=0x7f97e402a270 comp rx=0 tx=0).stop 2024-04-26T20:52:58.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.969+0000 7f97f636a640 1 -- 172.21.15.73:0/2023744783 shutdown_connections 2024-04-26T20:52:58.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.969+0000 7f97f636a640 1 --2- 172.21.15.73:0/2023744783 >> v2:172.21.15.120:3300/0 conn(0x7f97f00d1770 0x7f97f00d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:58.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.969+0000 7f97f636a640 1 --2- 172.21.15.73:0/2023744783 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f97f015f3a0 0x7f97f00d06f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:58.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.969+0000 7f97f636a640 1 -- 172.21.15.73:0/2023744783 >> 172.21.15.73:0/2023744783 conn(0x7f97f00d4f90 msgr2=0x7f97f00d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:58.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.969+0000 7f97f636a640 1 -- 172.21.15.73:0/2023744783 shutdown_connections 2024-04-26T20:52:58.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.969+0000 7f97f636a640 1 -- 172.21.15.73:0/2023744783 wait complete. 2024-04-26T20:52:58.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.970+0000 7f97f636a640 1 Processor -- start 2024-04-26T20:52:58.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.971+0000 7f97f636a640 1 -- start start 2024-04-26T20:52:58.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.971+0000 7f97f636a640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f97f00d1770 0x7f97f00b7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:58.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.971+0000 7f97f636a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97f00d2130 0x7f97f0150370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:58.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.971+0000 7f97f636a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f97f015f3a0 0x7f97f0150c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:58.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.971+0000 7f97f636a640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f97f015f900 con 0x7f97f00d2130 2024-04-26T20:52:58.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.971+0000 7f97f636a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f97f015f780 con 0x7f97f015f3a0 2024-04-26T20:52:58.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.971+0000 7f97f636a640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f97f00b3860 con 0x7f97f00d1770 2024-04-26T20:52:58.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.972+0000 7f97ef7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97f00d2130 0x7f97f0150370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:58.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.972+0000 7f97ef7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97f00d2130 0x7f97f0150370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47892/0 (socket says 172.21.15.73:47892) 2024-04-26T20:52:58.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.972+0000 7f97ef7fe640 1 -- 172.21.15.73:0/522223744 learned_addr learned my addr 172.21.15.73:0/522223744 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:58.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.972+0000 7f97effff640 1 --2- 172.21.15.73:0/522223744 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f97f00d1770 0x7f97f00b7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:58.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.972+0000 7f97f4907640 1 --2- 172.21.15.73:0/522223744 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f97f015f3a0 0x7f97f0150c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:58.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.972+0000 7f97ef7fe640 1 -- 172.21.15.73:0/522223744 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f97f00d1770 msgr2=0x7f97f00b7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:58.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.972+0000 7f97ef7fe640 1 --2- 172.21.15.73:0/522223744 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f97f00d1770 0x7f97f00b7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:58.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.972+0000 7f97ef7fe640 1 -- 172.21.15.73:0/522223744 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f97f015f3a0 msgr2=0x7f97f0150c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:58.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.972+0000 7f97ef7fe640 1 --2- 172.21.15.73:0/522223744 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f97f015f3a0 0x7f97f0150c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:58.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.972+0000 7f97ef7fe640 1 -- 172.21.15.73:0/522223744 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97f00cd3d0 con 0x7f97f00d2130 2024-04-26T20:52:58.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.972+0000 7f97effff640 1 --2- 172.21.15.73:0/522223744 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f97f00d1770 0x7f97f00b7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:52:58.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.972+0000 7f97ef7fe640 1 --2- 172.21.15.73:0/522223744 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97f00d2130 0x7f97f0150370 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f97e000e9b0 tx=0x7f97e000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:58.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.972+0000 7f97ecff9640 1 -- 172.21.15.73:0/522223744 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e0017070 con 0x7f97f00d2130 2024-04-26T20:52:58.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.972+0000 7f97ecff9640 1 -- 172.21.15.73:0/522223744 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97e000c9a0 con 0x7f97f00d2130 2024-04-26T20:52:58.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.972+0000 7f97f636a640 1 -- 172.21.15.73:0/522223744 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97f00cd6b0 con 0x7f97f00d2130 2024-04-26T20:52:58.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.972+0000 7f97ecff9640 1 -- 172.21.15.73:0/522223744 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e000cd90 con 0x7f97f00d2130 2024-04-26T20:52:58.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.972+0000 7f97f636a640 1 -- 172.21.15.73:0/522223744 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97f00cdc70 con 0x7f97f00d2130 2024-04-26T20:52:58.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.974+0000 7f97f636a640 1 -- 172.21.15.73:0/522223744 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97f00694f0 con 0x7f97f00d2130 2024-04-26T20:52:58.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.974+0000 7f97ecff9640 1 -- 172.21.15.73:0/522223744 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f97e001a070 con 0x7f97f00d2130 2024-04-26T20:52:58.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.974+0000 7f97ecff9640 1 --2- 172.21.15.73:0/522223744 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f97dc06c920 0x7f97dc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:58.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.974+0000 7f97ecff9640 1 -- 172.21.15.73:0/522223744 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6187+0+0 (secure 0 0 0) 0x7f97e008bfc0 con 0x7f97f00d2130 2024-04-26T20:52:58.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.974+0000 7f97effff640 1 --2- 172.21.15.73:0/522223744 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f97dc06c920 0x7f97dc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:58.977 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.975+0000 7f97effff640 1 --2- 172.21.15.73:0/522223744 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f97dc06c920 0x7f97dc06ede0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f97e4034000 tx=0x7f97e402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:58.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:58.979+0000 7f97ecff9640 1 -- 172.21.15.73:0/522223744 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f97e0059bd0 con 0x7f97f00d2130 2024-04-26T20:52:59.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.191+0000 7f97f636a640 1 -- 172.21.15.73:0/522223744 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f97f00b6860 con 0x7f97f00d2130 2024-04-26T20:52:59.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.192+0000 7f97ecff9640 1 -- 172.21.15.73:0/522223744 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5182 (secure 0 0 0) 0x7f97e005ded0 con 0x7f97f00d2130 2024-04-26T20:52:59.193 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:52:59.193 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":34,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5207,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/256948100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":256948100},{"type":"v1","addr":"172.21.15.120:6833","nonce":256948100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5632,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/4104096099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":4104096099},{"type":"v1","addr":"172.21.15.120:6835","nonce":4104096099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5641,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/272782804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":272782804},{"type":"v1","addr":"172.21.15.73:6837","nonce":272782804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:52:20.754736+0000","modified":"2024-04-26T20:52:57.564091+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6099":{"gid":6099,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":4,"addr":"172.21.15.73:6836/1169682578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1169682578},{"type":"v1","addr":"172.21.15.73:6836","nonce":1169682578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6099,"qdb_cluster":[6099]},"id":3}]} 2024-04-26T20:52:59.194 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 34 2024-04-26T20:52:59.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.194+0000 7f97f636a640 1 -- 172.21.15.73:0/522223744 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f97dc06c920 msgr2=0x7f97dc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:59.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.194+0000 7f97f636a640 1 --2- 172.21.15.73:0/522223744 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f97dc06c920 0x7f97dc06ede0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f97e4034000 tx=0x7f97e402e000 comp rx=0 tx=0).stop 2024-04-26T20:52:59.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.194+0000 7f97f636a640 1 -- 172.21.15.73:0/522223744 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97f00d2130 msgr2=0x7f97f0150370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:59.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.194+0000 7f97f636a640 1 --2- 172.21.15.73:0/522223744 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97f00d2130 0x7f97f0150370 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f97e000e9b0 tx=0x7f97e000ee70 comp rx=0 tx=0).stop 2024-04-26T20:52:59.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.194+0000 7f97f636a640 1 -- 172.21.15.73:0/522223744 shutdown_connections 2024-04-26T20:52:59.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.194+0000 7f97f636a640 1 --2- 172.21.15.73:0/522223744 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97f00d2130 0x7f97f0150370 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:59.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.194+0000 7f97f636a640 1 --2- 172.21.15.73:0/522223744 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f97dc06c920 0x7f97dc06ede0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:59.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.194+0000 7f97f636a640 1 --2- 172.21.15.73:0/522223744 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f97f015f3a0 0x7f97f0150c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:59.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.194+0000 7f97f636a640 1 --2- 172.21.15.73:0/522223744 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f97f00d1770 0x7f97f00b7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:59.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.194+0000 7f97f636a640 1 -- 172.21.15.73:0/522223744 >> 172.21.15.73:0/522223744 conn(0x7f97f00d4f90 msgr2=0x7f97f006cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:59.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.194+0000 7f97f636a640 1 -- 172.21.15.73:0/522223744 shutdown_connections 2024-04-26T20:52:59.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.194+0000 7f97f636a640 1 -- 172.21.15.73:0/522223744 wait complete. 2024-04-26T20:52:59.558 INFO:tasks.ceph.mds.c.smithi073.stderr:2024-04-26T20:52:59.557+0000 7f5ee69bf640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T20:52:59.608 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 34, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T20:52:20.754736+0000', 'modified': '2024-04-26T20:52:57.564091+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 56, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6099}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6099': {'gid': 6099, 'name': 'c', 'rank': 0, 'incarnation': 31, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.73:6836/1169682578', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6834', 'nonce': 1169682578}, {'type': 'v1', 'addr': '172.21.15.73:6836', 'nonce': 1169682578}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6099, 'qdb_cluster': [6099]} 2024-04-26T20:52:59.608 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T20:52:59.608 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-26T20:52:59.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.709+0000 7f98184f8640 1 Processor -- start 2024-04-26T20:52:59.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.709+0000 7f98184f8640 1 -- start start 2024-04-26T20:52:59.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.710+0000 7f98184f8640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f98100cf0e0 0x7f981006f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:59.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.710+0000 7f98184f8640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f981015b9e0 con 0x7f981015b600 2024-04-26T20:52:59.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.710+0000 7f98184f8640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98100d0740 con 0x7f98100cf5e0 2024-04-26T20:52:59.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.710+0000 7f98184f8640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f98100cb9b0 con 0x7f98100cf0e0 2024-04-26T20:52:59.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.710+0000 7f9816a95640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f98100cf0e0 0x7f981006f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:59.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.710+0000 7f9816a95640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f98100cf0e0 0x7f981006f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55122/0 (socket says 172.21.15.73:55122) 2024-04-26T20:52:59.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.710+0000 7f9816a95640 1 -- 172.21.15.73:0/2423466180 learned_addr learned my addr 172.21.15.73:0/2423466180 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:59.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.710+0000 7f9816294640 1 --1- 172.21.15.73:0/2423466180 >> v1:172.21.15.73:6789/0 conn(0x7f981015b600 0x7f9810071f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:50692/0 (socket says 172.21.15.73:50692) 2024-04-26T20:52:59.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.710+0000 7f9816a95640 1 -- 172.21.15.73:0/2423466180 >> v1:172.21.15.120:6789/0 conn(0x7f98100cf5e0 legacy=0x7f98100ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:59.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.710+0000 7f9816a95640 1 -- 172.21.15.73:0/2423466180 >> v1:172.21.15.73:6789/0 conn(0x7f981015b600 legacy=0x7f9810071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:59.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.710+0000 7f9816a95640 1 -- 172.21.15.73:0/2423466180 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98100cb360 con 0x7f98100cf0e0 2024-04-26T20:52:59.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.710+0000 7f9815292640 1 -- 172.21.15.73:0/2423466180 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2206779017 0 0) 0x7f981015b9e0 con 0x7f981015b600 2024-04-26T20:52:59.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.711+0000 7f9816a95640 1 --2- 172.21.15.73:0/2423466180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98100cf0e0 0x7f981006f370 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f980800d150 tx=0x7f980802c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac41477b3275586f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:59.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.711+0000 7f9815292640 1 -- 172.21.15.73:0/2423466180 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98080337a0 con 0x7f98100cf0e0 2024-04-26T20:52:59.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.711+0000 7f9815292640 1 -- 172.21.15.73:0/2423466180 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9808039070 con 0x7f98100cf0e0 2024-04-26T20:52:59.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.711+0000 7f9815292640 1 -- 172.21.15.73:0/2423466180 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9808033b60 con 0x7f98100cf0e0 2024-04-26T20:52:59.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.711+0000 7f98184f8640 1 -- 172.21.15.73:0/2423466180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98100cf0e0 msgr2=0x7f981006f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:59.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.711+0000 7f98184f8640 1 --2- 172.21.15.73:0/2423466180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98100cf0e0 0x7f981006f370 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f980800d150 tx=0x7f980802c970 comp rx=0 tx=0).stop 2024-04-26T20:52:59.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.712+0000 7f98184f8640 1 -- 172.21.15.73:0/2423466180 shutdown_connections 2024-04-26T20:52:59.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.712+0000 7f98184f8640 1 --2- 172.21.15.73:0/2423466180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98100cf0e0 0x7f981006f370 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:59.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.712+0000 7f98184f8640 1 -- 172.21.15.73:0/2423466180 >> 172.21.15.73:0/2423466180 conn(0x7f98100d0be0 msgr2=0x7f98100d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:59.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.712+0000 7f98184f8640 1 -- 172.21.15.73:0/2423466180 shutdown_connections 2024-04-26T20:52:59.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.712+0000 7f98184f8640 1 -- 172.21.15.73:0/2423466180 wait complete. 2024-04-26T20:52:59.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.713+0000 7f98184f8640 1 Processor -- start 2024-04-26T20:52:59.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.713+0000 7f98184f8640 1 -- start start 2024-04-26T20:52:59.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.713+0000 7f98184f8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98100cf0e0 0x7f9810157aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:59.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.713+0000 7f98184f8640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f98100cf5e0 0x7f981014eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:59.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.713+0000 7f98184f8640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f981015b600 0x7f981014f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:59.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.713+0000 7f98184f8640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f98100d0470 con 0x7f981015b600 2024-04-26T20:52:59.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.713+0000 7f98184f8640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f98100d02f0 con 0x7f98100cf0e0 2024-04-26T20:52:59.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.713+0000 7f98184f8640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f98100cbbd0 con 0x7f98100cf5e0 2024-04-26T20:52:59.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.714+0000 7f9816a95640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f981015b600 0x7f981014f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:59.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.714+0000 7f9816a95640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f981015b600 0x7f981014f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55126/0 (socket says 172.21.15.73:55126) 2024-04-26T20:52:59.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.714+0000 7f9816a95640 1 -- 172.21.15.73:0/3467946338 learned_addr learned my addr 172.21.15.73:0/3467946338 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:59.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.714+0000 7f9816a95640 1 -- 172.21.15.73:0/3467946338 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f98100cf5e0 msgr2=0x7f981014eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:59.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.714+0000 7f9816a95640 1 --2- 172.21.15.73:0/3467946338 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f98100cf5e0 0x7f981014eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:59.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.714+0000 7f9816a95640 1 -- 172.21.15.73:0/3467946338 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98100cf0e0 msgr2=0x7f9810157aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:59.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.714+0000 7f9816a95640 1 --2- 172.21.15.73:0/3467946338 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98100cf0e0 0x7f9810157aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:59.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.714+0000 7f9816a95640 1 -- 172.21.15.73:0/3467946338 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98100b38e0 con 0x7f981015b600 2024-04-26T20:52:59.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.714+0000 7f9816a95640 1 --2- 172.21.15.73:0/3467946338 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f981015b600 0x7f981014f3e0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f980800d150 tx=0x7f9808004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:59.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.714+0000 7f9806ffd640 1 -- 172.21.15.73:0/3467946338 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9808034630 con 0x7f981015b600 2024-04-26T20:52:59.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.714+0000 7f9806ffd640 1 -- 172.21.15.73:0/3467946338 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98080347d0 con 0x7f981015b600 2024-04-26T20:52:59.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.714+0000 7f98184f8640 1 -- 172.21.15.73:0/3467946338 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98100b3b80 con 0x7f981015b600 2024-04-26T20:52:59.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.714+0000 7f9806ffd640 1 -- 172.21.15.73:0/3467946338 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9808034ba0 con 0x7f981015b600 2024-04-26T20:52:59.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.714+0000 7f98184f8640 1 -- 172.21.15.73:0/3467946338 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f981015a150 con 0x7f981015b600 2024-04-26T20:52:59.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.716+0000 7f9806ffd640 1 -- 172.21.15.73:0/3467946338 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9808031070 con 0x7f981015b600 2024-04-26T20:52:59.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.716+0000 7f9806ffd640 1 --2- 172.21.15.73:0/3467946338 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f97f406c920 0x7f97f406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:59.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.716+0000 7f9806ffd640 1 -- 172.21.15.73:0/3467946338 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6187+0+0 (secure 0 0 0) 0x7f98080aebb0 con 0x7f981015b600 2024-04-26T20:52:59.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.716+0000 7f9816294640 1 --2- 172.21.15.73:0/3467946338 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f97f406c920 0x7f97f406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:59.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.716+0000 7f9816294640 1 --2- 172.21.15.73:0/3467946338 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f97f406c920 0x7f97f406ede0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f980c007fd0 tx=0x7f980c007980 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:59.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.717+0000 7f981b20e740 1 Processor -- start 2024-04-26T20:52:59.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.717+0000 7f981b20e740 1 -- start start 2024-04-26T20:52:59.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.718+0000 7f981b20e740 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x560594e879f0 0x560594e60050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:59.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.718+0000 7f981b20e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560594e87dd0 0x560594e8a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:59.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.718+0000 7f981b20e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560594e8a790 0x560594e5dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:59.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.718+0000 7f981b20e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x560594dc4740 con 0x560594e8a790 2024-04-26T20:52:59.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.718+0000 7f981b20e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560594d603a0 con 0x560594e87dd0 2024-04-26T20:52:59.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.718+0000 7f981b20e740 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x560594dc0700 con 0x560594e879f0 2024-04-26T20:52:59.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.718+0000 7f9816a95640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560594e8a790 0x560594e5dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:59.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.718+0000 7f9816a95640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560594e8a790 0x560594e5dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55142/0 (socket says 172.21.15.73:55142) 2024-04-26T20:52:59.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.718+0000 7f9816a95640 1 -- 172.21.15.73:0/1355400428 learned_addr learned my addr 172.21.15.73:0/1355400428 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:59.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.718+0000 7f9816294640 1 --2- 172.21.15.73:0/1355400428 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x560594e879f0 0x560594e60050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:59.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.718+0000 7f9815a93640 1 --2- 172.21.15.73:0/1355400428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560594e87dd0 0x560594e8a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:59.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.718+0000 7f9816a95640 1 -- 172.21.15.73:0/1355400428 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x560594e879f0 msgr2=0x560594e60050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:59.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.718+0000 7f9816a95640 1 --2- 172.21.15.73:0/1355400428 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x560594e879f0 0x560594e60050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:59.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.718+0000 7f9816a95640 1 -- 172.21.15.73:0/1355400428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560594e87dd0 msgr2=0x560594e8a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:59.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.718+0000 7f9816a95640 1 --2- 172.21.15.73:0/1355400428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560594e87dd0 0x560594e8a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:59.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.718+0000 7f9816a95640 1 -- 172.21.15.73:0/1355400428 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560594d7c750 con 0x560594e8a790 2024-04-26T20:52:59.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.718+0000 7f9816a95640 1 --2- 172.21.15.73:0/1355400428 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560594e8a790 0x560594e5dff0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f9808078bf0 tx=0x7f980807ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb8ad078bee330a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:59.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.719+0000 7f97f3fff640 1 -- 172.21.15.73:0/1355400428 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9808083070 con 0x560594e8a790 2024-04-26T20:52:59.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.719+0000 7f97f3fff640 1 -- 172.21.15.73:0/1355400428 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9808039040 con 0x560594e8a790 2024-04-26T20:52:59.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.719+0000 7f97f3fff640 1 -- 172.21.15.73:0/1355400428 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9808080b40 con 0x560594e8a790 2024-04-26T20:52:59.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.719+0000 7f981b20e740 1 -- 172.21.15.73:0/1355400428 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560594e8a790 msgr2=0x560594e5dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:59.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.719+0000 7f981b20e740 1 --2- 172.21.15.73:0/1355400428 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560594e8a790 0x560594e5dff0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f9808078bf0 tx=0x7f980807ce90 comp rx=0 tx=0).stop 2024-04-26T20:52:59.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.719+0000 7f981b20e740 1 -- 172.21.15.73:0/1355400428 shutdown_connections 2024-04-26T20:52:59.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.719+0000 7f981b20e740 1 --2- 172.21.15.73:0/1355400428 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560594e8a790 0x560594e5dff0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:59.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.719+0000 7f981b20e740 1 --2- 172.21.15.73:0/1355400428 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560594e87dd0 0x560594e8a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:59.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.719+0000 7f981b20e740 1 --2- 172.21.15.73:0/1355400428 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x560594e879f0 0x560594e60050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:59.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.719+0000 7f981b20e740 1 -- 172.21.15.73:0/1355400428 >> 172.21.15.73:0/1355400428 conn(0x560594e5ca50 msgr2=0x560594ea5250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:59.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.719+0000 7f981b20e740 1 -- 172.21.15.73:0/1355400428 shutdown_connections 2024-04-26T20:52:59.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.719+0000 7f981b20e740 1 -- 172.21.15.73:0/1355400428 wait complete. 2024-04-26T20:52:59.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.720+0000 7f981b20e740 1 -- start start 2024-04-26T20:52:59.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.720+0000 7f981b20e740 1 Processor -- start 2024-04-26T20:52:59.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.721+0000 7f981b20e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560594e87dd0 0x560594e71e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:59.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.721+0000 7f981b20e740 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x560594e8a790 0x560594e9c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:59.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.721+0000 7f981b20e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560594e8be60 0x560594e9c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:59.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.721+0000 7f981b20e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x560594d603a0 con 0x560594e87dd0 2024-04-26T20:52:59.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.721+0000 7f981b20e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560594d4bfe0 con 0x560594e8be60 2024-04-26T20:52:59.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.721+0000 7f981b20e740 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x560594dc4740 con 0x560594e8a790 2024-04-26T20:52:59.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.721+0000 7f981b20e740 10 client.0 _ll_register_callbacks cb 0x560594d62150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:52:59.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.721+0000 7f9816294640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560594e87dd0 0x560594e71e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:59.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.721+0000 7f9816294640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560594e87dd0 0x560594e71e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55146/0 (socket says 172.21.15.73:55146) 2024-04-26T20:52:59.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.721+0000 7f9816294640 1 -- 172.21.15.73:0/3845214207 learned_addr learned my addr 172.21.15.73:0/3845214207 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:59.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.721+0000 7f9815a93640 1 --2- 172.21.15.73:0/3845214207 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x560594e8a790 0x560594e9c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:59.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.721+0000 7f9816a95640 1 --2- 172.21.15.73:0/3845214207 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560594e8be60 0x560594e9c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:59.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.721+0000 7f9816294640 1 -- 172.21.15.73:0/3845214207 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x560594e8a790 msgr2=0x560594e9c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:59.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.721+0000 7f9816294640 1 --2- 172.21.15.73:0/3845214207 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x560594e8a790 0x560594e9c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:59.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.721+0000 7f9816294640 1 -- 172.21.15.73:0/3845214207 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560594e8be60 msgr2=0x560594e9c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:59.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.721+0000 7f9816294640 1 --2- 172.21.15.73:0/3845214207 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560594e8be60 0x560594e9c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:59.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.721+0000 7f9816294640 1 -- 172.21.15.73:0/3845214207 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560594ad62d0 con 0x560594e87dd0 2024-04-26T20:52:59.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.721+0000 7f9816294640 1 --2- 172.21.15.73:0/3845214207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560594e87dd0 0x560594e71e70 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f980c005e00 tx=0x7f980c0010f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e3da03426cd487a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:59.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.721+0000 7f97f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-04-26T20:52:59.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.722+0000 7f97f17fa640 1 -- 172.21.15.73:0/3845214207 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980c012850 con 0x560594e87dd0 2024-04-26T20:52:59.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.722+0000 7f97f17fa640 1 -- 172.21.15.73:0/3845214207 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(56..56 src has 1..56) ==== 6187+0+0 (secure 0 0 0) 0x7f980c0129f0 con 0x560594e87dd0 2024-04-26T20:52:59.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.722+0000 7f981b20e740 1 -- 172.21.15.73:0/3845214207 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9808039040 con 0x560594e87dd0 2024-04-26T20:52:59.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.722+0000 7f97f17fa640 1 -- 172.21.15.73:0/3845214207 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f980c012bf0 con 0x560594e87dd0 2024-04-26T20:52:59.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.722+0000 7f981b20e740 10 client.6138 fetch_fsmap learned FSMap version 34 2024-04-26T20:52:59.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.722+0000 7f981b20e740 1 -- 172.21.15.73:0/3845214207 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x560594ba06b0 con 0x560594e87dd0 2024-04-26T20:52:59.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.722+0000 7f97f17fa640 1 -- 172.21.15.73:0/3845214207 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9808039040 con 0x560594e87dd0 2024-04-26T20:52:59.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.722+0000 7f97f17fa640 1 -- 172.21.15.73:0/3845214207 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7f980c022430 con 0x560594e87dd0 2024-04-26T20:52:59.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.722+0000 7f981b20e740 10 client.6138 fetch_fsmap finished waiting for FSMap version 34 2024-04-26T20:52:59.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.722+0000 7f981b20e740 10 client.6138 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T20:52:59.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.722+0000 7f981b20e740 20 client.6138 populate_metadata read hostname 'smithi073' 2024-04-26T20:52:59.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.722+0000 7f981b20e740 1 --2- 172.21.15.73:0/3845214207 >> [v2:172.21.15.73:6834/1169682578,v1:172.21.15.73:6836/1169682578] conn(0x560594ea45b0 0x560594e8d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:59.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.722+0000 7f981b20e740 4 client.6138 mds_command: new command op to 6099 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:52:59.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.722+0000 7f981b20e740 1 -- 172.21.15.73:0/3845214207 --> [v2:172.21.15.73:6834/1169682578,v1:172.21.15.73:6836/1169682578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560594dd2350 con 0x560594ea45b0 2024-04-26T20:52:59.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.722+0000 7f9815a93640 1 --2- 172.21.15.73:0/3845214207 >> [v2:172.21.15.73:6834/1169682578,v1:172.21.15.73:6836/1169682578] conn(0x560594ea45b0 0x560594e8d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:59.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.722+0000 7f97f17fa640 10 client.6138 ms_handle_connect on v2:172.21.15.73:6834/1169682578 2024-04-26T20:52:59.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.722+0000 7f9815a93640 1 --2- 172.21.15.73:0/3845214207 >> [v2:172.21.15.73:6834/1169682578,v1:172.21.15.73:6836/1169682578] conn(0x560594ea45b0 0x560594e8d890 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7fbad3af869148f server_cookie=a1324587d21f2234 in_seq=0 out_seq=0 2024-04-26T20:52:59.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.723+0000 7f97f17fa640 1 -- 172.21.15.73:0/3845214207 <== mds.0 v2:172.21.15.73:6834/1169682578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560594dd2350 con 0x560594ea45b0 2024-04-26T20:52:59.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.723+0000 7f97f17fa640 10 client.6138 handle_command_reply: tid=0 2024-04-26T20:52:59.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.723+0000 7f981b20e740 1 client.6138 shutdown 2024-04-26T20:52:59.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.723+0000 7f981b20e740 1 -- 172.21.15.73:0/3845214207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560594e87dd0 msgr2=0x560594e71e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:59.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.723+0000 7f981b20e740 1 --2- 172.21.15.73:0/3845214207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560594e87dd0 0x560594e71e70 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f980c005e00 tx=0x7f980c0010f0 comp rx=0 tx=0).stop 2024-04-26T20:52:59.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.723+0000 7f981b20e740 1 -- 172.21.15.73:0/3845214207 shutdown_connections 2024-04-26T20:52:59.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.723+0000 7f981b20e740 1 --2- 172.21.15.73:0/3845214207 >> [v2:172.21.15.73:6834/1169682578,v1:172.21.15.73:6836/1169682578] conn(0x560594ea45b0 0x560594e8d890 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:59.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.723+0000 7f981b20e740 1 --2- 172.21.15.73:0/3845214207 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560594e8be60 0x560594e9c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:59.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.723+0000 7f981b20e740 1 --2- 172.21.15.73:0/3845214207 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x560594e8a790 0x560594e9c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:59.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.723+0000 7f97f17fa640 0 client.6138 ms_handle_reset on v2:172.21.15.73:6834/1169682578 2024-04-26T20:52:59.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.723+0000 7f981b20e740 1 --2- 172.21.15.73:0/3845214207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560594e87dd0 0x560594e71e70 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:59.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.723+0000 7f981b20e740 1 -- 172.21.15.73:0/3845214207 >> 172.21.15.73:0/3845214207 conn(0x560594e879f0 msgr2=0x560594e5dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:59.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.723+0000 7f981b20e740 1 -- 172.21.15.73:0/3845214207 shutdown_connections 2024-04-26T20:52:59.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.723+0000 7f981b20e740 1 -- 172.21.15.73:0/3845214207 wait complete. 2024-04-26T20:52:59.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.723+0000 7f981b20e740 20 client.6138 trim_cache size 0 max 16384 2024-04-26T20:52:59.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.741+0000 7f981b20e740 1 Processor -- start 2024-04-26T20:52:59.743 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.741+0000 7f981b20e740 1 -- start start 2024-04-26T20:52:59.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.742+0000 7f981b20e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560594e87dd0 0x560594e85eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:59.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.742+0000 7f981b20e740 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x560594e8a790 0x560594ec3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:59.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.742+0000 7f981b20e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560594e8be60 0x560594ec3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:59.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.742+0000 7f981b20e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f98101fc9e0 con 0x560594e8be60 2024-04-26T20:52:59.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.742+0000 7f981b20e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f98100b4f70 con 0x560594e87dd0 2024-04-26T20:52:59.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.742+0000 7f981b20e740 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f98101fc540 con 0x560594e8a790 2024-04-26T20:52:59.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.742+0000 7f9816a95640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560594e8be60 0x560594ec3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:59.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.742+0000 7f9816a95640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560594e8be60 0x560594ec3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55154/0 (socket says 172.21.15.73:55154) 2024-04-26T20:52:59.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.742+0000 7f9816a95640 1 -- 172.21.15.73:0/977264746 learned_addr learned my addr 172.21.15.73:0/977264746 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:59.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.742+0000 7f9815a93640 1 --2- 172.21.15.73:0/977264746 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x560594e8a790 0x560594ec3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:59.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.742+0000 7f9816294640 1 --2- 172.21.15.73:0/977264746 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560594e87dd0 0x560594e85eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:59.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.742+0000 7f9816a95640 1 -- 172.21.15.73:0/977264746 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x560594e8a790 msgr2=0x560594ec3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:59.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.742+0000 7f9816a95640 1 --2- 172.21.15.73:0/977264746 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x560594e8a790 0x560594ec3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:59.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.742+0000 7f9816a95640 1 -- 172.21.15.73:0/977264746 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560594e87dd0 msgr2=0x560594e85eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:59.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.742+0000 7f9816a95640 1 --2- 172.21.15.73:0/977264746 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560594e87dd0 0x560594e85eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:59.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.742+0000 7f9816a95640 1 -- 172.21.15.73:0/977264746 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560594e75170 con 0x560594e8be60 2024-04-26T20:52:59.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.742+0000 7f9815a93640 1 --2- 172.21.15.73:0/977264746 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x560594e8a790 0x560594ec3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:52:59.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.743+0000 7f9816a95640 1 --2- 172.21.15.73:0/977264746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560594e8be60 0x560594ec3eb0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f98080acf90 tx=0x7f9808083000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c461246279bcaf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:59.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.743+0000 7f97f37fe640 1 -- 172.21.15.73:0/977264746 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9808085370 con 0x560594e8be60 2024-04-26T20:52:59.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.743+0000 7f97f37fe640 1 -- 172.21.15.73:0/977264746 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9808039040 con 0x560594e8be60 2024-04-26T20:52:59.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.743+0000 7f97f37fe640 1 -- 172.21.15.73:0/977264746 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9808080b40 con 0x560594e8be60 2024-04-26T20:52:59.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.743+0000 7f981b20e740 1 -- 172.21.15.73:0/977264746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560594e8be60 msgr2=0x560594ec3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:59.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.743+0000 7f981b20e740 1 --2- 172.21.15.73:0/977264746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560594e8be60 0x560594ec3eb0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f98080acf90 tx=0x7f9808083000 comp rx=0 tx=0).stop 2024-04-26T20:52:59.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.743+0000 7f981b20e740 1 -- 172.21.15.73:0/977264746 shutdown_connections 2024-04-26T20:52:59.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.743+0000 7f981b20e740 1 --2- 172.21.15.73:0/977264746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560594e8be60 0x560594ec3eb0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:59.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.743+0000 7f981b20e740 1 --2- 172.21.15.73:0/977264746 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x560594e8a790 0x560594ec3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:59.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.743+0000 7f981b20e740 1 --2- 172.21.15.73:0/977264746 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560594e87dd0 0x560594e85eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:59.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.743+0000 7f981b20e740 1 -- 172.21.15.73:0/977264746 >> 172.21.15.73:0/977264746 conn(0x560594e879f0 msgr2=0x560594ea55f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:59.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.743+0000 7f981b20e740 1 -- 172.21.15.73:0/977264746 shutdown_connections 2024-04-26T20:52:59.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.743+0000 7f981b20e740 1 -- 172.21.15.73:0/977264746 wait complete. 2024-04-26T20:52:59.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.744+0000 7f981b20e740 1 -- start start 2024-04-26T20:52:59.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.744+0000 7f981b20e740 1 Processor -- start 2024-04-26T20:52:59.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.745+0000 7f981b20e740 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x560594e87dd0 0x560594e5f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:59.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.745+0000 7f981b20e740 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560594e8a790 0x560594e5f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:59.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.745+0000 7f981b20e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560594e8be60 0x560594e702e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:59.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.745+0000 7f981b20e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x560594ea4430 con 0x560594e8be60 2024-04-26T20:52:59.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.745+0000 7f981b20e740 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x560594e8d130 con 0x560594e8a790 2024-04-26T20:52:59.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.745+0000 7f981b20e740 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x560594ea4a50 con 0x560594e87dd0 2024-04-26T20:52:59.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.745+0000 7f981b20e740 10 client.0 _ll_register_callbacks cb 0x7f980c007170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:52:59.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.745+0000 7f9816a95640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560594e8be60 0x560594e702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:59.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.745+0000 7f9816a95640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560594e8be60 0x560594e702e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55160/0 (socket says 172.21.15.73:55160) 2024-04-26T20:52:59.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.745+0000 7f9816294640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x560594e87dd0 0x560594e5f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:59.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.745+0000 7f9816a95640 1 -- 172.21.15.73:0/1583866636 learned_addr learned my addr 172.21.15.73:0/1583866636 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:59.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.745+0000 7f9815a93640 1 --2- 172.21.15.73:0/1583866636 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560594e8a790 0x560594e5f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:59.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.745+0000 7f9816a95640 1 -- 172.21.15.73:0/1583866636 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x560594e87dd0 msgr2=0x560594e5f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:59.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.745+0000 7f9816a95640 1 --2- 172.21.15.73:0/1583866636 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x560594e87dd0 0x560594e5f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:59.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.745+0000 7f9816a95640 1 -- 172.21.15.73:0/1583866636 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560594e8a790 msgr2=0x560594e5f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:59.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.745+0000 7f9816a95640 1 --2- 172.21.15.73:0/1583866636 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560594e8a790 0x560594e5f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:59.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.745+0000 7f9816a95640 1 -- 172.21.15.73:0/1583866636 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560594e9b9e0 con 0x560594e8be60 2024-04-26T20:52:59.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.745+0000 7f9816a95640 1 --2- 172.21.15.73:0/1583866636 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560594e8be60 0x560594e702e0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f980802cf60 tx=0x7f9808036000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3daa706b0acc4421 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:59.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.745+0000 7f97f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-04-26T20:52:59.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.746+0000 7f97f17fa640 1 -- 172.21.15.73:0/1583866636 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9808080b40 con 0x560594e8be60 2024-04-26T20:52:59.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.746+0000 7f97f17fa640 1 -- 172.21.15.73:0/1583866636 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(56..56 src has 1..56) ==== 6187+0+0 (secure 0 0 0) 0x7f980807aa90 con 0x560594e8be60 2024-04-26T20:52:59.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.746+0000 7f981b20e740 1 -- 172.21.15.73:0/1583866636 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9808039040 con 0x560594e8be60 2024-04-26T20:52:59.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.746+0000 7f97f17fa640 1 -- 172.21.15.73:0/1583866636 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9808034d40 con 0x560594e8be60 2024-04-26T20:52:59.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.746+0000 7f981b20e740 10 client.6144 fetch_fsmap learned FSMap version 34 2024-04-26T20:52:59.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.746+0000 7f981b20e740 1 -- 172.21.15.73:0/1583866636 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x560594e9b9e0 con 0x560594e8be60 2024-04-26T20:52:59.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.746+0000 7f97f17fa640 1 -- 172.21.15.73:0/1583866636 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9808039040 con 0x560594e8be60 2024-04-26T20:52:59.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.746+0000 7f97f17fa640 1 -- 172.21.15.73:0/1583866636 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7f9808033070 con 0x560594e8be60 2024-04-26T20:52:59.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.746+0000 7f981b20e740 10 client.6144 fetch_fsmap finished waiting for FSMap version 34 2024-04-26T20:52:59.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.746+0000 7f981b20e740 10 client.6144 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T20:52:59.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.746+0000 7f981b20e740 20 client.6144 populate_metadata read hostname 'smithi073' 2024-04-26T20:52:59.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.746+0000 7f981b20e740 1 --2- 172.21.15.73:0/1583866636 >> [v2:172.21.15.73:6834/1169682578,v1:172.21.15.73:6836/1169682578] conn(0x560594e70820 0x560594e70c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:59.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.746+0000 7f981b20e740 4 client.6144 mds_command: new command op to 6099 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T20:52:59.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.746+0000 7f981b20e740 1 -- 172.21.15.73:0/1583866636 --> [v2:172.21.15.73:6834/1169682578,v1:172.21.15.73:6836/1169682578] -- command(tid 0: {"prefix": "status"}) -- 0x560594dcc1c0 con 0x560594e70820 2024-04-26T20:52:59.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.746+0000 7f9816294640 1 --2- 172.21.15.73:0/1583866636 >> [v2:172.21.15.73:6834/1169682578,v1:172.21.15.73:6836/1169682578] conn(0x560594e70820 0x560594e70c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:59.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.746+0000 7f97f17fa640 10 client.6144 ms_handle_connect on v2:172.21.15.73:6834/1169682578 2024-04-26T20:52:59.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.746+0000 7f9816294640 1 --2- 172.21.15.73:0/1583866636 >> [v2:172.21.15.73:6834/1169682578,v1:172.21.15.73:6836/1169682578] conn(0x560594e70820 0x560594e70c00 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2fc578b3dc12a25 server_cookie=6adc30f210b3b1cf in_seq=0 out_seq=0 2024-04-26T20:52:59.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.747+0000 7f97f17fa640 1 -- 172.21.15.73:0/1583866636 <== mds.0 v2:172.21.15.73:6834/1169682578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x560594dcc1c0 con 0x560594e70820 2024-04-26T20:52:59.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.747+0000 7f97f17fa640 10 client.6144 handle_command_reply: tid=0 2024-04-26T20:52:59.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.747+0000 7f981b20e740 1 client.6144 shutdown 2024-04-26T20:52:59.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.747+0000 7f981b20e740 1 -- 172.21.15.73:0/1583866636 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560594e8be60 msgr2=0x560594e702e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:59.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.747+0000 7f981b20e740 1 --2- 172.21.15.73:0/1583866636 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560594e8be60 0x560594e702e0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f980802cf60 tx=0x7f9808036000 comp rx=0 tx=0).stop 2024-04-26T20:52:59.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.747+0000 7f981b20e740 1 -- 172.21.15.73:0/1583866636 shutdown_connections 2024-04-26T20:52:59.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.747+0000 7f981b20e740 1 --2- 172.21.15.73:0/1583866636 >> [v2:172.21.15.73:6834/1169682578,v1:172.21.15.73:6836/1169682578] conn(0x560594e70820 0x560594e70c00 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:59.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.747+0000 7f981b20e740 1 --2- 172.21.15.73:0/1583866636 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560594e8be60 0x560594e702e0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:59.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.747+0000 7f981b20e740 1 --2- 172.21.15.73:0/1583866636 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x560594e8a790 0x560594e5f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:59.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.747+0000 7f97f17fa640 0 client.6144 ms_handle_reset on v2:172.21.15.73:6834/1169682578 2024-04-26T20:52:59.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.747+0000 7f981b20e740 1 --2- 172.21.15.73:0/1583866636 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x560594e87dd0 0x560594e5f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:59.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.747+0000 7f981b20e740 1 -- 172.21.15.73:0/1583866636 >> 172.21.15.73:0/1583866636 conn(0x560594e879f0 msgr2=0x560594ec3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:59.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.747+0000 7f981b20e740 1 -- 172.21.15.73:0/1583866636 shutdown_connections 2024-04-26T20:52:59.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.747+0000 7f981b20e740 1 -- 172.21.15.73:0/1583866636 wait complete. 2024-04-26T20:52:59.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.747+0000 7f981b20e740 20 client.6144 trim_cache size 0 max 16384 2024-04-26T20:52:59.755 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-26T20:52:59.755 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_fsid": "fbd3f548-b547-4713-856b-09f98f5d1764", 2024-04-26T20:52:59.755 INFO:teuthology.orchestra.run.smithi073.stdout: "whoami": 0, 2024-04-26T20:52:59.755 INFO:teuthology.orchestra.run.smithi073.stdout: "id": 6099, 2024-04-26T20:52:59.755 INFO:teuthology.orchestra.run.smithi073.stdout: "want_state": "up:active", 2024-04-26T20:52:59.755 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "up:active", 2024-04-26T20:52:59.755 INFO:teuthology.orchestra.run.smithi073.stdout: "fs_name": "cephfs", 2024-04-26T20:52:59.755 INFO:teuthology.orchestra.run.smithi073.stdout: "rank_uptime": 5.1906194149999996, 2024-04-26T20:52:59.755 INFO:teuthology.orchestra.run.smithi073.stdout: "mdsmap_epoch": 34, 2024-04-26T20:52:59.755 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch": 56, 2024-04-26T20:52:59.756 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch_barrier": 56, 2024-04-26T20:52:59.756 INFO:teuthology.orchestra.run.smithi073.stdout: "uptime": 6.0659275529999999 2024-04-26T20:52:59.756 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-26T20:52:59.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.747+0000 7f98184f8640 1 -- 172.21.15.73:0/3467946338 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f97f406c920 msgr2=0x7f97f406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:59.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.747+0000 7f98184f8640 1 --2- 172.21.15.73:0/3467946338 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f97f406c920 0x7f97f406ede0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f980c007fd0 tx=0x7f980c007980 comp rx=0 tx=0).stop 2024-04-26T20:52:59.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.747+0000 7f98184f8640 1 -- 172.21.15.73:0/3467946338 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f981015b600 msgr2=0x7f981014f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:59.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.747+0000 7f98184f8640 1 --2- 172.21.15.73:0/3467946338 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f981015b600 0x7f981014f3e0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f980800d150 tx=0x7f9808004a40 comp rx=0 tx=0).stop 2024-04-26T20:52:59.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.747+0000 7f98184f8640 1 -- 172.21.15.73:0/3467946338 shutdown_connections 2024-04-26T20:52:59.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.747+0000 7f98184f8640 1 --2- 172.21.15.73:0/3467946338 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f981015b600 0x7f981014f3e0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:59.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.747+0000 7f98184f8640 1 --2- 172.21.15.73:0/3467946338 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f97f406c920 0x7f97f406ede0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:59.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.747+0000 7f98184f8640 1 --2- 172.21.15.73:0/3467946338 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f98100cf5e0 0x7f981014eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:59.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.747+0000 7f98184f8640 1 --2- 172.21.15.73:0/3467946338 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f98100cf0e0 0x7f9810157aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:59.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.747+0000 7f98184f8640 1 -- 172.21.15.73:0/3467946338 >> 172.21.15.73:0/3467946338 conn(0x7f98100d0be0 msgr2=0x7f9810073020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:59.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.748+0000 7f98184f8640 1 -- 172.21.15.73:0/3467946338 shutdown_connections 2024-04-26T20:52:59.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.748+0000 7f98184f8640 1 -- 172.21.15.73:0/3467946338 wait complete. 2024-04-26T20:52:59.759 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-26T20:52:59.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.854+0000 7f301ed0a640 1 Processor -- start 2024-04-26T20:52:59.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.854+0000 7f301ed0a640 1 -- start start 2024-04-26T20:52:59.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.854+0000 7f301ed0a640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f301806fea0 0x7f3018166d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:59.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.854+0000 7f301ed0a640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3018160970 con 0x7f301815f2d0 2024-04-26T20:52:59.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.854+0000 7f301ed0a640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3018072f70 con 0x7f30181604e0 2024-04-26T20:52:59.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.854+0000 7f301ed0a640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f3018074010 con 0x7f301806fea0 2024-04-26T20:52:59.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.855+0000 7f301caa6640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f301815f2d0 0x7f3018072700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:50696/0 (socket says 172.21.15.73:50696) 2024-04-26T20:52:59.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.855+0000 7f301caa6640 1 -- 172.21.15.73:0/1990684515 learned_addr learned my addr 172.21.15.73:0/1990684515 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:52:59.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.855+0000 7f301d2a7640 1 --2- 172.21.15.73:0/1990684515 >> v2:172.21.15.120:3300/0 conn(0x7f301806fea0 0x7f3018166d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:59.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.855+0000 7f30177fe640 1 -- 172.21.15.73:0/1990684515 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4122531804 0 0) 0x7f3018160970 con 0x7f301815f2d0 2024-04-26T20:52:59.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.855+0000 7f30177fe640 1 -- 172.21.15.73:0/1990684515 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3008003520 con 0x7f301815f2d0 2024-04-26T20:52:59.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.855+0000 7f30177fe640 1 -- 172.21.15.73:0/1990684515 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1359199703 0 0) 0x7f3008003520 con 0x7f301815f2d0 2024-04-26T20:52:59.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.855+0000 7f30177fe640 1 -- 172.21.15.73:0/1990684515 >> v2:172.21.15.120:3300/0 conn(0x7f301806fea0 msgr2=0x7f3018166d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:59.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.855+0000 7f30177fe640 1 --2- 172.21.15.73:0/1990684515 >> v2:172.21.15.120:3300/0 conn(0x7f301806fea0 0x7f3018166d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:59.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.855+0000 7f30177fe640 1 -- 172.21.15.73:0/1990684515 >> v1:172.21.15.120:6789/0 conn(0x7f30181604e0 legacy=0x7f301806f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:59.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.855+0000 7f30177fe640 1 -- 172.21.15.73:0/1990684515 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3018073250 con 0x7f301815f2d0 2024-04-26T20:52:59.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.855+0000 7f30177fe640 1 -- 172.21.15.73:0/1990684515 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7f300c001670 con 0x7f301815f2d0 2024-04-26T20:52:59.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.855+0000 7f30177fe640 1 -- 172.21.15.73:0/1990684515 >> v1:172.21.15.73:6789/0 conn(0x7f301815f2d0 legacy=0x7f3018072700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:59.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.855+0000 7f30177fe640 1 --2- 172.21.15.73:0/1990684515 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f30080039b0 0x7f3008003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:59.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.855+0000 7f30177fe640 1 --2- 172.21.15.73:0/1990684515 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3008004800 0x7f3008004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:59.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.855+0000 7f30177fe640 1 --2- 172.21.15.73:0/1990684515 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f30080051c0 0x7f300800dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:59.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.855+0000 7f30177fe640 1 -- 172.21.15.73:0/1990684515 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3018074010 con 0x7f3008004800 2024-04-26T20:52:59.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.855+0000 7f30177fe640 1 -- 172.21.15.73:0/1990684515 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f300800e270 con 0x7f30080039b0 2024-04-26T20:52:59.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.855+0000 7f30177fe640 1 -- 172.21.15.73:0/1990684515 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f300800e470 con 0x7f30080051c0 2024-04-26T20:52:59.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.855+0000 7f3017fff640 1 --2- 172.21.15.73:0/1990684515 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3008004800 0x7f3008004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:59.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.855+0000 7f301caa6640 1 --2- 172.21.15.73:0/1990684515 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f30080039b0 0x7f3008003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:59.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.855+0000 7f301d2a7640 1 --2- 172.21.15.73:0/1990684515 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f30080051c0 0x7f300800dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:59.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.856+0000 7f3017fff640 1 -- 172.21.15.73:0/1990684515 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f30080051c0 msgr2=0x7f300800dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:59.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.856+0000 7f3017fff640 1 --2- 172.21.15.73:0/1990684515 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f30080051c0 0x7f300800dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:59.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.856+0000 7f3017fff640 1 -- 172.21.15.73:0/1990684515 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f30080039b0 msgr2=0x7f3008003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:52:59.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.856+0000 7f3017fff640 1 --2- 172.21.15.73:0/1990684515 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f30080039b0 0x7f3008003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:59.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.856+0000 7f301d2a7640 1 -- 172.21.15.73:0/1990684515 reap_dead start 2024-04-26T20:52:59.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.856+0000 7f3017fff640 1 -- 172.21.15.73:0/1990684515 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3008003520 con 0x7f3008004800 2024-04-26T20:52:59.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.856+0000 7f3017fff640 1 --2- 172.21.15.73:0/1990684515 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3008004800 0x7f3008004c80 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f3004001750 tx=0x7f3008005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a0776a877e4cbc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:59.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.856+0000 7f30177fe640 1 -- 172.21.15.73:0/1990684515 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3004001370 con 0x7f3008004800 2024-04-26T20:52:59.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.856+0000 7f30177fe640 1 -- 172.21.15.73:0/1990684515 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30040052d0 con 0x7f3008004800 2024-04-26T20:52:59.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.856+0000 7f30177fe640 1 -- 172.21.15.73:0/1990684515 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300402abe0 con 0x7f3008004800 2024-04-26T20:52:59.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.857+0000 7f301ed0a640 1 -- 172.21.15.73:0/1990684515 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3008004800 msgr2=0x7f3008004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:59.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.857+0000 7f301ed0a640 1 --2- 172.21.15.73:0/1990684515 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3008004800 0x7f3008004c80 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f3004001750 tx=0x7f3008005830 comp rx=0 tx=0).stop 2024-04-26T20:52:59.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.857+0000 7f301ed0a640 1 -- 172.21.15.73:0/1990684515 shutdown_connections 2024-04-26T20:52:59.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.857+0000 7f301ed0a640 1 --2- 172.21.15.73:0/1990684515 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3008004800 0x7f3008004c80 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:59.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.857+0000 7f301ed0a640 1 -- 172.21.15.73:0/1990684515 >> 172.21.15.73:0/1990684515 conn(0x7f30180d0be0 msgr2=0x7f30180d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:52:59.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.857+0000 7f301ed0a640 1 -- 172.21.15.73:0/1990684515 shutdown_connections 2024-04-26T20:52:59.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.857+0000 7f301ed0a640 1 -- 172.21.15.73:0/1990684515 wait complete. 2024-04-26T20:52:59.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.858+0000 7f301ed0a640 1 Processor -- start 2024-04-26T20:52:59.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.858+0000 7f301ed0a640 1 -- start start 2024-04-26T20:52:59.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.859+0000 7f301ed0a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3008004800 0x7f30180c4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:59.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.859+0000 7f301ed0a640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f30180b5ce0 0x7f30180c5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:59.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.859+0000 7f301ed0a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30180c61e0 0x7f301814dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:59.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.859+0000 7f301ed0a640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3018073af0 con 0x7f30180c61e0 2024-04-26T20:52:59.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.859+0000 7f301ed0a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f3018073810 con 0x7f3008004800 2024-04-26T20:52:59.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.859+0000 7f301ed0a640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f301815f6b0 con 0x7f30180b5ce0 2024-04-26T20:52:59.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.859+0000 7f301d2a7640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30180c61e0 0x7f301814dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:59.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.859+0000 7f301d2a7640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30180c61e0 0x7f301814dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55178/0 (socket says 172.21.15.73:55178) 2024-04-26T20:52:59.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.859+0000 7f301d2a7640 1 -- 172.21.15.73:0/2766091159 learned_addr learned my addr 172.21.15.73:0/2766091159 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:52:59.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.859+0000 7f301caa6640 1 --2- 172.21.15.73:0/2766091159 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3008004800 0x7f30180c4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:59.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.859+0000 7f3017fff640 1 --2- 172.21.15.73:0/2766091159 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f30180b5ce0 0x7f30180c5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:59.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.859+0000 7f301d2a7640 1 -- 172.21.15.73:0/2766091159 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f30180b5ce0 msgr2=0x7f30180c5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:59.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.859+0000 7f301d2a7640 1 --2- 172.21.15.73:0/2766091159 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f30180b5ce0 0x7f30180c5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:59.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.859+0000 7f301d2a7640 1 -- 172.21.15.73:0/2766091159 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3008004800 msgr2=0x7f30180c4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:52:59.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.859+0000 7f301d2a7640 1 --2- 172.21.15.73:0/2766091159 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3008004800 0x7f30180c4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:52:59.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.859+0000 7f301d2a7640 1 -- 172.21.15.73:0/2766091159 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30180c4c00 con 0x7f30180c61e0 2024-04-26T20:52:59.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.859+0000 7f301caa6640 1 --2- 172.21.15.73:0/2766091159 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3008004800 0x7f30180c4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:52:59.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.859+0000 7f301d2a7640 1 --2- 172.21.15.73:0/2766091159 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30180c61e0 0x7f301814dca0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f3018074ab0 tx=0x7f301000f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:59.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.860+0000 7f30157fa640 1 -- 172.21.15.73:0/2766091159 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301000fc80 con 0x7f30180c61e0 2024-04-26T20:52:59.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.860+0000 7f30157fa640 1 -- 172.21.15.73:0/2766091159 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f301000fe20 con 0x7f30180c61e0 2024-04-26T20:52:59.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.860+0000 7f30157fa640 1 -- 172.21.15.73:0/2766091159 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3010015670 con 0x7f30180c61e0 2024-04-26T20:52:59.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.860+0000 7f301ed0a640 1 -- 172.21.15.73:0/2766091159 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30180c4ee0 con 0x7f30180c61e0 2024-04-26T20:52:59.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.860+0000 7f301ed0a640 1 -- 172.21.15.73:0/2766091159 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30180c54a0 con 0x7f30180c61e0 2024-04-26T20:52:59.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.861+0000 7f301ed0a640 1 -- 172.21.15.73:0/2766091159 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30180694f0 con 0x7f30180c61e0 2024-04-26T20:52:59.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.861+0000 7f30157fa640 1 -- 172.21.15.73:0/2766091159 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3010015810 con 0x7f30180c61e0 2024-04-26T20:52:59.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.861+0000 7f30157fa640 1 --2- 172.21.15.73:0/2766091159 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f2ffc06c920 0x7f2ffc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:52:59.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.861+0000 7f30157fa640 1 -- 172.21.15.73:0/2766091159 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6187+0+0 (secure 0 0 0) 0x7f301008ca80 con 0x7f30180c61e0 2024-04-26T20:52:59.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.861+0000 7f301caa6640 1 --2- 172.21.15.73:0/2766091159 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f2ffc06c920 0x7f2ffc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:52:59.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.862+0000 7f301caa6640 1 --2- 172.21.15.73:0/2766091159 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f2ffc06c920 0x7f2ffc06ede0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f30180b26f0 tx=0x7f300c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:52:59.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:52:59.866+0000 7f30157fa640 1 -- 172.21.15.73:0/2766091159 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3010019030 con 0x7f30180c61e0 2024-04-26T20:53:00.084 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.083+0000 7f301ed0a640 1 -- 172.21.15.73:0/2766091159 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f30180c7a70 con 0x7f30180c61e0 2024-04-26T20:53:00.084 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.083+0000 7f30157fa640 1 -- 172.21.15.73:0/2766091159 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f3010010040 con 0x7f30180c61e0 2024-04-26T20:53:00.084 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:53:00.084 INFO:teuthology.orchestra.run.smithi073.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-26T20:53:00.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.085+0000 7f301ed0a640 1 -- 172.21.15.73:0/2766091159 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f2ffc06c920 msgr2=0x7f2ffc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:00.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.085+0000 7f301ed0a640 1 --2- 172.21.15.73:0/2766091159 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f2ffc06c920 0x7f2ffc06ede0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f30180b26f0 tx=0x7f300c009040 comp rx=0 tx=0).stop 2024-04-26T20:53:00.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.085+0000 7f301ed0a640 1 -- 172.21.15.73:0/2766091159 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30180c61e0 msgr2=0x7f301814dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:00.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.085+0000 7f301ed0a640 1 --2- 172.21.15.73:0/2766091159 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30180c61e0 0x7f301814dca0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f3018074ab0 tx=0x7f301000f1b0 comp rx=0 tx=0).stop 2024-04-26T20:53:00.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.085+0000 7f301ed0a640 1 -- 172.21.15.73:0/2766091159 shutdown_connections 2024-04-26T20:53:00.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.085+0000 7f301ed0a640 1 --2- 172.21.15.73:0/2766091159 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30180c61e0 0x7f301814dca0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:00.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.085+0000 7f301ed0a640 1 --2- 172.21.15.73:0/2766091159 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f2ffc06c920 0x7f2ffc06ede0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:00.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.085+0000 7f301ed0a640 1 --2- 172.21.15.73:0/2766091159 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f30180b5ce0 0x7f30180c5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:00.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.085+0000 7f301ed0a640 1 --2- 172.21.15.73:0/2766091159 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f3008004800 0x7f30180c4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:00.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.085+0000 7f301ed0a640 1 -- 172.21.15.73:0/2766091159 >> 172.21.15.73:0/2766091159 conn(0x7f30180d0be0 msgr2=0x7f3018152790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:00.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.085+0000 7f301ed0a640 1 -- 172.21.15.73:0/2766091159 shutdown_connections 2024-04-26T20:53:00.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.085+0000 7f301ed0a640 1 -- 172.21.15.73:0/2766091159 wait complete. 2024-04-26T20:53:00.097 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-26T20:53:00.097 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T20:53:00.097 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T20:53:00.137 DEBUG:teuthology.orchestra.run.smithi120:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T20:53:00.175 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T20:53:00.176 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T20:53:00.176 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nfd = os.open("/home/ubuntu/cephtest/mnt.0/background_file", os.O_RDWR | os.O_CREAT, 0o644)\ntry:\n while True:\n print("write_background: writing", file=sys.stderr)\n os.write(fd, b\'content\')\n if not True:\n break\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\nexcept IOError as e:\n pass\nos.close(fd)\n'], check_status=True, hostname='smithi073') 2024-04-26T20:53:00.176 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-26T20:53:00.176 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:53:00.193 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-26T20:53:00.193 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:53:00.249 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:53:00.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.384+0000 7fd48072b640 1 Processor -- start 2024-04-26T20:53:00.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.384+0000 7fd48072b640 1 -- start start 2024-04-26T20:53:00.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.385+0000 7fd48072b640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fd4780b2340 0x7fd4780cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:00.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.385+0000 7fd48072b640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd4780cd930 0x7fd4780d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:00.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.385+0000 7fd48072b640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4780d10d0 con 0x7fd478162810 2024-04-26T20:53:00.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.385+0000 7fd48072b640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fd4780d0750 con 0x7fd4780b2340 2024-04-26T20:53:00.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.385+0000 7fd48072b640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fd4780d16a0 con 0x7fd4780cd930 2024-04-26T20:53:00.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.385+0000 7fd47e4c7640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fd478162810 0x7fd478070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:50698/0 (socket says 172.21.15.73:50698) 2024-04-26T20:53:00.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.385+0000 7fd47e4c7640 1 -- 172.21.15.73:0/2352550029 learned_addr learned my addr 172.21.15.73:0/2352550029 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:53:00.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.385+0000 7fd47ecc8640 1 --2- 172.21.15.73:0/2352550029 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd4780cd930 0x7fd4780d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:00.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.385+0000 7fd47dcc6640 1 --2- 172.21.15.73:0/2352550029 >> v2:172.21.15.120:3300/0 conn(0x7fd4780b2340 0x7fd4780cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:00.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.385+0000 7fd47d4c5640 1 -- 172.21.15.73:0/2352550029 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1381237431 0 0) 0x7fd4780d10d0 con 0x7fd478162810 2024-04-26T20:53:00.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.385+0000 7fd47d4c5640 1 -- 172.21.15.73:0/2352550029 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd464003520 con 0x7fd478162810 2024-04-26T20:53:00.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.386+0000 7fd47d4c5640 1 -- 172.21.15.73:0/2352550029 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2030038481 0 0) 0x7fd464003520 con 0x7fd478162810 2024-04-26T20:53:00.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.386+0000 7fd47d4c5640 1 -- 172.21.15.73:0/2352550029 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd4780cd930 msgr2=0x7fd4780d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:00.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.386+0000 7fd47d4c5640 1 --2- 172.21.15.73:0/2352550029 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd4780cd930 0x7fd4780d1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:00.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.386+0000 7fd47d4c5640 1 -- 172.21.15.73:0/2352550029 >> v2:172.21.15.120:3300/0 conn(0x7fd4780b2340 msgr2=0x7fd4780cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:00.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.386+0000 7fd47d4c5640 1 --2- 172.21.15.73:0/2352550029 >> v2:172.21.15.120:3300/0 conn(0x7fd4780b2340 0x7fd4780cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:00.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.386+0000 7fd47d4c5640 1 -- 172.21.15.73:0/2352550029 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4780d1370 con 0x7fd478162810 2024-04-26T20:53:00.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.386+0000 7fd47d4c5640 1 -- 172.21.15.73:0/2352550029 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7fd474001670 con 0x7fd478162810 2024-04-26T20:53:00.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.386+0000 7fd47dcc6640 1 --2- 172.21.15.73:0/2352550029 >> v2:172.21.15.120:3300/0 conn(0x7fd4780b2340 0x7fd4780cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:53:00.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.386+0000 7fd47d4c5640 1 -- 172.21.15.73:0/2352550029 >> v1:172.21.15.73:6789/0 conn(0x7fd478162810 legacy=0x7fd478070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:00.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.386+0000 7fd47d4c5640 1 --2- 172.21.15.73:0/2352550029 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd4640039b0 0x7fd464003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:00.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.386+0000 7fd47d4c5640 1 --2- 172.21.15.73:0/2352550029 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4640043c0 0x7fd4640048a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:00.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.386+0000 7fd47d4c5640 1 --2- 172.21.15.73:0/2352550029 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd464004f10 0x7fd46400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:00.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.386+0000 7fd47d4c5640 1 -- 172.21.15.73:0/2352550029 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd4780d0750 con 0x7fd464004f10 2024-04-26T20:53:00.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.386+0000 7fd47d4c5640 1 -- 172.21.15.73:0/2352550029 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd4780d16a0 con 0x7fd4640043c0 2024-04-26T20:53:00.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.386+0000 7fd47d4c5640 1 -- 172.21.15.73:0/2352550029 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fd46400bf00 con 0x7fd4640039b0 2024-04-26T20:53:00.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.386+0000 7fd47ecc8640 1 --2- 172.21.15.73:0/2352550029 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd4780cd930 0x7fd4780d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:53:00.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.386+0000 7fd47ecc8640 1 --2- 172.21.15.73:0/2352550029 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd464004f10 0x7fd46400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:00.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.386+0000 7fd47dcc6640 1 --2- 172.21.15.73:0/2352550029 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4640043c0 0x7fd4640048a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:00.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.386+0000 7fd47e4c7640 1 --2- 172.21.15.73:0/2352550029 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd4640039b0 0x7fd464003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:00.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.386+0000 7fd47ecc8640 1 -- 172.21.15.73:0/2352550029 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd4640039b0 msgr2=0x7fd464003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:00.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.386+0000 7fd47ecc8640 1 --2- 172.21.15.73:0/2352550029 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd4640039b0 0x7fd464003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:00.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.386+0000 7fd47ecc8640 1 -- 172.21.15.73:0/2352550029 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4640043c0 msgr2=0x7fd4640048a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:00.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.386+0000 7fd47ecc8640 1 --2- 172.21.15.73:0/2352550029 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4640043c0 0x7fd4640048a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:00.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.386+0000 7fd47ecc8640 1 -- 172.21.15.73:0/2352550029 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd464003520 con 0x7fd464004f10 2024-04-26T20:53:00.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.387+0000 7fd47ecc8640 1 --2- 172.21.15.73:0/2352550029 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd464004f10 0x7fd46400b9c0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fd478074050 tx=0x7fd464005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38860b26ee74eb8d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:00.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.387+0000 7fd47d4c5640 1 -- 172.21.15.73:0/2352550029 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd470036070 con 0x7fd464004f10 2024-04-26T20:53:00.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.387+0000 7fd47d4c5640 1 -- 172.21.15.73:0/2352550029 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd470003c50 con 0x7fd464004f10 2024-04-26T20:53:00.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.387+0000 7fd47d4c5640 1 -- 172.21.15.73:0/2352550029 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd470009d40 con 0x7fd464004f10 2024-04-26T20:53:00.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.387+0000 7fd48072b640 1 -- 172.21.15.73:0/2352550029 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd464004f10 msgr2=0x7fd46400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:00.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.387+0000 7fd48072b640 1 --2- 172.21.15.73:0/2352550029 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd464004f10 0x7fd46400b9c0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fd478074050 tx=0x7fd464005830 comp rx=0 tx=0).stop 2024-04-26T20:53:00.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.387+0000 7fd47ecc8640 1 -- 172.21.15.73:0/2352550029 reap_dead start 2024-04-26T20:53:00.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.388+0000 7fd48072b640 1 -- 172.21.15.73:0/2352550029 shutdown_connections 2024-04-26T20:53:00.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.388+0000 7fd48072b640 1 -- 172.21.15.73:0/2352550029 >> 172.21.15.73:0/2352550029 conn(0x7fd4780d2da0 msgr2=0x7fd4780d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:00.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.388+0000 7fd48072b640 1 -- 172.21.15.73:0/2352550029 shutdown_connections 2024-04-26T20:53:00.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.388+0000 7fd48072b640 1 -- 172.21.15.73:0/2352550029 wait complete. 2024-04-26T20:53:00.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.389+0000 7fd48072b640 1 Processor -- start 2024-04-26T20:53:00.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.389+0000 7fd48072b640 1 -- start start 2024-04-26T20:53:00.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.389+0000 7fd48072b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd464004f10 0x7fd4780cc930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:00.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.389+0000 7fd48072b640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd47815a200 0x7fd478160890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:00.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.389+0000 7fd48072b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4780b3190 0x7fd4780b3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:00.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.389+0000 7fd48072b640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd4780d1b30 con 0x7fd464004f10 2024-04-26T20:53:00.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.389+0000 7fd48072b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd4780d0470 con 0x7fd4780b3190 2024-04-26T20:53:00.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.389+0000 7fd48072b640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fd4780d1cb0 con 0x7fd47815a200 2024-04-26T20:53:00.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.389+0000 7fd47e4c7640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd464004f10 0x7fd4780cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:00.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.389+0000 7fd47e4c7640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd464004f10 0x7fd4780cc930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55204/0 (socket says 172.21.15.73:55204) 2024-04-26T20:53:00.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.389+0000 7fd47e4c7640 1 -- 172.21.15.73:0/2333083236 learned_addr learned my addr 172.21.15.73:0/2333083236 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:00.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.390+0000 7fd47e4c7640 1 -- 172.21.15.73:0/2333083236 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd47815a200 msgr2=0x7fd478160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:00.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.390+0000 7fd47e4c7640 1 --2- 172.21.15.73:0/2333083236 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd47815a200 0x7fd478160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:00.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.390+0000 7fd47e4c7640 1 -- 172.21.15.73:0/2333083236 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4780b3190 msgr2=0x7fd4780b3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:00.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.390+0000 7fd47e4c7640 1 --2- 172.21.15.73:0/2333083236 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4780b3190 0x7fd4780b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:00.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.390+0000 7fd47e4c7640 1 -- 172.21.15.73:0/2333083236 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4781610e0 con 0x7fd464004f10 2024-04-26T20:53:00.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.390+0000 7fd47e4c7640 1 --2- 172.21.15.73:0/2333083236 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd464004f10 0x7fd4780cc930 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fd47400e850 tx=0x7fd47400ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:00.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.390+0000 7fd46effd640 1 -- 172.21.15.73:0/2333083236 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47400c7f0 con 0x7fd464004f10 2024-04-26T20:53:00.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.390+0000 7fd46effd640 1 -- 172.21.15.73:0/2333083236 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd474018070 con 0x7fd464004f10 2024-04-26T20:53:00.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.390+0000 7fd48072b640 1 -- 172.21.15.73:0/2333083236 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4781613c0 con 0x7fd464004f10 2024-04-26T20:53:00.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.390+0000 7fd46effd640 1 -- 172.21.15.73:0/2333083236 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd47400cbd0 con 0x7fd464004f10 2024-04-26T20:53:00.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.390+0000 7fd48072b640 1 -- 172.21.15.73:0/2333083236 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4780c9e70 con 0x7fd464004f10 2024-04-26T20:53:00.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.391+0000 7fd48072b640 1 -- 172.21.15.73:0/2333083236 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4780694f0 con 0x7fd464004f10 2024-04-26T20:53:00.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.391+0000 7fd46effd640 1 -- 172.21.15.73:0/2333083236 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd47401d460 con 0x7fd464004f10 2024-04-26T20:53:00.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.392+0000 7fd46effd640 1 --2- 172.21.15.73:0/2333083236 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd45c06c8f0 0x7fd45c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:00.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.392+0000 7fd46effd640 1 -- 172.21.15.73:0/2333083236 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6187+0+0 (secure 0 0 0) 0x7fd47408bc20 con 0x7fd464004f10 2024-04-26T20:53:00.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.392+0000 7fd47dcc6640 1 --2- 172.21.15.73:0/2333083236 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd45c06c8f0 0x7fd45c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:00.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.392+0000 7fd47dcc6640 1 --2- 172.21.15.73:0/2333083236 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd45c06c8f0 0x7fd45c06edb0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd47806f4b0 tx=0x7fd468009780 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:00.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.396+0000 7fd46effd640 1 -- 172.21.15.73:0/2333083236 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd474059f00 con 0x7fd464004f10 2024-04-26T20:53:00.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.575+0000 7fd48072b640 1 -- 172.21.15.73:0/2333083236 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd4780b1e10 con 0x7fd464004f10 2024-04-26T20:53:00.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.576+0000 7fd46effd640 1 -- 172.21.15.73:0/2333083236 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) ==== 74+0+17071 (secure 0 0 0) 0x7fd474059800 con 0x7fd464004f10 2024-04-26T20:53:00.577 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:53:00.577 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":56,"fsid":"fbd3f548-b547-4713-856b-09f98f5d1764","created":"2024-04-26T20:50:07.876786+0000","modified":"2024-04-26T20:52:53.540299+0000","last_up_change":"2024-04-26T20:50:19.082741+0000","last_in_change":"2024-04-26T20:50:10.018400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:50:19.993731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:52:18.960897+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-26T20:52:19.704261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d6f4f3-214a-4392-a02f-c33cbcab3c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6809","nonce":3461705124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6811","nonce":3461705124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6815","nonce":3461705124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6813","nonce":3461705124}]},"public_addr":"172.21.15.73:6809/3461705124","cluster_addr":"172.21.15.73:6811/3461705124","heartbeat_back_addr":"172.21.15.73:6815/3461705124","heartbeat_front_addr":"172.21.15.73:6813/3461705124","state":["exists","up"]},{"osd":1,"uuid":"786bf8ca-6ddc-4b3c-be6c-9152398bd5fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6817","nonce":2827619403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6819","nonce":2827619403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6825","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6827","nonce":2827619403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6821","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6823","nonce":2827619403}]},"public_addr":"172.21.15.73:6817/2827619403","cluster_addr":"172.21.15.73:6819/2827619403","heartbeat_back_addr":"172.21.15.73:6827/2827619403","heartbeat_front_addr":"172.21.15.73:6823/2827619403","state":["exists","up"]},{"osd":2,"uuid":"90963b1a-5437-4ef8-ad8b-6b74c534eeb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6801","nonce":2542681064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6803","nonce":2542681064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6808","nonce":2542681064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6805","nonce":2542681064}]},"public_addr":"172.21.15.73:6801/2542681064","cluster_addr":"172.21.15.73:6803/2542681064","heartbeat_back_addr":"172.21.15.73:6808/2542681064","heartbeat_front_addr":"172.21.15.73:6805/2542681064","state":["exists","up"]},{"osd":3,"uuid":"bd3d2148-6e38-41b4-a453-fe6e3b8b4ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6822","nonce":1604042718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6826","nonce":1604042718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6831","nonce":1604042718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6829","nonce":1604042718}]},"public_addr":"172.21.15.73:6822/1604042718","cluster_addr":"172.21.15.73:6826/1604042718","heartbeat_back_addr":"172.21.15.73:6831/1604042718","heartbeat_front_addr":"172.21.15.73:6829/1604042718","state":["exists","up"]},{"osd":4,"uuid":"e8668dc8-a23f-46d2-8311-cb882da08e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6817","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6819","nonce":3591055561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6824","nonce":3591055561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6831","nonce":3591055561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6828","nonce":3591055561}]},"public_addr":"172.21.15.120:6819/3591055561","cluster_addr":"172.21.15.120:6824/3591055561","heartbeat_back_addr":"172.21.15.120:6831/3591055561","heartbeat_front_addr":"172.21.15.120:6828/3591055561","state":["exists","up"]},{"osd":5,"uuid":"cbac0bdb-96bd-4852-bcce-e564495c3fcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6809","nonce":819450433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6811","nonce":819450433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6815","nonce":819450433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6813","nonce":819450433}]},"public_addr":"172.21.15.120:6809/819450433","cluster_addr":"172.21.15.120:6811/819450433","heartbeat_back_addr":"172.21.15.120:6815/819450433","heartbeat_front_addr":"172.21.15.120:6813/819450433","state":["exists","up"]},{"osd":6,"uuid":"1884b2a8-569f-4272-96a9-f25a09364dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6801","nonce":3946210727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6803","nonce":3946210727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6807","nonce":3946210727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6805","nonce":3946210727}]},"public_addr":"172.21.15.120:6801/3946210727","cluster_addr":"172.21.15.120:6803/3946210727","heartbeat_back_addr":"172.21.15.120:6807/3946210727","heartbeat_front_addr":"172.21.15.120:6805/3946210727","state":["exists","up"]},{"osd":7,"uuid":"c0c3bf6d-a4c5-4f1e-aa2d-330951830666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6818","nonce":2915574674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6821","nonce":2915574674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6827","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6829","nonce":2915574674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6823","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6825","nonce":2915574674}]},"public_addr":"172.21.15.120:6818/2915574674","cluster_addr":"172.21.15.120:6821/2915574674","heartbeat_back_addr":"172.21.15.120:6829/2915574674","heartbeat_front_addr":"172.21.15.120:6825/2915574674","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.218216+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.208710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.112705+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.182789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.164324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.174212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.225396+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.261559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:6837/2770303348":"2024-04-27T20:52:53.540263+0000","172.21.15.73:6836/2770303348":"2024-04-27T20:52:53.540263+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:53:00.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.578+0000 7fd48072b640 1 -- 172.21.15.73:0/2333083236 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd45c06c8f0 msgr2=0x7fd45c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:00.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.578+0000 7fd48072b640 1 --2- 172.21.15.73:0/2333083236 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd45c06c8f0 0x7fd45c06edb0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd47806f4b0 tx=0x7fd468009780 comp rx=0 tx=0).stop 2024-04-26T20:53:00.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.578+0000 7fd48072b640 1 -- 172.21.15.73:0/2333083236 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd464004f10 msgr2=0x7fd4780cc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:00.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.578+0000 7fd48072b640 1 --2- 172.21.15.73:0/2333083236 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd464004f10 0x7fd4780cc930 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fd47400e850 tx=0x7fd47400ed10 comp rx=0 tx=0).stop 2024-04-26T20:53:00.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.578+0000 7fd48072b640 1 -- 172.21.15.73:0/2333083236 shutdown_connections 2024-04-26T20:53:00.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.578+0000 7fd48072b640 1 --2- 172.21.15.73:0/2333083236 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd45c06c8f0 0x7fd45c06edb0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:00.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.578+0000 7fd48072b640 1 --2- 172.21.15.73:0/2333083236 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd4780b3190 0x7fd4780b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:00.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.578+0000 7fd48072b640 1 --2- 172.21.15.73:0/2333083236 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd47815a200 0x7fd478160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:00.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.578+0000 7fd48072b640 1 --2- 172.21.15.73:0/2333083236 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd464004f10 0x7fd4780cc930 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:00.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.578+0000 7fd48072b640 1 -- 172.21.15.73:0/2333083236 >> 172.21.15.73:0/2333083236 conn(0x7fd4780d2da0 msgr2=0x7fd4780b2720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:00.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.578+0000 7fd48072b640 1 -- 172.21.15.73:0/2333083236 shutdown_connections 2024-04-26T20:53:00.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.578+0000 7fd48072b640 1 -- 172.21.15.73:0/2333083236 wait complete. 2024-04-26T20:53:00.590 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:50:19.993731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:52:18.960897+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:52:19.704261+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:53:00.590 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:53:00.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.689+0000 7fe942c07640 1 Processor -- start 2024-04-26T20:53:00.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.689+0000 7fe942c07640 1 -- start start 2024-04-26T20:53:00.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.690+0000 7fe942c07640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe93c0d1a00 0x7fe93c0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:00.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.690+0000 7fe942c07640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe93c0b40d0 0x7fe93c0b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:00.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.690+0000 7fe942c07640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe93c0d0a10 con 0x7fe93c0d2320 2024-04-26T20:53:00.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.690+0000 7fe942c07640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7fe93c0d0cf0 con 0x7fe93c0d1a00 2024-04-26T20:53:00.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.690+0000 7fe942c07640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fe93c0d0fd0 con 0x7fe93c0b40d0 2024-04-26T20:53:00.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.690+0000 7fe9409a3640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe93c0d1a00 0x7fe93c0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:00.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.690+0000 7fe93bfff640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fe93c0d2320 0x7fe93c1625f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:50714/0 (socket says 172.21.15.73:50714) 2024-04-26T20:53:00.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.690+0000 7fe93bfff640 1 -- 172.21.15.73:0/3564173509 learned_addr learned my addr 172.21.15.73:0/3564173509 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:53:00.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.691+0000 7fe9411a4640 1 --2- 172.21.15.73:0/3564173509 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe93c0b40d0 0x7fe93c0b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:00.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.691+0000 7fe9409a3640 1 -- 172.21.15.73:0/3564173509 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe93c0b40d0 msgr2=0x7fe93c0b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:00.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.691+0000 7fe9409a3640 1 --2- 172.21.15.73:0/3564173509 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe93c0b40d0 0x7fe93c0b4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:00.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.691+0000 7fe9409a3640 1 -- 172.21.15.73:0/3564173509 >> v1:172.21.15.73:6789/0 conn(0x7fe93c0d2320 legacy=0x7fe93c1625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:00.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.691+0000 7fe9409a3640 1 -- 172.21.15.73:0/3564173509 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe93c0d40a0 con 0x7fe93c0d1a00 2024-04-26T20:53:00.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.691+0000 7fe9411a4640 1 --2- 172.21.15.73:0/3564173509 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe93c0b40d0 0x7fe93c0b4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:53:00.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.691+0000 7fe9409a3640 1 --2- 172.21.15.73:0/3564173509 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe93c0d1a00 0x7fe93c0d1de0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fe92c0097a0 tx=0x7fe92c02a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5655d0925e8aa101 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:00.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.691+0000 7fe93b7fe640 1 -- 172.21.15.73:0/3564173509 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92c037070 con 0x7fe93c0d1a00 2024-04-26T20:53:00.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.692+0000 7fe93b7fe640 1 -- 172.21.15.73:0/3564173509 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe92c02adb0 con 0x7fe93c0d1a00 2024-04-26T20:53:00.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.692+0000 7fe93b7fe640 1 -- 172.21.15.73:0/3564173509 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92c032660 con 0x7fe93c0d1a00 2024-04-26T20:53:00.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.692+0000 7fe942c07640 1 -- 172.21.15.73:0/3564173509 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe93c0d1a00 msgr2=0x7fe93c0d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:00.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.692+0000 7fe942c07640 1 --2- 172.21.15.73:0/3564173509 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe93c0d1a00 0x7fe93c0d1de0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fe92c0097a0 tx=0x7fe92c02a060 comp rx=0 tx=0).stop 2024-04-26T20:53:00.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.692+0000 7fe942c07640 1 -- 172.21.15.73:0/3564173509 shutdown_connections 2024-04-26T20:53:00.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.692+0000 7fe942c07640 1 --2- 172.21.15.73:0/3564173509 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe93c0b40d0 0x7fe93c0b4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:00.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.692+0000 7fe942c07640 1 --2- 172.21.15.73:0/3564173509 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe93c0d1a00 0x7fe93c0d1de0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:00.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.692+0000 7fe942c07640 1 -- 172.21.15.73:0/3564173509 >> 172.21.15.73:0/3564173509 conn(0x7fe93c0d4fd0 msgr2=0x7fe93c0d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:00.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.692+0000 7fe942c07640 1 -- 172.21.15.73:0/3564173509 shutdown_connections 2024-04-26T20:53:00.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.692+0000 7fe942c07640 1 -- 172.21.15.73:0/3564173509 wait complete. 2024-04-26T20:53:00.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.693+0000 7fe942c07640 1 Processor -- start 2024-04-26T20:53:00.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.693+0000 7fe942c07640 1 -- start start 2024-04-26T20:53:00.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.694+0000 7fe942c07640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe93c0b40d0 0x7fe93c0cee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:00.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.694+0000 7fe942c07640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe93c0d1a00 0x7fe93c15a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:00.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.694+0000 7fe942c07640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe93c0d2320 0x7fe93c15be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:00.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.694+0000 7fe942c07640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe93c0b5040 con 0x7fe93c0d2320 2024-04-26T20:53:00.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.694+0000 7fe942c07640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe93c162f50 con 0x7fe93c0d1a00 2024-04-26T20:53:00.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.694+0000 7fe942c07640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fe93c0b51c0 con 0x7fe93c0b40d0 2024-04-26T20:53:00.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.694+0000 7fe9411a4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe93c0d2320 0x7fe93c15be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:00.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.694+0000 7fe9411a4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe93c0d2320 0x7fe93c15be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55216/0 (socket says 172.21.15.73:55216) 2024-04-26T20:53:00.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.694+0000 7fe9411a4640 1 -- 172.21.15.73:0/628927805 learned_addr learned my addr 172.21.15.73:0/628927805 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:00.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.694+0000 7fe9409a3640 1 --2- 172.21.15.73:0/628927805 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe93c0b40d0 0x7fe93c0cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:00.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.694+0000 7fe93bfff640 1 --2- 172.21.15.73:0/628927805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe93c0d1a00 0x7fe93c15a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:00.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.695+0000 7fe9411a4640 1 -- 172.21.15.73:0/628927805 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe93c0b40d0 msgr2=0x7fe93c0cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:00.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.695+0000 7fe9411a4640 1 --2- 172.21.15.73:0/628927805 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe93c0b40d0 0x7fe93c0cee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:00.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.695+0000 7fe9411a4640 1 -- 172.21.15.73:0/628927805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe93c0d1a00 msgr2=0x7fe93c15a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:00.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.695+0000 7fe9411a4640 1 --2- 172.21.15.73:0/628927805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe93c0d1a00 0x7fe93c15a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:00.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.695+0000 7fe9411a4640 1 -- 172.21.15.73:0/628927805 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe93c15aa90 con 0x7fe93c0d2320 2024-04-26T20:53:00.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.695+0000 7fe93bfff640 1 --2- 172.21.15.73:0/628927805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe93c0d1a00 0x7fe93c15a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:53:00.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.695+0000 7fe9411a4640 1 --2- 172.21.15.73:0/628927805 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe93c0d2320 0x7fe93c15be40 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fe93400cf20 tx=0x7fe93400a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:00.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.695+0000 7fe9397fa640 1 -- 172.21.15.73:0/628927805 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93401a070 con 0x7fe93c0d2320 2024-04-26T20:53:00.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.695+0000 7fe9397fa640 1 -- 172.21.15.73:0/628927805 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe93400d840 con 0x7fe93c0d2320 2024-04-26T20:53:00.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.695+0000 7fe9397fa640 1 -- 172.21.15.73:0/628927805 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93400dc30 con 0x7fe93c0d2320 2024-04-26T20:53:00.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.695+0000 7fe942c07640 1 -- 172.21.15.73:0/628927805 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe93c15c720 con 0x7fe93c0d2320 2024-04-26T20:53:00.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.695+0000 7fe942c07640 1 -- 172.21.15.73:0/628927805 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe93c15cce0 con 0x7fe93c0d2320 2024-04-26T20:53:00.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.696+0000 7fe942c07640 1 -- 172.21.15.73:0/628927805 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe93c0694f0 con 0x7fe93c0d2320 2024-04-26T20:53:00.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.696+0000 7fe9397fa640 1 -- 172.21.15.73:0/628927805 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe93401d070 con 0x7fe93c0d2320 2024-04-26T20:53:00.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.697+0000 7fe9397fa640 1 --2- 172.21.15.73:0/628927805 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe92806c8f0 0x7fe92806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:00.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.697+0000 7fe9397fa640 1 -- 172.21.15.73:0/628927805 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6187+0+0 (secure 0 0 0) 0x7fe93408c4b0 con 0x7fe93c0d2320 2024-04-26T20:53:00.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.697+0000 7fe9409a3640 1 --2- 172.21.15.73:0/628927805 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe92806c8f0 0x7fe92806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:00.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.697+0000 7fe9409a3640 1 --2- 172.21.15.73:0/628927805 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe92806c8f0 0x7fe92806edb0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe92c034000 tx=0x7fe92c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:00.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.701+0000 7fe9397fa640 1 -- 172.21.15.73:0/628927805 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe93405a170 con 0x7fe93c0d2320 2024-04-26T20:53:00.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.875+0000 7fe942c07640 1 -- 172.21.15.73:0/628927805 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe93c074590 con 0x7fe93c0d2320 2024-04-26T20:53:00.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.876+0000 7fe9397fa640 1 -- 172.21.15.73:0/628927805 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v56) ==== 93+0+10 (secure 0 0 0) 0x7fe93405e470 con 0x7fe93c0d2320 2024-04-26T20:53:00.877 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-26T20:53:00.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.877+0000 7fe942c07640 1 -- 172.21.15.73:0/628927805 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe92806c8f0 msgr2=0x7fe92806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:00.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.877+0000 7fe942c07640 1 --2- 172.21.15.73:0/628927805 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe92806c8f0 0x7fe92806edb0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe92c034000 tx=0x7fe92c02e000 comp rx=0 tx=0).stop 2024-04-26T20:53:00.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.877+0000 7fe942c07640 1 -- 172.21.15.73:0/628927805 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe93c0d2320 msgr2=0x7fe93c15be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:00.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.877+0000 7fe942c07640 1 --2- 172.21.15.73:0/628927805 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe93c0d2320 0x7fe93c15be40 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fe93400cf20 tx=0x7fe93400a7d0 comp rx=0 tx=0).stop 2024-04-26T20:53:00.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.878+0000 7fe942c07640 1 -- 172.21.15.73:0/628927805 shutdown_connections 2024-04-26T20:53:00.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.878+0000 7fe942c07640 1 --2- 172.21.15.73:0/628927805 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe93c0d2320 0x7fe93c15be40 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:00.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.878+0000 7fe942c07640 1 --2- 172.21.15.73:0/628927805 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe92806c8f0 0x7fe92806edb0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:00.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.878+0000 7fe942c07640 1 --2- 172.21.15.73:0/628927805 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe93c0d1a00 0x7fe93c15a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:00.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.878+0000 7fe942c07640 1 --2- 172.21.15.73:0/628927805 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe93c0b40d0 0x7fe93c0cee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:00.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.878+0000 7fe942c07640 1 -- 172.21.15.73:0/628927805 >> 172.21.15.73:0/628927805 conn(0x7fe93c0d4fd0 msgr2=0x7fe93c0b5db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:00.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.878+0000 7fe942c07640 1 -- 172.21.15.73:0/628927805 shutdown_connections 2024-04-26T20:53:00.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.878+0000 7fe942c07640 1 -- 172.21.15.73:0/628927805 wait complete. 2024-04-26T20:53:00.889 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:53:00.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.988+0000 7fa8f6946640 1 Processor -- start 2024-04-26T20:53:00.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.988+0000 7fa8f6946640 1 -- start start 2024-04-26T20:53:00.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.989+0000 7fa8f6946640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa8f0155b00 0x7fa8f006e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:00.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.989+0000 7fa8f6946640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fa8f015f100 0x7fa8f006c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:00.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.989+0000 7fa8f6946640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8f015fae0 con 0x7fa8f015f580 2024-04-26T20:53:00.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.989+0000 7fa8f6946640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fa8f01600a0 con 0x7fa8f015f100 2024-04-26T20:53:00.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.989+0000 7fa8f6946640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fa8f015fdc0 con 0x7fa8f0155b00 2024-04-26T20:53:00.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.989+0000 7fa8f4ee3640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fa8f015f580 0x7fa8f00b4db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:50724/0 (socket says 172.21.15.73:50724) 2024-04-26T20:53:00.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.989+0000 7fa8f4ee3640 1 -- 172.21.15.73:0/2255570514 learned_addr learned my addr 172.21.15.73:0/2255570514 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:53:00.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.989+0000 7fa8effff640 1 --2- 172.21.15.73:0/2255570514 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa8f0155b00 0x7fa8f006e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:00.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.989+0000 7fa8ef7fe640 1 --2- 172.21.15.73:0/2255570514 >> v2:172.21.15.120:3300/0 conn(0x7fa8f015f100 0x7fa8f006c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:00.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.989+0000 7fa8eeffd640 1 -- 172.21.15.73:0/2255570514 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3938862872 0 0) 0x7fa8f015fae0 con 0x7fa8f015f580 2024-04-26T20:53:00.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.989+0000 7fa8eeffd640 1 -- 172.21.15.73:0/2255570514 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa8d8003520 con 0x7fa8f015f580 2024-04-26T20:53:00.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.990+0000 7fa8eeffd640 1 -- 172.21.15.73:0/2255570514 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3405190305 0 0) 0x7fa8d8003520 con 0x7fa8f015f580 2024-04-26T20:53:00.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.990+0000 7fa8eeffd640 1 -- 172.21.15.73:0/2255570514 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa8f0155b00 msgr2=0x7fa8f006e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:00.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.990+0000 7fa8eeffd640 1 --2- 172.21.15.73:0/2255570514 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa8f0155b00 0x7fa8f006e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:00.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.990+0000 7fa8eeffd640 1 -- 172.21.15.73:0/2255570514 >> v2:172.21.15.120:3300/0 conn(0x7fa8f015f100 msgr2=0x7fa8f006c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:00.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.990+0000 7fa8eeffd640 1 --2- 172.21.15.73:0/2255570514 >> v2:172.21.15.120:3300/0 conn(0x7fa8f015f100 0x7fa8f006c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:00.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.990+0000 7fa8eeffd640 1 -- 172.21.15.73:0/2255570514 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8f00b45b0 con 0x7fa8f015f580 2024-04-26T20:53:00.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.990+0000 7fa8eeffd640 1 -- 172.21.15.73:0/2255570514 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7fa8e80019f0 con 0x7fa8f015f580 2024-04-26T20:53:00.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.990+0000 7fa8effff640 1 --2- 172.21.15.73:0/2255570514 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa8f0155b00 0x7fa8f006e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:53:00.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.990+0000 7fa8eeffd640 1 -- 172.21.15.73:0/2255570514 >> v1:172.21.15.73:6789/0 conn(0x7fa8f015f580 legacy=0x7fa8f00b4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:00.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.990+0000 7fa8eeffd640 1 --2- 172.21.15.73:0/2255570514 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8d80039b0 0x7fa8d8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:00.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.990+0000 7fa8eeffd640 1 --2- 172.21.15.73:0/2255570514 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8d8004800 0x7fa8d8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:00.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.990+0000 7fa8eeffd640 1 --2- 172.21.15.73:0/2255570514 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa8f0155b00 0x7fa8d80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:00.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.990+0000 7fa8eeffd640 1 -- 172.21.15.73:0/2255570514 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f01600a0 con 0x7fa8d80039b0 2024-04-26T20:53:00.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.990+0000 7fa8eeffd640 1 -- 172.21.15.73:0/2255570514 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f015fdc0 con 0x7fa8d8004800 2024-04-26T20:53:00.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.990+0000 7fa8eeffd640 1 -- 172.21.15.73:0/2255570514 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fa8d800b9c0 con 0x7fa8f0155b00 2024-04-26T20:53:00.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.990+0000 7fa8effff640 1 --2- 172.21.15.73:0/2255570514 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8d80039b0 0x7fa8d8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:00.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.990+0000 7fa8ef7fe640 1 --2- 172.21.15.73:0/2255570514 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8d8004800 0x7fa8d8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:00.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.990+0000 7fa8f4ee3640 1 --2- 172.21.15.73:0/2255570514 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa8f0155b00 0x7fa8d80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:00.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.990+0000 7fa8effff640 1 -- 172.21.15.73:0/2255570514 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa8f0155b00 msgr2=0x7fa8d80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:00.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.990+0000 7fa8effff640 1 --2- 172.21.15.73:0/2255570514 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa8f0155b00 0x7fa8d80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:00.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.990+0000 7fa8effff640 1 -- 172.21.15.73:0/2255570514 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8d8004800 msgr2=0x7fa8d8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:00.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.990+0000 7fa8effff640 1 --2- 172.21.15.73:0/2255570514 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8d8004800 0x7fa8d8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:00.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.990+0000 7fa8effff640 1 -- 172.21.15.73:0/2255570514 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8d8003520 con 0x7fa8d80039b0 2024-04-26T20:53:00.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.990+0000 7fa8ef7fe640 1 --2- 172.21.15.73:0/2255570514 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8d8004800 0x7fa8d8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:53:00.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.991+0000 7fa8effff640 1 --2- 172.21.15.73:0/2255570514 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8d80039b0 0x7fa8d8003dd0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fa8e000ac90 tx=0x7fa8d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e6b5618583b4e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:00.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.991+0000 7fa8eeffd640 1 -- 172.21.15.73:0/2255570514 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e0039070 con 0x7fa8d80039b0 2024-04-26T20:53:00.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.991+0000 7fa8eeffd640 1 -- 172.21.15.73:0/2255570514 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8e002c950 con 0x7fa8d80039b0 2024-04-26T20:53:00.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.991+0000 7fa8eeffd640 1 -- 172.21.15.73:0/2255570514 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e000b040 con 0x7fa8d80039b0 2024-04-26T20:53:00.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.991+0000 7fa8f6946640 1 -- 172.21.15.73:0/2255570514 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8d80039b0 msgr2=0x7fa8d8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:00.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.991+0000 7fa8f6946640 1 --2- 172.21.15.73:0/2255570514 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8d80039b0 0x7fa8d8003dd0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fa8e000ac90 tx=0x7fa8d8005830 comp rx=0 tx=0).stop 2024-04-26T20:53:00.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.991+0000 7fa8f4ee3640 1 -- 172.21.15.73:0/2255570514 reap_dead start 2024-04-26T20:53:00.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.991+0000 7fa8f6946640 1 -- 172.21.15.73:0/2255570514 shutdown_connections 2024-04-26T20:53:00.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.991+0000 7fa8f6946640 1 -- 172.21.15.73:0/2255570514 >> 172.21.15.73:0/2255570514 conn(0x7fa8f00b3180 msgr2=0x7fa8f00b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:00.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.991+0000 7fa8f6946640 1 -- 172.21.15.73:0/2255570514 shutdown_connections 2024-04-26T20:53:00.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.991+0000 7fa8f6946640 1 -- 172.21.15.73:0/2255570514 wait complete. 2024-04-26T20:53:00.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.992+0000 7fa8f6946640 1 Processor -- start 2024-04-26T20:53:00.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.993+0000 7fa8f6946640 1 -- start start 2024-04-26T20:53:00.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.993+0000 7fa8f6946640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa8d80039b0 0x7fa8f01476c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:00.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.993+0000 7fa8f6946640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8f00d0e00 0x7fa8f01f06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:00.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.993+0000 7fa8f6946640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8f0149140 0x7fa8f01495c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:00.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.993+0000 7fa8f6946640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f00b0d70 con 0x7fa8f00d0e00 2024-04-26T20:53:00.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.993+0000 7fa8f6946640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f0155ee0 con 0x7fa8f0149140 2024-04-26T20:53:00.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.993+0000 7fa8f6946640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fa8f00b0ef0 con 0x7fa8d80039b0 2024-04-26T20:53:00.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.993+0000 7fa8ef7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8f00d0e00 0x7fa8f01f06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:00.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.994+0000 7fa8ef7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8f00d0e00 0x7fa8f01f06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55236/0 (socket says 172.21.15.73:55236) 2024-04-26T20:53:01.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.994+0000 7fa8ef7fe640 1 -- 172.21.15.73:0/973617677 learned_addr learned my addr 172.21.15.73:0/973617677 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:01.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.994+0000 7fa8effff640 1 --2- 172.21.15.73:0/973617677 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa8d80039b0 0x7fa8f01476c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:01.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.994+0000 7fa8f4ee3640 1 --2- 172.21.15.73:0/973617677 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8f0149140 0x7fa8f01495c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:01.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.994+0000 7fa8ef7fe640 1 -- 172.21.15.73:0/973617677 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa8d80039b0 msgr2=0x7fa8f01476c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:01.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.994+0000 7fa8ef7fe640 1 --2- 172.21.15.73:0/973617677 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa8d80039b0 0x7fa8f01476c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:01.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.994+0000 7fa8ef7fe640 1 -- 172.21.15.73:0/973617677 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8f0149140 msgr2=0x7fa8f01495c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:01.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.994+0000 7fa8ef7fe640 1 --2- 172.21.15.73:0/973617677 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8f0149140 0x7fa8f01495c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:01.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.994+0000 7fa8ef7fe640 1 -- 172.21.15.73:0/973617677 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8f00ce550 con 0x7fa8f00d0e00 2024-04-26T20:53:01.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.994+0000 7fa8effff640 1 --2- 172.21.15.73:0/973617677 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa8d80039b0 0x7fa8f01476c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:53:01.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.994+0000 7fa8ef7fe640 1 --2- 172.21.15.73:0/973617677 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8f00d0e00 0x7fa8f01f06d0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fa8e40116d0 tx=0x7fa8e4011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:01.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.994+0000 7fa8ecff9640 1 -- 172.21.15.73:0/973617677 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e400f9c0 con 0x7fa8f00d0e00 2024-04-26T20:53:01.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.994+0000 7fa8ecff9640 1 -- 172.21.15.73:0/973617677 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8e400fb60 con 0x7fa8f00d0e00 2024-04-26T20:53:01.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.994+0000 7fa8f6946640 1 -- 172.21.15.73:0/973617677 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8f014d170 con 0x7fa8f00d0e00 2024-04-26T20:53:01.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.994+0000 7fa8ecff9640 1 -- 172.21.15.73:0/973617677 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e4015460 con 0x7fa8f00d0e00 2024-04-26T20:53:01.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.994+0000 7fa8f6946640 1 -- 172.21.15.73:0/973617677 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8f00ceec0 con 0x7fa8f00d0e00 2024-04-26T20:53:01.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.995+0000 7fa8f6946640 1 -- 172.21.15.73:0/973617677 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8f00694f0 con 0x7fa8f00d0e00 2024-04-26T20:53:01.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.996+0000 7fa8ecff9640 1 -- 172.21.15.73:0/973617677 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa8e4015600 con 0x7fa8f00d0e00 2024-04-26T20:53:01.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.996+0000 7fa8ecff9640 1 --2- 172.21.15.73:0/973617677 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa8dc06c8f0 0x7fa8dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:01.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.996+0000 7fa8ecff9640 1 -- 172.21.15.73:0/973617677 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6187+0+0 (secure 0 0 0) 0x7fa8e408c610 con 0x7fa8f00d0e00 2024-04-26T20:53:01.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.996+0000 7fa8effff640 1 --2- 172.21.15.73:0/973617677 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa8dc06c8f0 0x7fa8dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:01.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:00.997+0000 7fa8effff640 1 --2- 172.21.15.73:0/973617677 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa8dc06c8f0 0x7fa8dc06edb0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0036000 tx=0x7fa8e0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:01.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.001+0000 7fa8ecff9640 1 -- 172.21.15.73:0/973617677 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa8e4019030 con 0x7fa8f00d0e00 2024-04-26T20:53:01.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.173+0000 7fa8f6946640 1 -- 172.21.15.73:0/973617677 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa8f006e700 con 0x7fa8f00d0e00 2024-04-26T20:53:01.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.173+0000 7fa8ecff9640 1 -- 172.21.15.73:0/973617677 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v56) ==== 104+0+11 (secure 0 0 0) 0x7fa8e405a8f0 con 0x7fa8f00d0e00 2024-04-26T20:53:01.174 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-26T20:53:01.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.175+0000 7fa8f6946640 1 -- 172.21.15.73:0/973617677 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa8dc06c8f0 msgr2=0x7fa8dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:01.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.175+0000 7fa8f6946640 1 --2- 172.21.15.73:0/973617677 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa8dc06c8f0 0x7fa8dc06edb0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0036000 tx=0x7fa8e0008040 comp rx=0 tx=0).stop 2024-04-26T20:53:01.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.175+0000 7fa8f6946640 1 -- 172.21.15.73:0/973617677 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8f00d0e00 msgr2=0x7fa8f01f06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:01.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.175+0000 7fa8f6946640 1 --2- 172.21.15.73:0/973617677 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8f00d0e00 0x7fa8f01f06d0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fa8e40116d0 tx=0x7fa8e4011b90 comp rx=0 tx=0).stop 2024-04-26T20:53:01.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.175+0000 7fa8f6946640 1 -- 172.21.15.73:0/973617677 shutdown_connections 2024-04-26T20:53:01.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.175+0000 7fa8f6946640 1 --2- 172.21.15.73:0/973617677 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8f00d0e00 0x7fa8f01f06d0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:01.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.175+0000 7fa8f6946640 1 --2- 172.21.15.73:0/973617677 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fa8dc06c8f0 0x7fa8dc06edb0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:01.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.175+0000 7fa8f6946640 1 --2- 172.21.15.73:0/973617677 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fa8f0149140 0x7fa8f01495c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:01.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.175+0000 7fa8f6946640 1 --2- 172.21.15.73:0/973617677 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fa8d80039b0 0x7fa8f01476c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:01.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.175+0000 7fa8f6946640 1 -- 172.21.15.73:0/973617677 >> 172.21.15.73:0/973617677 conn(0x7fa8f00b3180 msgr2=0x7fa8f00d1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:01.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.175+0000 7fa8f6946640 1 -- 172.21.15.73:0/973617677 shutdown_connections 2024-04-26T20:53:01.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.175+0000 7fa8f6946640 1 -- 172.21.15.73:0/973617677 wait complete. 2024-04-26T20:53:01.187 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:53:01.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.285+0000 7fb5b5da4640 1 Processor -- start 2024-04-26T20:53:01.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.285+0000 7fb5b5da4640 1 -- start start 2024-04-26T20:53:01.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.286+0000 7fb5b5da4640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb5b00d1240 0x7fb5b00d1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:01.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.286+0000 7fb5b5da4640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5b00d0690 con 0x7fb5b00b4360 2024-04-26T20:53:01.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.286+0000 7fb5b5da4640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5b00d1b60 con 0x7fb5b00b3610 2024-04-26T20:53:01.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.286+0000 7fb5b5da4640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7fb5b00cf210 con 0x7fb5b00d1240 2024-04-26T20:53:01.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.286+0000 7fb5af7fe640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb5b00d1240 0x7fb5b00d1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:01.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.286+0000 7fb5a6ffd640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fb5b00b4360 0x7fb5b00cf870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:50730/0 (socket says 172.21.15.73:50730) 2024-04-26T20:53:01.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.286+0000 7fb5a6ffd640 1 -- 172.21.15.73:0/945517155 learned_addr learned my addr 172.21.15.73:0/945517155 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:53:01.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.287+0000 7fb5af7fe640 1 -- 172.21.15.73:0/945517155 >> v1:172.21.15.120:6789/0 conn(0x7fb5b00b3610 legacy=0x7fb5b00cff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:01.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.287+0000 7fb5af7fe640 1 -- 172.21.15.73:0/945517155 >> v1:172.21.15.73:6789/0 conn(0x7fb5b00b4360 legacy=0x7fb5b00cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:01.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.287+0000 7fb5af7fe640 1 -- 172.21.15.73:0/945517155 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5b00ce5e0 con 0x7fb5b00d1240 2024-04-26T20:53:01.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.287+0000 7fb5aeffd640 1 -- 172.21.15.73:0/945517155 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 724617250 0 0) 0x7fb5b00d0690 con 0x7fb5b00b4360 2024-04-26T20:53:01.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.287+0000 7fb5af7fe640 1 --2- 172.21.15.73:0/945517155 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5b00d1240 0x7fb5b00d1620 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fb5a00097a0 tx=0x7fb5a002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8dba590d53ae051 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:01.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.287+0000 7fb5aeffd640 1 -- 172.21.15.73:0/945517155 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a0037070 con 0x7fb5b00d1240 2024-04-26T20:53:01.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.288+0000 7fb5aeffd640 1 -- 172.21.15.73:0/945517155 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5a0032440 con 0x7fb5b00d1240 2024-04-26T20:53:01.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.288+0000 7fb5aeffd640 1 -- 172.21.15.73:0/945517155 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a0032810 con 0x7fb5b00d1240 2024-04-26T20:53:01.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.288+0000 7fb5b5da4640 1 -- 172.21.15.73:0/945517155 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5b00d1240 msgr2=0x7fb5b00d1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:01.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.288+0000 7fb5b5da4640 1 --2- 172.21.15.73:0/945517155 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5b00d1240 0x7fb5b00d1620 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fb5a00097a0 tx=0x7fb5a002a1e0 comp rx=0 tx=0).stop 2024-04-26T20:53:01.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.288+0000 7fb5b5da4640 1 -- 172.21.15.73:0/945517155 shutdown_connections 2024-04-26T20:53:01.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.288+0000 7fb5b5da4640 1 --2- 172.21.15.73:0/945517155 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5b00d1240 0x7fb5b00d1620 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:01.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.288+0000 7fb5b5da4640 1 -- 172.21.15.73:0/945517155 >> 172.21.15.73:0/945517155 conn(0x7fb5b00d2e60 msgr2=0x7fb5b00d3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:01.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.288+0000 7fb5b5da4640 1 -- 172.21.15.73:0/945517155 shutdown_connections 2024-04-26T20:53:01.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.288+0000 7fb5b5da4640 1 -- 172.21.15.73:0/945517155 wait complete. 2024-04-26T20:53:01.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.289+0000 7fb5b5da4640 1 Processor -- start 2024-04-26T20:53:01.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.289+0000 7fb5b5da4640 1 -- start start 2024-04-26T20:53:01.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.290+0000 7fb5b5da4640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb5b00b3610 0x7fb5b00b8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:01.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.290+0000 7fb5b5da4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5b00d1240 0x7fb5b0151120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:01.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.290+0000 7fb5b5da4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb5b015a250 0x7fb5b01f2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:01.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.290+0000 7fb5b5da4640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b00cf460 con 0x7fb5b00d1240 2024-04-26T20:53:01.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.290+0000 7fb5b5da4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b0160ab0 con 0x7fb5b015a250 2024-04-26T20:53:01.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.290+0000 7fb5b5da4640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fb5b00c6610 con 0x7fb5b00b3610 2024-04-26T20:53:01.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.290+0000 7fb5a6ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5b00d1240 0x7fb5b0151120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:01.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.290+0000 7fb5a6ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5b00d1240 0x7fb5b0151120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55258/0 (socket says 172.21.15.73:55258) 2024-04-26T20:53:01.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.290+0000 7fb5a6ffd640 1 -- 172.21.15.73:0/1598282478 learned_addr learned my addr 172.21.15.73:0/1598282478 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:01.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.290+0000 7fb5af7fe640 1 --2- 172.21.15.73:0/1598282478 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb5b00b3610 0x7fb5b00b8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:01.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.290+0000 7fb5affff640 1 --2- 172.21.15.73:0/1598282478 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb5b015a250 0x7fb5b01f2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:01.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.291+0000 7fb5a6ffd640 1 -- 172.21.15.73:0/1598282478 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb5b00b3610 msgr2=0x7fb5b00b8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:01.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.291+0000 7fb5a6ffd640 1 --2- 172.21.15.73:0/1598282478 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb5b00b3610 0x7fb5b00b8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:01.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.291+0000 7fb5a6ffd640 1 -- 172.21.15.73:0/1598282478 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb5b015a250 msgr2=0x7fb5b01f2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:01.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.291+0000 7fb5a6ffd640 1 --2- 172.21.15.73:0/1598282478 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb5b015a250 0x7fb5b01f2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:01.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.291+0000 7fb5a6ffd640 1 -- 172.21.15.73:0/1598282478 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5b015cdb0 con 0x7fb5b00d1240 2024-04-26T20:53:01.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.291+0000 7fb5af7fe640 1 --2- 172.21.15.73:0/1598282478 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb5b00b3610 0x7fb5b00b8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:53:01.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.291+0000 7fb5affff640 1 --2- 172.21.15.73:0/1598282478 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb5b015a250 0x7fb5b01f2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:53:01.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.291+0000 7fb5a6ffd640 1 --2- 172.21.15.73:0/1598282478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5b00d1240 0x7fb5b0151120 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fb59800c4a0 tx=0x7fb59800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:01.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.291+0000 7fb5acff9640 1 -- 172.21.15.73:0/1598282478 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb598018070 con 0x7fb5b00d1240 2024-04-26T20:53:01.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.291+0000 7fb5acff9640 1 -- 172.21.15.73:0/1598282478 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5980074e0 con 0x7fb5b00d1240 2024-04-26T20:53:01.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.291+0000 7fb5acff9640 1 -- 172.21.15.73:0/1598282478 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb598004a20 con 0x7fb5b00d1240 2024-04-26T20:53:01.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.291+0000 7fb5b5da4640 1 -- 172.21.15.73:0/1598282478 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5b015c830 con 0x7fb5b00d1240 2024-04-26T20:53:01.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.291+0000 7fb5b5da4640 1 -- 172.21.15.73:0/1598282478 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5b015cad0 con 0x7fb5b00d1240 2024-04-26T20:53:01.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.292+0000 7fb5b5da4640 1 -- 172.21.15.73:0/1598282478 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5b00694f0 con 0x7fb5b00d1240 2024-04-26T20:53:01.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.292+0000 7fb5acff9640 1 -- 172.21.15.73:0/1598282478 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb598004bc0 con 0x7fb5b00d1240 2024-04-26T20:53:01.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.293+0000 7fb5acff9640 1 --2- 172.21.15.73:0/1598282478 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb59006c8f0 0x7fb59006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:01.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.293+0000 7fb5acff9640 1 -- 172.21.15.73:0/1598282478 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6187+0+0 (secure 0 0 0) 0x7fb59808afe0 con 0x7fb5b00d1240 2024-04-26T20:53:01.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.293+0000 7fb5af7fe640 1 --2- 172.21.15.73:0/1598282478 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb59006c8f0 0x7fb59006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:01.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.293+0000 7fb5af7fe640 1 --2- 172.21.15.73:0/1598282478 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb59006c8f0 0x7fb59006edb0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb5a0034000 tx=0x7fb5a002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:01.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.297+0000 7fb5acff9640 1 -- 172.21.15.73:0/1598282478 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb5980592c0 con 0x7fb5b00d1240 2024-04-26T20:53:01.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.477+0000 7fb5b5da4640 1 -- 172.21.15.73:0/1598282478 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb5b00c8b00 con 0x7fb5b00d1240 2024-04-26T20:53:01.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.478+0000 7fb5acff9640 1 -- 172.21.15.73:0/1598282478 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v56) ==== 100+0+11 (secure 0 0 0) 0x7fb598058bc0 con 0x7fb5b00d1240 2024-04-26T20:53:01.479 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-26T20:53:01.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.480+0000 7fb5b5da4640 1 -- 172.21.15.73:0/1598282478 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb59006c8f0 msgr2=0x7fb59006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:01.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.480+0000 7fb5b5da4640 1 --2- 172.21.15.73:0/1598282478 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb59006c8f0 0x7fb59006edb0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb5a0034000 tx=0x7fb5a002e000 comp rx=0 tx=0).stop 2024-04-26T20:53:01.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.480+0000 7fb5b5da4640 1 -- 172.21.15.73:0/1598282478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5b00d1240 msgr2=0x7fb5b0151120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:01.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.480+0000 7fb5b5da4640 1 --2- 172.21.15.73:0/1598282478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5b00d1240 0x7fb5b0151120 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fb59800c4a0 tx=0x7fb59800c960 comp rx=0 tx=0).stop 2024-04-26T20:53:01.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.480+0000 7fb5b5da4640 1 -- 172.21.15.73:0/1598282478 shutdown_connections 2024-04-26T20:53:01.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.480+0000 7fb5b5da4640 1 --2- 172.21.15.73:0/1598282478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5b00d1240 0x7fb5b0151120 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:01.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.480+0000 7fb5b5da4640 1 --2- 172.21.15.73:0/1598282478 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb59006c8f0 0x7fb59006edb0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:01.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.480+0000 7fb5b5da4640 1 --2- 172.21.15.73:0/1598282478 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb5b015a250 0x7fb5b01f2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:01.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.480+0000 7fb5b5da4640 1 --2- 172.21.15.73:0/1598282478 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb5b00b3610 0x7fb5b00b8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:01.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.480+0000 7fb5b5da4640 1 -- 172.21.15.73:0/1598282478 >> 172.21.15.73:0/1598282478 conn(0x7fb5b00d2e60 msgr2=0x7fb5b0070af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:01.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.480+0000 7fb5b5da4640 1 -- 172.21.15.73:0/1598282478 shutdown_connections 2024-04-26T20:53:01.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.480+0000 7fb5b5da4640 1 -- 172.21.15.73:0/1598282478 wait complete. 2024-04-26T20:53:01.492 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T20:53:01.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.589+0000 7fbbd1c2a640 1 Processor -- start 2024-04-26T20:53:01.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.589+0000 7fbbd1c2a640 1 -- start start 2024-04-26T20:53:01.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.590+0000 7fbbd1c2a640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbbcc0d3bc0 0x7fbbcc0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:01.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.590+0000 7fbbd1c2a640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbbcc0b5f20 0x7fbbcc0b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:01.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.590+0000 7fbbd1c2a640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbcc0b7e50 con 0x7fbbcc162880 2024-04-26T20:53:01.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.590+0000 7fbbd1c2a640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7fbbcc0d0580 con 0x7fbbcc0d3bc0 2024-04-26T20:53:01.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.590+0000 7fbbd1c2a640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fbbcc0d1800 con 0x7fbbcc0b5f20 2024-04-26T20:53:01.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.590+0000 7fbbcaffd640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbbcc0d3bc0 0x7fbbcc0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:01.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.590+0000 7fbbcaffd640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbbcc0d3bc0 0x7fbbcc0d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55260/0 (socket says 172.21.15.73:55260) 2024-04-26T20:53:01.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.590+0000 7fbbcaffd640 1 -- 172.21.15.73:0/1011695020 learned_addr learned my addr 172.21.15.73:0/1011695020 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:01.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.590+0000 7fbbcbfff640 1 --2- 172.21.15.73:0/1011695020 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbbcc0b5f20 0x7fbbcc0b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:01.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.590+0000 7fbbcaffd640 1 -- 172.21.15.73:0/1011695020 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbbcc0b5f20 msgr2=0x7fbbcc0b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:01.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.590+0000 7fbbcaffd640 1 --2- 172.21.15.73:0/1011695020 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbbcc0b5f20 0x7fbbcc0b8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:01.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.590+0000 7fbbcaffd640 1 -- 172.21.15.73:0/1011695020 >> v1:172.21.15.120:6789/0 conn(0x7fbbcc162880 legacy=0x7fbbcc0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:01.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.590+0000 7fbbcaffd640 1 -- 172.21.15.73:0/1011695020 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbcc0d07d0 con 0x7fbbcc0d3bc0 2024-04-26T20:53:01.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.590+0000 7fbbcbfff640 1 --2- 172.21.15.73:0/1011695020 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbbcc0b5f20 0x7fbbcc0b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:53:01.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.591+0000 7fbbcaffd640 1 --2- 172.21.15.73:0/1011695020 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbbcc0d3bc0 0x7fbbcc0d2cb0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0009870 tx=0x7fbbc002a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f8ee27a3d34961f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:01.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.591+0000 7fbbca7fc640 1 -- 172.21.15.73:0/1011695020 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0037070 con 0x7fbbcc0d3bc0 2024-04-26T20:53:01.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.591+0000 7fbbca7fc640 1 -- 172.21.15.73:0/1011695020 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbc0032440 con 0x7fbbcc0d3bc0 2024-04-26T20:53:01.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.591+0000 7fbbca7fc640 1 -- 172.21.15.73:0/1011695020 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0032810 con 0x7fbbcc0d3bc0 2024-04-26T20:53:01.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.592+0000 7fbbd1c2a640 1 -- 172.21.15.73:0/1011695020 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbbcc0d3bc0 msgr2=0x7fbbcc0d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:01.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.592+0000 7fbbd1c2a640 1 --2- 172.21.15.73:0/1011695020 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbbcc0d3bc0 0x7fbbcc0d2cb0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0009870 tx=0x7fbbc002a200 comp rx=0 tx=0).stop 2024-04-26T20:53:01.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.592+0000 7fbbd1c2a640 1 -- 172.21.15.73:0/1011695020 shutdown_connections 2024-04-26T20:53:01.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.592+0000 7fbbd1c2a640 1 --2- 172.21.15.73:0/1011695020 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbbcc0b5f20 0x7fbbcc0b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:01.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.592+0000 7fbbd1c2a640 1 --2- 172.21.15.73:0/1011695020 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbbcc0d3bc0 0x7fbbcc0d2cb0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:01.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.592+0000 7fbbd1c2a640 1 -- 172.21.15.73:0/1011695020 >> 172.21.15.73:0/1011695020 conn(0x7fbbcc0d4f60 msgr2=0x7fbbcc0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:01.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.592+0000 7fbbd1c2a640 1 -- 172.21.15.73:0/1011695020 shutdown_connections 2024-04-26T20:53:01.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.592+0000 7fbbd1c2a640 1 -- 172.21.15.73:0/1011695020 wait complete. 2024-04-26T20:53:01.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.593+0000 7fbbd1c2a640 1 Processor -- start 2024-04-26T20:53:01.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.593+0000 7fbbd1c2a640 1 -- start start 2024-04-26T20:53:01.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.594+0000 7fbbd1c2a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbcc0b5f20 0x7fbbcc15e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:01.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.594+0000 7fbbd1c2a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbbcc0d3bc0 0x7fbbcc0b4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:01.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.594+0000 7fbbd1c2a640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbbcc162880 0x7fbbcc06df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:01.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.594+0000 7fbbd1c2a640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbbcc162c60 con 0x7fbbcc0d3bc0 2024-04-26T20:53:01.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.594+0000 7fbbd1c2a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbbcc0d0d90 con 0x7fbbcc0b5f20 2024-04-26T20:53:01.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.594+0000 7fbbd1c2a640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fbbcc162de0 con 0x7fbbcc162880 2024-04-26T20:53:01.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.594+0000 7fbbcaffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbbcc0d3bc0 0x7fbbcc0b4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:01.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.594+0000 7fbbcaffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbbcc0d3bc0 0x7fbbcc0b4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55270/0 (socket says 172.21.15.73:55270) 2024-04-26T20:53:01.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.594+0000 7fbbcaffd640 1 -- 172.21.15.73:0/35602388 learned_addr learned my addr 172.21.15.73:0/35602388 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:01.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.594+0000 7fbbcaffd640 1 -- 172.21.15.73:0/35602388 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbbcc162880 msgr2=0x7fbbcc06df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:01.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.594+0000 7fbbcaffd640 1 --2- 172.21.15.73:0/35602388 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbbcc162880 0x7fbbcc06df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:01.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.594+0000 7fbbcaffd640 1 -- 172.21.15.73:0/35602388 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbcc0b5f20 msgr2=0x7fbbcc15e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:01.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.594+0000 7fbbcb7fe640 1 --2- 172.21.15.73:0/35602388 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbcc0b5f20 0x7fbbcc15e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:01.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.594+0000 7fbbcaffd640 1 --2- 172.21.15.73:0/35602388 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbcc0b5f20 0x7fbbcc15e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:01.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.594+0000 7fbbcaffd640 1 -- 172.21.15.73:0/35602388 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbcc09fe30 con 0x7fbbcc0d3bc0 2024-04-26T20:53:01.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.594+0000 7fbbcb7fe640 1 --2- 172.21.15.73:0/35602388 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbcc0b5f20 0x7fbbcc15e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:53:01.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.594+0000 7fbbcaffd640 1 --2- 172.21.15.73:0/35602388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbbcc0d3bc0 0x7fbbcc0b4c10 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0009870 tx=0x7fbbc00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:01.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.594+0000 7fbbb3fff640 1 -- 172.21.15.73:0/35602388 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0041020 con 0x7fbbcc0d3bc0 2024-04-26T20:53:01.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.594+0000 7fbbb3fff640 1 -- 172.21.15.73:0/35602388 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbc0030970 con 0x7fbbcc0d3bc0 2024-04-26T20:53:01.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.594+0000 7fbbd1c2a640 1 -- 172.21.15.73:0/35602388 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbcc0a0110 con 0x7fbbcc0d3bc0 2024-04-26T20:53:01.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.594+0000 7fbbb3fff640 1 -- 172.21.15.73:0/35602388 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0037040 con 0x7fbbcc0d3bc0 2024-04-26T20:53:01.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.594+0000 7fbbd1c2a640 1 -- 172.21.15.73:0/35602388 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbcc0a09b0 con 0x7fbbcc0d3bc0 2024-04-26T20:53:01.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.595+0000 7fbbd1c2a640 1 -- 172.21.15.73:0/35602388 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbcc0a0500 con 0x7fbbcc0d3bc0 2024-04-26T20:53:01.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.596+0000 7fbbb3fff640 1 -- 172.21.15.73:0/35602388 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbbc0044020 con 0x7fbbcc0d3bc0 2024-04-26T20:53:01.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.596+0000 7fbbb3fff640 1 --2- 172.21.15.73:0/35602388 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fbbb806c8f0 0x7fbbb806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:01.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.596+0000 7fbbb3fff640 1 -- 172.21.15.73:0/35602388 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6187+0+0 (secure 0 0 0) 0x7fbbc00ac700 con 0x7fbbcc0d3bc0 2024-04-26T20:53:01.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.596+0000 7fbbcb7fe640 1 --2- 172.21.15.73:0/35602388 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fbbb806c8f0 0x7fbbb806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:01.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.597+0000 7fbbcb7fe640 1 --2- 172.21.15.73:0/35602388 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fbbb806c8f0 0x7fbbb806edb0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc1f83e0 tx=0x7fbbbc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:01.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.600+0000 7fbbb3fff640 1 -- 172.21.15.73:0/35602388 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbbc004b030 con 0x7fbbcc0d3bc0 2024-04-26T20:53:01.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.786+0000 7fbbd1c2a640 1 -- 172.21.15.73:0/35602388 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbbcc0694f0 con 0x7fbbcc0d3bc0 2024-04-26T20:53:01.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.787+0000 7fbbb3fff640 1 -- 172.21.15.73:0/35602388 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v56) ==== 80+0+122 (secure 0 0 0) 0x7fbbc007a940 con 0x7fbbcc0d3bc0 2024-04-26T20:53:01.788 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73:6837/2770303348 2024-04-27T20:52:53.540263+0000 2024-04-26T20:53:01.788 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73:6836/2770303348 2024-04-27T20:52:53.540263+0000 2024-04-26T20:53:01.788 INFO:teuthology.orchestra.run.smithi073.stderr:listed 2 entries 2024-04-26T20:53:01.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.789+0000 7fbbd1c2a640 1 -- 172.21.15.73:0/35602388 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fbbb806c8f0 msgr2=0x7fbbb806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:01.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.789+0000 7fbbd1c2a640 1 --2- 172.21.15.73:0/35602388 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fbbb806c8f0 0x7fbbb806edb0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc1f83e0 tx=0x7fbbbc009040 comp rx=0 tx=0).stop 2024-04-26T20:53:01.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.789+0000 7fbbd1c2a640 1 -- 172.21.15.73:0/35602388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbbcc0d3bc0 msgr2=0x7fbbcc0b4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:01.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.789+0000 7fbbd1c2a640 1 --2- 172.21.15.73:0/35602388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbbcc0d3bc0 0x7fbbcc0b4c10 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0009870 tx=0x7fbbc00046c0 comp rx=0 tx=0).stop 2024-04-26T20:53:01.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.789+0000 7fbbd1c2a640 1 -- 172.21.15.73:0/35602388 shutdown_connections 2024-04-26T20:53:01.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.789+0000 7fbbd1c2a640 1 --2- 172.21.15.73:0/35602388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbbcc0d3bc0 0x7fbbcc0b4c10 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:01.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.789+0000 7fbbd1c2a640 1 --2- 172.21.15.73:0/35602388 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fbbb806c8f0 0x7fbbb806edb0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:01.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.789+0000 7fbbd1c2a640 1 --2- 172.21.15.73:0/35602388 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbbcc162880 0x7fbbcc06df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:01.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.789+0000 7fbbd1c2a640 1 --2- 172.21.15.73:0/35602388 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbbcc0b5f20 0x7fbbcc15e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:01.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.789+0000 7fbbd1c2a640 1 -- 172.21.15.73:0/35602388 >> 172.21.15.73:0/35602388 conn(0x7fbbcc0d4f60 msgr2=0x7fbbcc0d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:01.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.789+0000 7fbbd1c2a640 1 -- 172.21.15.73:0/35602388 shutdown_connections 2024-04-26T20:53:01.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:01.789+0000 7fbbd1c2a640 1 -- 172.21.15.73:0/35602388 wait complete. 2024-04-26T20:53:01.800 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T20:53:01.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:53:01.801 DEBUG:teuthology.orchestra.run.smithi073:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T20:53:01.827 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi073.front.sepia.ceph.com 2024-04-26T20:53:01.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:53:01.827 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T20:53:01.881 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T20:53:01.882 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-26T20:53:01.882 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:53:01.901 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056097,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19070 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T20:53:01.901 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T20:53:01.901 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-26T20:53:01.901 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:53:01.937 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19637 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T20:53:01.937 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T20:53:01.937 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-26T20:53:01.937 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:53:01.959 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056097,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19070 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T20:53:01.959 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:53:02.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.073+0000 7f46f9e9c640 1 Processor -- start 2024-04-26T20:53:02.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.073+0000 7f46f9e9c640 1 -- start start 2024-04-26T20:53:02.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.074+0000 7f46f9e9c640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46f40d1380 0x7f46f40d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:02.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.074+0000 7f46f9e9c640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f46f4070ae0 0x7f46f4070f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:02.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.074+0000 7f46f9e9c640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46f40d1ce0 con 0x7f46f415d1e0 2024-04-26T20:53:02.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.074+0000 7f46f9e9c640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f46f40cede0 con 0x7f46f4070ae0 2024-04-26T20:53:02.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.074+0000 7f46f9e9c640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f46f4070650 con 0x7f46f40d1380 2024-04-26T20:53:02.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.074+0000 7f46f37fe640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f46f415d1e0 0x7f46f40d09e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:50732/0 (socket says 172.21.15.73:50732) 2024-04-26T20:53:02.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.074+0000 7f46f37fe640 1 -- 172.21.15.73:0/1137928829 learned_addr learned my addr 172.21.15.73:0/1137928829 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:53:02.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.074+0000 7f46f27fc640 1 -- 172.21.15.73:0/1137928829 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3709210039 0 0) 0x7f46f40d1ce0 con 0x7f46f415d1e0 2024-04-26T20:53:02.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.074+0000 7f46f27fc640 1 -- 172.21.15.73:0/1137928829 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f46dc003520 con 0x7f46f415d1e0 2024-04-26T20:53:02.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.074+0000 7f46f2ffd640 1 --2- 172.21.15.73:0/1137928829 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46f40d1380 0x7f46f40d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:02.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.074+0000 7f46f3fff640 1 --2- 172.21.15.73:0/1137928829 >> v2:172.21.15.120:3300/0 conn(0x7f46f4070ae0 0x7f46f4070f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:02.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.074+0000 7f46f27fc640 1 -- 172.21.15.73:0/1137928829 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3092639809 0 0) 0x7f46dc003520 con 0x7f46f415d1e0 2024-04-26T20:53:02.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.074+0000 7f46f27fc640 1 -- 172.21.15.73:0/1137928829 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46f40d1380 msgr2=0x7f46f40d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:02.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.074+0000 7f46f27fc640 1 --2- 172.21.15.73:0/1137928829 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46f40d1380 0x7f46f40d17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:02.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.074+0000 7f46f27fc640 1 -- 172.21.15.73:0/1137928829 >> v2:172.21.15.120:3300/0 conn(0x7f46f4070ae0 msgr2=0x7f46f4070f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:02.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.074+0000 7f46f27fc640 1 --2- 172.21.15.73:0/1137928829 >> v2:172.21.15.120:3300/0 conn(0x7f46f4070ae0 0x7f46f4070f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:02.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.074+0000 7f46f27fc640 1 -- 172.21.15.73:0/1137928829 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46f40ce840 con 0x7f46f415d1e0 2024-04-26T20:53:02.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.074+0000 7f46f27fc640 1 -- 172.21.15.73:0/1137928829 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7f46e4001670 con 0x7f46f415d1e0 2024-04-26T20:53:02.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.074+0000 7f46f27fc640 1 -- 172.21.15.73:0/1137928829 >> v1:172.21.15.73:6789/0 conn(0x7f46f415d1e0 legacy=0x7f46f40d09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:02.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.074+0000 7f46f27fc640 1 --2- 172.21.15.73:0/1137928829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f46dc0039b0 0x7f46dc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:02.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.074+0000 7f46f27fc640 1 --2- 172.21.15.73:0/1137928829 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46f40d1380 0x7f46dc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:02.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.074+0000 7f46f27fc640 1 --2- 172.21.15.73:0/1137928829 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46dc004d40 0x7f46dc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:02.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.074+0000 7f46f27fc640 1 -- 172.21.15.73:0/1137928829 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f46f40cede0 con 0x7f46dc004d40 2024-04-26T20:53:02.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.074+0000 7f46f27fc640 1 -- 172.21.15.73:0/1137928829 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f46f4070650 con 0x7f46dc0039b0 2024-04-26T20:53:02.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.074+0000 7f46f27fc640 1 -- 172.21.15.73:0/1137928829 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f46dc00ba70 con 0x7f46f40d1380 2024-04-26T20:53:02.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.075+0000 7f46f3fff640 1 --2- 172.21.15.73:0/1137928829 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46dc004d40 0x7f46dc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:02.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.075+0000 7f46f37fe640 1 --2- 172.21.15.73:0/1137928829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f46dc0039b0 0x7f46dc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:02.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.075+0000 7f46f2ffd640 1 --2- 172.21.15.73:0/1137928829 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46f40d1380 0x7f46dc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:02.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.075+0000 7f46f3fff640 1 -- 172.21.15.73:0/1137928829 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46f40d1380 msgr2=0x7f46dc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:02.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.075+0000 7f46f3fff640 1 --2- 172.21.15.73:0/1137928829 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46f40d1380 0x7f46dc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:02.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.075+0000 7f46f3fff640 1 -- 172.21.15.73:0/1137928829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f46dc0039b0 msgr2=0x7f46dc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:02.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.075+0000 7f46f3fff640 1 --2- 172.21.15.73:0/1137928829 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f46dc0039b0 0x7f46dc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:02.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.075+0000 7f46f3fff640 1 -- 172.21.15.73:0/1137928829 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46dc003520 con 0x7f46dc004d40 2024-04-26T20:53:02.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.075+0000 7f46f2ffd640 1 --2- 172.21.15.73:0/1137928829 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46f40d1380 0x7f46dc004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:53:02.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.075+0000 7f46f3fff640 1 --2- 172.21.15.73:0/1137928829 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46dc004d40 0x7f46dc0051c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f46ec007390 tx=0x7f46dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5d2fec80760c582 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:02.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.076+0000 7f46f27fc640 1 -- 172.21.15.73:0/1137928829 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec009be0 con 0x7f46dc004d40 2024-04-26T20:53:02.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.076+0000 7f46f27fc640 1 -- 172.21.15.73:0/1137928829 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46ec02e640 con 0x7f46dc004d40 2024-04-26T20:53:02.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.076+0000 7f46f27fc640 1 -- 172.21.15.73:0/1137928829 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec02ea10 con 0x7f46dc004d40 2024-04-26T20:53:02.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.076+0000 7f46f9e9c640 1 -- 172.21.15.73:0/1137928829 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46dc004d40 msgr2=0x7f46dc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:02.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.076+0000 7f46f9e9c640 1 --2- 172.21.15.73:0/1137928829 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46dc004d40 0x7f46dc0051c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f46ec007390 tx=0x7f46dc005830 comp rx=0 tx=0).stop 2024-04-26T20:53:02.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.076+0000 7f46f3fff640 1 -- 172.21.15.73:0/1137928829 reap_dead start 2024-04-26T20:53:02.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.076+0000 7f46f9e9c640 1 -- 172.21.15.73:0/1137928829 shutdown_connections 2024-04-26T20:53:02.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.076+0000 7f46f9e9c640 1 -- 172.21.15.73:0/1137928829 >> 172.21.15.73:0/1137928829 conn(0x7f46f40d2da0 msgr2=0x7f46f40d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:02.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.076+0000 7f46f9e9c640 1 -- 172.21.15.73:0/1137928829 shutdown_connections 2024-04-26T20:53:02.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.076+0000 7f46f9e9c640 1 -- 172.21.15.73:0/1137928829 wait complete. 2024-04-26T20:53:02.084 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.077+0000 7f46f9e9c640 1 Processor -- start 2024-04-26T20:53:02.084 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.078+0000 7f46f9e9c640 1 -- start start 2024-04-26T20:53:02.084 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.078+0000 7f46f9e9c640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46f40c8b90 0x7f46f40b5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:02.084 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.078+0000 7f46f9e9c640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f46f409f890 0x7f46f40b71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:02.084 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.078+0000 7f46f9e9c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46f40b7730 0x7f46f409e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:02.084 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.078+0000 7f46f9e9c640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f46f40d10f0 con 0x7f46f40b7730 2024-04-26T20:53:02.084 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.078+0000 7f46f9e9c640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f46f40702e0 con 0x7f46f409f890 2024-04-26T20:53:02.084 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.078+0000 7f46f9e9c640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f46f40ce4c0 con 0x7f46f40c8b90 2024-04-26T20:53:02.084 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.078+0000 7f46f3fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46f40b7730 0x7f46f409e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:02.084 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.078+0000 7f46f3fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46f40b7730 0x7f46f409e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55290/0 (socket says 172.21.15.73:55290) 2024-04-26T20:53:02.084 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.078+0000 7f46f3fff640 1 -- 172.21.15.73:0/3872105379 learned_addr learned my addr 172.21.15.73:0/3872105379 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:02.084 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.079+0000 7f46f37fe640 1 --2- 172.21.15.73:0/3872105379 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46f40c8b90 0x7f46f40b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:02.084 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.079+0000 7f46f2ffd640 1 --2- 172.21.15.73:0/3872105379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f46f409f890 0x7f46f40b71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:02.084 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.079+0000 7f46f3fff640 1 -- 172.21.15.73:0/3872105379 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46f40c8b90 msgr2=0x7f46f40b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:02.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.079+0000 7f46f3fff640 1 --2- 172.21.15.73:0/3872105379 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46f40c8b90 0x7f46f40b5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:02.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.079+0000 7f46f3fff640 1 -- 172.21.15.73:0/3872105379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f46f409f890 msgr2=0x7f46f40b71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:02.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.079+0000 7f46f3fff640 1 --2- 172.21.15.73:0/3872105379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f46f409f890 0x7f46f40b71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:02.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.079+0000 7f46f3fff640 1 -- 172.21.15.73:0/3872105379 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46f41f2e00 con 0x7f46f40b7730 2024-04-26T20:53:02.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.079+0000 7f46f3fff640 1 --2- 172.21.15.73:0/3872105379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46f40b7730 0x7f46f409e090 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f46ec0074b0 tx=0x7f46ec004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:02.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.079+0000 7f46dbfff640 1 -- 172.21.15.73:0/3872105379 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec009be0 con 0x7f46f40b7730 2024-04-26T20:53:02.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.079+0000 7f46dbfff640 1 -- 172.21.15.73:0/3872105379 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46ec02e440 con 0x7f46f40b7730 2024-04-26T20:53:02.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.079+0000 7f46f9e9c640 1 -- 172.21.15.73:0/3872105379 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46f40b5ca0 con 0x7f46f40b7730 2024-04-26T20:53:02.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.079+0000 7f46dbfff640 1 -- 172.21.15.73:0/3872105379 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec038270 con 0x7f46f40b7730 2024-04-26T20:53:02.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.079+0000 7f46f9e9c640 1 -- 172.21.15.73:0/3872105379 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46f40b6260 con 0x7f46f40b7730 2024-04-26T20:53:02.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.080+0000 7f46f9e9c640 1 -- 172.21.15.73:0/3872105379 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46f40694f0 con 0x7f46f40b7730 2024-04-26T20:53:02.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.080+0000 7f46dbfff640 1 -- 172.21.15.73:0/3872105379 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f46ec040030 con 0x7f46f40b7730 2024-04-26T20:53:02.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.081+0000 7f46dbfff640 1 --2- 172.21.15.73:0/3872105379 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f46e006cac0 0x7f46e006ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:02.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.081+0000 7f46dbfff640 1 -- 172.21.15.73:0/3872105379 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6187+0+0 (secure 0 0 0) 0x7f46ec0abab0 con 0x7f46f40b7730 2024-04-26T20:53:02.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.081+0000 7f46f37fe640 1 --2- 172.21.15.73:0/3872105379 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f46e006cac0 0x7f46e006ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:02.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.081+0000 7f46f37fe640 1 --2- 172.21.15.73:0/3872105379 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f46e006cac0 0x7f46e006ef80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f46e4000c00 tx=0x7f46e4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:02.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.085+0000 7f46dbfff640 1 -- 172.21.15.73:0/3872105379 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f46f41f2e00 con 0x7f46f40b7730 2024-04-26T20:53:02.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.296+0000 7f46f9e9c640 1 -- 172.21.15.73:0/3872105379 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46f40b1480 con 0x7f46f40b7730 2024-04-26T20:53:02.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.297+0000 7f46dbfff640 1 -- 172.21.15.73:0/3872105379 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5182 (secure 0 0 0) 0x7f46ec032030 con 0x7f46f40b7730 2024-04-26T20:53:02.298 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:53:02.298 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":34,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5207,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/256948100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":256948100},{"type":"v1","addr":"172.21.15.120:6833","nonce":256948100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5632,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/4104096099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":4104096099},{"type":"v1","addr":"172.21.15.120:6835","nonce":4104096099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5641,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/272782804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":272782804},{"type":"v1","addr":"172.21.15.73:6837","nonce":272782804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:52:20.754736+0000","modified":"2024-04-26T20:52:57.564091+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6099":{"gid":6099,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":4,"addr":"172.21.15.73:6836/1169682578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1169682578},{"type":"v1","addr":"172.21.15.73:6836","nonce":1169682578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6099,"qdb_cluster":[6099]},"id":3}]} 2024-04-26T20:53:02.298 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 34 2024-04-26T20:53:02.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.299+0000 7f46f9e9c640 1 -- 172.21.15.73:0/3872105379 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f46e006cac0 msgr2=0x7f46e006ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:02.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.299+0000 7f46f9e9c640 1 --2- 172.21.15.73:0/3872105379 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f46e006cac0 0x7f46e006ef80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f46e4000c00 tx=0x7f46e4009040 comp rx=0 tx=0).stop 2024-04-26T20:53:02.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.299+0000 7f46f9e9c640 1 -- 172.21.15.73:0/3872105379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46f40b7730 msgr2=0x7f46f409e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:02.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.299+0000 7f46f9e9c640 1 --2- 172.21.15.73:0/3872105379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46f40b7730 0x7f46f409e090 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f46ec0074b0 tx=0x7f46ec004ab0 comp rx=0 tx=0).stop 2024-04-26T20:53:02.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.299+0000 7f46f9e9c640 1 -- 172.21.15.73:0/3872105379 shutdown_connections 2024-04-26T20:53:02.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.299+0000 7f46f9e9c640 1 --2- 172.21.15.73:0/3872105379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46f40b7730 0x7f46f409e090 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:02.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.299+0000 7f46f9e9c640 1 --2- 172.21.15.73:0/3872105379 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f46e006cac0 0x7f46e006ef80 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:02.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.299+0000 7f46f9e9c640 1 --2- 172.21.15.73:0/3872105379 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f46f409f890 0x7f46f40b71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:02.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.299+0000 7f46f9e9c640 1 --2- 172.21.15.73:0/3872105379 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46f40c8b90 0x7f46f40b5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:02.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.299+0000 7f46f9e9c640 1 -- 172.21.15.73:0/3872105379 >> 172.21.15.73:0/3872105379 conn(0x7f46f40d2da0 msgr2=0x7f46f40711a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:02.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.299+0000 7f46f9e9c640 1 -- 172.21.15.73:0/3872105379 shutdown_connections 2024-04-26T20:53:02.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.299+0000 7f46f9e9c640 1 -- 172.21.15.73:0/3872105379 wait complete. 2024-04-26T20:53:02.311 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:53:02.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.407+0000 7f57fdae1640 1 Processor -- start 2024-04-26T20:53:02.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.407+0000 7f57fdae1640 1 -- start start 2024-04-26T20:53:02.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.407+0000 7f57fdae1640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f57f80cb7e0 0x7f57f8162e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:02.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.407+0000 7f57fdae1640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f57f81633c0 0x7f57f80726c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:02.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.407+0000 7f57fdae1640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57f806f5a0 con 0x7f57f815f290 2024-04-26T20:53:02.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.407+0000 7f57fdae1640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f57f806f350 con 0x7f57f81633c0 2024-04-26T20:53:02.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.407+0000 7f57fdae1640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f57f80cbc20 con 0x7f57f80cb7e0 2024-04-26T20:53:02.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.408+0000 7f57f7fff640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f57f81633c0 0x7f57f80726c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:02.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.408+0000 7f57f77fe640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f57f815f290 0x7f57f80cd060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:50734/0 (socket says 172.21.15.73:50734) 2024-04-26T20:53:02.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.408+0000 7f57f77fe640 1 -- 172.21.15.73:0/4048908700 learned_addr learned my addr 172.21.15.73:0/4048908700 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:53:02.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.408+0000 7f57f6ffd640 1 --2- 172.21.15.73:0/4048908700 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f57f80cb7e0 0x7f57f8162e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:02.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.408+0000 7f57f7fff640 1 -- 172.21.15.73:0/4048908700 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f57f80cb7e0 msgr2=0x7f57f8162e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:02.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.408+0000 7f57f7fff640 1 --2- 172.21.15.73:0/4048908700 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f57f80cb7e0 0x7f57f8162e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:02.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.408+0000 7f57f7fff640 1 -- 172.21.15.73:0/4048908700 >> v1:172.21.15.73:6789/0 conn(0x7f57f815f290 legacy=0x7f57f80cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:02.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.408+0000 7f57f7fff640 1 -- 172.21.15.73:0/4048908700 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57f806f880 con 0x7f57f81633c0 2024-04-26T20:53:02.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.408+0000 7f57f67fc640 1 -- 172.21.15.73:0/4048908700 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2474449115 0 0) 0x7f57f806f5a0 con 0x7f57f815f290 2024-04-26T20:53:02.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.408+0000 7f57f6ffd640 1 --2- 172.21.15.73:0/4048908700 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f57f80cb7e0 0x7f57f8162e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:53:02.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.409+0000 7f57f7fff640 1 --2- 172.21.15.73:0/4048908700 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57f81633c0 0x7f57f80726c0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f57f0009d90 tx=0x7f57f002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a27761fe0daa0143 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:02.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.409+0000 7f57f67fc640 1 -- 172.21.15.73:0/4048908700 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f0007740 con 0x7f57f81633c0 2024-04-26T20:53:02.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.409+0000 7f57f67fc640 1 -- 172.21.15.73:0/4048908700 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57f00078e0 con 0x7f57f81633c0 2024-04-26T20:53:02.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.409+0000 7f57f67fc640 1 -- 172.21.15.73:0/4048908700 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f0007cb0 con 0x7f57f81633c0 2024-04-26T20:53:02.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.409+0000 7f57fdae1640 1 -- 172.21.15.73:0/4048908700 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57f81633c0 msgr2=0x7f57f80726c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:02.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.409+0000 7f57fdae1640 1 --2- 172.21.15.73:0/4048908700 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57f81633c0 0x7f57f80726c0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f57f0009d90 tx=0x7f57f002a6f0 comp rx=0 tx=0).stop 2024-04-26T20:53:02.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.409+0000 7f57fdae1640 1 -- 172.21.15.73:0/4048908700 shutdown_connections 2024-04-26T20:53:02.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.409+0000 7f57fdae1640 1 --2- 172.21.15.73:0/4048908700 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f57f80cb7e0 0x7f57f8162e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:02.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.409+0000 7f57fdae1640 1 --2- 172.21.15.73:0/4048908700 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57f81633c0 0x7f57f80726c0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:02.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.409+0000 7f57fdae1640 1 -- 172.21.15.73:0/4048908700 >> 172.21.15.73:0/4048908700 conn(0x7f57f80d0fd0 msgr2=0x7f57f80d13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:02.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.409+0000 7f57fdae1640 1 -- 172.21.15.73:0/4048908700 shutdown_connections 2024-04-26T20:53:02.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.409+0000 7f57fdae1640 1 -- 172.21.15.73:0/4048908700 wait complete. 2024-04-26T20:53:02.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.410+0000 7f57fdae1640 1 Processor -- start 2024-04-26T20:53:02.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.411+0000 7f57fdae1640 1 -- start start 2024-04-26T20:53:02.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.411+0000 7f57fdae1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57f80cb7e0 0x7f57f809ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:02.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.411+0000 7f57fdae1640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f57f815f290 0x7f57f815b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:02.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.411+0000 7f57fdae1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57f81633c0 0x7f57f815f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:02.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.411+0000 7f57fdae1640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f57f815f670 con 0x7f57f81633c0 2024-04-26T20:53:02.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.411+0000 7f57fdae1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f57f8072c00 con 0x7f57f80cb7e0 2024-04-26T20:53:02.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.411+0000 7f57fdae1640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f57f809c070 con 0x7f57f815f290 2024-04-26T20:53:02.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.412+0000 7f57f7fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57f81633c0 0x7f57f815f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:02.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.412+0000 7f57f7fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57f81633c0 0x7f57f815f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55302/0 (socket says 172.21.15.73:55302) 2024-04-26T20:53:02.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.412+0000 7f57f7fff640 1 -- 172.21.15.73:0/2771461842 learned_addr learned my addr 172.21.15.73:0/2771461842 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:02.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.412+0000 7f57f6ffd640 1 --2- 172.21.15.73:0/2771461842 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f57f815f290 0x7f57f815b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:02.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.412+0000 7f57f77fe640 1 --2- 172.21.15.73:0/2771461842 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57f80cb7e0 0x7f57f809ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:02.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.412+0000 7f57f7fff640 1 -- 172.21.15.73:0/2771461842 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f57f815f290 msgr2=0x7f57f815b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:02.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.412+0000 7f57f7fff640 1 --2- 172.21.15.73:0/2771461842 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f57f815f290 0x7f57f815b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:02.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.412+0000 7f57f7fff640 1 -- 172.21.15.73:0/2771461842 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57f80cb7e0 msgr2=0x7f57f809ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:02.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.412+0000 7f57f7fff640 1 --2- 172.21.15.73:0/2771461842 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57f80cb7e0 0x7f57f809ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:02.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.412+0000 7f57f7fff640 1 -- 172.21.15.73:0/2771461842 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57f80b5360 con 0x7f57f81633c0 2024-04-26T20:53:02.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.412+0000 7f57f6ffd640 1 --2- 172.21.15.73:0/2771461842 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f57f815f290 0x7f57f815b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:53:02.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.412+0000 7f57f77fe640 1 --2- 172.21.15.73:0/2771461842 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57f80cb7e0 0x7f57f809ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:53:02.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.412+0000 7f57f7fff640 1 --2- 172.21.15.73:0/2771461842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57f81633c0 0x7f57f815f980 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f57f002ace0 tx=0x7f57f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:02.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.412+0000 7f57dffff640 1 -- 172.21.15.73:0/2771461842 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f003f070 con 0x7f57f81633c0 2024-04-26T20:53:02.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.412+0000 7f57dffff640 1 -- 172.21.15.73:0/2771461842 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57f0031bc0 con 0x7f57f81633c0 2024-04-26T20:53:02.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.412+0000 7f57fdae1640 1 -- 172.21.15.73:0/2771461842 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57f80b5640 con 0x7f57f81633c0 2024-04-26T20:53:02.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.412+0000 7f57dffff640 1 -- 172.21.15.73:0/2771461842 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f0044460 con 0x7f57f81633c0 2024-04-26T20:53:02.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.412+0000 7f57fdae1640 1 -- 172.21.15.73:0/2771461842 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57f80b5c00 con 0x7f57f81633c0 2024-04-26T20:53:02.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.413+0000 7f57fdae1640 1 -- 172.21.15.73:0/2771461842 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57f80b5e00 con 0x7f57f81633c0 2024-04-26T20:53:02.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.414+0000 7f57dffff640 1 -- 172.21.15.73:0/2771461842 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f57f0007790 con 0x7f57f81633c0 2024-04-26T20:53:02.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.414+0000 7f57dffff640 1 --2- 172.21.15.73:0/2771461842 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f57e406c8f0 0x7f57e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:02.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.414+0000 7f57dffff640 1 -- 172.21.15.73:0/2771461842 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6187+0+0 (secure 0 0 0) 0x7f57f00ad5c0 con 0x7f57f81633c0 2024-04-26T20:53:02.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.414+0000 7f57f77fe640 1 --2- 172.21.15.73:0/2771461842 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f57e406c8f0 0x7f57e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:02.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.414+0000 7f57f77fe640 1 --2- 172.21.15.73:0/2771461842 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f57e406c8f0 0x7f57e406edb0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f57e8000f60 tx=0x7f57e8007c10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:02.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.418+0000 7f57dffff640 1 -- 172.21.15.73:0/2771461842 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f57f80b5e00 con 0x7f57f81633c0 2024-04-26T20:53:02.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.599+0000 7f57fdae1640 1 -- 172.21.15.73:0/2771461842 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f57f80694f0 con 0x7f57f81633c0 2024-04-26T20:53:02.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.600+0000 7f57dffff640 1 -- 172.21.15.73:0/2771461842 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) ==== 74+0+17071 (secure 0 0 0) 0x7f57f007b800 con 0x7f57f81633c0 2024-04-26T20:53:02.602 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:53:02.602 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":56,"fsid":"fbd3f548-b547-4713-856b-09f98f5d1764","created":"2024-04-26T20:50:07.876786+0000","modified":"2024-04-26T20:52:53.540299+0000","last_up_change":"2024-04-26T20:50:19.082741+0000","last_in_change":"2024-04-26T20:50:10.018400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:50:19.993731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:52:18.960897+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-26T20:52:19.704261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d6f4f3-214a-4392-a02f-c33cbcab3c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6809","nonce":3461705124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6811","nonce":3461705124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6815","nonce":3461705124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6813","nonce":3461705124}]},"public_addr":"172.21.15.73:6809/3461705124","cluster_addr":"172.21.15.73:6811/3461705124","heartbeat_back_addr":"172.21.15.73:6815/3461705124","heartbeat_front_addr":"172.21.15.73:6813/3461705124","state":["exists","up"]},{"osd":1,"uuid":"786bf8ca-6ddc-4b3c-be6c-9152398bd5fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6817","nonce":2827619403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6819","nonce":2827619403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6825","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6827","nonce":2827619403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6821","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6823","nonce":2827619403}]},"public_addr":"172.21.15.73:6817/2827619403","cluster_addr":"172.21.15.73:6819/2827619403","heartbeat_back_addr":"172.21.15.73:6827/2827619403","heartbeat_front_addr":"172.21.15.73:6823/2827619403","state":["exists","up"]},{"osd":2,"uuid":"90963b1a-5437-4ef8-ad8b-6b74c534eeb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6801","nonce":2542681064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6803","nonce":2542681064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6808","nonce":2542681064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6805","nonce":2542681064}]},"public_addr":"172.21.15.73:6801/2542681064","cluster_addr":"172.21.15.73:6803/2542681064","heartbeat_back_addr":"172.21.15.73:6808/2542681064","heartbeat_front_addr":"172.21.15.73:6805/2542681064","state":["exists","up"]},{"osd":3,"uuid":"bd3d2148-6e38-41b4-a453-fe6e3b8b4ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6822","nonce":1604042718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6826","nonce":1604042718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6831","nonce":1604042718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6829","nonce":1604042718}]},"public_addr":"172.21.15.73:6822/1604042718","cluster_addr":"172.21.15.73:6826/1604042718","heartbeat_back_addr":"172.21.15.73:6831/1604042718","heartbeat_front_addr":"172.21.15.73:6829/1604042718","state":["exists","up"]},{"osd":4,"uuid":"e8668dc8-a23f-46d2-8311-cb882da08e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6817","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6819","nonce":3591055561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6824","nonce":3591055561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6831","nonce":3591055561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6828","nonce":3591055561}]},"public_addr":"172.21.15.120:6819/3591055561","cluster_addr":"172.21.15.120:6824/3591055561","heartbeat_back_addr":"172.21.15.120:6831/3591055561","heartbeat_front_addr":"172.21.15.120:6828/3591055561","state":["exists","up"]},{"osd":5,"uuid":"cbac0bdb-96bd-4852-bcce-e564495c3fcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6809","nonce":819450433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6811","nonce":819450433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6815","nonce":819450433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6813","nonce":819450433}]},"public_addr":"172.21.15.120:6809/819450433","cluster_addr":"172.21.15.120:6811/819450433","heartbeat_back_addr":"172.21.15.120:6815/819450433","heartbeat_front_addr":"172.21.15.120:6813/819450433","state":["exists","up"]},{"osd":6,"uuid":"1884b2a8-569f-4272-96a9-f25a09364dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6801","nonce":3946210727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6803","nonce":3946210727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6807","nonce":3946210727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6805","nonce":3946210727}]},"public_addr":"172.21.15.120:6801/3946210727","cluster_addr":"172.21.15.120:6803/3946210727","heartbeat_back_addr":"172.21.15.120:6807/3946210727","heartbeat_front_addr":"172.21.15.120:6805/3946210727","state":["exists","up"]},{"osd":7,"uuid":"c0c3bf6d-a4c5-4f1e-aa2d-330951830666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6818","nonce":2915574674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6821","nonce":2915574674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6827","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6829","nonce":2915574674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6823","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6825","nonce":2915574674}]},"public_addr":"172.21.15.120:6818/2915574674","cluster_addr":"172.21.15.120:6821/2915574674","heartbeat_back_addr":"172.21.15.120:6829/2915574674","heartbeat_front_addr":"172.21.15.120:6825/2915574674","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.218216+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.208710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.112705+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.182789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.164324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.174212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.225396+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.261559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:6837/2770303348":"2024-04-27T20:52:53.540263+0000","172.21.15.73:6836/2770303348":"2024-04-27T20:52:53.540263+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:53:02.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.603+0000 7f57fdae1640 1 -- 172.21.15.73:0/2771461842 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f57e406c8f0 msgr2=0x7f57e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:02.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.603+0000 7f57fdae1640 1 --2- 172.21.15.73:0/2771461842 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f57e406c8f0 0x7f57e406edb0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f57e8000f60 tx=0x7f57e8007c10 comp rx=0 tx=0).stop 2024-04-26T20:53:02.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.603+0000 7f57fdae1640 1 -- 172.21.15.73:0/2771461842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57f81633c0 msgr2=0x7f57f815f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:02.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.603+0000 7f57fdae1640 1 --2- 172.21.15.73:0/2771461842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57f81633c0 0x7f57f815f980 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f57f002ace0 tx=0x7f57f0004a40 comp rx=0 tx=0).stop 2024-04-26T20:53:02.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.603+0000 7f57fdae1640 1 -- 172.21.15.73:0/2771461842 shutdown_connections 2024-04-26T20:53:02.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.603+0000 7f57fdae1640 1 --2- 172.21.15.73:0/2771461842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57f81633c0 0x7f57f815f980 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:02.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.603+0000 7f57fdae1640 1 --2- 172.21.15.73:0/2771461842 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f57e406c8f0 0x7f57e406edb0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:02.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.603+0000 7f57fdae1640 1 --2- 172.21.15.73:0/2771461842 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f57f815f290 0x7f57f815b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:02.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.603+0000 7f57fdae1640 1 --2- 172.21.15.73:0/2771461842 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f57f80cb7e0 0x7f57f809ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:02.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.603+0000 7f57fdae1640 1 -- 172.21.15.73:0/2771461842 >> 172.21.15.73:0/2771461842 conn(0x7f57f80d0fd0 msgr2=0x7f57f80b2380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:02.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.603+0000 7f57fdae1640 1 -- 172.21.15.73:0/2771461842 shutdown_connections 2024-04-26T20:53:02.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.603+0000 7f57fdae1640 1 -- 172.21.15.73:0/2771461842 wait complete. 2024-04-26T20:53:02.615 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:50:19.993731+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:52:18.960897+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:52:19.704261+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:53:02.615 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:53:02.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.715+0000 7fb37e55f640 1 Processor -- start 2024-04-26T20:53:02.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.715+0000 7fb37e55f640 1 -- start start 2024-04-26T20:53:02.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.716+0000 7fb37e55f640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb3780cfde0 0x7fb3780d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:02.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.716+0000 7fb37e55f640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fb378070350 0x7fb378070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:02.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.716+0000 7fb37e55f640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3780d2390 con 0x7fb378071090 2024-04-26T20:53:02.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.716+0000 7fb37e55f640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7fb3780afab0 con 0x7fb3780cfde0 2024-04-26T20:53:02.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.716+0000 7fb37e55f640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fb37806dd20 con 0x7fb378070350 2024-04-26T20:53:02.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.716+0000 7fb377fff640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb3780cfde0 0x7fb3780d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:02.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.716+0000 7fb37cafc640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fb378071090 0x7fb37815e9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:50740/0 (socket says 172.21.15.73:50740) 2024-04-26T20:53:02.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.716+0000 7fb37cafc640 1 -- 172.21.15.73:0/1965214616 learned_addr learned my addr 172.21.15.73:0/1965214616 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:53:02.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.716+0000 7fb3777fe640 1 --2- 172.21.15.73:0/1965214616 >> v2:172.21.15.120:3300/0 conn(0x7fb378070350 0x7fb378070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:02.719 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.716+0000 7fb377fff640 1 -- 172.21.15.73:0/1965214616 >> v2:172.21.15.120:3300/0 conn(0x7fb378070350 msgr2=0x7fb378070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:02.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.716+0000 7fb377fff640 1 --2- 172.21.15.73:0/1965214616 >> v2:172.21.15.120:3300/0 conn(0x7fb378070350 0x7fb378070b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:02.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.717+0000 7fb377fff640 1 -- 172.21.15.73:0/1965214616 >> v1:172.21.15.73:6789/0 conn(0x7fb378071090 legacy=0x7fb37815e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:02.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.717+0000 7fb377fff640 1 -- 172.21.15.73:0/1965214616 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb37806d3f0 con 0x7fb3780cfde0 2024-04-26T20:53:02.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.717+0000 7fb376ffd640 1 -- 172.21.15.73:0/1965214616 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3787051465 0 0) 0x7fb3780d2390 con 0x7fb378071090 2024-04-26T20:53:02.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.717+0000 7fb377fff640 1 --2- 172.21.15.73:0/1965214616 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3780cfde0 0x7fb3780d1b60 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fb37806cb80 tx=0x7fb36802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c9161bd070b4cc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:02.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.717+0000 7fb376ffd640 1 -- 172.21.15.73:0/1965214616 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb368037070 con 0x7fb3780cfde0 2024-04-26T20:53:02.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.717+0000 7fb376ffd640 1 -- 172.21.15.73:0/1965214616 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb368032440 con 0x7fb3780cfde0 2024-04-26T20:53:02.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.717+0000 7fb376ffd640 1 -- 172.21.15.73:0/1965214616 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36802ae60 con 0x7fb3780cfde0 2024-04-26T20:53:02.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.718+0000 7fb37e55f640 1 -- 172.21.15.73:0/1965214616 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3780cfde0 msgr2=0x7fb3780d1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:02.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.718+0000 7fb37e55f640 1 --2- 172.21.15.73:0/1965214616 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3780cfde0 0x7fb3780d1b60 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fb37806cb80 tx=0x7fb36802a110 comp rx=0 tx=0).stop 2024-04-26T20:53:02.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.718+0000 7fb37e55f640 1 -- 172.21.15.73:0/1965214616 shutdown_connections 2024-04-26T20:53:02.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.718+0000 7fb37e55f640 1 --2- 172.21.15.73:0/1965214616 >> v2:172.21.15.120:3300/0 conn(0x7fb378070350 0x7fb378070b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:02.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.718+0000 7fb37e55f640 1 --2- 172.21.15.73:0/1965214616 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3780cfde0 0x7fb3780d1b60 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:02.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.718+0000 7fb37e55f640 1 -- 172.21.15.73:0/1965214616 >> 172.21.15.73:0/1965214616 conn(0x7fb3780d2d80 msgr2=0x7fb3780d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:02.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.718+0000 7fb37e55f640 1 -- 172.21.15.73:0/1965214616 shutdown_connections 2024-04-26T20:53:02.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.718+0000 7fb37e55f640 1 -- 172.21.15.73:0/1965214616 wait complete. 2024-04-26T20:53:02.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.719+0000 7fb37e55f640 1 Processor -- start 2024-04-26T20:53:02.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.719+0000 7fb37e55f640 1 -- start start 2024-04-26T20:53:02.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.720+0000 7fb37e55f640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb378070350 0x7fb378151200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:02.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.720+0000 7fb37e55f640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb378071090 0x7fb37809ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:02.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.720+0000 7fb37e55f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3780cfde0 0x7fb3780b0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:02.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.720+0000 7fb37e55f640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb37806d9b0 con 0x7fb3780cfde0 2024-04-26T20:53:02.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.720+0000 7fb37e55f640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb3780714c0 con 0x7fb378070350 2024-04-26T20:53:02.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.720+0000 7fb37e55f640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fb3780d20a0 con 0x7fb378071090 2024-04-26T20:53:02.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.720+0000 7fb37cafc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3780cfde0 0x7fb3780b0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:02.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.720+0000 7fb37cafc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3780cfde0 0x7fb3780b0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55326/0 (socket says 172.21.15.73:55326) 2024-04-26T20:53:02.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.720+0000 7fb37cafc640 1 -- 172.21.15.73:0/3899467920 learned_addr learned my addr 172.21.15.73:0/3899467920 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:02.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.720+0000 7fb3777fe640 1 --2- 172.21.15.73:0/3899467920 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb378071090 0x7fb37809ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:02.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.720+0000 7fb377fff640 1 --2- 172.21.15.73:0/3899467920 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb378070350 0x7fb378151200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:02.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.720+0000 7fb37cafc640 1 -- 172.21.15.73:0/3899467920 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb378071090 msgr2=0x7fb37809ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:02.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.720+0000 7fb37cafc640 1 --2- 172.21.15.73:0/3899467920 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb378071090 0x7fb37809ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:02.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.720+0000 7fb37cafc640 1 -- 172.21.15.73:0/3899467920 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb378070350 msgr2=0x7fb378151200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:02.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.720+0000 7fb37cafc640 1 --2- 172.21.15.73:0/3899467920 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb378070350 0x7fb378151200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:02.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.720+0000 7fb37cafc640 1 -- 172.21.15.73:0/3899467920 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3780a09f0 con 0x7fb3780cfde0 2024-04-26T20:53:02.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.720+0000 7fb3777fe640 1 --2- 172.21.15.73:0/3899467920 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb378071090 0x7fb37809ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:53:02.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.720+0000 7fb377fff640 1 --2- 172.21.15.73:0/3899467920 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb378070350 0x7fb378151200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:53:02.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.721+0000 7fb37cafc640 1 --2- 172.21.15.73:0/3899467920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3780cfde0 0x7fb3780b0b10 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fb37000c700 tx=0x7fb37000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:02.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.721+0000 7fb374ff9640 1 -- 172.21.15.73:0/3899467920 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb370018070 con 0x7fb3780cfde0 2024-04-26T20:53:02.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.721+0000 7fb374ff9640 1 -- 172.21.15.73:0/3899467920 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb37000ce10 con 0x7fb3780cfde0 2024-04-26T20:53:02.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.721+0000 7fb374ff9640 1 -- 172.21.15.73:0/3899467920 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37000a860 con 0x7fb3780cfde0 2024-04-26T20:53:02.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.721+0000 7fb37e55f640 1 -- 172.21.15.73:0/3899467920 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb37809f950 con 0x7fb3780cfde0 2024-04-26T20:53:02.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.721+0000 7fb37e55f640 1 -- 172.21.15.73:0/3899467920 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb37809ff10 con 0x7fb3780cfde0 2024-04-26T20:53:02.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.722+0000 7fb37e55f640 1 -- 172.21.15.73:0/3899467920 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3780694f0 con 0x7fb3780cfde0 2024-04-26T20:53:02.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.722+0000 7fb374ff9640 1 -- 172.21.15.73:0/3899467920 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb37001b070 con 0x7fb3780cfde0 2024-04-26T20:53:02.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.722+0000 7fb374ff9640 1 --2- 172.21.15.73:0/3899467920 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb36406c920 0x7fb36406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:02.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.723+0000 7fb374ff9640 1 -- 172.21.15.73:0/3899467920 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6187+0+0 (secure 0 0 0) 0x7fb37008b290 con 0x7fb3780cfde0 2024-04-26T20:53:02.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.723+0000 7fb377fff640 1 --2- 172.21.15.73:0/3899467920 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb36406c920 0x7fb36406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:02.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.723+0000 7fb377fff640 1 --2- 172.21.15.73:0/3899467920 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb36406c920 0x7fb36406ede0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb368034000 tx=0x7fb36802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:02.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.727+0000 7fb374ff9640 1 -- 172.21.15.73:0/3899467920 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb370059570 con 0x7fb3780cfde0 2024-04-26T20:53:02.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.900+0000 7fb37e55f640 1 -- 172.21.15.73:0/3899467920 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb3780b1b60 con 0x7fb3780cfde0 2024-04-26T20:53:02.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.901+0000 7fb374ff9640 1 -- 172.21.15.73:0/3899467920 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v56) ==== 93+0+10 (secure 0 0 0) 0x7fb370058e70 con 0x7fb3780cfde0 2024-04-26T20:53:02.902 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-04-26T20:53:02.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.902+0000 7fb37e55f640 1 -- 172.21.15.73:0/3899467920 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb36406c920 msgr2=0x7fb36406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:02.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.902+0000 7fb37e55f640 1 --2- 172.21.15.73:0/3899467920 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb36406c920 0x7fb36406ede0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb368034000 tx=0x7fb36802e000 comp rx=0 tx=0).stop 2024-04-26T20:53:02.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.902+0000 7fb37e55f640 1 -- 172.21.15.73:0/3899467920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3780cfde0 msgr2=0x7fb3780b0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:02.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.902+0000 7fb37e55f640 1 --2- 172.21.15.73:0/3899467920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3780cfde0 0x7fb3780b0b10 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fb37000c700 tx=0x7fb37000cbc0 comp rx=0 tx=0).stop 2024-04-26T20:53:02.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.903+0000 7fb37e55f640 1 -- 172.21.15.73:0/3899467920 shutdown_connections 2024-04-26T20:53:02.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.903+0000 7fb37e55f640 1 --2- 172.21.15.73:0/3899467920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3780cfde0 0x7fb3780b0b10 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:02.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.903+0000 7fb37e55f640 1 --2- 172.21.15.73:0/3899467920 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb36406c920 0x7fb36406ede0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:02.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.903+0000 7fb37e55f640 1 --2- 172.21.15.73:0/3899467920 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb378071090 0x7fb37809ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:02.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.903+0000 7fb37e55f640 1 --2- 172.21.15.73:0/3899467920 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb378070350 0x7fb378151200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:02.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.903+0000 7fb37e55f640 1 -- 172.21.15.73:0/3899467920 >> 172.21.15.73:0/3899467920 conn(0x7fb3780d2d80 msgr2=0x7fb3780b4060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:02.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.903+0000 7fb37e55f640 1 -- 172.21.15.73:0/3899467920 shutdown_connections 2024-04-26T20:53:02.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:02.903+0000 7fb37e55f640 1 -- 172.21.15.73:0/3899467920 wait complete. 2024-04-26T20:53:02.914 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:53:03.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.013+0000 7ff5a74e1640 1 Processor -- start 2024-04-26T20:53:03.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.013+0000 7ff5a74e1640 1 -- start start 2024-04-26T20:53:03.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.014+0000 7ff5a74e1640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7ff5a0070350 0x7ff5a0070b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:03.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.014+0000 7ff5a74e1640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5a00d2390 con 0x7ff5a00cfde0 2024-04-26T20:53:03.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.014+0000 7ff5a74e1640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5a006d3f0 con 0x7ff5a0071090 2024-04-26T20:53:03.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.014+0000 7ff5a74e1640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7ff5a00afab0 con 0x7ff5a0070350 2024-04-26T20:53:03.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.014+0000 7ff5a527d640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7ff5a00cfde0 0x7ff5a00d1b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:50744/0 (socket says 172.21.15.73:50744) 2024-04-26T20:53:03.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.014+0000 7ff5a527d640 1 -- 172.21.15.73:0/1568818469 learned_addr learned my addr 172.21.15.73:0/1568818469 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:53:03.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.014+0000 7ff597fff640 1 -- 172.21.15.73:0/1568818469 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3770805985 0 0) 0x7ff5a00d2390 con 0x7ff5a00cfde0 2024-04-26T20:53:03.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.014+0000 7ff597fff640 1 -- 172.21.15.73:0/1568818469 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff58c003520 con 0x7ff5a00cfde0 2024-04-26T20:53:03.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.014+0000 7ff5a4a7c640 1 --2- 172.21.15.73:0/1568818469 >> v2:172.21.15.120:3300/0 conn(0x7ff5a0070350 0x7ff5a0070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:03.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.014+0000 7ff597fff640 1 -- 172.21.15.73:0/1568818469 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2182517475 0 0) 0x7ff58c003520 con 0x7ff5a00cfde0 2024-04-26T20:53:03.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.014+0000 7ff597fff640 1 -- 172.21.15.73:0/1568818469 >> v2:172.21.15.120:3300/0 conn(0x7ff5a0070350 msgr2=0x7ff5a0070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:03.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.014+0000 7ff597fff640 1 --2- 172.21.15.73:0/1568818469 >> v2:172.21.15.120:3300/0 conn(0x7ff5a0070350 0x7ff5a0070b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:03.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.014+0000 7ff597fff640 1 -- 172.21.15.73:0/1568818469 >> v1:172.21.15.120:6789/0 conn(0x7ff5a0071090 legacy=0x7ff5a015e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:03.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.014+0000 7ff597fff640 1 -- 172.21.15.73:0/1568818469 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5a006d6d0 con 0x7ff5a00cfde0 2024-04-26T20:53:03.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.014+0000 7ff597fff640 1 -- 172.21.15.73:0/1568818469 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7ff59c001670 con 0x7ff5a00cfde0 2024-04-26T20:53:03.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.014+0000 7ff597fff640 1 -- 172.21.15.73:0/1568818469 >> v1:172.21.15.73:6789/0 conn(0x7ff5a00cfde0 legacy=0x7ff5a00d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:03.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.014+0000 7ff597fff640 1 --2- 172.21.15.73:0/1568818469 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff58c0039b0 0x7ff58c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:03.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.014+0000 7ff597fff640 1 --2- 172.21.15.73:0/1568818469 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff58c004800 0x7ff58c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:03.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.014+0000 7ff597fff640 1 --2- 172.21.15.73:0/1568818469 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff58c0051c0 0x7ff58c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:03.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.014+0000 7ff597fff640 1 -- 172.21.15.73:0/1568818469 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c00c130 con 0x7ff58c0051c0 2024-04-26T20:53:03.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.014+0000 7ff597fff640 1 -- 172.21.15.73:0/1568818469 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff58c00c2e0 con 0x7ff58c004800 2024-04-26T20:53:03.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.014+0000 7ff597fff640 1 -- 172.21.15.73:0/1568818469 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7ff58c00c4e0 con 0x7ff58c0039b0 2024-04-26T20:53:03.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.015+0000 7ff5a5a7e640 1 --2- 172.21.15.73:0/1568818469 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff58c0051c0 0x7ff58c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:03.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.015+0000 7ff5a527d640 1 --2- 172.21.15.73:0/1568818469 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff58c0039b0 0x7ff58c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:03.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.015+0000 7ff5a4a7c640 1 --2- 172.21.15.73:0/1568818469 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff58c004800 0x7ff58c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:03.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.015+0000 7ff5a5a7e640 1 -- 172.21.15.73:0/1568818469 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff58c0039b0 msgr2=0x7ff58c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:03.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.015+0000 7ff5a5a7e640 1 --2- 172.21.15.73:0/1568818469 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff58c0039b0 0x7ff58c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:03.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.015+0000 7ff5a5a7e640 1 -- 172.21.15.73:0/1568818469 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff58c004800 msgr2=0x7ff58c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:03.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.015+0000 7ff5a5a7e640 1 --2- 172.21.15.73:0/1568818469 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff58c004800 0x7ff58c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:03.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.015+0000 7ff5a5a7e640 1 -- 172.21.15.73:0/1568818469 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff58c003520 con 0x7ff58c0051c0 2024-04-26T20:53:03.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.015+0000 7ff5a5a7e640 1 -- 172.21.15.73:0/1568818469 reap_dead start 2024-04-26T20:53:03.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.016+0000 7ff5a5a7e640 1 --2- 172.21.15.73:0/1568818469 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff58c0051c0 0x7ff58c00b9c0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7ff598001e50 tx=0x7ff58c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ff26d60ca0eb0a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:03.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.016+0000 7ff597fff640 1 -- 172.21.15.73:0/1568818469 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff598001a70 con 0x7ff58c0051c0 2024-04-26T20:53:03.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.016+0000 7ff597fff640 1 -- 172.21.15.73:0/1568818469 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff598003970 con 0x7ff58c0051c0 2024-04-26T20:53:03.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.016+0000 7ff597fff640 1 -- 172.21.15.73:0/1568818469 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff598003d60 con 0x7ff58c0051c0 2024-04-26T20:53:03.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.016+0000 7ff5a74e1640 1 -- 172.21.15.73:0/1568818469 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff58c0051c0 msgr2=0x7ff58c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:03.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.016+0000 7ff5a74e1640 1 --2- 172.21.15.73:0/1568818469 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff58c0051c0 0x7ff58c00b9c0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7ff598001e50 tx=0x7ff58c005830 comp rx=0 tx=0).stop 2024-04-26T20:53:03.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.016+0000 7ff5a74e1640 1 -- 172.21.15.73:0/1568818469 shutdown_connections 2024-04-26T20:53:03.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.016+0000 7ff5a74e1640 1 --2- 172.21.15.73:0/1568818469 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff58c0051c0 0x7ff58c00b9c0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:03.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.016+0000 7ff5a74e1640 1 -- 172.21.15.73:0/1568818469 >> 172.21.15.73:0/1568818469 conn(0x7ff5a00d2d80 msgr2=0x7ff5a00d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:03.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.016+0000 7ff5a74e1640 1 -- 172.21.15.73:0/1568818469 shutdown_connections 2024-04-26T20:53:03.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.016+0000 7ff5a74e1640 1 -- 172.21.15.73:0/1568818469 wait complete. 2024-04-26T20:53:03.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.017+0000 7ff5a74e1640 1 Processor -- start 2024-04-26T20:53:03.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.017+0000 7ff5a74e1640 1 -- start start 2024-04-26T20:53:03.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.018+0000 7ff5a74e1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff58c0051c0 0x7ff5a01f4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:03.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.018+0000 7ff5a74e1640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff5a00c7840 0x7ff5a00c96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:03.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.018+0000 7ff5a74e1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff5a015ae60 0x7ff5a015b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:03.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.018+0000 7ff5a74e1640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a00714c0 con 0x7ff5a015ae60 2024-04-26T20:53:03.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.018+0000 7ff5a74e1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a006dd20 con 0x7ff58c0051c0 2024-04-26T20:53:03.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.018+0000 7ff5a74e1640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7ff5a0070b50 con 0x7ff5a00c7840 2024-04-26T20:53:03.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.018+0000 7ff5a5a7e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff5a015ae60 0x7ff5a015b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:03.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.018+0000 7ff5a5a7e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff5a015ae60 0x7ff5a015b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55348/0 (socket says 172.21.15.73:55348) 2024-04-26T20:53:03.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.018+0000 7ff5a5a7e640 1 -- 172.21.15.73:0/2578337192 learned_addr learned my addr 172.21.15.73:0/2578337192 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:03.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.018+0000 7ff5a4a7c640 1 --2- 172.21.15.73:0/2578337192 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff5a00c7840 0x7ff5a00c96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:03.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.018+0000 7ff5a527d640 1 --2- 172.21.15.73:0/2578337192 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff58c0051c0 0x7ff5a01f4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:03.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.018+0000 7ff5a5a7e640 1 -- 172.21.15.73:0/2578337192 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff5a00c7840 msgr2=0x7ff5a00c96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:03.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.019+0000 7ff5a5a7e640 1 --2- 172.21.15.73:0/2578337192 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff5a00c7840 0x7ff5a00c96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:03.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.019+0000 7ff5a5a7e640 1 -- 172.21.15.73:0/2578337192 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff58c0051c0 msgr2=0x7ff5a01f4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:03.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.019+0000 7ff5a5a7e640 1 --2- 172.21.15.73:0/2578337192 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff58c0051c0 0x7ff5a01f4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:03.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.019+0000 7ff5a5a7e640 1 -- 172.21.15.73:0/2578337192 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5a00c9bf0 con 0x7ff5a015ae60 2024-04-26T20:53:03.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.019+0000 7ff5a4a7c640 1 --2- 172.21.15.73:0/2578337192 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff5a00c7840 0x7ff5a00c96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:53:03.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.019+0000 7ff5a527d640 1 --2- 172.21.15.73:0/2578337192 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff58c0051c0 0x7ff5a01f4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:53:03.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.019+0000 7ff5a5a7e640 1 --2- 172.21.15.73:0/2578337192 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff5a015ae60 0x7ff5a015b2e0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7ff598001f70 tx=0x7ff5980014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:03.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.019+0000 7ff595ffb640 1 -- 172.21.15.73:0/2578337192 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff598001a70 con 0x7ff5a015ae60 2024-04-26T20:53:03.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.019+0000 7ff595ffb640 1 -- 172.21.15.73:0/2578337192 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff59802ac70 con 0x7ff5a015ae60 2024-04-26T20:53:03.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.019+0000 7ff5a74e1640 1 -- 172.21.15.73:0/2578337192 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5a01f5120 con 0x7ff5a015ae60 2024-04-26T20:53:03.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.019+0000 7ff595ffb640 1 -- 172.21.15.73:0/2578337192 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff598006d70 con 0x7ff5a015ae60 2024-04-26T20:53:03.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.019+0000 7ff5a74e1640 1 -- 172.21.15.73:0/2578337192 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5a01f56e0 con 0x7ff5a015ae60 2024-04-26T20:53:03.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.020+0000 7ff5a74e1640 1 -- 172.21.15.73:0/2578337192 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5a015b820 con 0x7ff5a015ae60 2024-04-26T20:53:03.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.021+0000 7ff595ffb640 1 -- 172.21.15.73:0/2578337192 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff598007030 con 0x7ff5a015ae60 2024-04-26T20:53:03.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.021+0000 7ff595ffb640 1 --2- 172.21.15.73:0/2578337192 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff58406c920 0x7ff58406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:03.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.021+0000 7ff595ffb640 1 -- 172.21.15.73:0/2578337192 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6187+0+0 (secure 0 0 0) 0x7ff5980ab520 con 0x7ff5a015ae60 2024-04-26T20:53:03.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.021+0000 7ff5a527d640 1 --2- 172.21.15.73:0/2578337192 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff58406c920 0x7ff58406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:03.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.021+0000 7ff5a527d640 1 --2- 172.21.15.73:0/2578337192 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff58406c920 0x7ff58406ede0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff59c001050 tx=0x7ff59c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:03.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.025+0000 7ff595ffb640 1 -- 172.21.15.73:0/2578337192 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff598043060 con 0x7ff5a015ae60 2024-04-26T20:53:03.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.198+0000 7ff5a74e1640 1 -- 172.21.15.73:0/2578337192 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff5a00b2ff0 con 0x7ff5a015ae60 2024-04-26T20:53:03.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.199+0000 7ff595ffb640 1 -- 172.21.15.73:0/2578337192 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v56) ==== 104+0+11 (secure 0 0 0) 0x7ff598079800 con 0x7ff5a015ae60 2024-04-26T20:53:03.200 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-04-26T20:53:03.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.200+0000 7ff5a74e1640 1 -- 172.21.15.73:0/2578337192 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff58406c920 msgr2=0x7ff58406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:03.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.200+0000 7ff5a74e1640 1 --2- 172.21.15.73:0/2578337192 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff58406c920 0x7ff58406ede0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff59c001050 tx=0x7ff59c009040 comp rx=0 tx=0).stop 2024-04-26T20:53:03.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.200+0000 7ff5a74e1640 1 -- 172.21.15.73:0/2578337192 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff5a015ae60 msgr2=0x7ff5a015b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:03.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.200+0000 7ff5a74e1640 1 --2- 172.21.15.73:0/2578337192 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff5a015ae60 0x7ff5a015b2e0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7ff598001f70 tx=0x7ff5980014a0 comp rx=0 tx=0).stop 2024-04-26T20:53:03.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.200+0000 7ff5a74e1640 1 -- 172.21.15.73:0/2578337192 shutdown_connections 2024-04-26T20:53:03.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.200+0000 7ff5a74e1640 1 --2- 172.21.15.73:0/2578337192 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff5a015ae60 0x7ff5a015b2e0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:03.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.200+0000 7ff5a74e1640 1 --2- 172.21.15.73:0/2578337192 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff58406c920 0x7ff58406ede0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:03.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.200+0000 7ff5a74e1640 1 --2- 172.21.15.73:0/2578337192 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff5a00c7840 0x7ff5a00c96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:03.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.200+0000 7ff5a74e1640 1 --2- 172.21.15.73:0/2578337192 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff58c0051c0 0x7ff5a01f4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:03.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.200+0000 7ff5a74e1640 1 -- 172.21.15.73:0/2578337192 >> 172.21.15.73:0/2578337192 conn(0x7ff5a00d2d80 msgr2=0x7ff5a00b1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:03.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.201+0000 7ff5a74e1640 1 -- 172.21.15.73:0/2578337192 shutdown_connections 2024-04-26T20:53:03.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.201+0000 7ff5a74e1640 1 -- 172.21.15.73:0/2578337192 wait complete. 2024-04-26T20:53:03.212 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:53:03.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.310+0000 7fef1c167640 1 Processor -- start 2024-04-26T20:53:03.311 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.310+0000 7fef1c167640 1 -- start start 2024-04-26T20:53:03.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.311+0000 7fef1c167640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fef14162470 0x7fef1406c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:03.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.311+0000 7fef1c167640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fef14070c70 0x7fef14158f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:03.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.311+0000 7fef1c167640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef1415fc70 con 0x7fef14162850 2024-04-26T20:53:03.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.311+0000 7fef1c167640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fef1415ff50 con 0x7fef14070c70 2024-04-26T20:53:03.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.311+0000 7fef1c167640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fef14160230 con 0x7fef14162470 2024-04-26T20:53:03.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.312+0000 7fef19f03640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fef14162470 0x7fef1406c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:03.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.312+0000 7fef1a704640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fef14070c70 0x7fef14158f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:03.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.312+0000 7fef19f03640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fef14162470 0x7fef1406c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.73:45884/0 (socket says 172.21.15.73:45884) 2024-04-26T20:53:03.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.312+0000 7fef19f03640 1 -- 172.21.15.73:0/3721454760 learned_addr learned my addr 172.21.15.73:0/3721454760 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:03.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.312+0000 7fef19f03640 1 -- 172.21.15.73:0/3721454760 >> v2:172.21.15.120:3300/0 conn(0x7fef14070c70 msgr2=0x7fef14158f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:03.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.312+0000 7fef19f03640 1 --2- 172.21.15.73:0/3721454760 >> v2:172.21.15.120:3300/0 conn(0x7fef14070c70 0x7fef14158f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:03.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.312+0000 7fef19f03640 1 -- 172.21.15.73:0/3721454760 >> v1:172.21.15.120:6789/0 conn(0x7fef14162850 legacy=0x7fef14070560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:03.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.312+0000 7fef19f03640 1 -- 172.21.15.73:0/3721454760 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef14160510 con 0x7fef14162470 2024-04-26T20:53:03.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.313+0000 7fef19f03640 1 --2- 172.21.15.73:0/3721454760 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fef14162470 0x7fef1406c570 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fef04009e50 tx=0x7fef0402a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62d1357a1ed890df server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:03.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.313+0000 7fef18f01640 1 -- 172.21.15.73:0/3721454760 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef04036070 con 0x7fef14162470 2024-04-26T20:53:03.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.313+0000 7fef18f01640 1 -- 172.21.15.73:0/3721454760 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef04031600 con 0x7fef14162470 2024-04-26T20:53:03.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.313+0000 7fef18f01640 1 -- 172.21.15.73:0/3721454760 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef040319d0 con 0x7fef14162470 2024-04-26T20:53:03.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.313+0000 7fef1c167640 1 -- 172.21.15.73:0/3721454760 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fef14162470 msgr2=0x7fef1406c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:03.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.313+0000 7fef1c167640 1 --2- 172.21.15.73:0/3721454760 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fef14162470 0x7fef1406c570 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fef04009e50 tx=0x7fef0402a710 comp rx=0 tx=0).stop 2024-04-26T20:53:03.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.313+0000 7fef1c167640 1 -- 172.21.15.73:0/3721454760 shutdown_connections 2024-04-26T20:53:03.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.313+0000 7fef1c167640 1 --2- 172.21.15.73:0/3721454760 >> v2:172.21.15.120:3300/0 conn(0x7fef14070c70 0x7fef14158f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:03.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.313+0000 7fef1c167640 1 --2- 172.21.15.73:0/3721454760 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fef14162470 0x7fef1406c570 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:03.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.313+0000 7fef1c167640 1 -- 172.21.15.73:0/3721454760 >> 172.21.15.73:0/3721454760 conn(0x7fef140b3940 msgr2=0x7fef140b3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:03.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.313+0000 7fef1c167640 1 -- 172.21.15.73:0/3721454760 shutdown_connections 2024-04-26T20:53:03.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.313+0000 7fef1c167640 1 -- 172.21.15.73:0/3721454760 wait complete. 2024-04-26T20:53:03.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.315+0000 7fef1c167640 1 Processor -- start 2024-04-26T20:53:03.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.315+0000 7fef1c167640 1 -- start start 2024-04-26T20:53:03.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.315+0000 7fef1c167640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef14070c70 0x7fef140b66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:03.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.315+0000 7fef1c167640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fef14162470 0x7fef14146680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:03.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.315+0000 7fef1c167640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef14162850 0x7fef140b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:03.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.315+0000 7fef1c167640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fef14160a40 con 0x7fef14162850 2024-04-26T20:53:03.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.315+0000 7fef1c167640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fef141608c0 con 0x7fef14070c70 2024-04-26T20:53:03.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.315+0000 7fef1c167640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fef1415f700 con 0x7fef14162470 2024-04-26T20:53:03.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.315+0000 7fef1a704640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef14162850 0x7fef140b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:03.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.315+0000 7fef1a704640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef14162850 0x7fef140b4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55356/0 (socket says 172.21.15.73:55356) 2024-04-26T20:53:03.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.315+0000 7fef1a704640 1 -- 172.21.15.73:0/1400978629 learned_addr learned my addr 172.21.15.73:0/1400978629 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:03.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.316+0000 7fef19f03640 1 --2- 172.21.15.73:0/1400978629 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef14070c70 0x7fef140b66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:03.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.316+0000 7fef19702640 1 --2- 172.21.15.73:0/1400978629 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fef14162470 0x7fef14146680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:03.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.316+0000 7fef1a704640 1 -- 172.21.15.73:0/1400978629 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fef14162470 msgr2=0x7fef14146680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:03.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.316+0000 7fef1a704640 1 --2- 172.21.15.73:0/1400978629 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fef14162470 0x7fef14146680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:03.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.316+0000 7fef1a704640 1 -- 172.21.15.73:0/1400978629 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef14070c70 msgr2=0x7fef140b66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:03.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.316+0000 7fef1a704640 1 --2- 172.21.15.73:0/1400978629 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef14070c70 0x7fef140b66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:03.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.316+0000 7fef1a704640 1 -- 172.21.15.73:0/1400978629 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef140b5610 con 0x7fef14162850 2024-04-26T20:53:03.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.316+0000 7fef19f03640 1 --2- 172.21.15.73:0/1400978629 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef14070c70 0x7fef140b66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:53:03.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.316+0000 7fef19702640 1 --2- 172.21.15.73:0/1400978629 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fef14162470 0x7fef14146680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:53:03.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.316+0000 7fef1a704640 1 --2- 172.21.15.73:0/1400978629 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef14162850 0x7fef140b4a30 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fef0c00e930 tx=0x7fef0c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:03.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.316+0000 7fef127fc640 1 -- 172.21.15.73:0/1400978629 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0c017070 con 0x7fef14162850 2024-04-26T20:53:03.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.316+0000 7fef127fc640 1 -- 172.21.15.73:0/1400978629 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef0c00c980 con 0x7fef14162850 2024-04-26T20:53:03.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.316+0000 7fef1c167640 1 -- 172.21.15.73:0/1400978629 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef140b58f0 con 0x7fef14162850 2024-04-26T20:53:03.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.316+0000 7fef127fc640 1 -- 172.21.15.73:0/1400978629 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0c00cd70 con 0x7fef14162850 2024-04-26T20:53:03.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.316+0000 7fef1c167640 1 -- 172.21.15.73:0/1400978629 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef140b5eb0 con 0x7fef14162850 2024-04-26T20:53:03.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.317+0000 7fef1c167640 1 -- 172.21.15.73:0/1400978629 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef140694f0 con 0x7fef14162850 2024-04-26T20:53:03.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.317+0000 7fef127fc640 1 -- 172.21.15.73:0/1400978629 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fef0c01a070 con 0x7fef14162850 2024-04-26T20:53:03.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.318+0000 7fef127fc640 1 --2- 172.21.15.73:0/1400978629 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fef0006c8f0 0x7fef0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:03.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.318+0000 7fef19f03640 1 --2- 172.21.15.73:0/1400978629 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fef0006c8f0 0x7fef0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:03.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.318+0000 7fef127fc640 1 -- 172.21.15.73:0/1400978629 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6187+0+0 (secure 0 0 0) 0x7fef0c08c140 con 0x7fef14162850 2024-04-26T20:53:03.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.318+0000 7fef19f03640 1 --2- 172.21.15.73:0/1400978629 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fef0006c8f0 0x7fef0006edb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fef04033000 tx=0x7fef04009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:03.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.322+0000 7fef127fc640 1 -- 172.21.15.73:0/1400978629 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fef0c05a3c0 con 0x7fef14162850 2024-04-26T20:53:03.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.497+0000 7fef1c167640 1 -- 172.21.15.73:0/1400978629 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fef14073290 con 0x7fef14162850 2024-04-26T20:53:03.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.498+0000 7fef127fc640 1 -- 172.21.15.73:0/1400978629 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v56) ==== 100+0+11 (secure 0 0 0) 0x7fef0c059cc0 con 0x7fef14162850 2024-04-26T20:53:03.499 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-04-26T20:53:03.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.500+0000 7fef1c167640 1 -- 172.21.15.73:0/1400978629 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fef0006c8f0 msgr2=0x7fef0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:03.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.500+0000 7fef1c167640 1 --2- 172.21.15.73:0/1400978629 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fef0006c8f0 0x7fef0006edb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fef04033000 tx=0x7fef04009000 comp rx=0 tx=0).stop 2024-04-26T20:53:03.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.500+0000 7fef1c167640 1 -- 172.21.15.73:0/1400978629 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef14162850 msgr2=0x7fef140b4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:03.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.500+0000 7fef1c167640 1 --2- 172.21.15.73:0/1400978629 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef14162850 0x7fef140b4a30 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fef0c00e930 tx=0x7fef0c00edf0 comp rx=0 tx=0).stop 2024-04-26T20:53:03.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.500+0000 7fef1c167640 1 -- 172.21.15.73:0/1400978629 shutdown_connections 2024-04-26T20:53:03.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.500+0000 7fef1c167640 1 --2- 172.21.15.73:0/1400978629 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef14162850 0x7fef140b4a30 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:03.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.500+0000 7fef1c167640 1 --2- 172.21.15.73:0/1400978629 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fef0006c8f0 0x7fef0006edb0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:03.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.500+0000 7fef1c167640 1 --2- 172.21.15.73:0/1400978629 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fef14162470 0x7fef14146680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:03.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.500+0000 7fef1c167640 1 --2- 172.21.15.73:0/1400978629 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fef14070c70 0x7fef140b66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:03.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.500+0000 7fef1c167640 1 -- 172.21.15.73:0/1400978629 >> 172.21.15.73:0/1400978629 conn(0x7fef140b3940 msgr2=0x7fef140d0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:03.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.500+0000 7fef1c167640 1 -- 172.21.15.73:0/1400978629 shutdown_connections 2024-04-26T20:53:03.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.500+0000 7fef1c167640 1 -- 172.21.15.73:0/1400978629 wait complete. 2024-04-26T20:53:03.511 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:53:03.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.608+0000 7f8f2e99d640 1 Processor -- start 2024-04-26T20:53:03.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.608+0000 7f8f2e99d640 1 -- start start 2024-04-26T20:53:03.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.609+0000 7f8f2e99d640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8f2815d1e0 0x7f8f280d09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:03.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.609+0000 7f8f2e99d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8f280d1a90 0x7f8f28070ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:03.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.609+0000 7f8f2e99d640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f280ce840 con 0x7f8f280d0f20 2024-04-26T20:53:03.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.609+0000 7f8f2e99d640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f8f280cede0 con 0x7f8f2815d1e0 2024-04-26T20:53:03.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.609+0000 7f8f2e99d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f8f28070650 con 0x7f8f280d1a90 2024-04-26T20:53:03.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.609+0000 7f8f27fff640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8f2815d1e0 0x7f8f280d09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:03.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.609+0000 7f8f277fe640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f8f280d0f20 0x7f8f280d1380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:50750/0 (socket says 172.21.15.73:50750) 2024-04-26T20:53:03.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.609+0000 7f8f27fff640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8f2815d1e0 0x7f8f280d09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55368/0 (socket says 172.21.15.73:55368) 2024-04-26T20:53:03.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.609+0000 7f8f277fe640 1 -- 172.21.15.73:0/1891971853 learned_addr learned my addr 172.21.15.73:0/1891971853 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:53:03.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.609+0000 7f8f2cf3a640 1 --2- 172.21.15.73:0/1891971853 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8f280d1a90 0x7f8f28070ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:03.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.609+0000 7f8f27fff640 1 -- 172.21.15.73:0/1891971853 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8f280d1a90 msgr2=0x7f8f28070ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:03.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.609+0000 7f8f27fff640 1 --2- 172.21.15.73:0/1891971853 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8f280d1a90 0x7f8f28070ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:03.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.609+0000 7f8f27fff640 1 -- 172.21.15.73:0/1891971853 >> v1:172.21.15.73:6789/0 conn(0x7f8f280d0f20 legacy=0x7f8f280d1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:03.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.609+0000 7f8f27fff640 1 -- 172.21.15.73:0/1891971853 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f280ceb00 con 0x7f8f2815d1e0 2024-04-26T20:53:03.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.609+0000 7f8f26ffd640 1 -- 172.21.15.73:0/1891971853 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2498539622 0 0) 0x7f8f280ce840 con 0x7f8f280d0f20 2024-04-26T20:53:03.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.610+0000 7f8f27fff640 1 --2- 172.21.15.73:0/1891971853 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f2815d1e0 0x7f8f280d09e0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f8f180097a0 tx=0x7f8f1802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3779f9e1745c98b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:03.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.610+0000 7f8f26ffd640 1 -- 172.21.15.73:0/1891971853 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f18037070 con 0x7f8f2815d1e0 2024-04-26T20:53:03.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.610+0000 7f8f26ffd640 1 -- 172.21.15.73:0/1891971853 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f18032440 con 0x7f8f2815d1e0 2024-04-26T20:53:03.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.610+0000 7f8f26ffd640 1 -- 172.21.15.73:0/1891971853 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f18032810 con 0x7f8f2815d1e0 2024-04-26T20:53:03.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.611+0000 7f8f2e99d640 1 -- 172.21.15.73:0/1891971853 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f2815d1e0 msgr2=0x7f8f280d09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:03.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.611+0000 7f8f2e99d640 1 --2- 172.21.15.73:0/1891971853 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f2815d1e0 0x7f8f280d09e0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f8f180097a0 tx=0x7f8f1802a180 comp rx=0 tx=0).stop 2024-04-26T20:53:03.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.611+0000 7f8f2e99d640 1 -- 172.21.15.73:0/1891971853 shutdown_connections 2024-04-26T20:53:03.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.611+0000 7f8f2e99d640 1 --2- 172.21.15.73:0/1891971853 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8f280d1a90 0x7f8f28070ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:03.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.611+0000 7f8f2e99d640 1 --2- 172.21.15.73:0/1891971853 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f2815d1e0 0x7f8f280d09e0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:03.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.611+0000 7f8f2e99d640 1 -- 172.21.15.73:0/1891971853 >> 172.21.15.73:0/1891971853 conn(0x7f8f280d2da0 msgr2=0x7f8f280d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:03.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.611+0000 7f8f2e99d640 1 -- 172.21.15.73:0/1891971853 shutdown_connections 2024-04-26T20:53:03.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.611+0000 7f8f2e99d640 1 -- 172.21.15.73:0/1891971853 wait complete. 2024-04-26T20:53:03.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.612+0000 7f8f2e99d640 1 Processor -- start 2024-04-26T20:53:03.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.612+0000 7f8f2e99d640 1 -- start start 2024-04-26T20:53:03.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.613+0000 7f8f2e99d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f280d0f20 0x7f8f280c8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:03.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.613+0000 7f8f2e99d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8f280d1a90 0x7f8f281584e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:03.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.613+0000 7f8f2e99d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8f2815d1e0 0x7f8f28158a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:03.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.613+0000 7f8f2e99d640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8f28071020 con 0x7f8f280d0f20 2024-04-26T20:53:03.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.613+0000 7f8f2e99d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f8f280ce4c0 con 0x7f8f280d1a90 2024-04-26T20:53:03.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.613+0000 7f8f2e99d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f8f280711a0 con 0x7f8f2815d1e0 2024-04-26T20:53:03.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.613+0000 7f8f27fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f280d0f20 0x7f8f280c8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:03.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.613+0000 7f8f27fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f280d0f20 0x7f8f280c8080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55374/0 (socket says 172.21.15.73:55374) 2024-04-26T20:53:03.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.613+0000 7f8f27fff640 1 -- 172.21.15.73:0/3751932793 learned_addr learned my addr 172.21.15.73:0/3751932793 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:03.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.613+0000 7f8f27fff640 1 -- 172.21.15.73:0/3751932793 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8f2815d1e0 msgr2=0x7f8f28158a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:03.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.613+0000 7f8f27fff640 1 --2- 172.21.15.73:0/3751932793 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8f2815d1e0 0x7f8f28158a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:03.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.613+0000 7f8f27fff640 1 -- 172.21.15.73:0/3751932793 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8f280d1a90 msgr2=0x7f8f281584e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:03.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.613+0000 7f8f27fff640 1 --2- 172.21.15.73:0/3751932793 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8f280d1a90 0x7f8f281584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:03.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.613+0000 7f8f27fff640 1 -- 172.21.15.73:0/3751932793 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f280a0020 con 0x7f8f280d0f20 2024-04-26T20:53:03.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.613+0000 7f8f27fff640 1 --2- 172.21.15.73:0/3751932793 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f280d0f20 0x7f8f280c8080 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f8f18000c00 tx=0x7f8f180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:03.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.613+0000 7f8f24ff9640 1 -- 172.21.15.73:0/3751932793 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f18037040 con 0x7f8f280d0f20 2024-04-26T20:53:03.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.613+0000 7f8f24ff9640 1 -- 172.21.15.73:0/3751932793 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f18042070 con 0x7f8f280d0f20 2024-04-26T20:53:03.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.613+0000 7f8f2e99d640 1 -- 172.21.15.73:0/3751932793 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f280a0300 con 0x7f8f280d0f20 2024-04-26T20:53:03.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.613+0000 7f8f24ff9640 1 -- 172.21.15.73:0/3751932793 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f18030a70 con 0x7f8f280d0f20 2024-04-26T20:53:03.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.613+0000 7f8f2e99d640 1 -- 172.21.15.73:0/3751932793 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f280a08c0 con 0x7f8f280d0f20 2024-04-26T20:53:03.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.614+0000 7f8f2e99d640 1 -- 172.21.15.73:0/3751932793 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f280694f0 con 0x7f8f280d0f20 2024-04-26T20:53:03.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.615+0000 7f8f24ff9640 1 -- 172.21.15.73:0/3751932793 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8f18030c10 con 0x7f8f280d0f20 2024-04-26T20:53:03.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.615+0000 7f8f24ff9640 1 --2- 172.21.15.73:0/3751932793 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f8f1406c8f0 0x7f8f1406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:03.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.615+0000 7f8f24ff9640 1 -- 172.21.15.73:0/3751932793 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6187+0+0 (secure 0 0 0) 0x7f8f180ac6d0 con 0x7f8f280d0f20 2024-04-26T20:53:03.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.615+0000 7f8f277fe640 1 --2- 172.21.15.73:0/3751932793 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f8f1406c8f0 0x7f8f1406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:03.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.616+0000 7f8f277fe640 1 --2- 172.21.15.73:0/3751932793 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f8f1406c8f0 0x7f8f1406edb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c007fd0 tx=0x7f8f1c007920 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:03.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.619+0000 7f8f24ff9640 1 -- 172.21.15.73:0/3751932793 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8f1807a9b0 con 0x7f8f280d0f20 2024-04-26T20:53:03.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.831+0000 7f8f2e99d640 1 -- 172.21.15.73:0/3751932793 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8f280c9e50 con 0x7f8f280d0f20 2024-04-26T20:53:03.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.831+0000 7f8f24ff9640 1 -- 172.21.15.73:0/3751932793 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5182 (secure 0 0 0) 0x7f8f280c9e50 con 0x7f8f280d0f20 2024-04-26T20:53:03.832 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:53:03.833 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":34,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5207,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/256948100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":256948100},{"type":"v1","addr":"172.21.15.120:6833","nonce":256948100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5632,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/4104096099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":4104096099},{"type":"v1","addr":"172.21.15.120:6835","nonce":4104096099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5641,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/272782804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":272782804},{"type":"v1","addr":"172.21.15.73:6837","nonce":272782804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:52:20.754736+0000","modified":"2024-04-26T20:52:57.564091+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6099":{"gid":6099,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":4,"addr":"172.21.15.73:6836/1169682578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1169682578},{"type":"v1","addr":"172.21.15.73:6836","nonce":1169682578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6099,"qdb_cluster":[6099]},"id":3}]} 2024-04-26T20:53:03.833 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 34 2024-04-26T20:53:03.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.833+0000 7f8f2e99d640 1 -- 172.21.15.73:0/3751932793 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f8f1406c8f0 msgr2=0x7f8f1406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:03.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.833+0000 7f8f2e99d640 1 --2- 172.21.15.73:0/3751932793 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f8f1406c8f0 0x7f8f1406edb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c007fd0 tx=0x7f8f1c007920 comp rx=0 tx=0).stop 2024-04-26T20:53:03.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.833+0000 7f8f2e99d640 1 -- 172.21.15.73:0/3751932793 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f280d0f20 msgr2=0x7f8f280c8080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:03.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.833+0000 7f8f2e99d640 1 --2- 172.21.15.73:0/3751932793 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f280d0f20 0x7f8f280c8080 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f8f18000c00 tx=0x7f8f180046c0 comp rx=0 tx=0).stop 2024-04-26T20:53:03.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.833+0000 7f8f2e99d640 1 -- 172.21.15.73:0/3751932793 shutdown_connections 2024-04-26T20:53:03.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.833+0000 7f8f2e99d640 1 --2- 172.21.15.73:0/3751932793 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f8f1406c8f0 0x7f8f1406edb0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:03.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.833+0000 7f8f2e99d640 1 --2- 172.21.15.73:0/3751932793 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8f2815d1e0 0x7f8f28158a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:03.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.833+0000 7f8f2e99d640 1 --2- 172.21.15.73:0/3751932793 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f8f280d1a90 0x7f8f281584e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:03.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.833+0000 7f8f2e99d640 1 --2- 172.21.15.73:0/3751932793 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f280d0f20 0x7f8f280c8080 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:03.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.833+0000 7f8f2e99d640 1 -- 172.21.15.73:0/3751932793 >> 172.21.15.73:0/3751932793 conn(0x7f8f280d2da0 msgr2=0x7f8f280b2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:03.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.833+0000 7f8f2e99d640 1 -- 172.21.15.73:0/3751932793 shutdown_connections 2024-04-26T20:53:03.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.833+0000 7f8f2e99d640 1 -- 172.21.15.73:0/3751932793 wait complete. 2024-04-26T20:53:03.845 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:53:03.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.941+0000 7f0b12015640 1 Processor -- start 2024-04-26T20:53:03.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.941+0000 7f0b12015640 1 -- start start 2024-04-26T20:53:03.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.941+0000 7f0b12015640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f0b0c0d06f0 0x7f0b0c0d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:03.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.941+0000 7f0b12015640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b0c0d0ba0 con 0x7f0b0c0b2850 2024-04-26T20:53:03.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.941+0000 7f0b12015640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b0c0d0e20 con 0x7f0b0c15f3a0 2024-04-26T20:53:03.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.941+0000 7f0b12015640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f0b0c0b1d70 con 0x7f0b0c0d06f0 2024-04-26T20:53:03.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.942+0000 7f0b0bfff640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f0b0c0b2850 0x7f0b0c0b2c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:50754/0 (socket says 172.21.15.73:50754) 2024-04-26T20:53:03.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.942+0000 7f0b0bfff640 1 -- 172.21.15.73:0/1048122783 learned_addr learned my addr 172.21.15.73:0/1048122783 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:53:03.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.942+0000 7f0b0a7fc640 1 -- 172.21.15.73:0/1048122783 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2104963618 0 0) 0x7f0b04001490 con 0x7f0b0c0b2850 2024-04-26T20:53:03.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.942+0000 7f0b0a7fc640 1 -- 172.21.15.73:0/1048122783 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0af4003520 con 0x7f0b0c0b2850 2024-04-26T20:53:03.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.942+0000 7f0b0affd640 1 --2- 172.21.15.73:0/1048122783 >> v2:172.21.15.120:3300/0 conn(0x7f0b0c0d06f0 0x7f0b0c0d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:03.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.942+0000 7f0b0a7fc640 1 -- 172.21.15.73:0/1048122783 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 196822627 0 0) 0x7f0af4003520 con 0x7f0b0c0b2850 2024-04-26T20:53:03.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.942+0000 7f0b0a7fc640 1 -- 172.21.15.73:0/1048122783 >> v2:172.21.15.120:3300/0 conn(0x7f0b0c0d06f0 msgr2=0x7f0b0c0d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:03.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.942+0000 7f0b0a7fc640 1 --2- 172.21.15.73:0/1048122783 >> v2:172.21.15.120:3300/0 conn(0x7f0b0c0d06f0 0x7f0b0c0d1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:03.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.942+0000 7f0b0a7fc640 1 -- 172.21.15.73:0/1048122783 >> v1:172.21.15.120:6789/0 conn(0x7f0b0c15f3a0 legacy=0x7f0b0c0d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:03.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.942+0000 7f0b0a7fc640 1 -- 172.21.15.73:0/1048122783 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b0c0b2050 con 0x7f0b0c0b2850 2024-04-26T20:53:03.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.942+0000 7f0b0a7fc640 1 -- 172.21.15.73:0/1048122783 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7f0b04001bc0 con 0x7f0b0c0b2850 2024-04-26T20:53:03.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.942+0000 7f0b0a7fc640 1 -- 172.21.15.73:0/1048122783 >> v1:172.21.15.73:6789/0 conn(0x7f0b0c0b2850 legacy=0x7f0b0c0b2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:03.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.942+0000 7f0b0a7fc640 1 --2- 172.21.15.73:0/1048122783 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0af40039b0 0x7f0af4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:03.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.942+0000 7f0b0a7fc640 1 --2- 172.21.15.73:0/1048122783 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0af4004800 0x7f0af4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:03.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.942+0000 7f0b0a7fc640 1 --2- 172.21.15.73:0/1048122783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0af40051c0 0x7f0af400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:03.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.942+0000 7f0b0a7fc640 1 -- 172.21.15.73:0/1048122783 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c0b1d70 con 0x7f0af40039b0 2024-04-26T20:53:03.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.942+0000 7f0b0a7fc640 1 -- 172.21.15.73:0/1048122783 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0af400c060 con 0x7f0af40051c0 2024-04-26T20:53:03.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.942+0000 7f0b0a7fc640 1 -- 172.21.15.73:0/1048122783 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f0af400c260 con 0x7f0af4004800 2024-04-26T20:53:03.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.942+0000 7f0b0b7fe640 1 --2- 172.21.15.73:0/1048122783 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0af40039b0 0x7f0af4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:03.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.942+0000 7f0b0affd640 1 --2- 172.21.15.73:0/1048122783 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0af4004800 0x7f0af4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:03.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.942+0000 7f0b0bfff640 1 --2- 172.21.15.73:0/1048122783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0af40051c0 0x7f0af400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:03.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.943+0000 7f0b0b7fe640 1 -- 172.21.15.73:0/1048122783 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0af4004800 msgr2=0x7f0af4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:03.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.943+0000 7f0b0b7fe640 1 --2- 172.21.15.73:0/1048122783 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0af4004800 0x7f0af4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:03.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.943+0000 7f0b0b7fe640 1 -- 172.21.15.73:0/1048122783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0af40051c0 msgr2=0x7f0af400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:03.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.943+0000 7f0b0b7fe640 1 --2- 172.21.15.73:0/1048122783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0af40051c0 0x7f0af400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:03.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.943+0000 7f0b0b7fe640 1 -- 172.21.15.73:0/1048122783 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0af4003520 con 0x7f0af40039b0 2024-04-26T20:53:03.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.943+0000 7f0b0bfff640 1 --2- 172.21.15.73:0/1048122783 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0af40051c0 0x7f0af400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:53:03.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.943+0000 7f0b0bfff640 1 -- 172.21.15.73:0/1048122783 reap_dead start 2024-04-26T20:53:03.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.943+0000 7f0b0b7fe640 1 --2- 172.21.15.73:0/1048122783 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0af40039b0 0x7f0af4003dd0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f0afc001750 tx=0x7f0af4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=433d6519da7c9170 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:03.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.943+0000 7f0b0a7fc640 1 -- 172.21.15.73:0/1048122783 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0afc001370 con 0x7f0af40039b0 2024-04-26T20:53:03.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.943+0000 7f0b0a7fc640 1 -- 172.21.15.73:0/1048122783 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0afc0052d0 con 0x7f0af40039b0 2024-04-26T20:53:03.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.943+0000 7f0b0a7fc640 1 -- 172.21.15.73:0/1048122783 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0afc02abc0 con 0x7f0af40039b0 2024-04-26T20:53:03.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.944+0000 7f0b12015640 1 -- 172.21.15.73:0/1048122783 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0af40039b0 msgr2=0x7f0af4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:03.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.944+0000 7f0b12015640 1 --2- 172.21.15.73:0/1048122783 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0af40039b0 0x7f0af4003dd0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f0afc001750 tx=0x7f0af4005830 comp rx=0 tx=0).stop 2024-04-26T20:53:03.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.944+0000 7f0b12015640 1 -- 172.21.15.73:0/1048122783 shutdown_connections 2024-04-26T20:53:03.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.944+0000 7f0b12015640 1 --2- 172.21.15.73:0/1048122783 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0af40039b0 0x7f0af4003dd0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:03.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.944+0000 7f0b12015640 1 -- 172.21.15.73:0/1048122783 >> 172.21.15.73:0/1048122783 conn(0x7f0b0c0d4f90 msgr2=0x7f0b0c0d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:03.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.944+0000 7f0b12015640 1 -- 172.21.15.73:0/1048122783 shutdown_connections 2024-04-26T20:53:03.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.944+0000 7f0b12015640 1 -- 172.21.15.73:0/1048122783 wait complete. 2024-04-26T20:53:03.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.945+0000 7f0b12015640 1 Processor -- start 2024-04-26T20:53:03.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.945+0000 7f0b12015640 1 -- start start 2024-04-26T20:53:03.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.946+0000 7f0b12015640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0af40039b0 0x7f0b0c159700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:03.952 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.946+0000 7f0b12015640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b0c1f7ac0 0x7f0b0c1f5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:03.953 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.946+0000 7f0b12015640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b0c1f7f40 0x7f0b0c160b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:03.953 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.946+0000 7f0b12015640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c15f900 con 0x7f0b0c1f7ac0 2024-04-26T20:53:03.953 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.946+0000 7f0b12015640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c15f780 con 0x7f0b0c1f7f40 2024-04-26T20:53:03.953 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.946+0000 7f0b12015640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f0b0c0d1770 con 0x7f0af40039b0 2024-04-26T20:53:03.953 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.946+0000 7f0b0affd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b0c1f7ac0 0x7f0b0c1f5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:03.953 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.946+0000 7f0b0affd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b0c1f7ac0 0x7f0b0c1f5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55388/0 (socket says 172.21.15.73:55388) 2024-04-26T20:53:03.953 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.946+0000 7f0b0affd640 1 -- 172.21.15.73:0/1078316077 learned_addr learned my addr 172.21.15.73:0/1078316077 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:03.953 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.946+0000 7f0b0b7fe640 1 --2- 172.21.15.73:0/1078316077 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0af40039b0 0x7f0b0c159700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:03.953 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.946+0000 7f0b0bfff640 1 --2- 172.21.15.73:0/1078316077 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b0c1f7f40 0x7f0b0c160b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:03.953 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.946+0000 7f0b0affd640 1 -- 172.21.15.73:0/1078316077 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0af40039b0 msgr2=0x7f0b0c159700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:03.953 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.946+0000 7f0b0affd640 1 --2- 172.21.15.73:0/1078316077 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0af40039b0 0x7f0b0c159700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:03.953 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.946+0000 7f0b0affd640 1 -- 172.21.15.73:0/1078316077 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b0c1f7f40 msgr2=0x7f0b0c160b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:03.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.946+0000 7f0b0affd640 1 --2- 172.21.15.73:0/1078316077 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b0c1f7f40 0x7f0b0c160b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:03.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.946+0000 7f0b0affd640 1 -- 172.21.15.73:0/1078316077 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b0c1f6a20 con 0x7f0b0c1f7ac0 2024-04-26T20:53:03.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.946+0000 7f0b0b7fe640 1 --2- 172.21.15.73:0/1078316077 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0af40039b0 0x7f0b0c159700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:53:03.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.946+0000 7f0b0affd640 1 --2- 172.21.15.73:0/1078316077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b0c1f7ac0 0x7f0b0c1f5f60 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f0b00011ad0 tx=0x7f0b00011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:03.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.947+0000 7f0af3fff640 1 -- 172.21.15.73:0/1078316077 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0000fb50 con 0x7f0b0c1f7ac0 2024-04-26T20:53:03.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.947+0000 7f0af3fff640 1 -- 172.21.15.73:0/1078316077 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b0000fcf0 con 0x7f0b0c1f7ac0 2024-04-26T20:53:03.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.947+0000 7f0af3fff640 1 -- 172.21.15.73:0/1078316077 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0000fea0 con 0x7f0b0c1f7ac0 2024-04-26T20:53:03.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.947+0000 7f0b12015640 1 -- 172.21.15.73:0/1078316077 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b0c1f6d00 con 0x7f0b0c1f7ac0 2024-04-26T20:53:03.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.947+0000 7f0b12015640 1 -- 172.21.15.73:0/1078316077 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b0c1f72c0 con 0x7f0b0c1f7ac0 2024-04-26T20:53:03.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.948+0000 7f0b12015640 1 -- 172.21.15.73:0/1078316077 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b0c0694f0 con 0x7f0b0c1f7ac0 2024-04-26T20:53:03.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.948+0000 7f0af3fff640 1 -- 172.21.15.73:0/1078316077 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0b00012040 con 0x7f0b0c1f7ac0 2024-04-26T20:53:03.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.948+0000 7f0af3fff640 1 --2- 172.21.15.73:0/1078316077 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f0af806c8f0 0x7f0af806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:03.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.949+0000 7f0af3fff640 1 -- 172.21.15.73:0/1078316077 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6187+0+0 (secure 0 0 0) 0x7f0b0008c330 con 0x7f0b0c1f7ac0 2024-04-26T20:53:03.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.949+0000 7f0b0b7fe640 1 --2- 172.21.15.73:0/1078316077 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f0af806c8f0 0x7f0af806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:03.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.949+0000 7f0b0b7fe640 1 --2- 172.21.15.73:0/1078316077 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f0af806c8f0 0x7f0af806edb0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0afc032000 tx=0x7f0afc02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:03.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:03.953+0000 7f0af3fff640 1 -- 172.21.15.73:0/1078316077 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0b00019030 con 0x7f0b0c1f7ac0 2024-04-26T20:53:04.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.131+0000 7f0b12015640 1 -- 172.21.15.73:0/1078316077 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0b0c0b6860 con 0x7f0b0c1f7ac0 2024-04-26T20:53:04.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.132+0000 7f0af3fff640 1 -- 172.21.15.73:0/1078316077 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) ==== 74+0+17071 (secure 0 0 0) 0x7f0b0005a580 con 0x7f0b0c1f7ac0 2024-04-26T20:53:04.133 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:53:04.133 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":56,"fsid":"fbd3f548-b547-4713-856b-09f98f5d1764","created":"2024-04-26T20:50:07.876786+0000","modified":"2024-04-26T20:52:53.540299+0000","last_up_change":"2024-04-26T20:50:19.082741+0000","last_in_change":"2024-04-26T20:50:10.018400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:50:19.993731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:52:18.960897+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-26T20:52:19.704261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d6f4f3-214a-4392-a02f-c33cbcab3c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6809","nonce":3461705124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6811","nonce":3461705124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6815","nonce":3461705124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6813","nonce":3461705124}]},"public_addr":"172.21.15.73:6809/3461705124","cluster_addr":"172.21.15.73:6811/3461705124","heartbeat_back_addr":"172.21.15.73:6815/3461705124","heartbeat_front_addr":"172.21.15.73:6813/3461705124","state":["exists","up"]},{"osd":1,"uuid":"786bf8ca-6ddc-4b3c-be6c-9152398bd5fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6817","nonce":2827619403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6819","nonce":2827619403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6825","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6827","nonce":2827619403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6821","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6823","nonce":2827619403}]},"public_addr":"172.21.15.73:6817/2827619403","cluster_addr":"172.21.15.73:6819/2827619403","heartbeat_back_addr":"172.21.15.73:6827/2827619403","heartbeat_front_addr":"172.21.15.73:6823/2827619403","state":["exists","up"]},{"osd":2,"uuid":"90963b1a-5437-4ef8-ad8b-6b74c534eeb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6801","nonce":2542681064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6803","nonce":2542681064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6808","nonce":2542681064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6805","nonce":2542681064}]},"public_addr":"172.21.15.73:6801/2542681064","cluster_addr":"172.21.15.73:6803/2542681064","heartbeat_back_addr":"172.21.15.73:6808/2542681064","heartbeat_front_addr":"172.21.15.73:6805/2542681064","state":["exists","up"]},{"osd":3,"uuid":"bd3d2148-6e38-41b4-a453-fe6e3b8b4ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6822","nonce":1604042718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6826","nonce":1604042718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6831","nonce":1604042718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6829","nonce":1604042718}]},"public_addr":"172.21.15.73:6822/1604042718","cluster_addr":"172.21.15.73:6826/1604042718","heartbeat_back_addr":"172.21.15.73:6831/1604042718","heartbeat_front_addr":"172.21.15.73:6829/1604042718","state":["exists","up"]},{"osd":4,"uuid":"e8668dc8-a23f-46d2-8311-cb882da08e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6817","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6819","nonce":3591055561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6824","nonce":3591055561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6831","nonce":3591055561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6828","nonce":3591055561}]},"public_addr":"172.21.15.120:6819/3591055561","cluster_addr":"172.21.15.120:6824/3591055561","heartbeat_back_addr":"172.21.15.120:6831/3591055561","heartbeat_front_addr":"172.21.15.120:6828/3591055561","state":["exists","up"]},{"osd":5,"uuid":"cbac0bdb-96bd-4852-bcce-e564495c3fcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6809","nonce":819450433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6811","nonce":819450433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6815","nonce":819450433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6813","nonce":819450433}]},"public_addr":"172.21.15.120:6809/819450433","cluster_addr":"172.21.15.120:6811/819450433","heartbeat_back_addr":"172.21.15.120:6815/819450433","heartbeat_front_addr":"172.21.15.120:6813/819450433","state":["exists","up"]},{"osd":6,"uuid":"1884b2a8-569f-4272-96a9-f25a09364dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6801","nonce":3946210727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6803","nonce":3946210727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6807","nonce":3946210727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6805","nonce":3946210727}]},"public_addr":"172.21.15.120:6801/3946210727","cluster_addr":"172.21.15.120:6803/3946210727","heartbeat_back_addr":"172.21.15.120:6807/3946210727","heartbeat_front_addr":"172.21.15.120:6805/3946210727","state":["exists","up"]},{"osd":7,"uuid":"c0c3bf6d-a4c5-4f1e-aa2d-330951830666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6818","nonce":2915574674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6821","nonce":2915574674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6827","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6829","nonce":2915574674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6823","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6825","nonce":2915574674}]},"public_addr":"172.21.15.120:6818/2915574674","cluster_addr":"172.21.15.120:6821/2915574674","heartbeat_back_addr":"172.21.15.120:6829/2915574674","heartbeat_front_addr":"172.21.15.120:6825/2915574674","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.218216+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.208710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.112705+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.182789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.164324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.174212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.225396+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.261559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:6837/2770303348":"2024-04-27T20:52:53.540263+0000","172.21.15.73:6836/2770303348":"2024-04-27T20:52:53.540263+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:53:04.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.134+0000 7f0b12015640 1 -- 172.21.15.73:0/1078316077 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f0af806c8f0 msgr2=0x7f0af806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:04.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.134+0000 7f0b12015640 1 --2- 172.21.15.73:0/1078316077 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f0af806c8f0 0x7f0af806edb0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0afc032000 tx=0x7f0afc02e000 comp rx=0 tx=0).stop 2024-04-26T20:53:04.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.134+0000 7f0b12015640 1 -- 172.21.15.73:0/1078316077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b0c1f7ac0 msgr2=0x7f0b0c1f5f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:04.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.134+0000 7f0b12015640 1 --2- 172.21.15.73:0/1078316077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b0c1f7ac0 0x7f0b0c1f5f60 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f0b00011ad0 tx=0x7f0b00011f90 comp rx=0 tx=0).stop 2024-04-26T20:53:04.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.134+0000 7f0b12015640 1 -- 172.21.15.73:0/1078316077 shutdown_connections 2024-04-26T20:53:04.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.134+0000 7f0b12015640 1 --2- 172.21.15.73:0/1078316077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b0c1f7ac0 0x7f0b0c1f5f60 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:04.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.134+0000 7f0b12015640 1 --2- 172.21.15.73:0/1078316077 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f0af806c8f0 0x7f0af806edb0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:04.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.134+0000 7f0b12015640 1 --2- 172.21.15.73:0/1078316077 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f0b0c1f7f40 0x7f0b0c160b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:04.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.134+0000 7f0b12015640 1 --2- 172.21.15.73:0/1078316077 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f0af40039b0 0x7f0b0c159700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:04.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.134+0000 7f0b12015640 1 -- 172.21.15.73:0/1078316077 >> 172.21.15.73:0/1078316077 conn(0x7f0b0c0d4f90 msgr2=0x7f0b0c0d1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:04.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.134+0000 7f0b12015640 1 -- 172.21.15.73:0/1078316077 shutdown_connections 2024-04-26T20:53:04.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.134+0000 7f0b12015640 1 -- 172.21.15.73:0/1078316077 wait complete. 2024-04-26T20:53:04.146 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T20:53:04.146 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:53:04.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.242+0000 7fbf4ac93640 1 Processor -- start 2024-04-26T20:53:04.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.242+0000 7fbf4ac93640 1 -- start start 2024-04-26T20:53:04.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.243+0000 7fbf4ac93640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbf44162810 0x7fbf44070350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:04.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.243+0000 7fbf4ac93640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fbf440b2340 0x7fbf440cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:04.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.243+0000 7fbf4ac93640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbf440cd930 0x7fbf440d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:04.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.243+0000 7fbf4ac93640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7fbf440d0750 con 0x7fbf440cd930 2024-04-26T20:53:04.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.243+0000 7fbf4ac93640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fbf440d16a0 con 0x7fbf440b2340 2024-04-26T20:53:04.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.243+0000 7fbf4ac93640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fbf440d10d0 con 0x7fbf44162810 2024-04-26T20:53:04.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.243+0000 7fbf49230640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbf440cd930 0x7fbf440d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:04.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.243+0000 7fbf49230640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbf440cd930 0x7fbf440d1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55392/0 (socket says 172.21.15.73:55392) 2024-04-26T20:53:04.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.243+0000 7fbf49230640 1 -- 172.21.15.73:0/1899947277 learned_addr learned my addr 172.21.15.73:0/1899947277 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:04.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.243+0000 7fbf48a2f640 1 --2- 172.21.15.73:0/1899947277 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbf44162810 0x7fbf44070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:04.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.243+0000 7fbf43fff640 1 --2- 172.21.15.73:0/1899947277 >> v2:172.21.15.120:3300/0 conn(0x7fbf440b2340 0x7fbf440cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:04.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.243+0000 7fbf49230640 1 -- 172.21.15.73:0/1899947277 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbf44162810 msgr2=0x7fbf44070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:04.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.243+0000 7fbf49230640 1 --2- 172.21.15.73:0/1899947277 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbf44162810 0x7fbf44070350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:04.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.244+0000 7fbf49230640 1 -- 172.21.15.73:0/1899947277 >> v2:172.21.15.120:3300/0 conn(0x7fbf440b2340 msgr2=0x7fbf440cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:04.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.244+0000 7fbf49230640 1 --2- 172.21.15.73:0/1899947277 >> v2:172.21.15.120:3300/0 conn(0x7fbf440b2340 0x7fbf440cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:04.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.244+0000 7fbf49230640 1 -- 172.21.15.73:0/1899947277 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf440d1370 con 0x7fbf440cd930 2024-04-26T20:53:04.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.244+0000 7fbf48a2f640 1 --2- 172.21.15.73:0/1899947277 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbf44162810 0x7fbf44070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:53:04.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.244+0000 7fbf43fff640 1 --2- 172.21.15.73:0/1899947277 >> v2:172.21.15.120:3300/0 conn(0x7fbf440b2340 0x7fbf440cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:53:04.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.244+0000 7fbf49230640 1 --2- 172.21.15.73:0/1899947277 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf440cd930 0x7fbf440d1b30 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fbf4406c4e0 tx=0x7fbf3c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=809b61a4afe3b69 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:04.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.244+0000 7fbf437fe640 1 -- 172.21.15.73:0/1899947277 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c036070 con 0x7fbf440cd930 2024-04-26T20:53:04.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.244+0000 7fbf437fe640 1 -- 172.21.15.73:0/1899947277 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf3c031600 con 0x7fbf440cd930 2024-04-26T20:53:04.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.244+0000 7fbf437fe640 1 -- 172.21.15.73:0/1899947277 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c0319d0 con 0x7fbf440cd930 2024-04-26T20:53:04.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.245+0000 7fbf4ac93640 1 -- 172.21.15.73:0/1899947277 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf440cd930 msgr2=0x7fbf440d1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:04.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.245+0000 7fbf4ac93640 1 --2- 172.21.15.73:0/1899947277 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf440cd930 0x7fbf440d1b30 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fbf4406c4e0 tx=0x7fbf3c02a6a0 comp rx=0 tx=0).stop 2024-04-26T20:53:04.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.245+0000 7fbf4ac93640 1 -- 172.21.15.73:0/1899947277 shutdown_connections 2024-04-26T20:53:04.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.245+0000 7fbf4ac93640 1 --2- 172.21.15.73:0/1899947277 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf440cd930 0x7fbf440d1b30 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:04.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.245+0000 7fbf4ac93640 1 --2- 172.21.15.73:0/1899947277 >> v2:172.21.15.120:3300/0 conn(0x7fbf440b2340 0x7fbf440cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:04.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.245+0000 7fbf4ac93640 1 --2- 172.21.15.73:0/1899947277 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbf44162810 0x7fbf44070350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:04.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.245+0000 7fbf4ac93640 1 -- 172.21.15.73:0/1899947277 >> 172.21.15.73:0/1899947277 conn(0x7fbf440d2da0 msgr2=0x7fbf440d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:04.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.245+0000 7fbf4ac93640 1 -- 172.21.15.73:0/1899947277 shutdown_connections 2024-04-26T20:53:04.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.245+0000 7fbf4ac93640 1 -- 172.21.15.73:0/1899947277 wait complete. 2024-04-26T20:53:04.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.246+0000 7fbf4ac93640 1 Processor -- start 2024-04-26T20:53:04.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.246+0000 7fbf4ac93640 1 -- start start 2024-04-26T20:53:04.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.247+0000 7fbf4ac93640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf440b2340 0x7fbf440cb150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:04.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.247+0000 7fbf4ac93640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbf440cd930 0x7fbf44158420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:04.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.247+0000 7fbf4ac93640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbf44162810 0x7fbf4415eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:04.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.247+0000 7fbf4ac93640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbf44070890 con 0x7fbf440b2340 2024-04-26T20:53:04.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.247+0000 7fbf4ac93640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fbf440d0470 con 0x7fbf440cd930 2024-04-26T20:53:04.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.247+0000 7fbf4ac93640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fbf440d2070 con 0x7fbf44162810 2024-04-26T20:53:04.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.247+0000 7fbf48a2f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf440b2340 0x7fbf440cb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:04.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.247+0000 7fbf48a2f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf440b2340 0x7fbf440cb150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55404/0 (socket says 172.21.15.73:55404) 2024-04-26T20:53:04.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.247+0000 7fbf48a2f640 1 -- 172.21.15.73:0/1793380391 learned_addr learned my addr 172.21.15.73:0/1793380391 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:04.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.247+0000 7fbf49230640 1 --2- 172.21.15.73:0/1793380391 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbf44162810 0x7fbf4415eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:04.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.247+0000 7fbf43fff640 1 --2- 172.21.15.73:0/1793380391 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbf440cd930 0x7fbf44158420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:04.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.247+0000 7fbf48a2f640 1 -- 172.21.15.73:0/1793380391 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbf44162810 msgr2=0x7fbf4415eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:04.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.247+0000 7fbf48a2f640 1 --2- 172.21.15.73:0/1793380391 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbf44162810 0x7fbf4415eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:04.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.247+0000 7fbf48a2f640 1 -- 172.21.15.73:0/1793380391 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbf440cd930 msgr2=0x7fbf44158420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:04.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.247+0000 7fbf48a2f640 1 --2- 172.21.15.73:0/1793380391 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbf440cd930 0x7fbf44158420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:04.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.247+0000 7fbf48a2f640 1 -- 172.21.15.73:0/1793380391 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf441f4f80 con 0x7fbf440b2340 2024-04-26T20:53:04.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.247+0000 7fbf49230640 1 --2- 172.21.15.73:0/1793380391 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbf44162810 0x7fbf4415eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:53:04.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.247+0000 7fbf43fff640 1 --2- 172.21.15.73:0/1793380391 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbf440cd930 0x7fbf44158420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:53:04.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.248+0000 7fbf48a2f640 1 --2- 172.21.15.73:0/1793380391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf440b2340 0x7fbf440cb150 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fbf3400d7d0 tx=0x7fbf3400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:04.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.248+0000 7fbf417fa640 1 -- 172.21.15.73:0/1793380391 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf34004030 con 0x7fbf440b2340 2024-04-26T20:53:04.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.248+0000 7fbf417fa640 1 -- 172.21.15.73:0/1793380391 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf340041d0 con 0x7fbf440b2340 2024-04-26T20:53:04.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.248+0000 7fbf4ac93640 1 -- 172.21.15.73:0/1793380391 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf441f5260 con 0x7fbf440b2340 2024-04-26T20:53:04.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.248+0000 7fbf417fa640 1 -- 172.21.15.73:0/1793380391 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf340096a0 con 0x7fbf440b2340 2024-04-26T20:53:04.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.248+0000 7fbf4ac93640 1 -- 172.21.15.73:0/1793380391 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf441f5820 con 0x7fbf440b2340 2024-04-26T20:53:04.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.249+0000 7fbf4ac93640 1 -- 172.21.15.73:0/1793380391 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf440694f0 con 0x7fbf440b2340 2024-04-26T20:53:04.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.249+0000 7fbf417fa640 1 -- 172.21.15.73:0/1793380391 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbf34009840 con 0x7fbf440b2340 2024-04-26T20:53:04.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.250+0000 7fbf417fa640 1 --2- 172.21.15.73:0/1793380391 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fbf3006c8f0 0x7fbf3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:04.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.250+0000 7fbf417fa640 1 -- 172.21.15.73:0/1793380391 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6187+0+0 (secure 0 0 0) 0x7fbf3408c030 con 0x7fbf440b2340 2024-04-26T20:53:04.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.250+0000 7fbf43fff640 1 --2- 172.21.15.73:0/1793380391 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fbf3006c8f0 0x7fbf3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:04.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.250+0000 7fbf43fff640 1 --2- 172.21.15.73:0/1793380391 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fbf3006c8f0 0x7fbf3006edb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fbf440d36b0 tx=0x7fbf38008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:04.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.254+0000 7fbf417fa640 1 -- 172.21.15.73:0/1793380391 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbf3405a280 con 0x7fbf440b2340 2024-04-26T20:53:04.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.467+0000 7fbf4ac93640 1 -- 172.21.15.73:0/1793380391 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbf440d36e0 con 0x7fbf440b2340 2024-04-26T20:53:04.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.467+0000 7fbf417fa640 1 -- 172.21.15.73:0/1793380391 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5182 (secure 0 0 0) 0x7fbf34059b80 con 0x7fbf440b2340 2024-04-26T20:53:04.469 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:53:04.469 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":34,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5207,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/256948100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":256948100},{"type":"v1","addr":"172.21.15.120:6833","nonce":256948100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5632,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/4104096099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":4104096099},{"type":"v1","addr":"172.21.15.120:6835","nonce":4104096099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5641,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/272782804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":272782804},{"type":"v1","addr":"172.21.15.73:6837","nonce":272782804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:52:20.754736+0000","modified":"2024-04-26T20:52:57.564091+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6099":{"gid":6099,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":4,"addr":"172.21.15.73:6836/1169682578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1169682578},{"type":"v1","addr":"172.21.15.73:6836","nonce":1169682578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6099,"qdb_cluster":[6099]},"id":3}]} 2024-04-26T20:53:04.469 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 34 2024-04-26T20:53:04.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.469+0000 7fbf4ac93640 1 -- 172.21.15.73:0/1793380391 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fbf3006c8f0 msgr2=0x7fbf3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:04.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.469+0000 7fbf4ac93640 1 --2- 172.21.15.73:0/1793380391 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fbf3006c8f0 0x7fbf3006edb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fbf440d36b0 tx=0x7fbf38008040 comp rx=0 tx=0).stop 2024-04-26T20:53:04.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.469+0000 7fbf4ac93640 1 -- 172.21.15.73:0/1793380391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf440b2340 msgr2=0x7fbf440cb150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:04.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.469+0000 7fbf4ac93640 1 --2- 172.21.15.73:0/1793380391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf440b2340 0x7fbf440cb150 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fbf3400d7d0 tx=0x7fbf3400dc90 comp rx=0 tx=0).stop 2024-04-26T20:53:04.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.469+0000 7fbf4ac93640 1 -- 172.21.15.73:0/1793380391 shutdown_connections 2024-04-26T20:53:04.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.469+0000 7fbf4ac93640 1 --2- 172.21.15.73:0/1793380391 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fbf3006c8f0 0x7fbf3006edb0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:04.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.469+0000 7fbf4ac93640 1 --2- 172.21.15.73:0/1793380391 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fbf44162810 0x7fbf4415eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:04.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.469+0000 7fbf4ac93640 1 --2- 172.21.15.73:0/1793380391 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fbf440cd930 0x7fbf44158420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:04.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.469+0000 7fbf4ac93640 1 --2- 172.21.15.73:0/1793380391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf440b2340 0x7fbf440cb150 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:04.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.469+0000 7fbf4ac93640 1 -- 172.21.15.73:0/1793380391 >> 172.21.15.73:0/1793380391 conn(0x7fbf440d2da0 msgr2=0x7fbf440cf3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:04.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.470+0000 7fbf4ac93640 1 -- 172.21.15.73:0/1793380391 shutdown_connections 2024-04-26T20:53:04.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.470+0000 7fbf4ac93640 1 -- 172.21.15.73:0/1793380391 wait complete. 2024-04-26T20:53:04.481 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:53:04.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.578+0000 7fe92dfda640 1 Processor -- start 2024-04-26T20:53:04.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.578+0000 7fe92dfda640 1 -- start start 2024-04-26T20:53:04.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.579+0000 7fe92dfda640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe92815a9f0 0x7fe9280b4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:04.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.579+0000 7fe92dfda640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe9280726e0 0x7fe928072b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:04.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.579+0000 7fe92dfda640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9280ce060 con 0x7fe928161510 2024-04-26T20:53:04.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.579+0000 7fe92dfda640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7fe9280ce620 con 0x7fe92815a9f0 2024-04-26T20:53:04.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.579+0000 7fe92dfda640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fe9280ce340 con 0x7fe9280726e0 2024-04-26T20:53:04.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.579+0000 7fe9277fe640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe92815a9f0 0x7fe9280b4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:04.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.579+0000 7fe9277fe640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe92815a9f0 0x7fe9280b4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55408/0 (socket says 172.21.15.73:55408) 2024-04-26T20:53:04.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.579+0000 7fe9277fe640 1 -- 172.21.15.73:0/2755805062 learned_addr learned my addr 172.21.15.73:0/2755805062 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:04.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.579+0000 7fe927fff640 1 --2- 172.21.15.73:0/2755805062 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe9280726e0 0x7fe928072b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:04.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.579+0000 7fe9277fe640 1 -- 172.21.15.73:0/2755805062 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe9280726e0 msgr2=0x7fe928072b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:04.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.579+0000 7fe9277fe640 1 --2- 172.21.15.73:0/2755805062 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe9280726e0 0x7fe928072b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:04.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.579+0000 7fe9277fe640 1 -- 172.21.15.73:0/2755805062 >> v1:172.21.15.120:6789/0 conn(0x7fe928161510 legacy=0x7fe92815dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:04.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.579+0000 7fe9277fe640 1 -- 172.21.15.73:0/2755805062 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe92815d6b0 con 0x7fe92815a9f0 2024-04-26T20:53:04.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.580+0000 7fe9277fe640 1 --2- 172.21.15.73:0/2755805062 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe92815a9f0 0x7fe9280b4480 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fe918009870 tx=0x7fe91802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75d902339cdcbe83 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:04.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.580+0000 7fe9267fc640 1 -- 172.21.15.73:0/2755805062 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe918037070 con 0x7fe92815a9f0 2024-04-26T20:53:04.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.580+0000 7fe9267fc640 1 -- 172.21.15.73:0/2755805062 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe918032440 con 0x7fe92815a9f0 2024-04-26T20:53:04.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.580+0000 7fe9267fc640 1 -- 172.21.15.73:0/2755805062 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe918032810 con 0x7fe92815a9f0 2024-04-26T20:53:04.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.580+0000 7fe92dfda640 1 -- 172.21.15.73:0/2755805062 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe92815a9f0 msgr2=0x7fe9280b4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:04.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.580+0000 7fe92dfda640 1 --2- 172.21.15.73:0/2755805062 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe92815a9f0 0x7fe9280b4480 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fe918009870 tx=0x7fe91802a210 comp rx=0 tx=0).stop 2024-04-26T20:53:04.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.580+0000 7fe92dfda640 1 -- 172.21.15.73:0/2755805062 shutdown_connections 2024-04-26T20:53:04.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.580+0000 7fe92dfda640 1 --2- 172.21.15.73:0/2755805062 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe9280726e0 0x7fe928072b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:04.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.580+0000 7fe92dfda640 1 --2- 172.21.15.73:0/2755805062 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe92815a9f0 0x7fe9280b4480 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:04.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.580+0000 7fe92dfda640 1 -- 172.21.15.73:0/2755805062 >> 172.21.15.73:0/2755805062 conn(0x7fe9280b3180 msgr2=0x7fe9280b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:04.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.580+0000 7fe92dfda640 1 -- 172.21.15.73:0/2755805062 shutdown_connections 2024-04-26T20:53:04.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.581+0000 7fe92dfda640 1 -- 172.21.15.73:0/2755805062 wait complete. 2024-04-26T20:53:04.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.582+0000 7fe92dfda640 1 Processor -- start 2024-04-26T20:53:04.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.582+0000 7fe92dfda640 1 -- start start 2024-04-26T20:53:04.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.582+0000 7fe92dfda640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe9280726e0 0x7fe9281559a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:04.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.582+0000 7fe92dfda640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe92815a9f0 0x7fe9280cd270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:04.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.582+0000 7fe92dfda640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe928161510 0x7fe9280d0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:04.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.583+0000 7fe92dfda640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe92815e2e0 con 0x7fe92815a9f0 2024-04-26T20:53:04.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.583+0000 7fe92dfda640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fe92815add0 con 0x7fe928161510 2024-04-26T20:53:04.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.583+0000 7fe92dfda640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fe92815e460 con 0x7fe9280726e0 2024-04-26T20:53:04.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.583+0000 7fe926ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe92815a9f0 0x7fe9280cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:04.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.583+0000 7fe926ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe92815a9f0 0x7fe9280cd270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55414/0 (socket says 172.21.15.73:55414) 2024-04-26T20:53:04.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.583+0000 7fe926ffd640 1 -- 172.21.15.73:0/1642938730 learned_addr learned my addr 172.21.15.73:0/1642938730 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:04.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.583+0000 7fe927fff640 1 --2- 172.21.15.73:0/1642938730 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe928161510 0x7fe9280d0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:04.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.583+0000 7fe926ffd640 1 -- 172.21.15.73:0/1642938730 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe9280726e0 msgr2=0x7fe9281559a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:04.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.583+0000 7fe9277fe640 1 --2- 172.21.15.73:0/1642938730 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe9280726e0 0x7fe9281559a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:04.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.583+0000 7fe926ffd640 1 --2- 172.21.15.73:0/1642938730 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe9280726e0 0x7fe9281559a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:04.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.583+0000 7fe926ffd640 1 -- 172.21.15.73:0/1642938730 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe928161510 msgr2=0x7fe9280d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:04.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.583+0000 7fe926ffd640 1 --2- 172.21.15.73:0/1642938730 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe928161510 0x7fe9280d0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:04.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.583+0000 7fe926ffd640 1 -- 172.21.15.73:0/1642938730 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9280cc1f0 con 0x7fe92815a9f0 2024-04-26T20:53:04.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.583+0000 7fe927fff640 1 --2- 172.21.15.73:0/1642938730 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe928161510 0x7fe9280d0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:53:04.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.583+0000 7fe9277fe640 1 --2- 172.21.15.73:0/1642938730 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe9280726e0 0x7fe9281559a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:53:04.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.583+0000 7fe926ffd640 1 --2- 172.21.15.73:0/1642938730 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe92815a9f0 0x7fe9280cd270 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fe91c00c430 tx=0x7fe91c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:04.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.583+0000 7fe90ffff640 1 -- 172.21.15.73:0/1642938730 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91c011790 con 0x7fe92815a9f0 2024-04-26T20:53:04.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.583+0000 7fe90ffff640 1 -- 172.21.15.73:0/1642938730 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe91c017070 con 0x7fe92815a9f0 2024-04-26T20:53:04.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.583+0000 7fe90ffff640 1 -- 172.21.15.73:0/1642938730 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91c011bb0 con 0x7fe92815a9f0 2024-04-26T20:53:04.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.583+0000 7fe92dfda640 1 -- 172.21.15.73:0/1642938730 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9280cc4d0 con 0x7fe92815a9f0 2024-04-26T20:53:04.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.583+0000 7fe92dfda640 1 -- 172.21.15.73:0/1642938730 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9280cd7b0 con 0x7fe92815a9f0 2024-04-26T20:53:04.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.584+0000 7fe92dfda640 1 -- 172.21.15.73:0/1642938730 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe928068fc0 con 0x7fe92815a9f0 2024-04-26T20:53:04.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.585+0000 7fe90ffff640 1 -- 172.21.15.73:0/1642938730 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe91c01c460 con 0x7fe92815a9f0 2024-04-26T20:53:04.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.585+0000 7fe90ffff640 1 --2- 172.21.15.73:0/1642938730 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe91406c920 0x7fe91406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:04.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.585+0000 7fe90ffff640 1 -- 172.21.15.73:0/1642938730 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6187+0+0 (secure 0 0 0) 0x7fe91c08bb20 con 0x7fe92815a9f0 2024-04-26T20:53:04.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.585+0000 7fe9277fe640 1 --2- 172.21.15.73:0/1642938730 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe91406c920 0x7fe91406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:04.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.585+0000 7fe9277fe640 1 --2- 172.21.15.73:0/1642938730 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe91406c920 0x7fe91406ede0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe918034000 tx=0x7fe91802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:04.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.589+0000 7fe90ffff640 1 -- 172.21.15.73:0/1642938730 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe91c059e00 con 0x7fe92815a9f0 2024-04-26T20:53:04.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.803+0000 7fe92dfda640 1 -- 172.21.15.73:0/1642938730 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe9280694f0 con 0x7fe92815a9f0 2024-04-26T20:53:04.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.804+0000 7fe90ffff640 1 -- 172.21.15.73:0/1642938730 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5182 (secure 0 0 0) 0x7fe91c059700 con 0x7fe92815a9f0 2024-04-26T20:53:04.805 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:53:04.805 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":34,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5207,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/256948100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":256948100},{"type":"v1","addr":"172.21.15.120:6833","nonce":256948100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5632,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/4104096099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":4104096099},{"type":"v1","addr":"172.21.15.120:6835","nonce":4104096099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5641,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/272782804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":272782804},{"type":"v1","addr":"172.21.15.73:6837","nonce":272782804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:52:20.754736+0000","modified":"2024-04-26T20:52:57.564091+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6099":{"gid":6099,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":4,"addr":"172.21.15.73:6836/1169682578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1169682578},{"type":"v1","addr":"172.21.15.73:6836","nonce":1169682578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6099,"qdb_cluster":[6099]},"id":3}]} 2024-04-26T20:53:04.805 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 34 2024-04-26T20:53:04.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.806+0000 7fe92dfda640 1 -- 172.21.15.73:0/1642938730 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe91406c920 msgr2=0x7fe91406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:04.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.806+0000 7fe92dfda640 1 --2- 172.21.15.73:0/1642938730 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe91406c920 0x7fe91406ede0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe918034000 tx=0x7fe91802e000 comp rx=0 tx=0).stop 2024-04-26T20:53:04.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.806+0000 7fe92dfda640 1 -- 172.21.15.73:0/1642938730 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe92815a9f0 msgr2=0x7fe9280cd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:04.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.806+0000 7fe92dfda640 1 --2- 172.21.15.73:0/1642938730 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe92815a9f0 0x7fe9280cd270 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fe91c00c430 tx=0x7fe91c00c8f0 comp rx=0 tx=0).stop 2024-04-26T20:53:04.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.806+0000 7fe92dfda640 1 -- 172.21.15.73:0/1642938730 shutdown_connections 2024-04-26T20:53:04.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.806+0000 7fe92dfda640 1 --2- 172.21.15.73:0/1642938730 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe92815a9f0 0x7fe9280cd270 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:04.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.806+0000 7fe92dfda640 1 --2- 172.21.15.73:0/1642938730 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fe91406c920 0x7fe91406ede0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:04.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.806+0000 7fe92dfda640 1 --2- 172.21.15.73:0/1642938730 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fe928161510 0x7fe9280d0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:04.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.806+0000 7fe92dfda640 1 --2- 172.21.15.73:0/1642938730 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fe9280726e0 0x7fe9281559a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:04.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.806+0000 7fe92dfda640 1 -- 172.21.15.73:0/1642938730 >> 172.21.15.73:0/1642938730 conn(0x7fe9280b3180 msgr2=0x7fe928074ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:04.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.806+0000 7fe92dfda640 1 -- 172.21.15.73:0/1642938730 shutdown_connections 2024-04-26T20:53:04.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.806+0000 7fe92dfda640 1 -- 172.21.15.73:0/1642938730 wait complete. 2024-04-26T20:53:04.818 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:53:04.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.913+0000 7fdd181c6640 1 Processor -- start 2024-04-26T20:53:04.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.913+0000 7fdd181c6640 1 -- start start 2024-04-26T20:53:04.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.914+0000 7fdd181c6640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdd1015f3a0 0x7fdd100d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:04.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.914+0000 7fdd181c6640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fdd100d1770 0x7fdd100d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:04.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.914+0000 7fdd181c6640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd100d25a0 con 0x7fdd100d2130 2024-04-26T20:53:04.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.914+0000 7fdd181c6640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fdd100d0e20 con 0x7fdd100d1770 2024-04-26T20:53:04.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.914+0000 7fdd181c6640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fdd100b1d70 con 0x7fdd1015f3a0 2024-04-26T20:53:04.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.914+0000 7fdd16763640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fdd100d2130 0x7fdd100d2e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:50760/0 (socket says 172.21.15.73:50760) 2024-04-26T20:53:04.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.914+0000 7fdd16763640 1 -- 172.21.15.73:0/2344814663 learned_addr learned my addr 172.21.15.73:0/2344814663 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:53:04.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.915+0000 7fdd15f62640 1 --2- 172.21.15.73:0/2344814663 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdd1015f3a0 0x7fdd100d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:04.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.915+0000 7fdd15761640 1 --2- 172.21.15.73:0/2344814663 >> v2:172.21.15.120:3300/0 conn(0x7fdd100d1770 0x7fdd100d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:04.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.915+0000 7fdd14f60640 1 -- 172.21.15.73:0/2344814663 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 512206078 0 0) 0x7fdd08001110 con 0x7fdd100d2130 2024-04-26T20:53:04.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.915+0000 7fdd14f60640 1 -- 172.21.15.73:0/2344814663 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdcfc003520 con 0x7fdd100d2130 2024-04-26T20:53:04.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.915+0000 7fdd14f60640 1 -- 172.21.15.73:0/2344814663 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1371172873 0 0) 0x7fdcfc003520 con 0x7fdd100d2130 2024-04-26T20:53:04.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.915+0000 7fdd14f60640 1 -- 172.21.15.73:0/2344814663 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdd1015f3a0 msgr2=0x7fdd100d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:04.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.915+0000 7fdd14f60640 1 --2- 172.21.15.73:0/2344814663 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdd1015f3a0 0x7fdd100d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:04.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.915+0000 7fdd14f60640 1 -- 172.21.15.73:0/2344814663 >> v2:172.21.15.120:3300/0 conn(0x7fdd100d1770 msgr2=0x7fdd100d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:04.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.915+0000 7fdd14f60640 1 --2- 172.21.15.73:0/2344814663 >> v2:172.21.15.120:3300/0 conn(0x7fdd100d1770 0x7fdd100d1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:04.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.915+0000 7fdd14f60640 1 -- 172.21.15.73:0/2344814663 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd100b2050 con 0x7fdd100d2130 2024-04-26T20:53:04.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.915+0000 7fdd14f60640 1 -- 172.21.15.73:0/2344814663 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7fdd08001840 con 0x7fdd100d2130 2024-04-26T20:53:04.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.915+0000 7fdd14f60640 1 -- 172.21.15.73:0/2344814663 >> v1:172.21.15.73:6789/0 conn(0x7fdd100d2130 legacy=0x7fdd100d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:04.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.915+0000 7fdd14f60640 1 --2- 172.21.15.73:0/2344814663 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdd1015f3a0 0x7fdcfc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:04.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.915+0000 7fdd14f60640 1 --2- 172.21.15.73:0/2344814663 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdcfc003f30 0x7fdcfc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:04.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.915+0000 7fdd14f60640 1 --2- 172.21.15.73:0/2344814663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdcfc004d40 0x7fdcfc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:04.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.915+0000 7fdd14f60640 1 -- 172.21.15.73:0/2344814663 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdd100d0e20 con 0x7fdcfc004d40 2024-04-26T20:53:04.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.915+0000 7fdd14f60640 1 -- 172.21.15.73:0/2344814663 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdd100b1d70 con 0x7fdcfc003f30 2024-04-26T20:53:04.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.915+0000 7fdd14f60640 1 -- 172.21.15.73:0/2344814663 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fdcfc00b9c0 con 0x7fdd1015f3a0 2024-04-26T20:53:04.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.915+0000 7fdd16763640 1 --2- 172.21.15.73:0/2344814663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdcfc004d40 0x7fdcfc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:04.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.915+0000 7fdd15f62640 1 --2- 172.21.15.73:0/2344814663 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdd1015f3a0 0x7fdcfc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:04.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.915+0000 7fdd15761640 1 --2- 172.21.15.73:0/2344814663 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdcfc003f30 0x7fdcfc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:04.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.915+0000 7fdd16763640 1 -- 172.21.15.73:0/2344814663 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdd1015f3a0 msgr2=0x7fdcfc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:04.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.915+0000 7fdd16763640 1 --2- 172.21.15.73:0/2344814663 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdd1015f3a0 0x7fdcfc0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:04.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.915+0000 7fdd16763640 1 -- 172.21.15.73:0/2344814663 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdcfc003f30 msgr2=0x7fdcfc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:04.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.915+0000 7fdd16763640 1 --2- 172.21.15.73:0/2344814663 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdcfc003f30 0x7fdcfc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:04.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.915+0000 7fdd16763640 1 -- 172.21.15.73:0/2344814663 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcfc003520 con 0x7fdcfc004d40 2024-04-26T20:53:04.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.915+0000 7fdd15761640 1 --2- 172.21.15.73:0/2344814663 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdcfc003f30 0x7fdcfc004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:53:04.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.916+0000 7fdd16763640 1 --2- 172.21.15.73:0/2344814663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdcfc004d40 0x7fdcfc0051c0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fdd0800c490 tx=0x7fdcfc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d9b93aceb8c0018 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:04.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.916+0000 7fdd14f60640 1 -- 172.21.15.73:0/2344814663 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd08032020 con 0x7fdcfc004d40 2024-04-26T20:53:04.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.916+0000 7fdd14f60640 1 -- 172.21.15.73:0/2344814663 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd08005ad0 con 0x7fdcfc004d40 2024-04-26T20:53:04.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.916+0000 7fdd14f60640 1 -- 172.21.15.73:0/2344814663 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd0800c0e0 con 0x7fdcfc004d40 2024-04-26T20:53:04.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.917+0000 7fdd181c6640 1 -- 172.21.15.73:0/2344814663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdcfc004d40 msgr2=0x7fdcfc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:04.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.917+0000 7fdd181c6640 1 --2- 172.21.15.73:0/2344814663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdcfc004d40 0x7fdcfc0051c0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fdd0800c490 tx=0x7fdcfc005830 comp rx=0 tx=0).stop 2024-04-26T20:53:04.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.917+0000 7fdd16763640 1 -- 172.21.15.73:0/2344814663 reap_dead start 2024-04-26T20:53:04.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.917+0000 7fdd181c6640 1 -- 172.21.15.73:0/2344814663 shutdown_connections 2024-04-26T20:53:04.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.917+0000 7fdd181c6640 1 -- 172.21.15.73:0/2344814663 >> 172.21.15.73:0/2344814663 conn(0x7fdd100d4f90 msgr2=0x7fdd100d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:04.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.917+0000 7fdd181c6640 1 -- 172.21.15.73:0/2344814663 shutdown_connections 2024-04-26T20:53:04.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.917+0000 7fdd181c6640 1 -- 172.21.15.73:0/2344814663 wait complete. 2024-04-26T20:53:04.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.918+0000 7fdd181c6640 1 Processor -- start 2024-04-26T20:53:04.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.918+0000 7fdd181c6640 1 -- start start 2024-04-26T20:53:04.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.919+0000 7fdd181c6640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdd10162260 0x7fdd100b53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:04.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.919+0000 7fdd181c6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd10152860 0x7fdd100cac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:04.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.919+0000 7fdd181c6640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdd100cb150 0x7fdd100b7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:04.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.919+0000 7fdd181c6640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1015f900 con 0x7fdd10152860 2024-04-26T20:53:04.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.919+0000 7fdd181c6640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1015f780 con 0x7fdd10162260 2024-04-26T20:53:04.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.919+0000 7fdd181c6640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fdd100b3860 con 0x7fdd100cb150 2024-04-26T20:53:04.926 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.919+0000 7fdd15761640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd10152860 0x7fdd100cac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:04.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.919+0000 7fdd15761640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd10152860 0x7fdd100cac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55434/0 (socket says 172.21.15.73:55434) 2024-04-26T20:53:04.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.919+0000 7fdd15761640 1 -- 172.21.15.73:0/3461947036 learned_addr learned my addr 172.21.15.73:0/3461947036 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:04.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.919+0000 7fdd16763640 1 --2- 172.21.15.73:0/3461947036 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdd100cb150 0x7fdd100b7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:04.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.919+0000 7fdd15f62640 1 --2- 172.21.15.73:0/3461947036 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdd10162260 0x7fdd100b53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:04.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.919+0000 7fdd15761640 1 -- 172.21.15.73:0/3461947036 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdd100cb150 msgr2=0x7fdd100b7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:04.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.919+0000 7fdd15761640 1 --2- 172.21.15.73:0/3461947036 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdd100cb150 0x7fdd100b7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:04.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.919+0000 7fdd15761640 1 -- 172.21.15.73:0/3461947036 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdd10162260 msgr2=0x7fdd100b53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:04.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.919+0000 7fdd15761640 1 --2- 172.21.15.73:0/3461947036 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdd10162260 0x7fdd100b53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:04.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.919+0000 7fdd15761640 1 -- 172.21.15.73:0/3461947036 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd1014ff40 con 0x7fdd10152860 2024-04-26T20:53:04.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.919+0000 7fdd15761640 1 --2- 172.21.15.73:0/3461947036 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd10152860 0x7fdd100cac10 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fdd000115d0 tx=0x7fdd00011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:04.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.919+0000 7fdd067fc640 1 -- 172.21.15.73:0/3461947036 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd0000f8f0 con 0x7fdd10152860 2024-04-26T20:53:04.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.919+0000 7fdd067fc640 1 -- 172.21.15.73:0/3461947036 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd0000fa90 con 0x7fdd10152860 2024-04-26T20:53:04.927 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.919+0000 7fdd181c6640 1 -- 172.21.15.73:0/3461947036 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd10150220 con 0x7fdd10152860 2024-04-26T20:53:04.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.919+0000 7fdd067fc640 1 -- 172.21.15.73:0/3461947036 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd0000fe60 con 0x7fdd10152860 2024-04-26T20:53:04.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.919+0000 7fdd181c6640 1 -- 172.21.15.73:0/3461947036 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd101507e0 con 0x7fdd10152860 2024-04-26T20:53:04.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.920+0000 7fdd181c6640 1 -- 172.21.15.73:0/3461947036 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd100cb560 con 0x7fdd10152860 2024-04-26T20:53:04.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.921+0000 7fdd067fc640 1 -- 172.21.15.73:0/3461947036 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdd00015460 con 0x7fdd10152860 2024-04-26T20:53:04.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.921+0000 7fdd067fc640 1 --2- 172.21.15.73:0/3461947036 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fdcf406c8f0 0x7fdcf406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:04.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.921+0000 7fdd067fc640 1 -- 172.21.15.73:0/3461947036 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6187+0+0 (secure 0 0 0) 0x7fdd0008c630 con 0x7fdd10152860 2024-04-26T20:53:04.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.921+0000 7fdd15f62640 1 --2- 172.21.15.73:0/3461947036 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fdcf406c8f0 0x7fdcf406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:04.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.922+0000 7fdd15f62640 1 --2- 172.21.15.73:0/3461947036 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fdcf406c8f0 0x7fdcf406edb0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c007520 tx=0x7fdd0c00b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:04.928 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:04.925+0000 7fdd067fc640 1 -- 172.21.15.73:0/3461947036 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdd100cb560 con 0x7fdd10152860 2024-04-26T20:53:05.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.100+0000 7fdd181c6640 1 -- 172.21.15.73:0/3461947036 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdd100694f0 con 0x7fdd10152860 2024-04-26T20:53:05.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.101+0000 7fdd067fc640 1 -- 172.21.15.73:0/3461947036 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) ==== 74+0+17071 (secure 0 0 0) 0x7fdd00019030 con 0x7fdd10152860 2024-04-26T20:53:05.102 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:53:05.102 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":56,"fsid":"fbd3f548-b547-4713-856b-09f98f5d1764","created":"2024-04-26T20:50:07.876786+0000","modified":"2024-04-26T20:52:53.540299+0000","last_up_change":"2024-04-26T20:50:19.082741+0000","last_in_change":"2024-04-26T20:50:10.018400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:50:19.993731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:52:18.960897+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-26T20:52:19.704261+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d6f4f3-214a-4392-a02f-c33cbcab3c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6809","nonce":3461705124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6811","nonce":3461705124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6815","nonce":3461705124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6813","nonce":3461705124}]},"public_addr":"172.21.15.73:6809/3461705124","cluster_addr":"172.21.15.73:6811/3461705124","heartbeat_back_addr":"172.21.15.73:6815/3461705124","heartbeat_front_addr":"172.21.15.73:6813/3461705124","state":["exists","up"]},{"osd":1,"uuid":"786bf8ca-6ddc-4b3c-be6c-9152398bd5fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6817","nonce":2827619403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6819","nonce":2827619403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6825","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6827","nonce":2827619403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6821","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6823","nonce":2827619403}]},"public_addr":"172.21.15.73:6817/2827619403","cluster_addr":"172.21.15.73:6819/2827619403","heartbeat_back_addr":"172.21.15.73:6827/2827619403","heartbeat_front_addr":"172.21.15.73:6823/2827619403","state":["exists","up"]},{"osd":2,"uuid":"90963b1a-5437-4ef8-ad8b-6b74c534eeb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6801","nonce":2542681064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6803","nonce":2542681064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6808","nonce":2542681064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6805","nonce":2542681064}]},"public_addr":"172.21.15.73:6801/2542681064","cluster_addr":"172.21.15.73:6803/2542681064","heartbeat_back_addr":"172.21.15.73:6808/2542681064","heartbeat_front_addr":"172.21.15.73:6805/2542681064","state":["exists","up"]},{"osd":3,"uuid":"bd3d2148-6e38-41b4-a453-fe6e3b8b4ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6822","nonce":1604042718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6826","nonce":1604042718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6831","nonce":1604042718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6829","nonce":1604042718}]},"public_addr":"172.21.15.73:6822/1604042718","cluster_addr":"172.21.15.73:6826/1604042718","heartbeat_back_addr":"172.21.15.73:6831/1604042718","heartbeat_front_addr":"172.21.15.73:6829/1604042718","state":["exists","up"]},{"osd":4,"uuid":"e8668dc8-a23f-46d2-8311-cb882da08e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6817","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6819","nonce":3591055561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6824","nonce":3591055561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6831","nonce":3591055561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6828","nonce":3591055561}]},"public_addr":"172.21.15.120:6819/3591055561","cluster_addr":"172.21.15.120:6824/3591055561","heartbeat_back_addr":"172.21.15.120:6831/3591055561","heartbeat_front_addr":"172.21.15.120:6828/3591055561","state":["exists","up"]},{"osd":5,"uuid":"cbac0bdb-96bd-4852-bcce-e564495c3fcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6809","nonce":819450433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6811","nonce":819450433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6815","nonce":819450433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6813","nonce":819450433}]},"public_addr":"172.21.15.120:6809/819450433","cluster_addr":"172.21.15.120:6811/819450433","heartbeat_back_addr":"172.21.15.120:6815/819450433","heartbeat_front_addr":"172.21.15.120:6813/819450433","state":["exists","up"]},{"osd":6,"uuid":"1884b2a8-569f-4272-96a9-f25a09364dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6801","nonce":3946210727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6803","nonce":3946210727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6807","nonce":3946210727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6805","nonce":3946210727}]},"public_addr":"172.21.15.120:6801/3946210727","cluster_addr":"172.21.15.120:6803/3946210727","heartbeat_back_addr":"172.21.15.120:6807/3946210727","heartbeat_front_addr":"172.21.15.120:6805/3946210727","state":["exists","up"]},{"osd":7,"uuid":"c0c3bf6d-a4c5-4f1e-aa2d-330951830666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6818","nonce":2915574674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6821","nonce":2915574674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6827","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6829","nonce":2915574674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6823","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6825","nonce":2915574674}]},"public_addr":"172.21.15.120:6818/2915574674","cluster_addr":"172.21.15.120:6821/2915574674","heartbeat_back_addr":"172.21.15.120:6829/2915574674","heartbeat_front_addr":"172.21.15.120:6825/2915574674","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.218216+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.208710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.112705+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.182789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.164324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.174212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.225396+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.261559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:6837/2770303348":"2024-04-27T20:52:53.540263+0000","172.21.15.73:6836/2770303348":"2024-04-27T20:52:53.540263+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:53:05.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.103+0000 7fdd181c6640 1 -- 172.21.15.73:0/3461947036 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fdcf406c8f0 msgr2=0x7fdcf406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:05.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.103+0000 7fdd181c6640 1 --2- 172.21.15.73:0/3461947036 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fdcf406c8f0 0x7fdcf406edb0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c007520 tx=0x7fdd0c00b040 comp rx=0 tx=0).stop 2024-04-26T20:53:05.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.103+0000 7fdd181c6640 1 -- 172.21.15.73:0/3461947036 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd10152860 msgr2=0x7fdd100cac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:05.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.103+0000 7fdd181c6640 1 --2- 172.21.15.73:0/3461947036 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd10152860 0x7fdd100cac10 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fdd000115d0 tx=0x7fdd00011a90 comp rx=0 tx=0).stop 2024-04-26T20:53:05.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.103+0000 7fdd181c6640 1 -- 172.21.15.73:0/3461947036 shutdown_connections 2024-04-26T20:53:05.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.103+0000 7fdd181c6640 1 --2- 172.21.15.73:0/3461947036 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd10152860 0x7fdd100cac10 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:05.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.103+0000 7fdd181c6640 1 --2- 172.21.15.73:0/3461947036 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fdcf406c8f0 0x7fdcf406edb0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:05.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.103+0000 7fdd181c6640 1 --2- 172.21.15.73:0/3461947036 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdd100cb150 0x7fdd100b7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:05.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.103+0000 7fdd181c6640 1 --2- 172.21.15.73:0/3461947036 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdd10162260 0x7fdd100b53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:05.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.103+0000 7fdd181c6640 1 -- 172.21.15.73:0/3461947036 >> 172.21.15.73:0/3461947036 conn(0x7fdd100d4f90 msgr2=0x7fdd100d34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:05.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.103+0000 7fdd181c6640 1 -- 172.21.15.73:0/3461947036 shutdown_connections 2024-04-26T20:53:05.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.103+0000 7fdd181c6640 1 -- 172.21.15.73:0/3461947036 wait complete. 2024-04-26T20:53:05.115 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T20:53:05.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.211+0000 7fd17139d640 1 Processor -- start 2024-04-26T20:53:05.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.211+0000 7fd17139d640 1 -- start start 2024-04-26T20:53:05.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.212+0000 7fd17139d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fd16c15f3a0 0x7fd16c0d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:05.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.212+0000 7fd17139d640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd16c0d2330 0x7fd16c0b2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:05.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.212+0000 7fd17139d640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd16c0d0e20 con 0x7fd16c0d1770 2024-04-26T20:53:05.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.212+0000 7fd17139d640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7fd16c0b1d70 con 0x7fd16c0d2330 2024-04-26T20:53:05.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.212+0000 7fd17139d640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fd16c0b2050 con 0x7fd16c15f3a0 2024-04-26T20:53:05.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.212+0000 7fd16b7fe640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd16c0d2330 0x7fd16c0b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:05.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.212+0000 7fd16b7fe640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd16c0d2330 0x7fd16c0b2850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55450/0 (socket says 172.21.15.73:55450) 2024-04-26T20:53:05.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.212+0000 7fd16b7fe640 1 -- 172.21.15.73:0/3815119541 learned_addr learned my addr 172.21.15.73:0/3815119541 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:05.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.212+0000 7fd16b7fe640 1 -- 172.21.15.73:0/3815119541 >> v2:172.21.15.120:3300/0 conn(0x7fd16c15f3a0 msgr2=0x7fd16c0d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:05.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.212+0000 7fd16affd640 1 --2- 172.21.15.73:0/3815119541 >> v2:172.21.15.120:3300/0 conn(0x7fd16c15f3a0 0x7fd16c0d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:05.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.212+0000 7fd16b7fe640 1 --2- 172.21.15.73:0/3815119541 >> v2:172.21.15.120:3300/0 conn(0x7fd16c15f3a0 0x7fd16c0d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:05.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.212+0000 7fd16b7fe640 1 -- 172.21.15.73:0/3815119541 >> v1:172.21.15.120:6789/0 conn(0x7fd16c0d1770 legacy=0x7fd16c0d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:05.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.212+0000 7fd16b7fe640 1 -- 172.21.15.73:0/3815119541 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd16c0b2330 con 0x7fd16c0d2330 2024-04-26T20:53:05.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.212+0000 7fd16affd640 1 --2- 172.21.15.73:0/3815119541 >> v2:172.21.15.120:3300/0 conn(0x7fd16c15f3a0 0x7fd16c0d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:53:05.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.213+0000 7fd16b7fe640 1 --2- 172.21.15.73:0/3815119541 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd16c0d2330 0x7fd16c0b2850 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fd16c0c89f0 tx=0x7fd16402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f704137dfd2aebef server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:05.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.213+0000 7fd169ffb640 1 -- 172.21.15.73:0/3815119541 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd164037070 con 0x7fd16c0d2330 2024-04-26T20:53:05.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.213+0000 7fd169ffb640 1 -- 172.21.15.73:0/3815119541 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd164032440 con 0x7fd16c0d2330 2024-04-26T20:53:05.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.213+0000 7fd169ffb640 1 -- 172.21.15.73:0/3815119541 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd164032810 con 0x7fd16c0d2330 2024-04-26T20:53:05.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.213+0000 7fd17139d640 1 -- 172.21.15.73:0/3815119541 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd16c0d2330 msgr2=0x7fd16c0b2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:05.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.213+0000 7fd17139d640 1 --2- 172.21.15.73:0/3815119541 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd16c0d2330 0x7fd16c0b2850 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fd16c0c89f0 tx=0x7fd16402a200 comp rx=0 tx=0).stop 2024-04-26T20:53:05.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.213+0000 7fd17139d640 1 -- 172.21.15.73:0/3815119541 shutdown_connections 2024-04-26T20:53:05.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.213+0000 7fd17139d640 1 --2- 172.21.15.73:0/3815119541 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd16c0d2330 0x7fd16c0b2850 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:05.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.214+0000 7fd17139d640 1 --2- 172.21.15.73:0/3815119541 >> v2:172.21.15.120:3300/0 conn(0x7fd16c15f3a0 0x7fd16c0d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:05.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.214+0000 7fd17139d640 1 -- 172.21.15.73:0/3815119541 >> 172.21.15.73:0/3815119541 conn(0x7fd16c0d4f90 msgr2=0x7fd16c0d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:05.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.214+0000 7fd17139d640 1 -- 172.21.15.73:0/3815119541 shutdown_connections 2024-04-26T20:53:05.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.214+0000 7fd17139d640 1 -- 172.21.15.73:0/3815119541 wait complete. 2024-04-26T20:53:05.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.215+0000 7fd17139d640 1 Processor -- start 2024-04-26T20:53:05.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.215+0000 7fd17139d640 1 -- start start 2024-04-26T20:53:05.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.215+0000 7fd17139d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd16c0d1770 0x7fd16c14d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:05.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.216+0000 7fd17139d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd16c0d2330 0x7fd16c1502d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:05.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.216+0000 7fd17139d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd16c15f3a0 0x7fd16c0ba320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:05.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.216+0000 7fd17139d640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd16c0b2f10 con 0x7fd16c0d1770 2024-04-26T20:53:05.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.216+0000 7fd17139d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd16c0b2d90 con 0x7fd16c0d2330 2024-04-26T20:53:05.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.216+0000 7fd17139d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fd16c0b3090 con 0x7fd16c15f3a0 2024-04-26T20:53:05.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.216+0000 7fd16affd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd16c0d1770 0x7fd16c14d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:05.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.216+0000 7fd16affd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd16c0d1770 0x7fd16c14d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55452/0 (socket says 172.21.15.73:55452) 2024-04-26T20:53:05.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.216+0000 7fd16affd640 1 -- 172.21.15.73:0/1570835480 learned_addr learned my addr 172.21.15.73:0/1570835480 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:05.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.216+0000 7fd16b7fe640 1 --2- 172.21.15.73:0/1570835480 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd16c15f3a0 0x7fd16c0ba320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:05.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.216+0000 7fd16a7fc640 1 --2- 172.21.15.73:0/1570835480 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd16c0d2330 0x7fd16c1502d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:05.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.216+0000 7fd16affd640 1 -- 172.21.15.73:0/1570835480 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd16c15f3a0 msgr2=0x7fd16c0ba320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:05.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.216+0000 7fd16affd640 1 --2- 172.21.15.73:0/1570835480 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd16c15f3a0 0x7fd16c0ba320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:05.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.216+0000 7fd16affd640 1 -- 172.21.15.73:0/1570835480 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd16c0d2330 msgr2=0x7fd16c1502d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:05.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.216+0000 7fd16affd640 1 --2- 172.21.15.73:0/1570835480 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd16c0d2330 0x7fd16c1502d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:05.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.216+0000 7fd16affd640 1 -- 172.21.15.73:0/1570835480 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd16c0baa50 con 0x7fd16c0d1770 2024-04-26T20:53:05.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.216+0000 7fd16a7fc640 1 --2- 172.21.15.73:0/1570835480 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd16c0d2330 0x7fd16c1502d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:53:05.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.216+0000 7fd16affd640 1 --2- 172.21.15.73:0/1570835480 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd16c0d1770 0x7fd16c14d080 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fd15c00ce00 tx=0x7fd15c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:05.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.216+0000 7fd1537fe640 1 -- 172.21.15.73:0/1570835480 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15c00f890 con 0x7fd16c0d1770 2024-04-26T20:53:05.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.216+0000 7fd1537fe640 1 -- 172.21.15.73:0/1570835480 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd15c00fa30 con 0x7fd16c0d1770 2024-04-26T20:53:05.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.216+0000 7fd17139d640 1 -- 172.21.15.73:0/1570835480 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd16c0bad30 con 0x7fd16c0d1770 2024-04-26T20:53:05.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.216+0000 7fd1537fe640 1 -- 172.21.15.73:0/1570835480 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15c00fe40 con 0x7fd16c0d1770 2024-04-26T20:53:05.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.216+0000 7fd17139d640 1 -- 172.21.15.73:0/1570835480 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd16c14da60 con 0x7fd16c0d1770 2024-04-26T20:53:05.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.217+0000 7fd17139d640 1 -- 172.21.15.73:0/1570835480 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd16c0694f0 con 0x7fd16c0d1770 2024-04-26T20:53:05.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.218+0000 7fd1537fe640 1 -- 172.21.15.73:0/1570835480 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd15c018400 con 0x7fd16c0d1770 2024-04-26T20:53:05.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.218+0000 7fd1537fe640 1 --2- 172.21.15.73:0/1570835480 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd15806c8f0 0x7fd15806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:05.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.218+0000 7fd1537fe640 1 -- 172.21.15.73:0/1570835480 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6187+0+0 (secure 0 0 0) 0x7fd15c08cb60 con 0x7fd16c0d1770 2024-04-26T20:53:05.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.218+0000 7fd16a7fc640 1 --2- 172.21.15.73:0/1570835480 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd15806c8f0 0x7fd15806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:05.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.218+0000 7fd16a7fc640 1 --2- 172.21.15.73:0/1570835480 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd15806c8f0 0x7fd15806edb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd16c1f87b0 tx=0x7fd160009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:05.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.222+0000 7fd1537fe640 1 -- 172.21.15.73:0/1570835480 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd15c014030 con 0x7fd16c0d1770 2024-04-26T20:53:05.435 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.434+0000 7fd17139d640 1 -- 172.21.15.73:0/1570835480 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd16c0b6860 con 0x7fd16c0d1770 2024-04-26T20:53:05.435 INFO:tasks.ceph.mon.a.smithi073.stderr:2024-04-26T20:53:05.434+0000 7f712aaa2640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T20:53:05.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.436+0000 7fd1537fe640 1 -- 172.21.15.73:0/1570835480 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v35) ==== 161+0+0 (secure 0 0 0) 0x7fd15c00ae20 con 0x7fd16c0d1770 2024-04-26T20:53:05.436 INFO:teuthology.orchestra.run.smithi073.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T20:53:05.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.438+0000 7fd17139d640 1 -- 172.21.15.73:0/1570835480 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd15806c8f0 msgr2=0x7fd15806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:05.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.438+0000 7fd17139d640 1 --2- 172.21.15.73:0/1570835480 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd15806c8f0 0x7fd15806edb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd16c1f87b0 tx=0x7fd160009040 comp rx=0 tx=0).stop 2024-04-26T20:53:05.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.438+0000 7fd17139d640 1 -- 172.21.15.73:0/1570835480 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd16c0d1770 msgr2=0x7fd16c14d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:05.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.438+0000 7fd17139d640 1 --2- 172.21.15.73:0/1570835480 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd16c0d1770 0x7fd16c14d080 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fd15c00ce00 tx=0x7fd15c00a650 comp rx=0 tx=0).stop 2024-04-26T20:53:05.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.439+0000 7fd17139d640 1 -- 172.21.15.73:0/1570835480 shutdown_connections 2024-04-26T20:53:05.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.439+0000 7fd17139d640 1 --2- 172.21.15.73:0/1570835480 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd15806c8f0 0x7fd15806edb0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:05.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.439+0000 7fd17139d640 1 --2- 172.21.15.73:0/1570835480 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd16c15f3a0 0x7fd16c0ba320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:05.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.439+0000 7fd17139d640 1 --2- 172.21.15.73:0/1570835480 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd16c0d2330 0x7fd16c1502d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:05.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.439+0000 7fd17139d640 1 --2- 172.21.15.73:0/1570835480 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd16c0d1770 0x7fd16c14d080 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:05.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.439+0000 7fd17139d640 1 -- 172.21.15.73:0/1570835480 >> 172.21.15.73:0/1570835480 conn(0x7fd16c0d4f90 msgr2=0x7fd16c0d1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:05.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.439+0000 7fd17139d640 1 -- 172.21.15.73:0/1570835480 shutdown_connections 2024-04-26T20:53:05.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.439+0000 7fd17139d640 1 -- 172.21.15.73:0/1570835480 wait complete. 2024-04-26T20:53:05.441 INFO:tasks.ceph.mds.c.smithi073.stderr: -402> 2024-04-26T20:52:59.557+0000 7f5ee69bf640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T20:53:05.451 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T20:53:05.465 INFO:tasks.ceph.mds.c.smithi073.stdout:starting mds.c at 2024-04-26T20:53:05.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.549+0000 7eff365a4640 1 Processor -- start 2024-04-26T20:53:05.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.549+0000 7eff365a4640 1 -- start start 2024-04-26T20:53:05.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.550+0000 7eff365a4640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7eff30156460 0x7eff300cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:05.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.550+0000 7eff365a4640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7eff3006fea0 0x7eff30072c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:05.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.550+0000 7eff365a4640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff30156840 con 0x7eff300cba10 2024-04-26T20:53:05.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.550+0000 7eff365a4640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7eff300cbf20 con 0x7eff30156460 2024-04-26T20:53:05.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.550+0000 7eff365a4640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7eff300cc200 con 0x7eff3006fea0 2024-04-26T20:53:05.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.550+0000 7eff2f7fe640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7eff300cba10 0x7eff3006f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:50764/0 (socket says 172.21.15.73:50764) 2024-04-26T20:53:05.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.550+0000 7eff2f7fe640 1 -- 172.21.15.73:0/2224900811 learned_addr learned my addr 172.21.15.73:0/2224900811 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:53:05.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.550+0000 7eff2effd640 1 -- 172.21.15.73:0/2224900811 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2824847688 0 0) 0x7eff30156840 con 0x7eff300cba10 2024-04-26T20:53:05.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.550+0000 7eff2effd640 1 -- 172.21.15.73:0/2224900811 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7eff18005730 con 0x7eff300cba10 2024-04-26T20:53:05.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.550+0000 7eff34b41640 1 --2- 172.21.15.73:0/2224900811 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7eff3006fea0 0x7eff30072c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:05.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.550+0000 7eff2ffff640 1 --2- 172.21.15.73:0/2224900811 >> v2:172.21.15.120:3300/0 conn(0x7eff30156460 0x7eff300cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:05.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.550+0000 7eff2effd640 1 -- 172.21.15.73:0/2224900811 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2926129106 0 0) 0x7eff18005730 con 0x7eff300cba10 2024-04-26T20:53:05.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.550+0000 7eff2effd640 1 -- 172.21.15.73:0/2224900811 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7eff3006fea0 msgr2=0x7eff30072c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:05.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.550+0000 7eff2effd640 1 --2- 172.21.15.73:0/2224900811 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7eff3006fea0 0x7eff30072c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:05.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.550+0000 7eff2effd640 1 -- 172.21.15.73:0/2224900811 >> v2:172.21.15.120:3300/0 conn(0x7eff30156460 msgr2=0x7eff300cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:05.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.550+0000 7eff2effd640 1 --2- 172.21.15.73:0/2224900811 >> v2:172.21.15.120:3300/0 conn(0x7eff30156460 0x7eff300cb4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:05.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.550+0000 7eff2effd640 1 -- 172.21.15.73:0/2224900811 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff300cc4e0 con 0x7eff300cba10 2024-04-26T20:53:05.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.550+0000 7eff2effd640 1 -- 172.21.15.73:0/2224900811 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7eff24001670 con 0x7eff300cba10 2024-04-26T20:53:05.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.550+0000 7eff2effd640 1 -- 172.21.15.73:0/2224900811 >> v1:172.21.15.73:6789/0 conn(0x7eff300cba10 legacy=0x7eff3006f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:05.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.550+0000 7eff2effd640 1 --2- 172.21.15.73:0/2224900811 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7eff18005bc0 0x7eff18005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:05.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.550+0000 7eff2effd640 1 --2- 172.21.15.73:0/2224900811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff18006a10 0x7eff18006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:05.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.550+0000 7eff2effd640 1 --2- 172.21.15.73:0/2224900811 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff180073d0 0x7eff1800dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:05.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.551+0000 7eff2effd640 1 -- 172.21.15.73:0/2224900811 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7eff300cbf20 con 0x7eff18006a10 2024-04-26T20:53:05.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.551+0000 7eff2effd640 1 -- 172.21.15.73:0/2224900811 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7eff300cc200 con 0x7eff180073d0 2024-04-26T20:53:05.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.551+0000 7eff2effd640 1 -- 172.21.15.73:0/2224900811 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7eff1800e210 con 0x7eff18005bc0 2024-04-26T20:53:05.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.551+0000 7eff34b41640 1 --2- 172.21.15.73:0/2224900811 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7eff3006fea0 0x7eff30072c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:53:05.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.551+0000 7eff2ffff640 1 --2- 172.21.15.73:0/2224900811 >> v2:172.21.15.120:3300/0 conn(0x7eff30156460 0x7eff300cb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:53:05.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.551+0000 7eff2f7fe640 1 --2- 172.21.15.73:0/2224900811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff18006a10 0x7eff18006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:05.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.551+0000 7eff34b41640 1 --2- 172.21.15.73:0/2224900811 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff180073d0 0x7eff1800dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:05.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.551+0000 7eff2ffff640 1 --2- 172.21.15.73:0/2224900811 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7eff18005bc0 0x7eff18005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:05.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.551+0000 7eff2f7fe640 1 -- 172.21.15.73:0/2224900811 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7eff18005bc0 msgr2=0x7eff18005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:05.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.551+0000 7eff2f7fe640 1 --2- 172.21.15.73:0/2224900811 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7eff18005bc0 0x7eff18005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:05.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.551+0000 7eff2f7fe640 1 -- 172.21.15.73:0/2224900811 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff180073d0 msgr2=0x7eff1800dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:05.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.551+0000 7eff2f7fe640 1 --2- 172.21.15.73:0/2224900811 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff180073d0 0x7eff1800dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:05.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.551+0000 7eff2f7fe640 1 -- 172.21.15.73:0/2224900811 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff18005730 con 0x7eff18006a10 2024-04-26T20:53:05.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.551+0000 7eff2ffff640 1 --2- 172.21.15.73:0/2224900811 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7eff18005bc0 0x7eff18005fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:53:05.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.552+0000 7eff2f7fe640 1 --2- 172.21.15.73:0/2224900811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff18006a10 0x7eff18006e90 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7eff2400c270 tx=0x7eff18007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8032424dc67721f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:05.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.552+0000 7eff2effd640 1 -- 172.21.15.73:0/2224900811 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff24007d30 con 0x7eff18006a10 2024-04-26T20:53:05.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.552+0000 7eff2effd640 1 -- 172.21.15.73:0/2224900811 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff24005ad0 con 0x7eff18006a10 2024-04-26T20:53:05.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.552+0000 7eff2effd640 1 -- 172.21.15.73:0/2224900811 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff24007070 con 0x7eff18006a10 2024-04-26T20:53:05.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.552+0000 7eff365a4640 1 -- 172.21.15.73:0/2224900811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff18006a10 msgr2=0x7eff18006e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:05.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.552+0000 7eff365a4640 1 --2- 172.21.15.73:0/2224900811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff18006a10 0x7eff18006e90 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7eff2400c270 tx=0x7eff18007a40 comp rx=0 tx=0).stop 2024-04-26T20:53:05.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.552+0000 7eff34b41640 1 -- 172.21.15.73:0/2224900811 reap_dead start 2024-04-26T20:53:05.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.552+0000 7eff365a4640 1 -- 172.21.15.73:0/2224900811 shutdown_connections 2024-04-26T20:53:05.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.552+0000 7eff365a4640 1 -- 172.21.15.73:0/2224900811 >> 172.21.15.73:0/2224900811 conn(0x7eff300d0be0 msgr2=0x7eff300d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:05.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.552+0000 7eff365a4640 1 -- 172.21.15.73:0/2224900811 shutdown_connections 2024-04-26T20:53:05.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.552+0000 7eff365a4640 1 -- 172.21.15.73:0/2224900811 wait complete. 2024-04-26T20:53:05.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.553+0000 7eff365a4640 1 Processor -- start 2024-04-26T20:53:05.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.554+0000 7eff365a4640 1 -- start start 2024-04-26T20:53:05.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.554+0000 7eff365a4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff18006a10 0x7eff30151560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:05.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.554+0000 7eff365a4640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7eff3015eb30 0x7eff3015efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:05.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.554+0000 7eff365a4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff3014dcb0 0x7eff3014e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:05.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.554+0000 7eff365a4640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7eff3006e260 con 0x7eff3014dcb0 2024-04-26T20:53:05.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.554+0000 7eff365a4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7eff30072700 con 0x7eff18006a10 2024-04-26T20:53:05.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.554+0000 7eff365a4640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7eff3006e3e0 con 0x7eff3015eb30 2024-04-26T20:53:05.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.554+0000 7eff34b41640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff3014dcb0 0x7eff3014e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:05.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.554+0000 7eff34b41640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff3014dcb0 0x7eff3014e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55478/0 (socket says 172.21.15.73:55478) 2024-04-26T20:53:05.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.554+0000 7eff34b41640 1 -- 172.21.15.73:0/3417918881 learned_addr learned my addr 172.21.15.73:0/3417918881 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:05.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.555+0000 7eff2f7fe640 1 --2- 172.21.15.73:0/3417918881 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7eff3015eb30 0x7eff3015efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:05.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.555+0000 7eff2ffff640 1 --2- 172.21.15.73:0/3417918881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff18006a10 0x7eff30151560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:05.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.555+0000 7eff34b41640 1 -- 172.21.15.73:0/3417918881 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7eff3015eb30 msgr2=0x7eff3015efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:05.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.555+0000 7eff34b41640 1 --2- 172.21.15.73:0/3417918881 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7eff3015eb30 0x7eff3015efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:05.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.555+0000 7eff34b41640 1 -- 172.21.15.73:0/3417918881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff18006a10 msgr2=0x7eff30151560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:05.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.555+0000 7eff34b41640 1 --2- 172.21.15.73:0/3417918881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff18006a10 0x7eff30151560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:05.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.555+0000 7eff34b41640 1 -- 172.21.15.73:0/3417918881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff301f1850 con 0x7eff3014dcb0 2024-04-26T20:53:05.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.555+0000 7eff34b41640 1 --2- 172.21.15.73:0/3417918881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff3014dcb0 0x7eff3014e130 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7eff300cb1e0 tx=0x7eff28010530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:05.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.555+0000 7eff2cff9640 1 -- 172.21.15.73:0/3417918881 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2801c070 con 0x7eff3014dcb0 2024-04-26T20:53:05.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.555+0000 7eff2cff9640 1 -- 172.21.15.73:0/3417918881 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff28010cf0 con 0x7eff3014dcb0 2024-04-26T20:53:05.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.555+0000 7eff365a4640 1 -- 172.21.15.73:0/3417918881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff3015d280 con 0x7eff3014dcb0 2024-04-26T20:53:05.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.555+0000 7eff2cff9640 1 -- 172.21.15.73:0/3417918881 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff28010ea0 con 0x7eff3014dcb0 2024-04-26T20:53:05.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.555+0000 7eff365a4640 1 -- 172.21.15.73:0/3417918881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff301f1da0 con 0x7eff3014dcb0 2024-04-26T20:53:05.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.556+0000 7eff365a4640 1 -- 172.21.15.73:0/3417918881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff301f1fa0 con 0x7eff3014dcb0 2024-04-26T20:53:05.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.556+0000 7eff2cff9640 1 -- 172.21.15.73:0/3417918881 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7eff2801f070 con 0x7eff3014dcb0 2024-04-26T20:53:05.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.557+0000 7eff2cff9640 1 --2- 172.21.15.73:0/3417918881 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7eff1c06c8f0 0x7eff1c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:05.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.557+0000 7eff2cff9640 1 -- 172.21.15.73:0/3417918881 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 6273+0+0 (secure 0 0 0) 0x7eff2808be70 con 0x7eff3014dcb0 2024-04-26T20:53:05.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.557+0000 7eff2ffff640 1 --2- 172.21.15.73:0/3417918881 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7eff1c06c8f0 0x7eff1c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:05.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.557+0000 7eff2ffff640 1 --2- 172.21.15.73:0/3417918881 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7eff1c06c8f0 0x7eff1c06edb0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7eff20006fd0 tx=0x7eff20008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:05.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.561+0000 7eff2cff9640 1 -- 172.21.15.73:0/3417918881 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7eff301f1fa0 con 0x7eff3014dcb0 2024-04-26T20:53:05.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:05.775+0000 7eff365a4640 1 -- 172.21.15.73:0/3417918881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7eff300694f0 con 0x7eff3014dcb0 2024-04-26T20:53:06.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.437+0000 7eff2cff9640 1 -- 172.21.15.73:0/3417918881 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v36) ==== 104+0+0 (secure 0 0 0) 0x7eff2805a060 con 0x7eff3014dcb0 2024-04-26T20:53:06.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.439+0000 7eff365a4640 1 -- 172.21.15.73:0/3417918881 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7eff1c06c8f0 msgr2=0x7eff1c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:06.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.439+0000 7eff365a4640 1 --2- 172.21.15.73:0/3417918881 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7eff1c06c8f0 0x7eff1c06edb0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7eff20006fd0 tx=0x7eff20008040 comp rx=0 tx=0).stop 2024-04-26T20:53:06.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.439+0000 7eff365a4640 1 -- 172.21.15.73:0/3417918881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff3014dcb0 msgr2=0x7eff3014e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:06.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.439+0000 7eff365a4640 1 --2- 172.21.15.73:0/3417918881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff3014dcb0 0x7eff3014e130 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7eff300cb1e0 tx=0x7eff28010530 comp rx=0 tx=0).stop 2024-04-26T20:53:06.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.439+0000 7eff365a4640 1 -- 172.21.15.73:0/3417918881 shutdown_connections 2024-04-26T20:53:06.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.439+0000 7eff365a4640 1 --2- 172.21.15.73:0/3417918881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff3014dcb0 0x7eff3014e130 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:06.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.439+0000 7eff365a4640 1 --2- 172.21.15.73:0/3417918881 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7eff1c06c8f0 0x7eff1c06edb0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:06.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.439+0000 7eff365a4640 1 --2- 172.21.15.73:0/3417918881 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7eff3015eb30 0x7eff3015efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:06.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.439+0000 7eff365a4640 1 --2- 172.21.15.73:0/3417918881 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7eff18006a10 0x7eff30151560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:06.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.439+0000 7eff365a4640 1 -- 172.21.15.73:0/3417918881 >> 172.21.15.73:0/3417918881 conn(0x7eff300d0be0 msgr2=0x7eff300b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:06.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.440+0000 7eff365a4640 1 -- 172.21.15.73:0/3417918881 shutdown_connections 2024-04-26T20:53:06.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.440+0000 7eff365a4640 1 -- 172.21.15.73:0/3417918881 wait complete. 2024-04-26T20:53:06.453 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T20:53:06.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.594+0000 7ff9bd2d1640 1 Processor -- start 2024-04-26T20:53:06.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.594+0000 7ff9bd2d1640 1 -- start start 2024-04-26T20:53:06.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.595+0000 7ff9bd2d1640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff9b8162860 0x7ff9b80b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:06.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.595+0000 7ff9bd2d1640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff9b80b3f80 0x7ff9b80d3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:06.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.595+0000 7ff9bd2d1640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9b80d5880 con 0x7ff9b8162c40 2024-04-26T20:53:06.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.595+0000 7ff9bd2d1640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7ff9b80d5b60 con 0x7ff9b8162860 2024-04-26T20:53:06.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.595+0000 7ff9bd2d1640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7ff9b80d5e40 con 0x7ff9b80b3f80 2024-04-26T20:53:06.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.595+0000 7ff9b6ffd640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff9b8162860 0x7ff9b80b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:06.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.595+0000 7ff9b6ffd640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff9b8162860 0x7ff9b80b1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55490/0 (socket says 172.21.15.73:55490) 2024-04-26T20:53:06.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.595+0000 7ff9b6ffd640 1 -- 172.21.15.73:0/2108039279 learned_addr learned my addr 172.21.15.73:0/2108039279 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:06.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.595+0000 7ff9b6ffd640 1 -- 172.21.15.73:0/2108039279 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff9b80b3f80 msgr2=0x7ff9b80d3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:06.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.595+0000 7ff9b77fe640 1 --2- 172.21.15.73:0/2108039279 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff9b80b3f80 0x7ff9b80d3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:06.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.595+0000 7ff9b6ffd640 1 --2- 172.21.15.73:0/2108039279 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff9b80b3f80 0x7ff9b80d3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:06.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.595+0000 7ff9b6ffd640 1 -- 172.21.15.73:0/2108039279 >> v1:172.21.15.120:6789/0 conn(0x7ff9b8162c40 legacy=0x7ff9b80d2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:06.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.596+0000 7ff9b6ffd640 1 -- 172.21.15.73:0/2108039279 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9b80cfc70 con 0x7ff9b8162860 2024-04-26T20:53:06.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.596+0000 7ff9b77fe640 1 --2- 172.21.15.73:0/2108039279 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff9b80b3f80 0x7ff9b80d3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:53:06.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.596+0000 7ff9b6ffd640 1 --2- 172.21.15.73:0/2108039279 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9b8162860 0x7ff9b80b1d40 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8009e50 tx=0x7ff9a802a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=328c6a82f7188a04 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:06.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.598+0000 7ff9b5ffb640 1 -- 172.21.15.73:0/2108039279 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a8036070 con 0x7ff9b8162860 2024-04-26T20:53:06.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.599+0000 7ff9b5ffb640 1 -- 172.21.15.73:0/2108039279 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9a8031600 con 0x7ff9b8162860 2024-04-26T20:53:06.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.599+0000 7ff9b5ffb640 1 -- 172.21.15.73:0/2108039279 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a80319d0 con 0x7ff9b8162860 2024-04-26T20:53:06.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.599+0000 7ff9bd2d1640 1 -- 172.21.15.73:0/2108039279 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9b8162860 msgr2=0x7ff9b80b1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:06.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.599+0000 7ff9bd2d1640 1 --2- 172.21.15.73:0/2108039279 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9b8162860 0x7ff9b80b1d40 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8009e50 tx=0x7ff9a802a630 comp rx=0 tx=0).stop 2024-04-26T20:53:06.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.599+0000 7ff9bd2d1640 1 -- 172.21.15.73:0/2108039279 shutdown_connections 2024-04-26T20:53:06.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.599+0000 7ff9bd2d1640 1 --2- 172.21.15.73:0/2108039279 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff9b80b3f80 0x7ff9b80d3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:06.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.599+0000 7ff9bd2d1640 1 --2- 172.21.15.73:0/2108039279 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9b8162860 0x7ff9b80b1d40 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:06.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.599+0000 7ff9bd2d1640 1 -- 172.21.15.73:0/2108039279 >> 172.21.15.73:0/2108039279 conn(0x7ff9b809e3e0 msgr2=0x7ff9b80d50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:06.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.599+0000 7ff9bd2d1640 1 -- 172.21.15.73:0/2108039279 shutdown_connections 2024-04-26T20:53:06.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.599+0000 7ff9bd2d1640 1 -- 172.21.15.73:0/2108039279 wait complete. 2024-04-26T20:53:06.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.601+0000 7ff9bd2d1640 1 Processor -- start 2024-04-26T20:53:06.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.601+0000 7ff9bd2d1640 1 -- start start 2024-04-26T20:53:06.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.601+0000 7ff9bd2d1640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff9b80b3f80 0x7ff9b80ba630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:06.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.601+0000 7ff9bd2d1640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9b8162860 0x7ff9b8157140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:06.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.601+0000 7ff9bd2d1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9b8162c40 0x7ff9b80c8be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:06.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.601+0000 7ff9bd2d1640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b80d3ce0 con 0x7ff9b8162c40 2024-04-26T20:53:06.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.601+0000 7ff9bd2d1640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b80d3b60 con 0x7ff9b8162860 2024-04-26T20:53:06.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.601+0000 7ff9bd2d1640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7ff9b80d0570 con 0x7ff9b80b3f80 2024-04-26T20:53:06.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.601+0000 7ff9b77fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9b8162c40 0x7ff9b80c8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:06.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.601+0000 7ff9b77fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9b8162c40 0x7ff9b80c8be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55502/0 (socket says 172.21.15.73:55502) 2024-04-26T20:53:06.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.601+0000 7ff9b77fe640 1 -- 172.21.15.73:0/2791300042 learned_addr learned my addr 172.21.15.73:0/2791300042 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:06.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.602+0000 7ff9b6ffd640 1 --2- 172.21.15.73:0/2791300042 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff9b80b3f80 0x7ff9b80ba630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:06.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.602+0000 7ff9b67fc640 1 --2- 172.21.15.73:0/2791300042 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9b8162860 0x7ff9b8157140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:06.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.602+0000 7ff9b77fe640 1 -- 172.21.15.73:0/2791300042 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff9b80b3f80 msgr2=0x7ff9b80ba630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:06.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.602+0000 7ff9b77fe640 1 --2- 172.21.15.73:0/2791300042 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff9b80b3f80 0x7ff9b80ba630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:06.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.602+0000 7ff9b77fe640 1 -- 172.21.15.73:0/2791300042 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9b8162860 msgr2=0x7ff9b8157140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:06.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.602+0000 7ff9b77fe640 1 --2- 172.21.15.73:0/2791300042 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9b8162860 0x7ff9b8157140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:06.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.602+0000 7ff9b77fe640 1 -- 172.21.15.73:0/2791300042 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9b80c99f0 con 0x7ff9b8162c40 2024-04-26T20:53:06.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.602+0000 7ff9b77fe640 1 --2- 172.21.15.73:0/2791300042 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9b8162c40 0x7ff9b80c8be0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7ff9b000c7f0 tx=0x7ff9b000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:06.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.603+0000 7ff99f7fe640 1 -- 172.21.15.73:0/2791300042 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b0015070 con 0x7ff9b8162c40 2024-04-26T20:53:06.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.603+0000 7ff99f7fe640 1 -- 172.21.15.73:0/2791300042 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9b0007820 con 0x7ff9b8162c40 2024-04-26T20:53:06.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.603+0000 7ff9bd2d1640 1 -- 172.21.15.73:0/2791300042 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9b80c9cd0 con 0x7ff9b8162c40 2024-04-26T20:53:06.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.603+0000 7ff99f7fe640 1 -- 172.21.15.73:0/2791300042 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b0007c50 con 0x7ff9b8162c40 2024-04-26T20:53:06.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.603+0000 7ff9bd2d1640 1 -- 172.21.15.73:0/2791300042 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9b80ba110 con 0x7ff9b8162c40 2024-04-26T20:53:06.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.604+0000 7ff9bd2d1640 1 -- 172.21.15.73:0/2791300042 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9b80b9c30 con 0x7ff9b8162c40 2024-04-26T20:53:06.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.604+0000 7ff99f7fe640 1 -- 172.21.15.73:0/2791300042 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff9b0018070 con 0x7ff9b8162c40 2024-04-26T20:53:06.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.604+0000 7ff99f7fe640 1 --2- 172.21.15.73:0/2791300042 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff9a406c8f0 0x7ff9a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:06.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.604+0000 7ff99f7fe640 1 -- 172.21.15.73:0/2791300042 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 6273+0+0 (secure 0 0 0) 0x7ff9b008be20 con 0x7ff9b8162c40 2024-04-26T20:53:06.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.604+0000 7ff9b6ffd640 1 --2- 172.21.15.73:0/2791300042 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff9a406c8f0 0x7ff9a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:06.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.605+0000 7ff9b6ffd640 1 --2- 172.21.15.73:0/2791300042 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff9a406c8f0 0x7ff9a406edb0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8033000 tx=0x7ff9a8009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:06.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.609+0000 7ff99f7fe640 1 -- 172.21.15.73:0/2791300042 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff9b0059a50 con 0x7ff9b8162c40 2024-04-26T20:53:06.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:06.782+0000 7ff9bd2d1640 1 -- 172.21.15.73:0/2791300042 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff9b8069600 con 0x7ff9b8162c40 2024-04-26T20:53:07.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.608+0000 7ff99f7fe640 1 -- 172.21.15.73:0/2791300042 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v58) ==== 181+0+0 (secure 0 0 0) 0x7ff9b005dd50 con 0x7ff9b8162c40 2024-04-26T20:53:07.610 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_metadata' removed 2024-04-26T20:53:07.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.611+0000 7ff9bd2d1640 1 -- 172.21.15.73:0/2791300042 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff9a406c8f0 msgr2=0x7ff9a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:07.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.611+0000 7ff9bd2d1640 1 --2- 172.21.15.73:0/2791300042 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff9a406c8f0 0x7ff9a406edb0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8033000 tx=0x7ff9a8009000 comp rx=0 tx=0).stop 2024-04-26T20:53:07.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.611+0000 7ff9bd2d1640 1 -- 172.21.15.73:0/2791300042 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9b8162c40 msgr2=0x7ff9b80c8be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:07.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.611+0000 7ff9bd2d1640 1 --2- 172.21.15.73:0/2791300042 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9b8162c40 0x7ff9b80c8be0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7ff9b000c7f0 tx=0x7ff9b000ccb0 comp rx=0 tx=0).stop 2024-04-26T20:53:07.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.611+0000 7ff9bd2d1640 1 -- 172.21.15.73:0/2791300042 shutdown_connections 2024-04-26T20:53:07.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.611+0000 7ff9bd2d1640 1 --2- 172.21.15.73:0/2791300042 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9b8162c40 0x7ff9b80c8be0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:07.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.611+0000 7ff9bd2d1640 1 --2- 172.21.15.73:0/2791300042 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff9a406c8f0 0x7ff9a406edb0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:07.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.611+0000 7ff9bd2d1640 1 --2- 172.21.15.73:0/2791300042 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff9b8162860 0x7ff9b8157140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:07.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.611+0000 7ff9bd2d1640 1 --2- 172.21.15.73:0/2791300042 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff9b80b3f80 0x7ff9b80ba630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:07.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.611+0000 7ff9bd2d1640 1 -- 172.21.15.73:0/2791300042 >> 172.21.15.73:0/2791300042 conn(0x7ff9b809e3e0 msgr2=0x7ff9b80b2310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:07.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.611+0000 7ff9bd2d1640 1 -- 172.21.15.73:0/2791300042 shutdown_connections 2024-04-26T20:53:07.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.612+0000 7ff9bd2d1640 1 -- 172.21.15.73:0/2791300042 wait complete. 2024-04-26T20:53:07.626 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T20:53:07.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.728+0000 7f827a524640 1 Processor -- start 2024-04-26T20:53:07.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.729+0000 7f827a524640 1 -- start start 2024-04-26T20:53:07.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.729+0000 7f827a524640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8274145630 0x7f82740cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:07.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.729+0000 7f827a524640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f827415f980 0x7f82740b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:07.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.729+0000 7f827a524640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f82740b2fc0 0x7f82740730d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:07.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.729+0000 7f827a524640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f82740b33c0 con 0x7f8274145630 2024-04-26T20:53:07.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.729+0000 7f827a524640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f82740cfd70 con 0x7f827415f980 2024-04-26T20:53:07.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.729+0000 7f827a524640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f82740d0050 con 0x7f82740b2fc0 2024-04-26T20:53:07.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.729+0000 7f8273fff640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8274145630 0x7f82740cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:07.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.730+0000 7f8273fff640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8274145630 0x7f82740cf500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55516/0 (socket says 172.21.15.73:55516) 2024-04-26T20:53:07.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.730+0000 7f8273fff640 1 -- 172.21.15.73:0/3871647628 learned_addr learned my addr 172.21.15.73:0/3871647628 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:07.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.730+0000 7f8278ac1640 1 --2- 172.21.15.73:0/3871647628 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f82740b2fc0 0x7f82740730d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:07.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.730+0000 7f82737fe640 1 --2- 172.21.15.73:0/3871647628 >> v2:172.21.15.120:3300/0 conn(0x7f827415f980 0x7f82740b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:07.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.730+0000 7f8273fff640 1 -- 172.21.15.73:0/3871647628 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f82740b2fc0 msgr2=0x7f82740730d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:07.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.730+0000 7f8273fff640 1 --2- 172.21.15.73:0/3871647628 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f82740b2fc0 0x7f82740730d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:07.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.730+0000 7f8273fff640 1 -- 172.21.15.73:0/3871647628 >> v2:172.21.15.120:3300/0 conn(0x7f827415f980 msgr2=0x7f82740b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:07.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.730+0000 7f8273fff640 1 --2- 172.21.15.73:0/3871647628 >> v2:172.21.15.120:3300/0 conn(0x7f827415f980 0x7f82740b2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:07.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.730+0000 7f8273fff640 1 -- 172.21.15.73:0/3871647628 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82740d0330 con 0x7f8274145630 2024-04-26T20:53:07.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.730+0000 7f8273fff640 1 --2- 172.21.15.73:0/3871647628 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8274145630 0x7f82740cf500 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f8274070470 tx=0x7f826402c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b17f2b1861d2c1d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:07.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.731+0000 7f8272ffd640 1 -- 172.21.15.73:0/3871647628 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8264032790 con 0x7f8274145630 2024-04-26T20:53:07.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.731+0000 7f8272ffd640 1 -- 172.21.15.73:0/3871647628 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8264038070 con 0x7f8274145630 2024-04-26T20:53:07.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.731+0000 7f8272ffd640 1 -- 172.21.15.73:0/3871647628 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8264032b50 con 0x7f8274145630 2024-04-26T20:53:07.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.731+0000 7f827a524640 1 -- 172.21.15.73:0/3871647628 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8274145630 msgr2=0x7f82740cf500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:07.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.731+0000 7f827a524640 1 --2- 172.21.15.73:0/3871647628 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8274145630 0x7f82740cf500 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f8274070470 tx=0x7f826402c890 comp rx=0 tx=0).stop 2024-04-26T20:53:07.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.731+0000 7f827a524640 1 -- 172.21.15.73:0/3871647628 shutdown_connections 2024-04-26T20:53:07.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.731+0000 7f827a524640 1 --2- 172.21.15.73:0/3871647628 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f82740b2fc0 0x7f82740730d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:07.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.731+0000 7f827a524640 1 --2- 172.21.15.73:0/3871647628 >> v2:172.21.15.120:3300/0 conn(0x7f827415f980 0x7f82740b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:07.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.731+0000 7f827a524640 1 --2- 172.21.15.73:0/3871647628 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8274145630 0x7f82740cf500 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:07.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.731+0000 7f827a524640 1 -- 172.21.15.73:0/3871647628 >> 172.21.15.73:0/3871647628 conn(0x7f827406f310 msgr2=0x7f827406f720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:07.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.731+0000 7f827a524640 1 -- 172.21.15.73:0/3871647628 shutdown_connections 2024-04-26T20:53:07.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.731+0000 7f827a524640 1 -- 172.21.15.73:0/3871647628 wait complete. 2024-04-26T20:53:07.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.732+0000 7f827a524640 1 Processor -- start 2024-04-26T20:53:07.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.733+0000 7f827a524640 1 -- start start 2024-04-26T20:53:07.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.733+0000 7f827a524640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f82740b2fc0 0x7f8274150fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:07.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.733+0000 7f827a524640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8274145630 0x7f827413ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:07.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.733+0000 7f827a524640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f827415f980 0x7f82741441b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:07.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.733+0000 7f827a524640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f82740cfa40 con 0x7f827415f980 2024-04-26T20:53:07.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.733+0000 7f827a524640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f82740d08f0 con 0x7f82740b2fc0 2024-04-26T20:53:07.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.733+0000 7f827a524640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f8274145a10 con 0x7f8274145630 2024-04-26T20:53:07.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.733+0000 7f8278ac1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f827415f980 0x7f82741441b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:07.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.733+0000 7f8278ac1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f827415f980 0x7f82741441b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55528/0 (socket says 172.21.15.73:55528) 2024-04-26T20:53:07.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.733+0000 7f8278ac1640 1 -- 172.21.15.73:0/2295483084 learned_addr learned my addr 172.21.15.73:0/2295483084 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:07.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.734+0000 7f82737fe640 1 --2- 172.21.15.73:0/2295483084 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8274145630 0x7f827413ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:07.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.734+0000 7f8278ac1640 1 -- 172.21.15.73:0/2295483084 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8274145630 msgr2=0x7f827413ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:07.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.734+0000 7f8278ac1640 1 --2- 172.21.15.73:0/2295483084 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8274145630 0x7f827413ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:07.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.734+0000 7f8273fff640 1 --2- 172.21.15.73:0/2295483084 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f82740b2fc0 0x7f8274150fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:07.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.734+0000 7f8278ac1640 1 -- 172.21.15.73:0/2295483084 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f82740b2fc0 msgr2=0x7f8274150fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:07.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.734+0000 7f8278ac1640 1 --2- 172.21.15.73:0/2295483084 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f82740b2fc0 0x7f8274150fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:07.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.734+0000 7f8278ac1640 1 -- 172.21.15.73:0/2295483084 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82740b5420 con 0x7f827415f980 2024-04-26T20:53:07.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.734+0000 7f8278ac1640 1 --2- 172.21.15.73:0/2295483084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f827415f980 0x7f82741441b0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f826c00efc0 tx=0x7f826c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:07.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.734+0000 7f8270ff9640 1 -- 172.21.15.73:0/2295483084 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826c018020 con 0x7f827415f980 2024-04-26T20:53:07.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.734+0000 7f8270ff9640 1 -- 172.21.15.73:0/2295483084 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f826c014480 con 0x7f827415f980 2024-04-26T20:53:07.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.734+0000 7f827a524640 1 -- 172.21.15.73:0/2295483084 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82740b56c0 con 0x7f827415f980 2024-04-26T20:53:07.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.734+0000 7f8270ff9640 1 -- 172.21.15.73:0/2295483084 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826c014870 con 0x7f827415f980 2024-04-26T20:53:07.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.734+0000 7f827a524640 1 -- 172.21.15.73:0/2295483084 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82740b5c80 con 0x7f827415f980 2024-04-26T20:53:07.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.735+0000 7f827a524640 1 -- 172.21.15.73:0/2295483084 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8274069600 con 0x7f827415f980 2024-04-26T20:53:07.738 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.735+0000 7f8270ff9640 1 -- 172.21.15.73:0/2295483084 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f826c005d10 con 0x7f827415f980 2024-04-26T20:53:07.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.736+0000 7f8270ff9640 1 --2- 172.21.15.73:0/2295483084 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f826006c8f0 0x7f826006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:07.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.736+0000 7f8270ff9640 1 -- 172.21.15.73:0/2295483084 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5835+0+0 (secure 0 0 0) 0x7f826c08c640 con 0x7f827415f980 2024-04-26T20:53:07.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.736+0000 7f8273fff640 1 --2- 172.21.15.73:0/2295483084 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f826006c8f0 0x7f826006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:07.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.736+0000 7f8273fff640 1 --2- 172.21.15.73:0/2295483084 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f826006c8f0 0x7f826006edb0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8264037000 tx=0x7f8264008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:07.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.740+0000 7f8270ff9640 1 -- 172.21.15.73:0/2295483084 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f826c022070 con 0x7f827415f980 2024-04-26T20:53:07.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:07.916+0000 7f827a524640 1 -- 172.21.15.73:0/2295483084 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f827415efc0 con 0x7f827415f980 2024-04-26T20:53:08.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.616+0000 7f8270ff9640 1 -- 172.21.15.73:0/2295483084 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v59) ==== 169+0+0 (secure 0 0 0) 0x7f826c010040 con 0x7f827415f980 2024-04-26T20:53:08.617 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data' removed 2024-04-26T20:53:08.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.619+0000 7f827a524640 1 -- 172.21.15.73:0/2295483084 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f826006c8f0 msgr2=0x7f826006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:08.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.619+0000 7f827a524640 1 --2- 172.21.15.73:0/2295483084 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f826006c8f0 0x7f826006edb0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8264037000 tx=0x7f8264008040 comp rx=0 tx=0).stop 2024-04-26T20:53:08.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.619+0000 7f827a524640 1 -- 172.21.15.73:0/2295483084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f827415f980 msgr2=0x7f82741441b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:08.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.619+0000 7f827a524640 1 --2- 172.21.15.73:0/2295483084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f827415f980 0x7f82741441b0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f826c00efc0 tx=0x7f826c00c490 comp rx=0 tx=0).stop 2024-04-26T20:53:08.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.619+0000 7f827a524640 1 -- 172.21.15.73:0/2295483084 shutdown_connections 2024-04-26T20:53:08.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.619+0000 7f827a524640 1 --2- 172.21.15.73:0/2295483084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f827415f980 0x7f82741441b0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:08.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.619+0000 7f827a524640 1 --2- 172.21.15.73:0/2295483084 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f826006c8f0 0x7f826006edb0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:08.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.619+0000 7f827a524640 1 --2- 172.21.15.73:0/2295483084 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f8274145630 0x7f827413ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:08.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.619+0000 7f827a524640 1 --2- 172.21.15.73:0/2295483084 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f82740b2fc0 0x7f8274150fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:08.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.619+0000 7f827a524640 1 -- 172.21.15.73:0/2295483084 >> 172.21.15.73:0/2295483084 conn(0x7f827406f310 msgr2=0x7f827406f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:08.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.619+0000 7f827a524640 1 -- 172.21.15.73:0/2295483084 shutdown_connections 2024-04-26T20:53:08.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.619+0000 7f827a524640 1 -- 172.21.15.73:0/2295483084 wait complete. 2024-04-26T20:53:08.632 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-26T20:53:08.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.731+0000 7ff7484c8640 1 Processor -- start 2024-04-26T20:53:08.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.731+0000 7ff7484c8640 1 -- start start 2024-04-26T20:53:08.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.732+0000 7ff7484c8640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff7400d06f0 0x7ff7400d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:08.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.732+0000 7ff7484c8640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7400b2850 0x7ff7400b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:08.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.732+0000 7ff7484c8640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7400d0ba0 con 0x7ff74015f3a0 2024-04-26T20:53:08.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.732+0000 7ff7484c8640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7ff7400d0e20 con 0x7ff7400d06f0 2024-04-26T20:53:08.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.732+0000 7ff7484c8640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7ff7400b1d70 con 0x7ff7400b2850 2024-04-26T20:53:08.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.732+0000 7ff745a63640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff7400d06f0 0x7ff7400d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:08.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.732+0000 7ff746264640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7ff74015f3a0 0x7ff7400d1770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:50772/0 (socket says 172.21.15.73:50772) 2024-04-26T20:53:08.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.732+0000 7ff746264640 1 -- 172.21.15.73:0/1355779496 learned_addr learned my addr 172.21.15.73:0/1355779496 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:53:08.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.733+0000 7ff746a65640 1 --2- 172.21.15.73:0/1355779496 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7400b2850 0x7ff7400b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:08.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.733+0000 7ff745a63640 1 -- 172.21.15.73:0/1355779496 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7400b2850 msgr2=0x7ff7400b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:08.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.733+0000 7ff745a63640 1 --2- 172.21.15.73:0/1355779496 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7400b2850 0x7ff7400b2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:08.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.733+0000 7ff745a63640 1 -- 172.21.15.73:0/1355779496 >> v1:172.21.15.73:6789/0 conn(0x7ff74015f3a0 legacy=0x7ff7400d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:08.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.733+0000 7ff745a63640 1 -- 172.21.15.73:0/1355779496 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7400b2050 con 0x7ff7400d06f0 2024-04-26T20:53:08.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.733+0000 7ff746a65640 1 --2- 172.21.15.73:0/1355779496 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7400b2850 0x7ff7400b2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:53:08.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.733+0000 7ff745a63640 1 --2- 172.21.15.73:0/1355779496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7400d06f0 0x7ff7400d1e80 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7ff7300097a0 tx=0x7ff73002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fb54ef011cf5c34 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:08.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.733+0000 7ff745262640 1 -- 172.21.15.73:0/1355779496 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff730037070 con 0x7ff7400d06f0 2024-04-26T20:53:08.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.734+0000 7ff745262640 1 -- 172.21.15.73:0/1355779496 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff730032440 con 0x7ff7400d06f0 2024-04-26T20:53:08.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.734+0000 7ff745262640 1 -- 172.21.15.73:0/1355779496 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff730032810 con 0x7ff7400d06f0 2024-04-26T20:53:08.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.734+0000 7ff7484c8640 1 -- 172.21.15.73:0/1355779496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7400d06f0 msgr2=0x7ff7400d1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:08.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.734+0000 7ff7484c8640 1 --2- 172.21.15.73:0/1355779496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7400d06f0 0x7ff7400d1e80 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7ff7300097a0 tx=0x7ff73002a180 comp rx=0 tx=0).stop 2024-04-26T20:53:08.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.734+0000 7ff7484c8640 1 -- 172.21.15.73:0/1355779496 shutdown_connections 2024-04-26T20:53:08.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.734+0000 7ff7484c8640 1 --2- 172.21.15.73:0/1355779496 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7400b2850 0x7ff7400b2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:08.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.734+0000 7ff7484c8640 1 --2- 172.21.15.73:0/1355779496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7400d06f0 0x7ff7400d1e80 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:08.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.734+0000 7ff7484c8640 1 -- 172.21.15.73:0/1355779496 >> 172.21.15.73:0/1355779496 conn(0x7ff7400d4f90 msgr2=0x7ff7400d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:08.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.734+0000 7ff7484c8640 1 -- 172.21.15.73:0/1355779496 shutdown_connections 2024-04-26T20:53:08.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.734+0000 7ff7484c8640 1 -- 172.21.15.73:0/1355779496 wait complete. 2024-04-26T20:53:08.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.735+0000 7ff7484c8640 1 Processor -- start 2024-04-26T20:53:08.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.735+0000 7ff7484c8640 1 -- start start 2024-04-26T20:53:08.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.736+0000 7ff7484c8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7400b2850 0x7ff7400caea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:08.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.736+0000 7ff7484c8640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7400d06f0 0x7ff7400cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:08.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.736+0000 7ff7484c8640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff74015f3a0 0x7ff7400c8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:08.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.736+0000 7ff7484c8640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff74015f900 con 0x7ff74015f3a0 2024-04-26T20:53:08.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.736+0000 7ff7484c8640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ff74015f780 con 0x7ff7400b2850 2024-04-26T20:53:08.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.736+0000 7ff7484c8640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7ff7400d23c0 con 0x7ff7400d06f0 2024-04-26T20:53:08.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.736+0000 7ff746a65640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff74015f3a0 0x7ff7400c8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:08.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.736+0000 7ff746a65640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff74015f3a0 0x7ff7400c8c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55552/0 (socket says 172.21.15.73:55552) 2024-04-26T20:53:08.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.736+0000 7ff746a65640 1 -- 172.21.15.73:0/311158442 learned_addr learned my addr 172.21.15.73:0/311158442 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:08.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.736+0000 7ff746264640 1 --2- 172.21.15.73:0/311158442 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7400b2850 0x7ff7400caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:08.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.736+0000 7ff745a63640 1 --2- 172.21.15.73:0/311158442 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7400d06f0 0x7ff7400cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:08.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.736+0000 7ff746a65640 1 -- 172.21.15.73:0/311158442 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7400d06f0 msgr2=0x7ff7400cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:08.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.736+0000 7ff746a65640 1 --2- 172.21.15.73:0/311158442 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7400d06f0 0x7ff7400cb3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:08.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.736+0000 7ff746a65640 1 -- 172.21.15.73:0/311158442 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7400b2850 msgr2=0x7ff7400caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:08.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.736+0000 7ff746a65640 1 --2- 172.21.15.73:0/311158442 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7400b2850 0x7ff7400caea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:08.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.736+0000 7ff746a65640 1 -- 172.21.15.73:0/311158442 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7400c99f0 con 0x7ff74015f3a0 2024-04-26T20:53:08.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.737+0000 7ff746a65640 1 --2- 172.21.15.73:0/311158442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff74015f3a0 0x7ff7400c8c80 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7ff73800c7f0 tx=0x7ff73800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:08.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.737+0000 7ff736ffd640 1 -- 172.21.15.73:0/311158442 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff738015020 con 0x7ff74015f3a0 2024-04-26T20:53:08.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.737+0000 7ff736ffd640 1 -- 172.21.15.73:0/311158442 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff738004230 con 0x7ff74015f3a0 2024-04-26T20:53:08.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.737+0000 7ff7484c8640 1 -- 172.21.15.73:0/311158442 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7400c9cd0 con 0x7ff74015f3a0 2024-04-26T20:53:08.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.737+0000 7ff736ffd640 1 -- 172.21.15.73:0/311158442 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff738007620 con 0x7ff74015f3a0 2024-04-26T20:53:08.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.737+0000 7ff7484c8640 1 -- 172.21.15.73:0/311158442 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7400ca290 con 0x7ff74015f3a0 2024-04-26T20:53:08.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.738+0000 7ff7484c8640 1 -- 172.21.15.73:0/311158442 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7400cc1a0 con 0x7ff74015f3a0 2024-04-26T20:53:08.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.738+0000 7ff736ffd640 1 -- 172.21.15.73:0/311158442 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff738018020 con 0x7ff74015f3a0 2024-04-26T20:53:08.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.739+0000 7ff736ffd640 1 --2- 172.21.15.73:0/311158442 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff72406c8f0 0x7ff72406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:08.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.739+0000 7ff736ffd640 1 -- 172.21.15.73:0/311158442 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5421+0+0 (secure 0 0 0) 0x7ff73808ba70 con 0x7ff74015f3a0 2024-04-26T20:53:08.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.739+0000 7ff746264640 1 --2- 172.21.15.73:0/311158442 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff72406c8f0 0x7ff72406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:08.742 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.739+0000 7ff746264640 1 --2- 172.21.15.73:0/311158442 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff72406c8f0 0x7ff72406edb0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff7400d1ef0 tx=0x7ff73c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:08.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.743+0000 7ff736ffd640 1 -- 172.21.15.73:0/311158442 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff7400cc1a0 con 0x7ff74015f3a0 2024-04-26T20:53:08.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.917+0000 7ff7484c8640 1 -- 172.21.15.73:0/311158442 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7ff7400694f0 con 0x7ff74015f3a0 2024-04-26T20:53:08.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.921+0000 7ff736ffd640 1 -- 172.21.15.73:0/311158442 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) ==== 104+0+0 (secure 0 0 0) 0x7ff738091020 con 0x7ff74015f3a0 2024-04-26T20:53:08.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.923+0000 7ff7484c8640 1 -- 172.21.15.73:0/311158442 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff72406c8f0 msgr2=0x7ff72406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:08.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.923+0000 7ff7484c8640 1 --2- 172.21.15.73:0/311158442 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff72406c8f0 0x7ff72406edb0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff7400d1ef0 tx=0x7ff73c009040 comp rx=0 tx=0).stop 2024-04-26T20:53:08.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.923+0000 7ff7484c8640 1 -- 172.21.15.73:0/311158442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff74015f3a0 msgr2=0x7ff7400c8c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:08.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.923+0000 7ff7484c8640 1 --2- 172.21.15.73:0/311158442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff74015f3a0 0x7ff7400c8c80 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7ff73800c7f0 tx=0x7ff73800ccb0 comp rx=0 tx=0).stop 2024-04-26T20:53:08.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.923+0000 7ff7484c8640 1 -- 172.21.15.73:0/311158442 shutdown_connections 2024-04-26T20:53:08.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.923+0000 7ff7484c8640 1 --2- 172.21.15.73:0/311158442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff74015f3a0 0x7ff7400c8c80 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:08.924 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.923+0000 7ff7484c8640 1 --2- 172.21.15.73:0/311158442 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff72406c8f0 0x7ff72406edb0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:08.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.923+0000 7ff7484c8640 1 --2- 172.21.15.73:0/311158442 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ff7400d06f0 0x7ff7400cb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:08.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.923+0000 7ff7484c8640 1 --2- 172.21.15.73:0/311158442 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ff7400b2850 0x7ff7400caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:08.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.923+0000 7ff7484c8640 1 -- 172.21.15.73:0/311158442 >> 172.21.15.73:0/311158442 conn(0x7ff7400d4f90 msgr2=0x7ff7400b2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:08.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.923+0000 7ff7484c8640 1 -- 172.21.15.73:0/311158442 shutdown_connections 2024-04-26T20:53:08.925 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:08.923+0000 7ff7484c8640 1 -- 172.21.15.73:0/311158442 wait complete. 2024-04-26T20:53:08.935 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_auto_repair.TestMDSAutoRepair.test_mds_readonly' 2024-04-26T20:53:09.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.031+0000 7f1d653b4640 1 Processor -- start 2024-04-26T20:53:09.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.031+0000 7f1d653b4640 1 -- start start 2024-04-26T20:53:09.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.031+0000 7f1d653b4640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1d6015f3a0 0x7f1d600d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:09.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.031+0000 7f1d653b4640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f1d600d1770 0x7f1d600d1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:09.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.031+0000 7f1d653b4640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1d600d2160 0x7f1d600d2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:09.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.031+0000 7f1d653b4640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f1d600d25d0 con 0x7f1d6015f3a0 2024-04-26T20:53:09.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.031+0000 7f1d653b4640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f1d600d0e20 con 0x7f1d600d1770 2024-04-26T20:53:09.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.031+0000 7f1d653b4640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f1d600b1d70 con 0x7f1d600d2160 2024-04-26T20:53:09.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.031+0000 7f1d5effd640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1d6015f3a0 0x7f1d600d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:09.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.032+0000 7f1d5effd640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1d6015f3a0 0x7f1d600d06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55560/0 (socket says 172.21.15.73:55560) 2024-04-26T20:53:09.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.032+0000 7f1d5effd640 1 -- 172.21.15.73:0/120053117 learned_addr learned my addr 172.21.15.73:0/120053117 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:09.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.032+0000 7f1d5effd640 1 -- 172.21.15.73:0/120053117 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1d600d2160 msgr2=0x7f1d600d2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:09.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.032+0000 7f1d5effd640 1 --2- 172.21.15.73:0/120053117 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1d600d2160 0x7f1d600d2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:09.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.032+0000 7f1d5effd640 1 -- 172.21.15.73:0/120053117 >> v2:172.21.15.120:3300/0 conn(0x7f1d600d1770 msgr2=0x7f1d600d1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:09.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.032+0000 7f1d5effd640 1 --2- 172.21.15.73:0/120053117 >> v2:172.21.15.120:3300/0 conn(0x7f1d600d1770 0x7f1d600d1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:09.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.032+0000 7f1d5effd640 1 -- 172.21.15.73:0/120053117 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d600b2050 con 0x7f1d6015f3a0 2024-04-26T20:53:09.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.033+0000 7f1d5effd640 1 --2- 172.21.15.73:0/120053117 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d6015f3a0 0x7f1d600d06f0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f1d600b7b80 tx=0x7f1d5002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c99e97ec6fe994 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:09.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.033+0000 7f1d5dffb640 1 -- 172.21.15.73:0/120053117 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d50036070 con 0x7f1d6015f3a0 2024-04-26T20:53:09.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.033+0000 7f1d5dffb640 1 -- 172.21.15.73:0/120053117 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d50031600 con 0x7f1d6015f3a0 2024-04-26T20:53:09.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.033+0000 7f1d5dffb640 1 -- 172.21.15.73:0/120053117 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d500319d0 con 0x7f1d6015f3a0 2024-04-26T20:53:09.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.033+0000 7f1d653b4640 1 -- 172.21.15.73:0/120053117 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d6015f3a0 msgr2=0x7f1d600d06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:09.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.033+0000 7f1d653b4640 1 --2- 172.21.15.73:0/120053117 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d6015f3a0 0x7f1d600d06f0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f1d600b7b80 tx=0x7f1d5002a6a0 comp rx=0 tx=0).stop 2024-04-26T20:53:09.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.033+0000 7f1d653b4640 1 -- 172.21.15.73:0/120053117 shutdown_connections 2024-04-26T20:53:09.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.033+0000 7f1d653b4640 1 --2- 172.21.15.73:0/120053117 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1d600d2160 0x7f1d600d2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:09.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.033+0000 7f1d653b4640 1 --2- 172.21.15.73:0/120053117 >> v2:172.21.15.120:3300/0 conn(0x7f1d600d1770 0x7f1d600d1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:09.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.033+0000 7f1d653b4640 1 --2- 172.21.15.73:0/120053117 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d6015f3a0 0x7f1d600d06f0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:09.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.033+0000 7f1d653b4640 1 -- 172.21.15.73:0/120053117 >> 172.21.15.73:0/120053117 conn(0x7f1d600d4f90 msgr2=0x7f1d600d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:09.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.033+0000 7f1d653b4640 1 -- 172.21.15.73:0/120053117 shutdown_connections 2024-04-26T20:53:09.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.033+0000 7f1d653b4640 1 -- 172.21.15.73:0/120053117 wait complete. 2024-04-26T20:53:09.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.035+0000 7f1d653b4640 1 Processor -- start 2024-04-26T20:53:09.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.035+0000 7f1d653b4640 1 -- start start 2024-04-26T20:53:09.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.035+0000 7f1d653b4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d600d1770 0x7f1d601f95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:09.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.035+0000 7f1d653b4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d600d2160 0x7f1d600cd760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:09.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.035+0000 7f1d653b4640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1d6015f3a0 0x7f1d600ce450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:09.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.035+0000 7f1d653b4640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6015f900 con 0x7f1d600d1770 2024-04-26T20:53:09.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.035+0000 7f1d653b4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6015f780 con 0x7f1d600d2160 2024-04-26T20:53:09.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.035+0000 7f1d653b4640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f1d600d3390 con 0x7f1d6015f3a0 2024-04-26T20:53:09.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.035+0000 7f1d5effd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d600d1770 0x7f1d601f95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:09.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.035+0000 7f1d5effd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d600d1770 0x7f1d601f95b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55576/0 (socket says 172.21.15.73:55576) 2024-04-26T20:53:09.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.035+0000 7f1d5effd640 1 -- 172.21.15.73:0/1230139873 learned_addr learned my addr 172.21.15.73:0/1230139873 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:09.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.036+0000 7f1d5f7fe640 1 --2- 172.21.15.73:0/1230139873 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1d6015f3a0 0x7f1d600ce450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:09.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.036+0000 7f1d5effd640 1 -- 172.21.15.73:0/1230139873 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1d6015f3a0 msgr2=0x7f1d600ce450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:09.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.036+0000 7f1d5effd640 1 --2- 172.21.15.73:0/1230139873 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1d6015f3a0 0x7f1d600ce450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:09.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.036+0000 7f1d5effd640 1 -- 172.21.15.73:0/1230139873 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d600d2160 msgr2=0x7f1d600cd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:09.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.036+0000 7f1d5effd640 1 --2- 172.21.15.73:0/1230139873 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d600d2160 0x7f1d600cd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:09.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.036+0000 7f1d5effd640 1 -- 172.21.15.73:0/1230139873 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d60161600 con 0x7f1d600d1770 2024-04-26T20:53:09.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.036+0000 7f1d5f7fe640 1 --2- 172.21.15.73:0/1230139873 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1d6015f3a0 0x7f1d600ce450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:53:09.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.036+0000 7f1d5effd640 1 --2- 172.21.15.73:0/1230139873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d600d1770 0x7f1d601f95b0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f1d500093e0 tx=0x7f1d50004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:09.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.036+0000 7f1d477fe640 1 -- 172.21.15.73:0/1230139873 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d50041020 con 0x7f1d600d1770 2024-04-26T20:53:09.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.036+0000 7f1d653b4640 1 -- 172.21.15.73:0/1230139873 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d601618e0 con 0x7f1d600d1770 2024-04-26T20:53:09.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.036+0000 7f1d477fe640 1 -- 172.21.15.73:0/1230139873 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d50007820 con 0x7f1d600d1770 2024-04-26T20:53:09.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.036+0000 7f1d477fe640 1 -- 172.21.15.73:0/1230139873 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d50036040 con 0x7f1d600d1770 2024-04-26T20:53:09.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.036+0000 7f1d653b4640 1 -- 172.21.15.73:0/1230139873 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d60161ea0 con 0x7f1d600d1770 2024-04-26T20:53:09.040 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.037+0000 7f1d653b4640 1 -- 172.21.15.73:0/1230139873 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d600694f0 con 0x7f1d600d1770 2024-04-26T20:53:09.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.037+0000 7f1d477fe640 1 -- 172.21.15.73:0/1230139873 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1d50044020 con 0x7f1d600d1770 2024-04-26T20:53:09.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.038+0000 7f1d477fe640 1 --2- 172.21.15.73:0/1230139873 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1d4c06c8f0 0x7f1d4c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:09.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.038+0000 7f1d477fe640 1 -- 172.21.15.73:0/1230139873 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5421+0+0 (secure 0 0 0) 0x7f1d500ac980 con 0x7f1d600d1770 2024-04-26T20:53:09.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.038+0000 7f1d5e7fc640 1 --2- 172.21.15.73:0/1230139873 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1d4c06c8f0 0x7f1d4c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:09.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.038+0000 7f1d5e7fc640 1 --2- 172.21.15.73:0/1230139873 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1d4c06c8f0 0x7f1d4c06edb0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1d600b5570 tx=0x7f1d54006c90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:09.043 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.042+0000 7f1d477fe640 1 -- 172.21.15.73:0/1230139873 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1d5004b030 con 0x7f1d600d1770 2024-04-26T20:53:09.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.286+0000 7f1d653b4640 1 -- 172.21.15.73:0/1230139873 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_auto_repair.TestMDSAutoRepair.test_mds_readonly"]} v 0) -- 0x7f1d600b6860 con 0x7f1d600d1770 2024-04-26T20:53:09.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.617+0000 7f1d477fe640 1 -- 172.21.15.73:0/1230139873 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_auto_repair.TestMDSAutoRepair.test_mds_readonly"]}]=0 v165) ==== 144+0+0 (secure 0 0 0) 0x7f1d5007aee0 con 0x7f1d600d1770 2024-04-26T20:53:09.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.619+0000 7f1d653b4640 1 -- 172.21.15.73:0/1230139873 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1d4c06c8f0 msgr2=0x7f1d4c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:09.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.619+0000 7f1d653b4640 1 --2- 172.21.15.73:0/1230139873 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1d4c06c8f0 0x7f1d4c06edb0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1d600b5570 tx=0x7f1d54006c90 comp rx=0 tx=0).stop 2024-04-26T20:53:09.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.619+0000 7f1d653b4640 1 -- 172.21.15.73:0/1230139873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d600d1770 msgr2=0x7f1d601f95b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:09.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.619+0000 7f1d653b4640 1 --2- 172.21.15.73:0/1230139873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d600d1770 0x7f1d601f95b0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f1d500093e0 tx=0x7f1d50004a40 comp rx=0 tx=0).stop 2024-04-26T20:53:09.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.619+0000 7f1d653b4640 1 -- 172.21.15.73:0/1230139873 shutdown_connections 2024-04-26T20:53:09.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.619+0000 7f1d653b4640 1 --2- 172.21.15.73:0/1230139873 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1d4c06c8f0 0x7f1d4c06edb0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:09.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.619+0000 7f1d653b4640 1 --2- 172.21.15.73:0/1230139873 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1d6015f3a0 0x7f1d600ce450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:09.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.619+0000 7f1d653b4640 1 --2- 172.21.15.73:0/1230139873 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1d600d2160 0x7f1d600cd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:09.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.619+0000 7f1d653b4640 1 --2- 172.21.15.73:0/1230139873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d600d1770 0x7f1d601f95b0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:09.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.619+0000 7f1d653b4640 1 -- 172.21.15.73:0/1230139873 >> 172.21.15.73:0/1230139873 conn(0x7f1d600d4f90 msgr2=0x7f1d600d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:09.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.619+0000 7f1d653b4640 1 -- 172.21.15.73:0/1230139873 shutdown_connections 2024-04-26T20:53:09.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:09.619+0000 7f1d653b4640 1 -- 172.21.15.73:0/1230139873 wait complete. 2024-04-26T20:53:09.631 INFO:tasks.cephfs_test_runner:test if MDS behave correct when it's readonly ... ok 2024-04-26T20:53:09.631 INFO:tasks.cephfs_test_runner: 2024-04-26T20:53:09.631 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-26T20:53:09.632 INFO:tasks.cephfs_test_runner:Ran 2 tests in 110.519s 2024-04-26T20:53:09.632 INFO:tasks.cephfs_test_runner: 2024-04-26T20:53:09.632 INFO:tasks.cephfs_test_runner:OK 2024-04-26T20:53:09.632 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-26T20:53:09.642 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-26T20:53:09.650 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-26T20:53:09.650 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-26T20:53:09.650 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:53:09.664 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19637 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T20:53:09.665 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-26T20:53:09.665 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:53:09.720 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056091,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19637 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248832k,nr_inodes=812208,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497668k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T20:53:09.720 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-26T20:53:09.720 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:53:09.738 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056097,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19070 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T20:53:09.739 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-26T20:53:09.739 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:53:09.796 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056097,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19070 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497676k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T20:53:09.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:53:09.796 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list 2024-04-26T20:53:09.811 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T20:53:09.811 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T20:53:09.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:53:09.811 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:53:09.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:53:09.875 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:53:09.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:53:09.942 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link delete ceph-brx 2024-04-26T20:53:10.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:53:10.052 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list 2024-04-26T20:53:10.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:53:10.067 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link delete ceph-brx 2024-04-26T20:53:10.132 INFO:teuthology.orchestra.run.smithi073.stderr:Cannot find device "ceph-brx" 2024-04-26T20:53:10.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:53:10.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:53:10.134 DEBUG:teuthology.orchestra.run.smithi120:> ip netns list 2024-04-26T20:53:10.152 INFO:teuthology.orchestra.run.smithi120.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-26T20:53:10.152 INFO:teuthology.orchestra.run.smithi120.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-26T20:53:10.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:53:10.152 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T20:53:10.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:53:10.221 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T20:53:10.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:53:10.288 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip link delete ceph-brx 2024-04-26T20:53:10.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:53:10.373 DEBUG:teuthology.orchestra.run.smithi120:> ip netns list 2024-04-26T20:53:10.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:53:10.429 DEBUG:teuthology.orchestra.run.smithi120:> sudo ip link delete ceph-brx 2024-04-26T20:53:10.494 INFO:teuthology.orchestra.run.smithi120.stderr:Cannot find device "ceph-brx" 2024-04-26T20:53:10.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:53:10.496 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-26T20:53:10.506 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-26T20:53:10.506 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:53:10.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.603+0000 7fd90efa4640 1 Processor -- start 2024-04-26T20:53:10.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.603+0000 7fd90efa4640 1 -- start start 2024-04-26T20:53:10.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.604+0000 7fd90efa4640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd9080cf5e0 0x7fd9080ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:10.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.604+0000 7fd90efa4640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd90815b9e0 con 0x7fd90815b600 2024-04-26T20:53:10.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.604+0000 7fd90efa4640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9080d0740 con 0x7fd9080cbbd0 2024-04-26T20:53:10.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.604+0000 7fd90efa4640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fd9080cb9b0 con 0x7fd9080cf5e0 2024-04-26T20:53:10.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.604+0000 7fd90cd40640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fd90815b600 0x7fd908071f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51722/0 (socket says 172.21.15.73:51722) 2024-04-26T20:53:10.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.604+0000 7fd90cd40640 1 -- 172.21.15.73:0/2694922410 learned_addr learned my addr 172.21.15.73:0/2694922410 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:53:10.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.605+0000 7fd9077fe640 1 -- 172.21.15.73:0/2694922410 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2105593100 0 0) 0x7fd90815b9e0 con 0x7fd90815b600 2024-04-26T20:53:10.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.605+0000 7fd9077fe640 1 -- 172.21.15.73:0/2694922410 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd8f4003440 con 0x7fd90815b600 2024-04-26T20:53:10.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.605+0000 7fd907fff640 1 --2- 172.21.15.73:0/2694922410 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd9080cf5e0 0x7fd9080ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:10.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.605+0000 7fd9077fe640 1 -- 172.21.15.73:0/2694922410 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2976481737 0 0) 0x7fd8f4003440 con 0x7fd90815b600 2024-04-26T20:53:10.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.605+0000 7fd9077fe640 1 -- 172.21.15.73:0/2694922410 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd9080cf5e0 msgr2=0x7fd9080ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:10.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.605+0000 7fd9077fe640 1 --2- 172.21.15.73:0/2694922410 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd9080cf5e0 0x7fd9080ce9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:10.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.605+0000 7fd9077fe640 1 -- 172.21.15.73:0/2694922410 >> v1:172.21.15.120:6789/0 conn(0x7fd9080cbbd0 legacy=0x7fd90806f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:10.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.605+0000 7fd9077fe640 1 -- 172.21.15.73:0/2694922410 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9080cb360 con 0x7fd90815b600 2024-04-26T20:53:10.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.605+0000 7fd9077fe640 1 -- 172.21.15.73:0/2694922410 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7fd9000019f0 con 0x7fd90815b600 2024-04-26T20:53:10.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.605+0000 7fd9077fe640 1 -- 172.21.15.73:0/2694922410 >> v1:172.21.15.73:6789/0 conn(0x7fd90815b600 legacy=0x7fd908071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:10.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.605+0000 7fd9077fe640 1 --2- 172.21.15.73:0/2694922410 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd9080cf5e0 0x7fd8f4003ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:10.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.605+0000 7fd9077fe640 1 --2- 172.21.15.73:0/2694922410 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8f4004050 0x7fd8f400a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:10.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.605+0000 7fd9077fe640 1 --2- 172.21.15.73:0/2694922410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd8f400aff0 0x7fd8f400d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:10.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.605+0000 7fd9077fe640 1 -- 172.21.15.73:0/2694922410 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd9080cb9b0 con 0x7fd8f4004050 2024-04-26T20:53:10.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.605+0000 7fd9077fe640 1 -- 172.21.15.73:0/2694922410 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f400dcc0 con 0x7fd8f400aff0 2024-04-26T20:53:10.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.605+0000 7fd9077fe640 1 -- 172.21.15.73:0/2694922410 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fd8f400dec0 con 0x7fd9080cf5e0 2024-04-26T20:53:10.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.605+0000 7fd907fff640 1 --2- 172.21.15.73:0/2694922410 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8f4004050 0x7fd8f400a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:10.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.605+0000 7fd90cd40640 1 --2- 172.21.15.73:0/2694922410 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd9080cf5e0 0x7fd8f4003ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:10.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.605+0000 7fd90d541640 1 --2- 172.21.15.73:0/2694922410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd8f400aff0 0x7fd8f400d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:10.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.605+0000 7fd907fff640 1 -- 172.21.15.73:0/2694922410 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd9080cf5e0 msgr2=0x7fd8f4003ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:10.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.605+0000 7fd907fff640 1 --2- 172.21.15.73:0/2694922410 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd9080cf5e0 0x7fd8f4003ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:10.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.605+0000 7fd907fff640 1 -- 172.21.15.73:0/2694922410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd8f400aff0 msgr2=0x7fd8f400d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:10.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.605+0000 7fd907fff640 1 --2- 172.21.15.73:0/2694922410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd8f400aff0 0x7fd8f400d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:10.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.605+0000 7fd907fff640 1 -- 172.21.15.73:0/2694922410 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8f4003440 con 0x7fd8f4004050 2024-04-26T20:53:10.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.605+0000 7fd90d541640 1 --2- 172.21.15.73:0/2694922410 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd8f400aff0 0x7fd8f400d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:53:10.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.606+0000 7fd907fff640 1 --2- 172.21.15.73:0/2694922410 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8f4004050 0x7fd8f400a860 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0009ac0 tx=0x7fd8f4003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=830b638c353dbc6e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:10.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.606+0000 7fd9077fe640 1 -- 172.21.15.73:0/2694922410 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f0036070 con 0x7fd8f4004050 2024-04-26T20:53:10.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.606+0000 7fd9077fe640 1 -- 172.21.15.73:0/2694922410 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8f0031400 con 0x7fd8f4004050 2024-04-26T20:53:10.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.606+0000 7fd9077fe640 1 -- 172.21.15.73:0/2694922410 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f0009690 con 0x7fd8f4004050 2024-04-26T20:53:10.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.607+0000 7fd90efa4640 1 -- 172.21.15.73:0/2694922410 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8f4004050 msgr2=0x7fd8f400a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:10.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.607+0000 7fd90efa4640 1 --2- 172.21.15.73:0/2694922410 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8f4004050 0x7fd8f400a860 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0009ac0 tx=0x7fd8f4003730 comp rx=0 tx=0).stop 2024-04-26T20:53:10.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.607+0000 7fd90d541640 1 -- 172.21.15.73:0/2694922410 reap_dead start 2024-04-26T20:53:10.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.607+0000 7fd90efa4640 1 -- 172.21.15.73:0/2694922410 shutdown_connections 2024-04-26T20:53:10.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.607+0000 7fd90efa4640 1 -- 172.21.15.73:0/2694922410 >> 172.21.15.73:0/2694922410 conn(0x7fd9080d0be0 msgr2=0x7fd9080d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:10.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.607+0000 7fd90efa4640 1 -- 172.21.15.73:0/2694922410 shutdown_connections 2024-04-26T20:53:10.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.607+0000 7fd90efa4640 1 -- 172.21.15.73:0/2694922410 wait complete. 2024-04-26T20:53:10.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.608+0000 7fd90efa4640 1 Processor -- start 2024-04-26T20:53:10.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.608+0000 7fd90efa4640 1 -- start start 2024-04-26T20:53:10.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.609+0000 7fd90efa4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8f4004050 0x7fd9080b0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:10.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.609+0000 7fd90efa4640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd908156140 0x7fd9080c9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:10.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.609+0000 7fd90efa4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9080ca3c0 0x7fd9081f57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:10.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.609+0000 7fd90efa4640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd9080d0470 con 0x7fd8f4004050 2024-04-26T20:53:10.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.609+0000 7fd90efa4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fd9080d02f0 con 0x7fd9080ca3c0 2024-04-26T20:53:10.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.609+0000 7fd90efa4640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fd90806fa80 con 0x7fd908156140 2024-04-26T20:53:10.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.609+0000 7fd90cd40640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8f4004050 0x7fd9080b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:10.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.609+0000 7fd90cd40640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8f4004050 0x7fd9080b0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56806/0 (socket says 172.21.15.73:56806) 2024-04-26T20:53:10.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.609+0000 7fd90cd40640 1 -- 172.21.15.73:0/1308737063 learned_addr learned my addr 172.21.15.73:0/1308737063 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:10.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.609+0000 7fd907fff640 1 --2- 172.21.15.73:0/1308737063 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd908156140 0x7fd9080c9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:10.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.609+0000 7fd90d541640 1 --2- 172.21.15.73:0/1308737063 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9080ca3c0 0x7fd9081f57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:10.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.609+0000 7fd90cd40640 1 -- 172.21.15.73:0/1308737063 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd908156140 msgr2=0x7fd9080c9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:10.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.609+0000 7fd90cd40640 1 --2- 172.21.15.73:0/1308737063 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd908156140 0x7fd9080c9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:10.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.609+0000 7fd90cd40640 1 -- 172.21.15.73:0/1308737063 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9080ca3c0 msgr2=0x7fd9081f57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:10.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.609+0000 7fd90cd40640 1 --2- 172.21.15.73:0/1308737063 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9080ca3c0 0x7fd9081f57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:10.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.609+0000 7fd90cd40640 1 -- 172.21.15.73:0/1308737063 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9081f65b0 con 0x7fd8f4004050 2024-04-26T20:53:10.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.609+0000 7fd90cd40640 1 --2- 172.21.15.73:0/1308737063 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8f4004050 0x7fd9080b0090 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fd9000077f0 tx=0x7fd900007bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:10.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.609+0000 7fd9057fa640 1 -- 172.21.15.73:0/1308737063 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90000ad30 con 0x7fd8f4004050 2024-04-26T20:53:10.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.610+0000 7fd9057fa640 1 -- 172.21.15.73:0/1308737063 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd900018070 con 0x7fd8f4004050 2024-04-26T20:53:10.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.610+0000 7fd90efa4640 1 -- 172.21.15.73:0/1308737063 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9080af2d0 con 0x7fd8f4004050 2024-04-26T20:53:10.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.610+0000 7fd9057fa640 1 -- 172.21.15.73:0/1308737063 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd900013630 con 0x7fd8f4004050 2024-04-26T20:53:10.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.610+0000 7fd90efa4640 1 -- 172.21.15.73:0/1308737063 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9080af890 con 0x7fd8f4004050 2024-04-26T20:53:10.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.610+0000 7fd90efa4640 1 -- 172.21.15.73:0/1308737063 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9080694f0 con 0x7fd8f4004050 2024-04-26T20:53:10.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.611+0000 7fd9057fa640 1 -- 172.21.15.73:0/1308737063 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd9000137d0 con 0x7fd8f4004050 2024-04-26T20:53:10.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.611+0000 7fd9057fa640 1 --2- 172.21.15.73:0/1308737063 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd8e806c8f0 0x7fd8e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:10.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.612+0000 7fd9057fa640 1 -- 172.21.15.73:0/1308737063 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5421+0+0 (secure 0 0 0) 0x7fd90008c1b0 con 0x7fd8f4004050 2024-04-26T20:53:10.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.612+0000 7fd907fff640 1 --2- 172.21.15.73:0/1308737063 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd8e806c8f0 0x7fd8e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:10.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.612+0000 7fd907fff640 1 --2- 172.21.15.73:0/1308737063 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd8e806c8f0 0x7fd8e806edb0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0033000 tx=0x7fd8f0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:10.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.616+0000 7fd9057fa640 1 -- 172.21.15.73:0/1308737063 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd90005a700 con 0x7fd8f4004050 2024-04-26T20:53:10.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.797+0000 7fd90efa4640 1 -- 172.21.15.73:0/1308737063 --> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd9080ca7a0 con 0x7fd8e806c8f0 2024-04-26T20:53:10.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.799+0000 7fd9057fa640 1 -- 172.21.15.73:0/1308737063 <== mgr.4107 v2:172.21.15.73:6832/666905549 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13383 (secure 0 0 0) 0x7fd9080ca7a0 con 0x7fd8e806c8f0 2024-04-26T20:53:10.800 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:53:10.800 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-26T20:53:10.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.801+0000 7fd90efa4640 1 -- 172.21.15.73:0/1308737063 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd8e806c8f0 msgr2=0x7fd8e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:10.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.801+0000 7fd90efa4640 1 --2- 172.21.15.73:0/1308737063 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd8e806c8f0 0x7fd8e806edb0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0033000 tx=0x7fd8f0008030 comp rx=0 tx=0).stop 2024-04-26T20:53:10.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.801+0000 7fd90efa4640 1 -- 172.21.15.73:0/1308737063 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8f4004050 msgr2=0x7fd9080b0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:10.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.801+0000 7fd90efa4640 1 --2- 172.21.15.73:0/1308737063 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8f4004050 0x7fd9080b0090 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fd9000077f0 tx=0x7fd900007bf0 comp rx=0 tx=0).stop 2024-04-26T20:53:10.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.801+0000 7fd90efa4640 1 -- 172.21.15.73:0/1308737063 shutdown_connections 2024-04-26T20:53:10.802 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.801+0000 7fd90efa4640 1 --2- 172.21.15.73:0/1308737063 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fd8e806c8f0 0x7fd8e806edb0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:10.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.801+0000 7fd90efa4640 1 --2- 172.21.15.73:0/1308737063 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fd9080ca3c0 0x7fd9081f57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:10.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.801+0000 7fd90efa4640 1 --2- 172.21.15.73:0/1308737063 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fd908156140 0x7fd9080c9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:10.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.801+0000 7fd90efa4640 1 --2- 172.21.15.73:0/1308737063 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8f4004050 0x7fd9080b0090 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:10.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.801+0000 7fd90efa4640 1 -- 172.21.15.73:0/1308737063 >> 172.21.15.73:0/1308737063 conn(0x7fd9080d0be0 msgr2=0x7fd908073730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:10.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.801+0000 7fd90efa4640 1 -- 172.21.15.73:0/1308737063 shutdown_connections 2024-04-26T20:53:10.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.801+0000 7fd90efa4640 1 -- 172.21.15.73:0/1308737063 wait complete. 2024-04-26T20:53:10.812 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":137,"stamp":"2024-04-26T20:53:10.014378+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222628,"kb_used_data":6480,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752092,"statfs":{"total":773094113280,"available":772866142208,"internally_reserved":0,"allocated":6635520,"data_stored":3644364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17778,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-80,"num_read_kb":-94,"num_write":-66,"num_write_kb":-40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000938"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":235,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T20:52:54.451283+0000","last_change":"2024-04-26T20:50:21.150132+0000","last_active":"2024-04-26T20:52:54.451283+0000","last_peered":"2024-04-26T20:52:54.451283+0000","last_clean":"2024-04-26T20:52:54.451283+0000","last_became_active":"2024-04-26T20:50:21.147439+0000","last_became_peered":"2024-04-26T20:50:21.147439+0000","last_unstale":"2024-04-26T20:52:54.451283+0000","last_undegraded":"2024-04-26T20:52:54.451283+0000","last_fullsized":"2024-04-26T20:52:54.451283+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:20.084499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:20.084499+0000","last_clean_scrub_stamp":"2024-04-26T20:50:20.084499+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:19:29.395477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28256,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343584,"statfs":{"total":96636764160,"available":96607830016,"internally_reserved":0,"allocated":1269760,"data_stored":893389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574885,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27680,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":675840,"data_stored":303549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574885,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27676,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344164,"statfs":{"total":96636764160,"available":96608423936,"internally_reserved":0,"allocated":667648,"data_stored":302897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574885,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27672,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344168,"statfs":{"total":96636764160,"available":96608428032,"internally_reserved":0,"allocated":671744,"data_stored":303479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574885,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27704,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":704512,"data_stored":322347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574884,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27696,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":696320,"data_stored":322283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574884,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27688,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344152,"statfs":{"total":96636764160,"available":96608411648,"internally_reserved":0,"allocated":679936,"data_stored":303099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28256,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343584,"statfs":{"total":96636764160,"available":96607830016,"internally_reserved":0,"allocated":1269760,"data_stored":893321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:53:10.813 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:53:10.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.910+0000 7f46a6dda640 1 Processor -- start 2024-04-26T20:53:10.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.910+0000 7f46a6dda640 1 -- start start 2024-04-26T20:53:10.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.911+0000 7f46a6dda640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46a00d0d00 0x7f46a00d11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:10.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.911+0000 7f46a6dda640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f46a00cf180 0x7f46a00d1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:10.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.911+0000 7f46a6dda640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46a00d0710 con 0x7f46a0160720 2024-04-26T20:53:10.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.911+0000 7f46a6dda640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f46a00d01e0 con 0x7f46a00cf180 2024-04-26T20:53:10.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.911+0000 7f46a6dda640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f46a00d0430 con 0x7f46a00d0d00 2024-04-26T20:53:10.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.911+0000 7f469ffff640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46a00d0d00 0x7f46a00d11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:10.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.911+0000 7f46a5377640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f46a00cf180 0x7f46a00d1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:10.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.911+0000 7f46a4b76640 1 --1- >> v1:172.21.15.120:6789/0 conn(0x7f46a0160720 0x7f46a00ce620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.120:6789/0 says I am v1:172.21.15.73:58884/0 (socket says 172.21.15.73:58884) 2024-04-26T20:53:10.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.911+0000 7f46a4b76640 1 -- 172.21.15.73:0/160847100 learned_addr learned my addr 172.21.15.73:0/160847100 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:53:10.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.912+0000 7f469f7fe640 1 -- 172.21.15.73:0/160847100 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3768362200 0 0) 0x7f46a00d0710 con 0x7f46a0160720 2024-04-26T20:53:10.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.912+0000 7f469ffff640 1 -- 172.21.15.73:0/160847100 >> v2:172.21.15.120:3300/0 conn(0x7f46a00cf180 msgr2=0x7f46a00d1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:10.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.912+0000 7f469ffff640 1 --2- 172.21.15.73:0/160847100 >> v2:172.21.15.120:3300/0 conn(0x7f46a00cf180 0x7f46a00d1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:10.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.912+0000 7f469ffff640 1 -- 172.21.15.73:0/160847100 >> v1:172.21.15.120:6789/0 conn(0x7f46a0160720 legacy=0x7f46a00ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:10.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.912+0000 7f469ffff640 1 -- 172.21.15.73:0/160847100 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46a00cdb40 con 0x7f46a00d0d00 2024-04-26T20:53:10.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.913+0000 7f469ffff640 1 --2- 172.21.15.73:0/160847100 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46a00d0d00 0x7f46a00d11e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f468c0097a0 tx=0x7f468c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad380fe6425a095a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:10.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.913+0000 7f469f7fe640 1 -- 172.21.15.73:0/160847100 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468c037070 con 0x7f46a00d0d00 2024-04-26T20:53:10.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.913+0000 7f469f7fe640 1 -- 172.21.15.73:0/160847100 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f468c032440 con 0x7f46a00d0d00 2024-04-26T20:53:10.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.913+0000 7f469f7fe640 1 -- 172.21.15.73:0/160847100 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468c032810 con 0x7f46a00d0d00 2024-04-26T20:53:10.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.913+0000 7f46a6dda640 1 -- 172.21.15.73:0/160847100 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46a00d0d00 msgr2=0x7f46a00d11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:10.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.913+0000 7f46a6dda640 1 --2- 172.21.15.73:0/160847100 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46a00d0d00 0x7f46a00d11e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f468c0097a0 tx=0x7f468c02a180 comp rx=0 tx=0).stop 2024-04-26T20:53:10.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.914+0000 7f46a6dda640 1 -- 172.21.15.73:0/160847100 shutdown_connections 2024-04-26T20:53:10.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.914+0000 7f46a6dda640 1 --2- 172.21.15.73:0/160847100 >> v2:172.21.15.120:3300/0 conn(0x7f46a00cf180 0x7f46a00d1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:10.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.914+0000 7f46a6dda640 1 --2- 172.21.15.73:0/160847100 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46a00d0d00 0x7f46a00d11e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:10.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.914+0000 7f46a6dda640 1 -- 172.21.15.73:0/160847100 >> 172.21.15.73:0/160847100 conn(0x7f46a00d2e00 msgr2=0x7f46a00d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:10.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.914+0000 7f46a6dda640 1 -- 172.21.15.73:0/160847100 shutdown_connections 2024-04-26T20:53:10.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.914+0000 7f46a6dda640 1 -- 172.21.15.73:0/160847100 wait complete. 2024-04-26T20:53:10.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.915+0000 7f46a6dda640 1 Processor -- start 2024-04-26T20:53:10.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.915+0000 7f46a6dda640 1 -- start start 2024-04-26T20:53:10.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.916+0000 7f46a6dda640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46a00cf180 0x7f46a0159520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:10.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.916+0000 7f46a6dda640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f46a00d0d00 0x7f46a00b7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:10.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.916+0000 7f46a6dda640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46a0160720 0x7f46a00b7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:10.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.916+0000 7f46a6dda640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f46a00ced30 con 0x7f46a00cf180 2024-04-26T20:53:10.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.916+0000 7f46a6dda640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f46a00ce100 con 0x7f46a00d0d00 2024-04-26T20:53:10.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.916+0000 7f46a6dda640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f46a0160b00 con 0x7f46a0160720 2024-04-26T20:53:10.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.916+0000 7f46a4b76640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46a00cf180 0x7f46a0159520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:10.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.916+0000 7f46a4b76640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46a00cf180 0x7f46a0159520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56822/0 (socket says 172.21.15.73:56822) 2024-04-26T20:53:10.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.916+0000 7f46a4b76640 1 -- 172.21.15.73:0/41207707 learned_addr learned my addr 172.21.15.73:0/41207707 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:10.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.916+0000 7f469ffff640 1 --2- 172.21.15.73:0/41207707 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f46a00d0d00 0x7f46a00b7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:10.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.916+0000 7f46a5377640 1 --2- 172.21.15.73:0/41207707 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46a0160720 0x7f46a00b7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:10.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.916+0000 7f46a4b76640 1 -- 172.21.15.73:0/41207707 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46a0160720 msgr2=0x7f46a00b7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:10.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.916+0000 7f46a4b76640 1 --2- 172.21.15.73:0/41207707 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46a0160720 0x7f46a00b7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:10.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.916+0000 7f46a4b76640 1 -- 172.21.15.73:0/41207707 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f46a00d0d00 msgr2=0x7f46a00b7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:10.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.916+0000 7f46a4b76640 1 --2- 172.21.15.73:0/41207707 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f46a00d0d00 0x7f46a00b7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:10.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.916+0000 7f46a4b76640 1 -- 172.21.15.73:0/41207707 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46a00b6860 con 0x7f46a00cf180 2024-04-26T20:53:10.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.916+0000 7f46a4b76640 1 --2- 172.21.15.73:0/41207707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46a00cf180 0x7f46a0159520 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f469400c500 tx=0x7f469400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:10.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.916+0000 7f469d7fa640 1 -- 172.21.15.73:0/41207707 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4694018070 con 0x7f46a00cf180 2024-04-26T20:53:10.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.916+0000 7f469d7fa640 1 -- 172.21.15.73:0/41207707 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4694007580 con 0x7f46a00cf180 2024-04-26T20:53:10.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.916+0000 7f46a6dda640 1 -- 172.21.15.73:0/41207707 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46a00b6b40 con 0x7f46a00cf180 2024-04-26T20:53:10.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.916+0000 7f469d7fa640 1 -- 172.21.15.73:0/41207707 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4694004a20 con 0x7f46a00cf180 2024-04-26T20:53:10.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.916+0000 7f46a6dda640 1 -- 172.21.15.73:0/41207707 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46a00b7100 con 0x7f46a00cf180 2024-04-26T20:53:10.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.917+0000 7f46a6dda640 1 -- 172.21.15.73:0/41207707 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46a00694f0 con 0x7f46a00cf180 2024-04-26T20:53:10.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.918+0000 7f469d7fa640 1 -- 172.21.15.73:0/41207707 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f469401b070 con 0x7f46a00cf180 2024-04-26T20:53:10.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.918+0000 7f469d7fa640 1 --2- 172.21.15.73:0/41207707 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f468406c8f0 0x7f468406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:10.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.918+0000 7f469d7fa640 1 -- 172.21.15.73:0/41207707 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5421+0+0 (secure 0 0 0) 0x7f469408ac30 con 0x7f46a00cf180 2024-04-26T20:53:10.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.918+0000 7f469ffff640 1 --2- 172.21.15.73:0/41207707 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f468406c8f0 0x7f468406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:10.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.919+0000 7f469ffff640 1 --2- 172.21.15.73:0/41207707 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f468406c8f0 0x7f468406edb0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f468c034000 tx=0x7f468c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:10.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:10.923+0000 7f469d7fa640 1 -- 172.21.15.73:0/41207707 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4694058bc0 con 0x7f46a00cf180 2024-04-26T20:53:11.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.102+0000 7f46a6dda640 1 -- 172.21.15.73:0/41207707 --> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f46a00b8d80 con 0x7f468406c8f0 2024-04-26T20:53:11.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.103+0000 7f469d7fa640 1 -- 172.21.15.73:0/41207707 <== mgr.4107 v2:172.21.15.73:6832/666905549 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13383 (secure 0 0 0) 0x7f46a00b8d80 con 0x7f468406c8f0 2024-04-26T20:53:11.104 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:53:11.105 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-26T20:53:11.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.105+0000 7f46a6dda640 1 -- 172.21.15.73:0/41207707 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f468406c8f0 msgr2=0x7f468406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:11.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.105+0000 7f46a6dda640 1 --2- 172.21.15.73:0/41207707 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f468406c8f0 0x7f468406edb0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f468c034000 tx=0x7f468c02e000 comp rx=0 tx=0).stop 2024-04-26T20:53:11.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.105+0000 7f46a6dda640 1 -- 172.21.15.73:0/41207707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46a00cf180 msgr2=0x7f46a0159520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:11.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.105+0000 7f46a6dda640 1 --2- 172.21.15.73:0/41207707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46a00cf180 0x7f46a0159520 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f469400c500 tx=0x7f469400c9c0 comp rx=0 tx=0).stop 2024-04-26T20:53:11.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.105+0000 7f46a6dda640 1 -- 172.21.15.73:0/41207707 shutdown_connections 2024-04-26T20:53:11.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.105+0000 7f46a6dda640 1 --2- 172.21.15.73:0/41207707 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f468406c8f0 0x7f468406edb0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:11.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.105+0000 7f46a6dda640 1 --2- 172.21.15.73:0/41207707 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f46a0160720 0x7f46a00b7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:11.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.105+0000 7f46a6dda640 1 --2- 172.21.15.73:0/41207707 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f46a00d0d00 0x7f46a00b7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:11.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.105+0000 7f46a6dda640 1 --2- 172.21.15.73:0/41207707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46a00cf180 0x7f46a0159520 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:11.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.106+0000 7f46a6dda640 1 -- 172.21.15.73:0/41207707 >> 172.21.15.73:0/41207707 conn(0x7f46a00d2e00 msgr2=0x7f46a0071210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:11.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.106+0000 7f46a6dda640 1 -- 172.21.15.73:0/41207707 shutdown_connections 2024-04-26T20:53:11.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.106+0000 7f46a6dda640 1 -- 172.21.15.73:0/41207707 wait complete. 2024-04-26T20:53:11.117 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":137,"stamp":"2024-04-26T20:53:10.014378+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222628,"kb_used_data":6480,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752092,"statfs":{"total":773094113280,"available":772866142208,"internally_reserved":0,"allocated":6635520,"data_stored":3644364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17778,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-80,"num_read_kb":-94,"num_write":-66,"num_write_kb":-40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000938"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":235,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T20:52:54.451283+0000","last_change":"2024-04-26T20:50:21.150132+0000","last_active":"2024-04-26T20:52:54.451283+0000","last_peered":"2024-04-26T20:52:54.451283+0000","last_clean":"2024-04-26T20:52:54.451283+0000","last_became_active":"2024-04-26T20:50:21.147439+0000","last_became_peered":"2024-04-26T20:50:21.147439+0000","last_unstale":"2024-04-26T20:52:54.451283+0000","last_undegraded":"2024-04-26T20:52:54.451283+0000","last_fullsized":"2024-04-26T20:52:54.451283+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:20.084499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:20.084499+0000","last_clean_scrub_stamp":"2024-04-26T20:50:20.084499+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:19:29.395477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28256,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343584,"statfs":{"total":96636764160,"available":96607830016,"internally_reserved":0,"allocated":1269760,"data_stored":893389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574885,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27680,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":675840,"data_stored":303549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574885,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27676,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344164,"statfs":{"total":96636764160,"available":96608423936,"internally_reserved":0,"allocated":667648,"data_stored":302897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574885,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27672,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344168,"statfs":{"total":96636764160,"available":96608428032,"internally_reserved":0,"allocated":671744,"data_stored":303479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574885,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27704,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":704512,"data_stored":322347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574884,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27696,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":696320,"data_stored":322283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574884,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27688,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344152,"statfs":{"total":96636764160,"available":96608411648,"internally_reserved":0,"allocated":679936,"data_stored":303099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28256,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343584,"statfs":{"total":96636764160,"available":96607830016,"internally_reserved":0,"allocated":1269760,"data_stored":893321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:53:11.118 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-26T20:53:11.118 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:53:11.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.214+0000 7f7e33c07640 1 Processor -- start 2024-04-26T20:53:11.215 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.214+0000 7f7e33c07640 1 -- start start 2024-04-26T20:53:11.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.215+0000 7f7e33c07640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7e2c0d06f0 0x7f7e2c0d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:11.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.215+0000 7f7e33c07640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7e2c0b2850 0x7f7e2c0b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:11.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.215+0000 7f7e33c07640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e2c0d0ba0 con 0x7f7e2c15f3a0 2024-04-26T20:53:11.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.215+0000 7f7e33c07640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f7e2c0d0e20 con 0x7f7e2c0d06f0 2024-04-26T20:53:11.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.215+0000 7f7e33c07640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f7e2c0b1d70 con 0x7f7e2c0b2850 2024-04-26T20:53:11.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.215+0000 7f7e311a2640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7e2c0d06f0 0x7f7e2c0d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:11.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.215+0000 7f7e319a3640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f7e2c15f3a0 0x7f7e2c0d1770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51730/0 (socket says 172.21.15.73:51730) 2024-04-26T20:53:11.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.215+0000 7f7e319a3640 1 -- 172.21.15.73:0/1753243823 learned_addr learned my addr 172.21.15.73:0/1753243823 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:53:11.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.216+0000 7f7e321a4640 1 --2- 172.21.15.73:0/1753243823 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7e2c0b2850 0x7f7e2c0b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:11.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.216+0000 7f7e311a2640 1 -- 172.21.15.73:0/1753243823 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7e2c0b2850 msgr2=0x7f7e2c0b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:11.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.216+0000 7f7e311a2640 1 --2- 172.21.15.73:0/1753243823 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7e2c0b2850 0x7f7e2c0b2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:11.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.216+0000 7f7e311a2640 1 -- 172.21.15.73:0/1753243823 >> v1:172.21.15.73:6789/0 conn(0x7f7e2c15f3a0 legacy=0x7f7e2c0d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:11.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.216+0000 7f7e311a2640 1 -- 172.21.15.73:0/1753243823 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e2c0b2050 con 0x7f7e2c0d06f0 2024-04-26T20:53:11.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.216+0000 7f7e309a1640 1 -- 172.21.15.73:0/1753243823 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1076958702 0 0) 0x7f7e24001110 con 0x7f7e2c15f3a0 2024-04-26T20:53:11.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.216+0000 7f7e311a2640 1 --2- 172.21.15.73:0/1753243823 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e2c0d06f0 0x7f7e2c0d1e80 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f7e20009d90 tx=0x7f7e2002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=412527ebd82c55fe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:11.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.217+0000 7f7e309a1640 1 -- 172.21.15.73:0/1753243823 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e20007740 con 0x7f7e2c0d06f0 2024-04-26T20:53:11.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.217+0000 7f7e309a1640 1 -- 172.21.15.73:0/1753243823 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e200078e0 con 0x7f7e2c0d06f0 2024-04-26T20:53:11.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.217+0000 7f7e309a1640 1 -- 172.21.15.73:0/1753243823 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e20007cb0 con 0x7f7e2c0d06f0 2024-04-26T20:53:11.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.217+0000 7f7e33c07640 1 -- 172.21.15.73:0/1753243823 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e2c0d06f0 msgr2=0x7f7e2c0d1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:11.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.217+0000 7f7e33c07640 1 --2- 172.21.15.73:0/1753243823 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e2c0d06f0 0x7f7e2c0d1e80 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f7e20009d90 tx=0x7f7e2002a660 comp rx=0 tx=0).stop 2024-04-26T20:53:11.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.217+0000 7f7e33c07640 1 -- 172.21.15.73:0/1753243823 shutdown_connections 2024-04-26T20:53:11.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.217+0000 7f7e33c07640 1 --2- 172.21.15.73:0/1753243823 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7e2c0b2850 0x7f7e2c0b2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:11.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.217+0000 7f7e33c07640 1 --2- 172.21.15.73:0/1753243823 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e2c0d06f0 0x7f7e2c0d1e80 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:11.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.217+0000 7f7e33c07640 1 -- 172.21.15.73:0/1753243823 >> 172.21.15.73:0/1753243823 conn(0x7f7e2c0d4f90 msgr2=0x7f7e2c0d53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:11.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.217+0000 7f7e33c07640 1 -- 172.21.15.73:0/1753243823 shutdown_connections 2024-04-26T20:53:11.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.217+0000 7f7e33c07640 1 -- 172.21.15.73:0/1753243823 wait complete. 2024-04-26T20:53:11.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.218+0000 7f7e33c07640 1 Processor -- start 2024-04-26T20:53:11.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.218+0000 7f7e33c07640 1 -- start start 2024-04-26T20:53:11.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.219+0000 7f7e33c07640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e2c0b2850 0x7f7e2c15e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:11.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.219+0000 7f7e33c07640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7e2c0d06f0 0x7f7e2c0cd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:11.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.219+0000 7f7e33c07640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e2c15f3a0 0x7f7e2c0cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:11.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.219+0000 7f7e33c07640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2c15f900 con 0x7f7e2c0b2850 2024-04-26T20:53:11.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.219+0000 7f7e33c07640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2c15f780 con 0x7f7e2c15f3a0 2024-04-26T20:53:11.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.219+0000 7f7e33c07640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f7e2c0d23c0 con 0x7f7e2c0d06f0 2024-04-26T20:53:11.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.219+0000 7f7e319a3640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e2c0b2850 0x7f7e2c15e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:11.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.219+0000 7f7e319a3640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e2c0b2850 0x7f7e2c15e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56838/0 (socket says 172.21.15.73:56838) 2024-04-26T20:53:11.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.219+0000 7f7e319a3640 1 -- 172.21.15.73:0/447350019 learned_addr learned my addr 172.21.15.73:0/447350019 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:11.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.219+0000 7f7e319a3640 1 -- 172.21.15.73:0/447350019 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7e2c0d06f0 msgr2=0x7f7e2c0cd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:11.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.219+0000 7f7e319a3640 1 --2- 172.21.15.73:0/447350019 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7e2c0d06f0 0x7f7e2c0cd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:11.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.219+0000 7f7e319a3640 1 -- 172.21.15.73:0/447350019 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e2c15f3a0 msgr2=0x7f7e2c0cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:11.223 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.219+0000 7f7e319a3640 1 --2- 172.21.15.73:0/447350019 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e2c15f3a0 0x7f7e2c0cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:11.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.219+0000 7f7e319a3640 1 -- 172.21.15.73:0/447350019 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e2c0cc200 con 0x7f7e2c0b2850 2024-04-26T20:53:11.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.220+0000 7f7e319a3640 1 --2- 172.21.15.73:0/447350019 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e2c0b2850 0x7f7e2c15e270 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f7e2400c780 tx=0x7f7e2400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:11.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.220+0000 7f7e2a7fc640 1 -- 172.21.15.73:0/447350019 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e24018070 con 0x7f7e2c0b2850 2024-04-26T20:53:11.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.220+0000 7f7e2a7fc640 1 -- 172.21.15.73:0/447350019 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e240072d0 con 0x7f7e2c0b2850 2024-04-26T20:53:11.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.220+0000 7f7e2a7fc640 1 -- 172.21.15.73:0/447350019 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e240047e0 con 0x7f7e2c0b2850 2024-04-26T20:53:11.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.220+0000 7f7e33c07640 1 -- 172.21.15.73:0/447350019 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e2c0cc4e0 con 0x7f7e2c0b2850 2024-04-26T20:53:11.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.220+0000 7f7e33c07640 1 -- 172.21.15.73:0/447350019 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e2c0ccaa0 con 0x7f7e2c0b2850 2024-04-26T20:53:11.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.221+0000 7f7e33c07640 1 -- 172.21.15.73:0/447350019 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e2c0694f0 con 0x7f7e2c0b2850 2024-04-26T20:53:11.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.221+0000 7f7e2a7fc640 1 -- 172.21.15.73:0/447350019 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7e24004980 con 0x7f7e2c0b2850 2024-04-26T20:53:11.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.222+0000 7f7e2a7fc640 1 --2- 172.21.15.73:0/447350019 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f7e1806c8f0 0x7f7e1806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:11.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.222+0000 7f7e2a7fc640 1 -- 172.21.15.73:0/447350019 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5421+0+0 (secure 0 0 0) 0x7f7e2408adf0 con 0x7f7e2c0b2850 2024-04-26T20:53:11.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.222+0000 7f7e311a2640 1 --2- 172.21.15.73:0/447350019 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f7e1806c8f0 0x7f7e1806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:11.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.222+0000 7f7e311a2640 1 --2- 172.21.15.73:0/447350019 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f7e1806c8f0 0x7f7e1806edb0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7e20034000 tx=0x7f7e2002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:11.226 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.226+0000 7f7e2a7fc640 1 -- 172.21.15.73:0/447350019 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7e2c0d0ba0 con 0x7f7e2c0b2850 2024-04-26T20:53:11.399 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.398+0000 7f7e33c07640 1 -- 172.21.15.73:0/447350019 --> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7e2c0b6860 con 0x7f7e1806c8f0 2024-04-26T20:53:11.399 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.399+0000 7f7e2a7fc640 1 -- 172.21.15.73:0/447350019 <== mgr.4107 v2:172.21.15.73:6832/666905549 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13383 (secure 0 0 0) 0x7f7e2c0b6860 con 0x7f7e1806c8f0 2024-04-26T20:53:11.400 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:53:11.400 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-26T20:53:11.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.401+0000 7f7e33c07640 1 -- 172.21.15.73:0/447350019 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f7e1806c8f0 msgr2=0x7f7e1806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:11.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.401+0000 7f7e33c07640 1 --2- 172.21.15.73:0/447350019 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f7e1806c8f0 0x7f7e1806edb0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7e20034000 tx=0x7f7e2002e000 comp rx=0 tx=0).stop 2024-04-26T20:53:11.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.401+0000 7f7e33c07640 1 -- 172.21.15.73:0/447350019 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e2c0b2850 msgr2=0x7f7e2c15e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:11.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.401+0000 7f7e33c07640 1 --2- 172.21.15.73:0/447350019 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e2c0b2850 0x7f7e2c15e270 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f7e2400c780 tx=0x7f7e2400cc40 comp rx=0 tx=0).stop 2024-04-26T20:53:11.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.401+0000 7f7e33c07640 1 -- 172.21.15.73:0/447350019 shutdown_connections 2024-04-26T20:53:11.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.401+0000 7f7e33c07640 1 --2- 172.21.15.73:0/447350019 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f7e1806c8f0 0x7f7e1806edb0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:11.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.401+0000 7f7e33c07640 1 --2- 172.21.15.73:0/447350019 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7e2c15f3a0 0x7f7e2c0cd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:11.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.401+0000 7f7e33c07640 1 --2- 172.21.15.73:0/447350019 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7e2c0d06f0 0x7f7e2c0cd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:11.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.401+0000 7f7e33c07640 1 --2- 172.21.15.73:0/447350019 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e2c0b2850 0x7f7e2c15e270 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:11.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.401+0000 7f7e33c07640 1 -- 172.21.15.73:0/447350019 >> 172.21.15.73:0/447350019 conn(0x7f7e2c0d4f90 msgr2=0x7f7e2c06d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:11.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.401+0000 7f7e33c07640 1 -- 172.21.15.73:0/447350019 shutdown_connections 2024-04-26T20:53:11.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.401+0000 7f7e33c07640 1 -- 172.21.15.73:0/447350019 wait complete. 2024-04-26T20:53:11.412 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":137,"stamp":"2024-04-26T20:53:10.014378+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222628,"kb_used_data":6480,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752092,"statfs":{"total":773094113280,"available":772866142208,"internally_reserved":0,"allocated":6635520,"data_stored":3644364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17778,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-80,"num_read_kb":-94,"num_write":-66,"num_write_kb":-40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000938"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":235,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T20:52:54.451283+0000","last_change":"2024-04-26T20:50:21.150132+0000","last_active":"2024-04-26T20:52:54.451283+0000","last_peered":"2024-04-26T20:52:54.451283+0000","last_clean":"2024-04-26T20:52:54.451283+0000","last_became_active":"2024-04-26T20:50:21.147439+0000","last_became_peered":"2024-04-26T20:50:21.147439+0000","last_unstale":"2024-04-26T20:52:54.451283+0000","last_undegraded":"2024-04-26T20:52:54.451283+0000","last_fullsized":"2024-04-26T20:52:54.451283+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:20.084499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:20.084499+0000","last_clean_scrub_stamp":"2024-04-26T20:50:20.084499+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:19:29.395477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28256,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343584,"statfs":{"total":96636764160,"available":96607830016,"internally_reserved":0,"allocated":1269760,"data_stored":893389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574885,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27680,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":675840,"data_stored":303549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574885,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27676,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344164,"statfs":{"total":96636764160,"available":96608423936,"internally_reserved":0,"allocated":667648,"data_stored":302897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574885,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27672,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344168,"statfs":{"total":96636764160,"available":96608428032,"internally_reserved":0,"allocated":671744,"data_stored":303479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574885,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27704,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":704512,"data_stored":322347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574884,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27696,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":696320,"data_stored":322283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574884,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27688,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344152,"statfs":{"total":96636764160,"available":96608411648,"internally_reserved":0,"allocated":679936,"data_stored":303099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28256,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343584,"statfs":{"total":96636764160,"available":96607830016,"internally_reserved":0,"allocated":1269760,"data_stored":893321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:53:11.413 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:53:11.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.509+0000 7f196a6c4640 1 Processor -- start 2024-04-26T20:53:11.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.509+0000 7f196a6c4640 1 -- start start 2024-04-26T20:53:11.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.509+0000 7f196a6c4640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1964156460 0x7f19640cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:11.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.509+0000 7f196a6c4640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f19640cba10 0x7f196406f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:11.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.509+0000 7f196a6c4640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1964156840 con 0x7f196406fcd0 2024-04-26T20:53:11.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.509+0000 7f196a6c4640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f19640cbf20 con 0x7f1964156460 2024-04-26T20:53:11.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.509+0000 7f196a6c4640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f19640cc200 con 0x7f19640cba10 2024-04-26T20:53:11.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.510+0000 7f1963fff640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1964156460 0x7f19640cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:11.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.510+0000 7f1963fff640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1964156460 0x7f19640cb4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56842/0 (socket says 172.21.15.73:56842) 2024-04-26T20:53:11.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.510+0000 7f1963fff640 1 -- 172.21.15.73:0/98682876 learned_addr learned my addr 172.21.15.73:0/98682876 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:11.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.510+0000 7f1968c61640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f196406fcd0 0x7f196415f980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51740/0 (socket says 172.21.15.73:51740) 2024-04-26T20:53:11.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.510+0000 7f19637fe640 1 --2- 172.21.15.73:0/98682876 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f19640cba10 0x7f196406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:11.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.510+0000 7f1963fff640 1 -- 172.21.15.73:0/98682876 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f19640cba10 msgr2=0x7f196406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:11.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.510+0000 7f1963fff640 1 --2- 172.21.15.73:0/98682876 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f19640cba10 0x7f196406f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:11.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.510+0000 7f1963fff640 1 -- 172.21.15.73:0/98682876 >> v1:172.21.15.73:6789/0 conn(0x7f196406fcd0 legacy=0x7f196415f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:11.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.510+0000 7f1962ffd640 1 -- 172.21.15.73:0/98682876 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2916152310 0 0) 0x7f1964156840 con 0x7f196406fcd0 2024-04-26T20:53:11.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.510+0000 7f1963fff640 1 -- 172.21.15.73:0/98682876 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19640cc4e0 con 0x7f1964156460 2024-04-26T20:53:11.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.510+0000 7f19637fe640 1 --2- 172.21.15.73:0/98682876 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f19640cba10 0x7f196406f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:53:11.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.511+0000 7f1963fff640 1 --2- 172.21.15.73:0/98682876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1964156460 0x7f19640cb4d0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f1954009870 tx=0x7f195402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f63253c9eb706510 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:11.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.511+0000 7f1962ffd640 1 -- 172.21.15.73:0/98682876 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1954037070 con 0x7f1964156460 2024-04-26T20:53:11.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.511+0000 7f1962ffd640 1 -- 172.21.15.73:0/98682876 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1954032440 con 0x7f1964156460 2024-04-26T20:53:11.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.511+0000 7f1962ffd640 1 -- 172.21.15.73:0/98682876 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1954032810 con 0x7f1964156460 2024-04-26T20:53:11.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.511+0000 7f196a6c4640 1 -- 172.21.15.73:0/98682876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1964156460 msgr2=0x7f19640cb4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:11.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.511+0000 7f196a6c4640 1 --2- 172.21.15.73:0/98682876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1964156460 0x7f19640cb4d0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f1954009870 tx=0x7f195402a270 comp rx=0 tx=0).stop 2024-04-26T20:53:11.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.511+0000 7f196a6c4640 1 -- 172.21.15.73:0/98682876 shutdown_connections 2024-04-26T20:53:11.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.511+0000 7f196a6c4640 1 --2- 172.21.15.73:0/98682876 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f19640cba10 0x7f196406f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:11.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.511+0000 7f196a6c4640 1 --2- 172.21.15.73:0/98682876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1964156460 0x7f19640cb4d0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:11.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.511+0000 7f196a6c4640 1 -- 172.21.15.73:0/98682876 >> 172.21.15.73:0/98682876 conn(0x7f19640d0be0 msgr2=0x7f19640d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:11.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.511+0000 7f196a6c4640 1 -- 172.21.15.73:0/98682876 shutdown_connections 2024-04-26T20:53:11.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.511+0000 7f196a6c4640 1 -- 172.21.15.73:0/98682876 wait complete. 2024-04-26T20:53:11.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.512+0000 7f196a6c4640 1 Processor -- start 2024-04-26T20:53:11.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.513+0000 7f196a6c4640 1 -- start start 2024-04-26T20:53:11.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.513+0000 7f196a6c4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f196406fcd0 0x7f19640c5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:11.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.513+0000 7f196a6c4640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f19640cba10 0x7f19640b5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:11.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.513+0000 7f196a6c4640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1964156460 0x7f19641e7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:11.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.513+0000 7f196a6c4640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f196406e260 con 0x7f196406fcd0 2024-04-26T20:53:11.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.513+0000 7f196a6c4640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1964072700 con 0x7f1964156460 2024-04-26T20:53:11.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.513+0000 7f196a6c4640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f196406e3e0 con 0x7f19640cba10 2024-04-26T20:53:11.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.513+0000 7f1963fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f196406fcd0 0x7f19640c5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:11.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.513+0000 7f1963fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f196406fcd0 0x7f19640c5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56846/0 (socket says 172.21.15.73:56846) 2024-04-26T20:53:11.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.513+0000 7f1963fff640 1 -- 172.21.15.73:0/3204568138 learned_addr learned my addr 172.21.15.73:0/3204568138 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:11.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.514+0000 7f1963fff640 1 -- 172.21.15.73:0/3204568138 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f19640cba10 msgr2=0x7f19640b5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:11.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.514+0000 7f1963fff640 1 --2- 172.21.15.73:0/3204568138 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f19640cba10 0x7f19640b5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:11.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.514+0000 7f1963fff640 1 -- 172.21.15.73:0/3204568138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1964156460 msgr2=0x7f19641e7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:11.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.514+0000 7f1963fff640 1 --2- 172.21.15.73:0/3204568138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1964156460 0x7f19641e7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:11.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.514+0000 7f1963fff640 1 -- 172.21.15.73:0/3204568138 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19640c60b0 con 0x7f196406fcd0 2024-04-26T20:53:11.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.514+0000 7f1963fff640 1 --2- 172.21.15.73:0/3204568138 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f196406fcd0 0x7f19640c5760 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f1954035040 tx=0x7f19540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:11.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.514+0000 7f1960ff9640 1 -- 172.21.15.73:0/3204568138 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1954037040 con 0x7f196406fcd0 2024-04-26T20:53:11.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.514+0000 7f1960ff9640 1 -- 172.21.15.73:0/3204568138 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1954004230 con 0x7f196406fcd0 2024-04-26T20:53:11.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.514+0000 7f1960ff9640 1 -- 172.21.15.73:0/3204568138 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195403c670 con 0x7f196406fcd0 2024-04-26T20:53:11.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.514+0000 7f196a6c4640 1 -- 172.21.15.73:0/3204568138 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19640c6390 con 0x7f196406fcd0 2024-04-26T20:53:11.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.514+0000 7f196a6c4640 1 -- 172.21.15.73:0/3204568138 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19640c6950 con 0x7f196406fcd0 2024-04-26T20:53:11.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.515+0000 7f196a6c4640 1 -- 172.21.15.73:0/3204568138 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19640694f0 con 0x7f196406fcd0 2024-04-26T20:53:11.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.515+0000 7f1960ff9640 1 -- 172.21.15.73:0/3204568138 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f195403a070 con 0x7f196406fcd0 2024-04-26T20:53:11.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.516+0000 7f1960ff9640 1 --2- 172.21.15.73:0/3204568138 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f195006c8f0 0x7f195006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:11.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.516+0000 7f1960ff9640 1 -- 172.21.15.73:0/3204568138 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5421+0+0 (secure 0 0 0) 0x7f19540ac120 con 0x7f196406fcd0 2024-04-26T20:53:11.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.516+0000 7f19637fe640 1 --2- 172.21.15.73:0/3204568138 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f195006c8f0 0x7f195006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:11.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.516+0000 7f19637fe640 1 --2- 172.21.15.73:0/3204568138 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f195006c8f0 0x7f195006edb0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1958006fd0 tx=0x7f1958008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:11.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.520+0000 7f1960ff9640 1 -- 172.21.15.73:0/3204568138 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1954041020 con 0x7f196406fcd0 2024-04-26T20:53:11.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.696+0000 7f196a6c4640 1 -- 172.21.15.73:0/3204568138 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f196406de80 con 0x7f196406fcd0 2024-04-26T20:53:11.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.697+0000 7f1960ff9640 1 -- 172.21.15.73:0/3204568138 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+13423 (secure 0 0 0) 0x7f195407a700 con 0x7f196406fcd0 2024-04-26T20:53:11.698 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:53:11.699 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":59,"fsid":"fbd3f548-b547-4713-856b-09f98f5d1764","created":"2024-04-26T20:50:07.876786+0000","modified":"2024-04-26T20:53:08.609856+0000","last_up_change":"2024-04-26T20:50:19.082741+0000","last_in_change":"2024-04-26T20:50:10.018400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:50:19.993731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"72d6f4f3-214a-4392-a02f-c33cbcab3c7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6809","nonce":3461705124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6811","nonce":3461705124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6815","nonce":3461705124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3461705124},{"type":"v1","addr":"172.21.15.73:6813","nonce":3461705124}]},"public_addr":"172.21.15.73:6809/3461705124","cluster_addr":"172.21.15.73:6811/3461705124","heartbeat_back_addr":"172.21.15.73:6815/3461705124","heartbeat_front_addr":"172.21.15.73:6813/3461705124","state":["exists","up"]},{"osd":1,"uuid":"786bf8ca-6ddc-4b3c-be6c-9152398bd5fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6817","nonce":2827619403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6819","nonce":2827619403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6825","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6827","nonce":2827619403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6821","nonce":2827619403},{"type":"v1","addr":"172.21.15.73:6823","nonce":2827619403}]},"public_addr":"172.21.15.73:6817/2827619403","cluster_addr":"172.21.15.73:6819/2827619403","heartbeat_back_addr":"172.21.15.73:6827/2827619403","heartbeat_front_addr":"172.21.15.73:6823/2827619403","state":["exists","up"]},{"osd":2,"uuid":"90963b1a-5437-4ef8-ad8b-6b74c534eeb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6801","nonce":2542681064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6803","nonce":2542681064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6808","nonce":2542681064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":2542681064},{"type":"v1","addr":"172.21.15.73:6805","nonce":2542681064}]},"public_addr":"172.21.15.73:6801/2542681064","cluster_addr":"172.21.15.73:6803/2542681064","heartbeat_back_addr":"172.21.15.73:6808/2542681064","heartbeat_front_addr":"172.21.15.73:6805/2542681064","state":["exists","up"]},{"osd":3,"uuid":"bd3d2148-6e38-41b4-a453-fe6e3b8b4ceb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6822","nonce":1604042718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6826","nonce":1604042718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6831","nonce":1604042718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":1604042718},{"type":"v1","addr":"172.21.15.73:6829","nonce":1604042718}]},"public_addr":"172.21.15.73:6822/1604042718","cluster_addr":"172.21.15.73:6826/1604042718","heartbeat_back_addr":"172.21.15.73:6831/1604042718","heartbeat_front_addr":"172.21.15.73:6829/1604042718","state":["exists","up"]},{"osd":4,"uuid":"e8668dc8-a23f-46d2-8311-cb882da08e74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6817","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6819","nonce":3591055561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6824","nonce":3591055561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6831","nonce":3591055561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":3591055561},{"type":"v1","addr":"172.21.15.120:6828","nonce":3591055561}]},"public_addr":"172.21.15.120:6819/3591055561","cluster_addr":"172.21.15.120:6824/3591055561","heartbeat_back_addr":"172.21.15.120:6831/3591055561","heartbeat_front_addr":"172.21.15.120:6828/3591055561","state":["exists","up"]},{"osd":5,"uuid":"cbac0bdb-96bd-4852-bcce-e564495c3fcc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6809","nonce":819450433}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6811","nonce":819450433}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6815","nonce":819450433}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":819450433},{"type":"v1","addr":"172.21.15.120:6813","nonce":819450433}]},"public_addr":"172.21.15.120:6809/819450433","cluster_addr":"172.21.15.120:6811/819450433","heartbeat_back_addr":"172.21.15.120:6815/819450433","heartbeat_front_addr":"172.21.15.120:6813/819450433","state":["exists","up"]},{"osd":6,"uuid":"1884b2a8-569f-4272-96a9-f25a09364dc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6801","nonce":3946210727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6803","nonce":3946210727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6807","nonce":3946210727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":3946210727},{"type":"v1","addr":"172.21.15.120:6805","nonce":3946210727}]},"public_addr":"172.21.15.120:6801/3946210727","cluster_addr":"172.21.15.120:6803/3946210727","heartbeat_back_addr":"172.21.15.120:6807/3946210727","heartbeat_front_addr":"172.21.15.120:6805/3946210727","state":["exists","up"]},{"osd":7,"uuid":"c0c3bf6d-a4c5-4f1e-aa2d-330951830666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6816","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6818","nonce":2915574674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6821","nonce":2915574674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6827","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6829","nonce":2915574674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6823","nonce":2915574674},{"type":"v1","addr":"172.21.15.120:6825","nonce":2915574674}]},"public_addr":"172.21.15.120:6818/2915574674","cluster_addr":"172.21.15.120:6821/2915574674","heartbeat_back_addr":"172.21.15.120:6829/2915574674","heartbeat_front_addr":"172.21.15.120:6825/2915574674","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.218216+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.208710+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.112705+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.182789+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.164324+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.174212+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.225396+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:50:15.261559+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:6836/1169682578":"2024-04-27T20:53:05.435545+0000","172.21.15.73:6834/1169682578":"2024-04-27T20:53:05.435545+0000","172.21.15.73:6837/2770303348":"2024-04-27T20:52:53.540263+0000","172.21.15.73:6836/2770303348":"2024-04-27T20:52:53.540263+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:53:11.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.699+0000 7f196a6c4640 1 -- 172.21.15.73:0/3204568138 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f195006c8f0 msgr2=0x7f195006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:11.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.699+0000 7f196a6c4640 1 --2- 172.21.15.73:0/3204568138 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f195006c8f0 0x7f195006edb0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1958006fd0 tx=0x7f1958008040 comp rx=0 tx=0).stop 2024-04-26T20:53:11.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.699+0000 7f196a6c4640 1 -- 172.21.15.73:0/3204568138 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f196406fcd0 msgr2=0x7f19640c5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:11.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.699+0000 7f196a6c4640 1 --2- 172.21.15.73:0/3204568138 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f196406fcd0 0x7f19640c5760 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f1954035040 tx=0x7f19540046c0 comp rx=0 tx=0).stop 2024-04-26T20:53:11.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.699+0000 7f196a6c4640 1 -- 172.21.15.73:0/3204568138 shutdown_connections 2024-04-26T20:53:11.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.699+0000 7f196a6c4640 1 --2- 172.21.15.73:0/3204568138 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f195006c8f0 0x7f195006edb0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:11.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.699+0000 7f196a6c4640 1 --2- 172.21.15.73:0/3204568138 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1964156460 0x7f19641e7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:11.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.699+0000 7f196a6c4640 1 --2- 172.21.15.73:0/3204568138 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f19640cba10 0x7f19640b5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:11.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.699+0000 7f196a6c4640 1 --2- 172.21.15.73:0/3204568138 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f196406fcd0 0x7f19640c5760 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:11.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.699+0000 7f196a6c4640 1 -- 172.21.15.73:0/3204568138 >> 172.21.15.73:0/3204568138 conn(0x7f19640d0be0 msgr2=0x7f19640cf970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:11.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.699+0000 7f196a6c4640 1 -- 172.21.15.73:0/3204568138 shutdown_connections 2024-04-26T20:53:11.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:11.699+0000 7f196a6c4640 1 -- 172.21.15.73:0/3204568138 wait complete. 2024-04-26T20:53:12.712 INFO:tasks.ceph:Scrubbing osd.0 2024-04-26T20:53:12.712 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-26T20:53:12.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.824+0000 7f299dc7e640 1 Processor -- start 2024-04-26T20:53:12.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.825+0000 7f299dc7e640 1 -- start start 2024-04-26T20:53:12.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.825+0000 7f299dc7e640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f29980d2d80 0x7f29980d31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:12.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.825+0000 7f299dc7e640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29980d3700 con 0x7f29980b15b0 2024-04-26T20:53:12.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.825+0000 7f299dc7e640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29980b0510 con 0x7f299815d0d0 2024-04-26T20:53:12.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.825+0000 7f299dc7e640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f29980b1120 con 0x7f29980d2d80 2024-04-26T20:53:12.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.826+0000 7f2997fff640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f29980b15b0 0x7f29980b19e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51748/0 (socket says 172.21.15.73:51748) 2024-04-26T20:53:12.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.826+0000 7f2997fff640 1 -- 172.21.15.73:0/763700795 learned_addr learned my addr 172.21.15.73:0/763700795 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:53:12.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.826+0000 7f2996ffd640 1 --2- 172.21.15.73:0/763700795 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f29980d2d80 0x7f29980d31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:12.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.826+0000 7f29967fc640 1 -- 172.21.15.73:0/763700795 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 450319260 0 0) 0x7f29980d3700 con 0x7f29980b15b0 2024-04-26T20:53:12.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.826+0000 7f29967fc640 1 -- 172.21.15.73:0/763700795 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2980003520 con 0x7f29980b15b0 2024-04-26T20:53:12.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.826+0000 7f29967fc640 1 -- 172.21.15.73:0/763700795 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2439794142 0 0) 0x7f29980b0510 con 0x7f299815d0d0 2024-04-26T20:53:12.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.826+0000 7f29967fc640 1 -- 172.21.15.73:0/763700795 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f29980d3700 con 0x7f299815d0d0 2024-04-26T20:53:12.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.826+0000 7f29967fc640 1 -- 172.21.15.73:0/763700795 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2489033115 0 0) 0x7f2980003520 con 0x7f29980b15b0 2024-04-26T20:53:12.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.826+0000 7f2996ffd640 1 -- 172.21.15.73:0/763700795 >> v1:172.21.15.120:6789/0 conn(0x7f299815d0d0 legacy=0x7f29980afb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:12.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.826+0000 7f2996ffd640 1 -- 172.21.15.73:0/763700795 >> v1:172.21.15.73:6789/0 conn(0x7f29980b15b0 legacy=0x7f29980b19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:12.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.826+0000 7f2996ffd640 1 -- 172.21.15.73:0/763700795 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29980b07f0 con 0x7f29980d2d80 2024-04-26T20:53:12.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.827+0000 7f2996ffd640 1 --2- 172.21.15.73:0/763700795 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f29980d2d80 0x7f29980d31c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f298c0097d0 tx=0x7f298c02a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=516326bd437d855c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:12.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.827+0000 7f29967fc640 1 -- 172.21.15.73:0/763700795 <== mon.2 v2:172.21.15.120:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298c037070 con 0x7f29980d2d80 2024-04-26T20:53:12.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.827+0000 7f29967fc640 1 -- 172.21.15.73:0/763700795 <== mon.2 v2:172.21.15.120:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f298c02ac30 con 0x7f29980d2d80 2024-04-26T20:53:12.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.827+0000 7f29967fc640 1 -- 172.21.15.73:0/763700795 <== mon.2 v2:172.21.15.120:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29900019f0 con 0x7f29980d2d80 2024-04-26T20:53:12.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.827+0000 7f299dc7e640 1 -- 172.21.15.73:0/763700795 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f29980d2d80 msgr2=0x7f29980d31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:12.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.827+0000 7f299dc7e640 1 --2- 172.21.15.73:0/763700795 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f29980d2d80 0x7f29980d31c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f298c0097d0 tx=0x7f298c02a1a0 comp rx=0 tx=0).stop 2024-04-26T20:53:12.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.828+0000 7f299dc7e640 1 -- 172.21.15.73:0/763700795 shutdown_connections 2024-04-26T20:53:12.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.828+0000 7f299dc7e640 1 --2- 172.21.15.73:0/763700795 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f29980d2d80 0x7f29980d31c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:12.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.828+0000 7f299dc7e640 1 -- 172.21.15.73:0/763700795 >> 172.21.15.73:0/763700795 conn(0x7f299809ba50 msgr2=0x7f299809be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:12.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.828+0000 7f299dc7e640 1 -- 172.21.15.73:0/763700795 shutdown_connections 2024-04-26T20:53:12.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.828+0000 7f299dc7e640 1 -- 172.21.15.73:0/763700795 wait complete. 2024-04-26T20:53:12.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.829+0000 7f299dc7e640 1 Processor -- start 2024-04-26T20:53:12.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.829+0000 7f299dc7e640 1 -- start start 2024-04-26T20:53:12.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.830+0000 7f299dc7e640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f29980b15b0 0x7f29980c7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:12.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.830+0000 7f299dc7e640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29980d2d80 0x7f299815b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:12.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.830+0000 7f299dc7e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f299815d0d0 0x7f29980b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:12.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.830+0000 7f299dc7e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f29980b0230 con 0x7f299815d0d0 2024-04-26T20:53:12.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.830+0000 7f299dc7e640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f29980b0db0 con 0x7f29980d2d80 2024-04-26T20:53:12.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.830+0000 7f299dc7e640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f299815d4b0 con 0x7f29980b15b0 2024-04-26T20:53:12.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.830+0000 7f2997fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f299815d0d0 0x7f29980b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:12.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.830+0000 7f2997fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f299815d0d0 0x7f29980b3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56852/0 (socket says 172.21.15.73:56852) 2024-04-26T20:53:12.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.830+0000 7f2997fff640 1 -- 172.21.15.73:0/3769622750 learned_addr learned my addr 172.21.15.73:0/3769622750 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:12.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.830+0000 7f2996ffd640 1 --2- 172.21.15.73:0/3769622750 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29980d2d80 0x7f299815b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:12.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.830+0000 7f29977fe640 1 --2- 172.21.15.73:0/3769622750 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f29980b15b0 0x7f29980c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:12.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.830+0000 7f2997fff640 1 -- 172.21.15.73:0/3769622750 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f29980b15b0 msgr2=0x7f29980c7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:12.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.830+0000 7f2997fff640 1 --2- 172.21.15.73:0/3769622750 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f29980b15b0 0x7f29980c7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:12.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.830+0000 7f2997fff640 1 -- 172.21.15.73:0/3769622750 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29980d2d80 msgr2=0x7f299815b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:12.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.830+0000 7f2997fff640 1 --2- 172.21.15.73:0/3769622750 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29980d2d80 0x7f299815b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:12.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.830+0000 7f2997fff640 1 -- 172.21.15.73:0/3769622750 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f299815b760 con 0x7f299815d0d0 2024-04-26T20:53:12.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.830+0000 7f29977fe640 1 --2- 172.21.15.73:0/3769622750 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f29980b15b0 0x7f29980c7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:53:12.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.830+0000 7f2997fff640 1 --2- 172.21.15.73:0/3769622750 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f299815d0d0 0x7f29980b3b20 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f2990001350 tx=0x7f299000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:12.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.830+0000 7f297ffff640 1 -- 172.21.15.73:0/3769622750 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2990017070 con 0x7f299815d0d0 2024-04-26T20:53:12.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.830+0000 7f297ffff640 1 -- 172.21.15.73:0/3769622750 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2990012400 con 0x7f299815d0d0 2024-04-26T20:53:12.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.830+0000 7f299dc7e640 1 -- 172.21.15.73:0/3769622750 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29980c8700 con 0x7f299815d0d0 2024-04-26T20:53:12.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.830+0000 7f297ffff640 1 -- 172.21.15.73:0/3769622750 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2990012830 con 0x7f299815d0d0 2024-04-26T20:53:12.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.831+0000 7f299dc7e640 1 -- 172.21.15.73:0/3769622750 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29980c8cc0 con 0x7f299815d0d0 2024-04-26T20:53:12.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.831+0000 7f299dc7e640 1 -- 172.21.15.73:0/3769622750 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2998069600 con 0x7f299815d0d0 2024-04-26T20:53:12.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.832+0000 7f297ffff640 1 -- 172.21.15.73:0/3769622750 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f299001a070 con 0x7f299815d0d0 2024-04-26T20:53:12.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.832+0000 7f297ffff640 1 --2- 172.21.15.73:0/3769622750 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f298406c8f0 0x7f298406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:12.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.832+0000 7f297ffff640 1 -- 172.21.15.73:0/3769622750 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5421+0+0 (secure 0 0 0) 0x7f299008b7e0 con 0x7f299815d0d0 2024-04-26T20:53:12.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.832+0000 7f29977fe640 1 --2- 172.21.15.73:0/3769622750 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f298406c8f0 0x7f298406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:12.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.832+0000 7f297ffff640 1 --2- 172.21.15.73:0/3769622750 >> [v2:172.21.15.73:6807/3461705124,v1:172.21.15.73:6809/3461705124] conn(0x7f2984072e30 0x7f2984075270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:12.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.833+0000 7f297ffff640 1 -- 172.21.15.73:0/3769622750 --> [v2:172.21.15.73:6807/3461705124,v1:172.21.15.73:6809/3461705124] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f299000de50 con 0x7f2984072e30 2024-04-26T20:53:12.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.833+0000 7f297ffff640 1 -- 172.21.15.73:0/3769622750 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f29900902c0 con 0x7f299815d0d0 2024-04-26T20:53:12.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.833+0000 7f2996ffd640 1 --2- 172.21.15.73:0/3769622750 >> [v2:172.21.15.73:6807/3461705124,v1:172.21.15.73:6809/3461705124] conn(0x7f2984072e30 0x7f2984075270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:12.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.833+0000 7f29977fe640 1 --2- 172.21.15.73:0/3769622750 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f298406c8f0 0x7f298406edb0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f29880011e0 tx=0x7f2988009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:12.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.833+0000 7f2996ffd640 1 --2- 172.21.15.73:0/3769622750 >> [v2:172.21.15.73:6807/3461705124,v1:172.21.15.73:6809/3461705124] conn(0x7f2984072e30 0x7f2984075270 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:12.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.834+0000 7f297ffff640 1 -- 172.21.15.73:0/3769622750 <== osd.0 v2:172.21.15.73:6807/3461705124 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f299000de50 con 0x7f2984072e30 2024-04-26T20:53:12.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.847+0000 7f299dc7e640 1 -- 172.21.15.73:0/3769622750 --> [v2:172.21.15.73:6807/3461705124,v1:172.21.15.73:6809/3461705124] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f29980d1b70 con 0x7f2984072e30 2024-04-26T20:53:12.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.847+0000 7f297ffff640 1 -- 172.21.15.73:0/3769622750 <== osd.0 v2:172.21.15.73:6807/3461705124 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f29980d1b70 con 0x7f2984072e30 2024-04-26T20:53:12.848 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-26T20:53:12.848 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T20:53:12.848 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-26T20:53:12.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.847+0000 7f299dc7e640 1 -- 172.21.15.73:0/3769622750 >> [v2:172.21.15.73:6807/3461705124,v1:172.21.15.73:6809/3461705124] conn(0x7f2984072e30 msgr2=0x7f2984075270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:12.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.847+0000 7f299dc7e640 1 --2- 172.21.15.73:0/3769622750 >> [v2:172.21.15.73:6807/3461705124,v1:172.21.15.73:6809/3461705124] conn(0x7f2984072e30 0x7f2984075270 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:12.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.847+0000 7f299dc7e640 1 -- 172.21.15.73:0/3769622750 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f298406c8f0 msgr2=0x7f298406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:12.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.847+0000 7f299dc7e640 1 --2- 172.21.15.73:0/3769622750 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f298406c8f0 0x7f298406edb0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f29880011e0 tx=0x7f2988009040 comp rx=0 tx=0).stop 2024-04-26T20:53:12.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.847+0000 7f299dc7e640 1 -- 172.21.15.73:0/3769622750 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f299815d0d0 msgr2=0x7f29980b3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:12.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.847+0000 7f299dc7e640 1 --2- 172.21.15.73:0/3769622750 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f299815d0d0 0x7f29980b3b20 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f2990001350 tx=0x7f299000cf90 comp rx=0 tx=0).stop 2024-04-26T20:53:12.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.847+0000 7f2997fff640 1 -- 172.21.15.73:0/3769622750 reap_dead start 2024-04-26T20:53:12.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.847+0000 7f299dc7e640 1 -- 172.21.15.73:0/3769622750 shutdown_connections 2024-04-26T20:53:12.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.847+0000 7f299dc7e640 1 -- 172.21.15.73:0/3769622750 >> 172.21.15.73:0/3769622750 conn(0x7f299809ba50 msgr2=0x7f299815dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:12.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.847+0000 7f299dc7e640 1 -- 172.21.15.73:0/3769622750 shutdown_connections 2024-04-26T20:53:12.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.847+0000 7f299dc7e640 1 -- 172.21.15.73:0/3769622750 wait complete. 2024-04-26T20:53:12.859 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-26T20:53:12.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.957+0000 7f29b516a640 1 Processor -- start 2024-04-26T20:53:12.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.957+0000 7f29b516a640 1 -- start start 2024-04-26T20:53:12.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.958+0000 7f29b516a640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f29b00b0380 0x7f29b00ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:12.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.958+0000 7f29b516a640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29b00cfd80 con 0x7f29b00b10b0 2024-04-26T20:53:12.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.958+0000 7f29b516a640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29b00d0020 con 0x7f29b006c950 2024-04-26T20:53:12.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.958+0000 7f29b516a640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f29b00af8e0 con 0x7f29b00b0380 2024-04-26T20:53:12.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.958+0000 7f29ae59c640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f29b00b10b0 0x7f29b006c240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51756/0 (socket says 172.21.15.73:51756) 2024-04-26T20:53:12.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.958+0000 7f29ae59c640 1 -- 172.21.15.73:0/3610626906 learned_addr learned my addr 172.21.15.73:0/3610626906 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:53:12.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.958+0000 7f29aed9d640 1 --2- 172.21.15.73:0/3610626906 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f29b00b0380 0x7f29b00ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:12.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.958+0000 7f29add9b640 1 -- 172.21.15.73:0/3610626906 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1495600015 0 0) 0x7f29b00cfd80 con 0x7f29b00b10b0 2024-04-26T20:53:12.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.958+0000 7f29add9b640 1 -- 172.21.15.73:0/3610626906 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2998003520 con 0x7f29b00b10b0 2024-04-26T20:53:12.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.958+0000 7f29add9b640 1 -- 172.21.15.73:0/3610626906 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3746172135 0 0) 0x7f2998003520 con 0x7f29b00b10b0 2024-04-26T20:53:12.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.958+0000 7f29add9b640 1 -- 172.21.15.73:0/3610626906 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f29b00b0380 msgr2=0x7f29b00ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:12.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.958+0000 7f29add9b640 1 --2- 172.21.15.73:0/3610626906 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f29b00b0380 0x7f29b00ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:12.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.958+0000 7f29add9b640 1 -- 172.21.15.73:0/3610626906 >> v1:172.21.15.120:6789/0 conn(0x7f29b006c950 legacy=0x7f29b015e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:12.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.958+0000 7f29add9b640 1 -- 172.21.15.73:0/3610626906 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29b00d0300 con 0x7f29b00b10b0 2024-04-26T20:53:12.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.958+0000 7f29add9b640 1 -- 172.21.15.73:0/3610626906 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7f29a4001670 con 0x7f29b00b10b0 2024-04-26T20:53:12.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.958+0000 7f29add9b640 1 -- 172.21.15.73:0/3610626906 >> v1:172.21.15.73:6789/0 conn(0x7f29b00b10b0 legacy=0x7f29b006c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:12.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.958+0000 7f29add9b640 1 --2- 172.21.15.73:0/3610626906 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29980039e0 0x7f2998003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:12.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.959+0000 7f29add9b640 1 --2- 172.21.15.73:0/3610626906 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2998004800 0x7f2998004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:12.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.959+0000 7f29add9b640 1 --2- 172.21.15.73:0/3610626906 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f29b00b0380 0x7f2998005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:12.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.959+0000 7f29add9b640 1 -- 172.21.15.73:0/3610626906 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f29b00af8e0 con 0x7f2998004800 2024-04-26T20:53:12.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.959+0000 7f29add9b640 1 -- 172.21.15.73:0/3610626906 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f299800baf0 con 0x7f29980039e0 2024-04-26T20:53:12.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.959+0000 7f29add9b640 1 -- 172.21.15.73:0/3610626906 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f299800bcf0 con 0x7f29b00b0380 2024-04-26T20:53:12.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.959+0000 7f29ae59c640 1 --2- 172.21.15.73:0/3610626906 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2998004800 0x7f2998004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:12.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.959+0000 7f29aed9d640 1 --2- 172.21.15.73:0/3610626906 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29980039e0 0x7f2998003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:12.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.959+0000 7f29af59e640 1 --2- 172.21.15.73:0/3610626906 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f29b00b0380 0x7f2998005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:12.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.959+0000 7f29ae59c640 1 -- 172.21.15.73:0/3610626906 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f29b00b0380 msgr2=0x7f2998005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:12.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.959+0000 7f29ae59c640 1 --2- 172.21.15.73:0/3610626906 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f29b00b0380 0x7f2998005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:12.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.959+0000 7f29ae59c640 1 -- 172.21.15.73:0/3610626906 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29980039e0 msgr2=0x7f2998003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:12.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.959+0000 7f29ae59c640 1 --2- 172.21.15.73:0/3610626906 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f29980039e0 0x7f2998003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:12.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.959+0000 7f29ae59c640 1 -- 172.21.15.73:0/3610626906 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2998003520 con 0x7f2998004800 2024-04-26T20:53:12.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.959+0000 7f29af59e640 1 --2- 172.21.15.73:0/3610626906 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f29b00b0380 0x7f2998005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:53:12.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.960+0000 7f29ae59c640 1 --2- 172.21.15.73:0/3610626906 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2998004800 0x7f2998004c80 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f29a400c240 tx=0x7f2998005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bd2b724dea2447b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:12.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.960+0000 7f29add9b640 1 -- 172.21.15.73:0/3610626906 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a4007cf0 con 0x7f2998004800 2024-04-26T20:53:12.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.960+0000 7f29add9b640 1 -- 172.21.15.73:0/3610626906 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29a40058d0 con 0x7f2998004800 2024-04-26T20:53:12.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.960+0000 7f29add9b640 1 -- 172.21.15.73:0/3610626906 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a4005ce0 con 0x7f2998004800 2024-04-26T20:53:12.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.960+0000 7f29b516a640 1 -- 172.21.15.73:0/3610626906 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2998004800 msgr2=0x7f2998004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:12.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.960+0000 7f29b516a640 1 --2- 172.21.15.73:0/3610626906 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2998004800 0x7f2998004c80 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f29a400c240 tx=0x7f2998005830 comp rx=0 tx=0).stop 2024-04-26T20:53:12.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.960+0000 7f29af59e640 1 -- 172.21.15.73:0/3610626906 reap_dead start 2024-04-26T20:53:12.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.960+0000 7f29b516a640 1 -- 172.21.15.73:0/3610626906 shutdown_connections 2024-04-26T20:53:12.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.960+0000 7f29b516a640 1 -- 172.21.15.73:0/3610626906 >> 172.21.15.73:0/3610626906 conn(0x7f29b00d1cf0 msgr2=0x7f29b00d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:12.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.960+0000 7f29b516a640 1 -- 172.21.15.73:0/3610626906 shutdown_connections 2024-04-26T20:53:12.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.960+0000 7f29b516a640 1 -- 172.21.15.73:0/3610626906 wait complete. 2024-04-26T20:53:12.967 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.962+0000 7f29b516a640 1 Processor -- start 2024-04-26T20:53:12.967 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.962+0000 7f29b516a640 1 -- start start 2024-04-26T20:53:12.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.962+0000 7f29b516a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2998004800 0x7f29b01f4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:12.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.962+0000 7f29b516a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29b015d2e0 0x7f29b01607f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:12.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.962+0000 7f29b516a640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f29b006fd80 0x7f29b0153b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:12.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.962+0000 7f29b516a640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f29b00b0760 con 0x7f29b015d2e0 2024-04-26T20:53:12.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.962+0000 7f29b516a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f29b00d1980 con 0x7f2998004800 2024-04-26T20:53:12.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.962+0000 7f29b516a640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f29b00d15c0 con 0x7f29b006fd80 2024-04-26T20:53:12.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.962+0000 7f29ae59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29b015d2e0 0x7f29b01607f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:12.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.962+0000 7f29ae59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29b015d2e0 0x7f29b01607f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56862/0 (socket says 172.21.15.73:56862) 2024-04-26T20:53:12.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.962+0000 7f29ae59c640 1 -- 172.21.15.73:0/215938119 learned_addr learned my addr 172.21.15.73:0/215938119 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:12.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.963+0000 7f29af59e640 1 --2- 172.21.15.73:0/215938119 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f29b006fd80 0x7f29b0153b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:12.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.963+0000 7f29aed9d640 1 --2- 172.21.15.73:0/215938119 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2998004800 0x7f29b01f4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:12.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.963+0000 7f29ae59c640 1 -- 172.21.15.73:0/215938119 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f29b006fd80 msgr2=0x7f29b0153b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:12.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.963+0000 7f29ae59c640 1 --2- 172.21.15.73:0/215938119 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f29b006fd80 0x7f29b0153b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:12.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.963+0000 7f29ae59c640 1 -- 172.21.15.73:0/215938119 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2998004800 msgr2=0x7f29b01f4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:12.968 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.963+0000 7f29ae59c640 1 --2- 172.21.15.73:0/215938119 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2998004800 0x7f29b01f4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:12.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.963+0000 7f29ae59c640 1 -- 172.21.15.73:0/215938119 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29b015de20 con 0x7f29b015d2e0 2024-04-26T20:53:12.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.963+0000 7f29ae59c640 1 --2- 172.21.15.73:0/215938119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29b015d2e0 0x7f29b01607f0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f29a400c360 tx=0x7f29b00d04d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:12.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.963+0000 7f29977fe640 1 -- 172.21.15.73:0/215938119 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a4038040 con 0x7f29b015d2e0 2024-04-26T20:53:12.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.963+0000 7f29977fe640 1 -- 172.21.15.73:0/215938119 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29a4042070 con 0x7f29b015d2e0 2024-04-26T20:53:12.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.963+0000 7f29b516a640 1 -- 172.21.15.73:0/215938119 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29b00b5fc0 con 0x7f29b015d2e0 2024-04-26T20:53:12.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.963+0000 7f29977fe640 1 -- 172.21.15.73:0/215938119 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a40049e0 con 0x7f29b015d2e0 2024-04-26T20:53:12.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.963+0000 7f29b516a640 1 -- 172.21.15.73:0/215938119 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29b00b6580 con 0x7f29b015d2e0 2024-04-26T20:53:12.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.964+0000 7f29b516a640 1 -- 172.21.15.73:0/215938119 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29b00694f0 con 0x7f29b015d2e0 2024-04-26T20:53:12.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.965+0000 7f29977fe640 1 -- 172.21.15.73:0/215938119 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f29a4004b80 con 0x7f29b015d2e0 2024-04-26T20:53:12.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.965+0000 7f29977fe640 1 --2- 172.21.15.73:0/215938119 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f299c06c8f0 0x7f299c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:12.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.965+0000 7f29977fe640 1 -- 172.21.15.73:0/215938119 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5421+0+0 (secure 0 0 0) 0x7f29a40aba90 con 0x7f29b015d2e0 2024-04-26T20:53:12.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.965+0000 7f29aed9d640 1 --2- 172.21.15.73:0/215938119 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f299c06c8f0 0x7f299c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:12.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.965+0000 7f29aed9d640 1 --2- 172.21.15.73:0/215938119 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f299c06c8f0 0x7f299c06edb0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f29a00099e0 tx=0x7f29a0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:12.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:12.969+0000 7f29977fe640 1 -- 172.21.15.73:0/215938119 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f29a407a190 con 0x7f29b015d2e0 2024-04-26T20:53:13.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.144+0000 7f29b516a640 1 -- 172.21.15.73:0/215938119 --> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f29b0160d30 con 0x7f299c06c8f0 2024-04-26T20:53:13.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.144+0000 7f29977fe640 1 -- 172.21.15.73:0/215938119 <== mgr.4107 v2:172.21.15.73:6832/666905549 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f29b0160d30 con 0x7f299c06c8f0 2024-04-26T20:53:13.145 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 0 to deep-scrub 2024-04-26T20:53:13.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.146+0000 7f29b516a640 1 -- 172.21.15.73:0/215938119 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f299c06c8f0 msgr2=0x7f299c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:13.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.146+0000 7f29b516a640 1 --2- 172.21.15.73:0/215938119 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f299c06c8f0 0x7f299c06edb0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f29a00099e0 tx=0x7f29a0008040 comp rx=0 tx=0).stop 2024-04-26T20:53:13.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.146+0000 7f29b516a640 1 -- 172.21.15.73:0/215938119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29b015d2e0 msgr2=0x7f29b01607f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:13.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.146+0000 7f29b516a640 1 --2- 172.21.15.73:0/215938119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29b015d2e0 0x7f29b01607f0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f29a400c360 tx=0x7f29b00d04d0 comp rx=0 tx=0).stop 2024-04-26T20:53:13.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.146+0000 7f29b516a640 1 -- 172.21.15.73:0/215938119 shutdown_connections 2024-04-26T20:53:13.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.146+0000 7f29b516a640 1 --2- 172.21.15.73:0/215938119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29b015d2e0 0x7f29b01607f0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:13.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.146+0000 7f29b516a640 1 --2- 172.21.15.73:0/215938119 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f299c06c8f0 0x7f299c06edb0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:13.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.146+0000 7f29b516a640 1 --2- 172.21.15.73:0/215938119 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f29b006fd80 0x7f29b0153b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:13.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.146+0000 7f29b516a640 1 --2- 172.21.15.73:0/215938119 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f2998004800 0x7f29b01f4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:13.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.146+0000 7f29b516a640 1 -- 172.21.15.73:0/215938119 >> 172.21.15.73:0/215938119 conn(0x7f29b00d1cf0 msgr2=0x7f29b00ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:13.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.146+0000 7f29b516a640 1 -- 172.21.15.73:0/215938119 shutdown_connections 2024-04-26T20:53:13.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.146+0000 7f29b516a640 1 -- 172.21.15.73:0/215938119 wait complete. 2024-04-26T20:53:13.158 INFO:tasks.ceph:Scrubbing osd.1 2024-04-26T20:53:13.158 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-26T20:53:13.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.258+0000 7f7f32551640 1 Processor -- start 2024-04-26T20:53:13.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.258+0000 7f7f32551640 1 -- start start 2024-04-26T20:53:13.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.259+0000 7f7f32551640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7f2c145630 0x7f7f2c0cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:13.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.259+0000 7f7f32551640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f7f2c15f980 0x7f7f2c0b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:13.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.259+0000 7f7f32551640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f2c0b33c0 con 0x7f7f2c0b2fc0 2024-04-26T20:53:13.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.259+0000 7f7f32551640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f7f2c0cfd70 con 0x7f7f2c145630 2024-04-26T20:53:13.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.259+0000 7f7f32551640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f7f2c0d0050 con 0x7f7f2c15f980 2024-04-26T20:53:13.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.259+0000 7f7f2bfff640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7f2c145630 0x7f7f2c0cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:13.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.259+0000 7f7f30aee640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f7f2c0b2fc0 0x7f7f2c0730d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51760/0 (socket says 172.21.15.73:51760) 2024-04-26T20:53:13.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.259+0000 7f7f30aee640 1 -- 172.21.15.73:0/1436598876 learned_addr learned my addr 172.21.15.73:0/1436598876 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:53:13.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.259+0000 7f7f2bfff640 1 -- 172.21.15.73:0/1436598876 >> v2:172.21.15.120:3300/0 conn(0x7f7f2c15f980 msgr2=0x7f7f2c0b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:13.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.259+0000 7f7f2bfff640 1 --2- 172.21.15.73:0/1436598876 >> v2:172.21.15.120:3300/0 conn(0x7f7f2c15f980 0x7f7f2c0b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:13.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.259+0000 7f7f2bfff640 1 -- 172.21.15.73:0/1436598876 >> v1:172.21.15.73:6789/0 conn(0x7f7f2c0b2fc0 legacy=0x7f7f2c0730d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:13.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.259+0000 7f7f2affd640 1 -- 172.21.15.73:0/1436598876 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1913657026 0 0) 0x7f7f2c0b33c0 con 0x7f7f2c0b2fc0 2024-04-26T20:53:13.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.259+0000 7f7f2bfff640 1 -- 172.21.15.73:0/1436598876 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f2c0d0330 con 0x7f7f2c145630 2024-04-26T20:53:13.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.260+0000 7f7f2bfff640 1 --2- 172.21.15.73:0/1436598876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f2c145630 0x7f7f2c0cf500 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c070470 tx=0x7f7f1c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=619e2ebda9a627a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:13.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.260+0000 7f7f2affd640 1 -- 172.21.15.73:0/1436598876 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c037070 con 0x7f7f2c145630 2024-04-26T20:53:13.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.260+0000 7f7f2affd640 1 -- 172.21.15.73:0/1436598876 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f1c032440 con 0x7f7f2c145630 2024-04-26T20:53:13.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.260+0000 7f7f2affd640 1 -- 172.21.15.73:0/1436598876 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c032810 con 0x7f7f2c145630 2024-04-26T20:53:13.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.261+0000 7f7f32551640 1 -- 172.21.15.73:0/1436598876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f2c145630 msgr2=0x7f7f2c0cf500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:13.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.261+0000 7f7f32551640 1 --2- 172.21.15.73:0/1436598876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f2c145630 0x7f7f2c0cf500 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c070470 tx=0x7f7f1c02a1a0 comp rx=0 tx=0).stop 2024-04-26T20:53:13.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.261+0000 7f7f32551640 1 -- 172.21.15.73:0/1436598876 shutdown_connections 2024-04-26T20:53:13.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.261+0000 7f7f32551640 1 --2- 172.21.15.73:0/1436598876 >> v2:172.21.15.120:3300/0 conn(0x7f7f2c15f980 0x7f7f2c0b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:13.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.261+0000 7f7f32551640 1 --2- 172.21.15.73:0/1436598876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f2c145630 0x7f7f2c0cf500 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:13.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.261+0000 7f7f32551640 1 -- 172.21.15.73:0/1436598876 >> 172.21.15.73:0/1436598876 conn(0x7f7f2c06f310 msgr2=0x7f7f2c06f720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:13.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.261+0000 7f7f32551640 1 -- 172.21.15.73:0/1436598876 shutdown_connections 2024-04-26T20:53:13.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.261+0000 7f7f32551640 1 -- 172.21.15.73:0/1436598876 wait complete. 2024-04-26T20:53:13.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.262+0000 7f7f32551640 1 Processor -- start 2024-04-26T20:53:13.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.262+0000 7f7f32551640 1 -- start start 2024-04-26T20:53:13.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.263+0000 7f7f32551640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f2c0b2fc0 0x7f7f2c1407a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:13.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.263+0000 7f7f32551640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7f2c145630 0x7f7f2c1f8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:13.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.263+0000 7f7f32551640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f2c15f980 0x7f7f2c1f9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:13.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.263+0000 7f7f32551640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2c0cfa40 con 0x7f7f2c0b2fc0 2024-04-26T20:53:13.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.263+0000 7f7f32551640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2c0d08f0 con 0x7f7f2c15f980 2024-04-26T20:53:13.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.263+0000 7f7f32551640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f7f2c145a10 con 0x7f7f2c145630 2024-04-26T20:53:13.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.263+0000 7f7f2bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f2c0b2fc0 0x7f7f2c1407a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:13.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.263+0000 7f7f2bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f2c0b2fc0 0x7f7f2c1407a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56878/0 (socket says 172.21.15.73:56878) 2024-04-26T20:53:13.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.263+0000 7f7f2bfff640 1 -- 172.21.15.73:0/3789352339 learned_addr learned my addr 172.21.15.73:0/3789352339 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:13.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.263+0000 7f7f2bfff640 1 -- 172.21.15.73:0/3789352339 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7f2c145630 msgr2=0x7f7f2c1f8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:13.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.263+0000 7f7f2b7fe640 1 --2- 172.21.15.73:0/3789352339 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7f2c145630 0x7f7f2c1f8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:13.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.263+0000 7f7f2bfff640 1 --2- 172.21.15.73:0/3789352339 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f7f2c145630 0x7f7f2c1f8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:13.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.263+0000 7f7f2bfff640 1 -- 172.21.15.73:0/3789352339 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f2c15f980 msgr2=0x7f7f2c1f9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:13.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.263+0000 7f7f2bfff640 1 --2- 172.21.15.73:0/3789352339 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f7f2c15f980 0x7f7f2c1f9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:13.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.263+0000 7f7f2bfff640 1 -- 172.21.15.73:0/3789352339 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f2c15b7d0 con 0x7f7f2c0b2fc0 2024-04-26T20:53:13.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.263+0000 7f7f2bfff640 1 --2- 172.21.15.73:0/3789352339 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f2c0b2fc0 0x7f7f2c1407a0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c000c00 tx=0x7f7f1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:13.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.264+0000 7f7f28ff9640 1 -- 172.21.15.73:0/3789352339 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c041070 con 0x7f7f2c0b2fc0 2024-04-26T20:53:13.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.264+0000 7f7f28ff9640 1 -- 172.21.15.73:0/3789352339 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f1c03c950 con 0x7f7f2c0b2fc0 2024-04-26T20:53:13.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.264+0000 7f7f32551640 1 -- 172.21.15.73:0/3789352339 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f2c1e6bb0 con 0x7f7f2c0b2fc0 2024-04-26T20:53:13.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.264+0000 7f7f28ff9640 1 -- 172.21.15.73:0/3789352339 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c037040 con 0x7f7f2c0b2fc0 2024-04-26T20:53:13.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.264+0000 7f7f32551640 1 -- 172.21.15.73:0/3789352339 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f2c1e7170 con 0x7f7f2c0b2fc0 2024-04-26T20:53:13.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.264+0000 7f7f32551640 1 -- 172.21.15.73:0/3789352339 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7f2c069600 con 0x7f7f2c0b2fc0 2024-04-26T20:53:13.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.265+0000 7f7f28ff9640 1 -- 172.21.15.73:0/3789352339 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7f1c044070 con 0x7f7f2c0b2fc0 2024-04-26T20:53:13.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.265+0000 7f7f28ff9640 1 --2- 172.21.15.73:0/3789352339 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f7f1806c8f0 0x7f7f1806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:13.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.265+0000 7f7f28ff9640 1 -- 172.21.15.73:0/3789352339 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5421+0+0 (secure 0 0 0) 0x7f7f1c0ac6a0 con 0x7f7f2c0b2fc0 2024-04-26T20:53:13.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.265+0000 7f7f2b7fe640 1 --2- 172.21.15.73:0/3789352339 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f7f1806c8f0 0x7f7f1806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:13.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.266+0000 7f7f28ff9640 1 --2- 172.21.15.73:0/3789352339 >> [v2:172.21.15.73:6816/2827619403,v1:172.21.15.73:6817/2827619403] conn(0x7f7f18072e30 0x7f7f18075270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:13.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.266+0000 7f7f28ff9640 1 -- 172.21.15.73:0/3789352339 --> [v2:172.21.15.73:6816/2827619403,v1:172.21.15.73:6817/2827619403] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7f1c001d30 con 0x7f7f18072e30 2024-04-26T20:53:13.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.266+0000 7f7f28ff9640 1 -- 172.21.15.73:0/3789352339 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f7f1c0b3050 con 0x7f7f2c0b2fc0 2024-04-26T20:53:13.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.266+0000 7f7f30aee640 1 --2- 172.21.15.73:0/3789352339 >> [v2:172.21.15.73:6816/2827619403,v1:172.21.15.73:6817/2827619403] conn(0x7f7f18072e30 0x7f7f18075270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:13.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.266+0000 7f7f2b7fe640 1 --2- 172.21.15.73:0/3789352339 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f7f1806c8f0 0x7f7f1806edb0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c15bf50 tx=0x7f7f20008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:13.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.266+0000 7f7f30aee640 1 --2- 172.21.15.73:0/3789352339 >> [v2:172.21.15.73:6816/2827619403,v1:172.21.15.73:6817/2827619403] conn(0x7f7f18072e30 0x7f7f18075270 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:13.271 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.267+0000 7f7f28ff9640 1 -- 172.21.15.73:0/3789352339 <== osd.1 v2:172.21.15.73:6816/2827619403 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7f1c001d30 con 0x7f7f18072e30 2024-04-26T20:53:13.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.280+0000 7f7f32551640 1 -- 172.21.15.73:0/3789352339 --> [v2:172.21.15.73:6816/2827619403,v1:172.21.15.73:6817/2827619403] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f7f2c15f580 con 0x7f7f18072e30 2024-04-26T20:53:13.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.280+0000 7f7f28ff9640 1 -- 172.21.15.73:0/3789352339 <== osd.1 v2:172.21.15.73:6816/2827619403 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f7f2c15f580 con 0x7f7f18072e30 2024-04-26T20:53:13.281 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-26T20:53:13.281 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T20:53:13.281 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-26T20:53:13.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.280+0000 7f7f32551640 1 -- 172.21.15.73:0/3789352339 >> [v2:172.21.15.73:6816/2827619403,v1:172.21.15.73:6817/2827619403] conn(0x7f7f18072e30 msgr2=0x7f7f18075270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:13.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.280+0000 7f7f32551640 1 --2- 172.21.15.73:0/3789352339 >> [v2:172.21.15.73:6816/2827619403,v1:172.21.15.73:6817/2827619403] conn(0x7f7f18072e30 0x7f7f18075270 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:13.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.280+0000 7f7f32551640 1 -- 172.21.15.73:0/3789352339 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f7f1806c8f0 msgr2=0x7f7f1806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:13.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.280+0000 7f7f32551640 1 --2- 172.21.15.73:0/3789352339 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f7f1806c8f0 0x7f7f1806edb0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c15bf50 tx=0x7f7f20008040 comp rx=0 tx=0).stop 2024-04-26T20:53:13.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.280+0000 7f7f32551640 1 -- 172.21.15.73:0/3789352339 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f2c0b2fc0 msgr2=0x7f7f2c1407a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:13.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.280+0000 7f7f32551640 1 --2- 172.21.15.73:0/3789352339 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f2c0b2fc0 0x7f7f2c1407a0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c000c00 tx=0x7f7f1c0046c0 comp rx=0 tx=0).stop 2024-04-26T20:53:13.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.280+0000 7f7f30aee640 1 -- 172.21.15.73:0/3789352339 reap_dead start 2024-04-26T20:53:13.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.280+0000 7f7f32551640 1 -- 172.21.15.73:0/3789352339 shutdown_connections 2024-04-26T20:53:13.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.280+0000 7f7f32551640 1 -- 172.21.15.73:0/3789352339 >> 172.21.15.73:0/3789352339 conn(0x7f7f2c06f310 msgr2=0x7f7f2c0b5960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:13.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.280+0000 7f7f32551640 1 -- 172.21.15.73:0/3789352339 shutdown_connections 2024-04-26T20:53:13.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.280+0000 7f7f32551640 1 -- 172.21.15.73:0/3789352339 wait complete. 2024-04-26T20:53:13.291 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-26T20:53:13.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.389+0000 7fb2776c5640 1 Processor -- start 2024-04-26T20:53:13.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.389+0000 7fb2776c5640 1 -- start start 2024-04-26T20:53:13.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.390+0000 7fb2776c5640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb27015a9f0 0x7fb2700ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:13.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.390+0000 7fb2776c5640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fb27015d4b0 0x7fb270072e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:13.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.390+0000 7fb2776c5640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb27015dd30 con 0x7fb270073390 2024-04-26T20:53:13.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.390+0000 7fb2776c5640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fb2700726e0 con 0x7fb27015d4b0 2024-04-26T20:53:13.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.390+0000 7fb2776c5640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fb27015e010 con 0x7fb27015a9f0 2024-04-26T20:53:13.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.390+0000 7fb275c62640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fb270073390 0x7fb2700b4790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51774/0 (socket says 172.21.15.73:51774) 2024-04-26T20:53:13.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.390+0000 7fb275c62640 1 -- 172.21.15.73:0/3697987913 learned_addr learned my addr 172.21.15.73:0/3697987913 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:53:13.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.390+0000 7fb275461640 1 --2- 172.21.15.73:0/3697987913 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb27015a9f0 0x7fb2700ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:13.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.390+0000 7fb274c60640 1 --2- 172.21.15.73:0/3697987913 >> v2:172.21.15.120:3300/0 conn(0x7fb27015d4b0 0x7fb270072e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:13.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.390+0000 7fb267fff640 1 -- 172.21.15.73:0/3697987913 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3546970749 0 0) 0x7fb27015dd30 con 0x7fb270073390 2024-04-26T20:53:13.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.390+0000 7fb267fff640 1 -- 172.21.15.73:0/3697987913 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb25c003520 con 0x7fb270073390 2024-04-26T20:53:13.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.390+0000 7fb274c60640 1 -- 172.21.15.73:0/3697987913 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb27015a9f0 msgr2=0x7fb2700ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:13.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.390+0000 7fb274c60640 1 --2- 172.21.15.73:0/3697987913 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb27015a9f0 0x7fb2700ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:13.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.390+0000 7fb274c60640 1 -- 172.21.15.73:0/3697987913 >> v1:172.21.15.73:6789/0 conn(0x7fb270073390 legacy=0x7fb2700b4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:13.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.390+0000 7fb274c60640 1 -- 172.21.15.73:0/3697987913 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb27015e2f0 con 0x7fb27015d4b0 2024-04-26T20:53:13.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.390+0000 7fb275461640 1 --2- 172.21.15.73:0/3697987913 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb27015a9f0 0x7fb2700ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:53:13.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.391+0000 7fb274c60640 1 --2- 172.21.15.73:0/3697987913 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb27015d4b0 0x7fb270072e50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb2600097a0 tx=0x7fb26002a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61a30081a2621654 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:13.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.391+0000 7fb267fff640 1 -- 172.21.15.73:0/3697987913 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb260037070 con 0x7fb27015d4b0 2024-04-26T20:53:13.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.391+0000 7fb267fff640 1 -- 172.21.15.73:0/3697987913 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb26002ae50 con 0x7fb27015d4b0 2024-04-26T20:53:13.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.391+0000 7fb267fff640 1 -- 172.21.15.73:0/3697987913 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb260032640 con 0x7fb27015d4b0 2024-04-26T20:53:13.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.392+0000 7fb2776c5640 1 -- 172.21.15.73:0/3697987913 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb27015d4b0 msgr2=0x7fb270072e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:13.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.392+0000 7fb2776c5640 1 --2- 172.21.15.73:0/3697987913 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb27015d4b0 0x7fb270072e50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb2600097a0 tx=0x7fb26002a100 comp rx=0 tx=0).stop 2024-04-26T20:53:13.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.392+0000 7fb2776c5640 1 -- 172.21.15.73:0/3697987913 shutdown_connections 2024-04-26T20:53:13.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.392+0000 7fb2776c5640 1 --2- 172.21.15.73:0/3697987913 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb27015d4b0 0x7fb270072e50 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:13.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.392+0000 7fb2776c5640 1 --2- 172.21.15.73:0/3697987913 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb27015a9f0 0x7fb2700ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:13.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.392+0000 7fb2776c5640 1 -- 172.21.15.73:0/3697987913 >> 172.21.15.73:0/3697987913 conn(0x7fb2700b3180 msgr2=0x7fb2700b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:13.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.392+0000 7fb2776c5640 1 -- 172.21.15.73:0/3697987913 shutdown_connections 2024-04-26T20:53:13.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.392+0000 7fb2776c5640 1 -- 172.21.15.73:0/3697987913 wait complete. 2024-04-26T20:53:13.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.393+0000 7fb2776c5640 1 Processor -- start 2024-04-26T20:53:13.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.393+0000 7fb2776c5640 1 -- start start 2024-04-26T20:53:13.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.394+0000 7fb2776c5640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb270073390 0x7fb27014c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:13.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.394+0000 7fb2776c5640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb27015a9f0 0x7fb27006fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:13.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.394+0000 7fb2776c5640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb27015d4b0 0x7fb2700af890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:13.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.394+0000 7fb2776c5640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb27015af50 con 0x7fb27015d4b0 2024-04-26T20:53:13.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.394+0000 7fb2776c5640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb27015add0 con 0x7fb270073390 2024-04-26T20:53:13.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.394+0000 7fb2776c5640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fb2700b4ea0 con 0x7fb27015a9f0 2024-04-26T20:53:13.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.394+0000 7fb275c62640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb27015d4b0 0x7fb2700af890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:13.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.394+0000 7fb275c62640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb27015d4b0 0x7fb2700af890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56888/0 (socket says 172.21.15.73:56888) 2024-04-26T20:53:13.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.394+0000 7fb275c62640 1 -- 172.21.15.73:0/3120899792 learned_addr learned my addr 172.21.15.73:0/3120899792 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:13.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.394+0000 7fb275461640 1 --2- 172.21.15.73:0/3120899792 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb270073390 0x7fb27014c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:13.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.394+0000 7fb274c60640 1 --2- 172.21.15.73:0/3120899792 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb27015a9f0 0x7fb27006fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:13.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.394+0000 7fb275c62640 1 -- 172.21.15.73:0/3120899792 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb27015a9f0 msgr2=0x7fb27006fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:13.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.394+0000 7fb275c62640 1 --2- 172.21.15.73:0/3120899792 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb27015a9f0 0x7fb27006fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:13.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.394+0000 7fb275c62640 1 -- 172.21.15.73:0/3120899792 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb270073390 msgr2=0x7fb27014c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:13.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.394+0000 7fb275c62640 1 --2- 172.21.15.73:0/3120899792 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb270073390 0x7fb27014c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:13.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.394+0000 7fb275c62640 1 -- 172.21.15.73:0/3120899792 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb27006ed40 con 0x7fb27015d4b0 2024-04-26T20:53:13.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.394+0000 7fb274c60640 1 --2- 172.21.15.73:0/3120899792 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb27015a9f0 0x7fb27006fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:53:13.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.394+0000 7fb275c62640 1 --2- 172.21.15.73:0/3120899792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb27015d4b0 0x7fb2700af890 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fb26800cb70 tx=0x7fb26800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:13.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.394+0000 7fb265ffb640 1 -- 172.21.15.73:0/3120899792 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb268017070 con 0x7fb27015d4b0 2024-04-26T20:53:13.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.394+0000 7fb265ffb640 1 -- 172.21.15.73:0/3120899792 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb26800ad40 con 0x7fb27015d4b0 2024-04-26T20:53:13.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.394+0000 7fb2776c5640 1 -- 172.21.15.73:0/3120899792 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb27006f020 con 0x7fb27015d4b0 2024-04-26T20:53:13.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.394+0000 7fb265ffb640 1 -- 172.21.15.73:0/3120899792 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb268012620 con 0x7fb27015d4b0 2024-04-26T20:53:13.399 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.395+0000 7fb2776c5640 1 -- 172.21.15.73:0/3120899792 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb27006f5e0 con 0x7fb27015d4b0 2024-04-26T20:53:13.399 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.396+0000 7fb2776c5640 1 -- 172.21.15.73:0/3120899792 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb270068fc0 con 0x7fb27015d4b0 2024-04-26T20:53:13.399 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.396+0000 7fb265ffb640 1 -- 172.21.15.73:0/3120899792 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb26801a070 con 0x7fb27015d4b0 2024-04-26T20:53:13.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.396+0000 7fb265ffb640 1 --2- 172.21.15.73:0/3120899792 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb25406c8f0 0x7fb25406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:13.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.397+0000 7fb275461640 1 --2- 172.21.15.73:0/3120899792 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb25406c8f0 0x7fb25406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:13.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.397+0000 7fb265ffb640 1 -- 172.21.15.73:0/3120899792 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5421+0+0 (secure 0 0 0) 0x7fb26808b850 con 0x7fb27015d4b0 2024-04-26T20:53:13.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.397+0000 7fb275461640 1 --2- 172.21.15.73:0/3120899792 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb25406c8f0 0x7fb25406edb0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb26c009560 tx=0x7fb26c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:13.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.401+0000 7fb265ffb640 1 -- 172.21.15.73:0/3120899792 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb268059750 con 0x7fb27015d4b0 2024-04-26T20:53:13.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.576+0000 7fb2776c5640 1 -- 172.21.15.73:0/3120899792 --> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7fb27006e6a0 con 0x7fb25406c8f0 2024-04-26T20:53:13.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.576+0000 7fb265ffb640 1 -- 172.21.15.73:0/3120899792 <== mgr.4107 v2:172.21.15.73:6832/666905549 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb27006e6a0 con 0x7fb25406c8f0 2024-04-26T20:53:13.577 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 1 to deep-scrub 2024-04-26T20:53:13.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.578+0000 7fb2776c5640 1 -- 172.21.15.73:0/3120899792 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb25406c8f0 msgr2=0x7fb25406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:13.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.578+0000 7fb2776c5640 1 --2- 172.21.15.73:0/3120899792 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb25406c8f0 0x7fb25406edb0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb26c009560 tx=0x7fb26c009190 comp rx=0 tx=0).stop 2024-04-26T20:53:13.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.578+0000 7fb2776c5640 1 -- 172.21.15.73:0/3120899792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb27015d4b0 msgr2=0x7fb2700af890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:13.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.578+0000 7fb2776c5640 1 --2- 172.21.15.73:0/3120899792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb27015d4b0 0x7fb2700af890 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fb26800cb70 tx=0x7fb26800a430 comp rx=0 tx=0).stop 2024-04-26T20:53:13.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.578+0000 7fb2776c5640 1 -- 172.21.15.73:0/3120899792 shutdown_connections 2024-04-26T20:53:13.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.578+0000 7fb2776c5640 1 --2- 172.21.15.73:0/3120899792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb27015d4b0 0x7fb2700af890 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:13.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.578+0000 7fb2776c5640 1 --2- 172.21.15.73:0/3120899792 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb25406c8f0 0x7fb25406edb0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:13.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.578+0000 7fb2776c5640 1 --2- 172.21.15.73:0/3120899792 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb27015a9f0 0x7fb27006fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:13.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.578+0000 7fb2776c5640 1 --2- 172.21.15.73:0/3120899792 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb270073390 0x7fb27014c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:13.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.578+0000 7fb2776c5640 1 -- 172.21.15.73:0/3120899792 >> 172.21.15.73:0/3120899792 conn(0x7fb2700b3180 msgr2=0x7fb2700b4790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:13.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.578+0000 7fb2776c5640 1 -- 172.21.15.73:0/3120899792 shutdown_connections 2024-04-26T20:53:13.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.578+0000 7fb2776c5640 1 -- 172.21.15.73:0/3120899792 wait complete. 2024-04-26T20:53:13.589 INFO:tasks.ceph:Scrubbing osd.2 2024-04-26T20:53:13.589 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-26T20:53:13.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.691+0000 7ffa171af640 1 Processor -- start 2024-04-26T20:53:13.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.692+0000 7ffa171af640 1 -- start start 2024-04-26T20:53:13.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.692+0000 7ffa171af640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ffa1015f3c0 0x7ffa100d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:13.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.692+0000 7ffa171af640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7ffa100d3da0 0x7ffa100b7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:13.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.692+0000 7ffa171af640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa1015f7a0 con 0x7ffa100d1f30 2024-04-26T20:53:13.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.692+0000 7ffa171af640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7ffa100d2390 con 0x7ffa1015f3c0 2024-04-26T20:53:13.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.692+0000 7ffa171af640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7ffa100cfef0 con 0x7ffa100d3da0 2024-04-26T20:53:13.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.692+0000 7ffa14f4b640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ffa1015f3c0 0x7ffa100d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:13.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.692+0000 7ffa0ffff640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7ffa100d1f30 0x7ffa100d3690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51782/0 (socket says 172.21.15.73:51782) 2024-04-26T20:53:13.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.692+0000 7ffa0ffff640 1 -- 172.21.15.73:0/61502262 learned_addr learned my addr 172.21.15.73:0/61502262 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:53:13.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.693+0000 7ffa1574c640 1 --2- 172.21.15.73:0/61502262 >> v2:172.21.15.120:3300/0 conn(0x7ffa100d3da0 0x7ffa100b7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:13.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.693+0000 7ffa14f4b640 1 -- 172.21.15.73:0/61502262 >> v2:172.21.15.120:3300/0 conn(0x7ffa100d3da0 msgr2=0x7ffa100b7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:13.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.693+0000 7ffa14f4b640 1 --2- 172.21.15.73:0/61502262 >> v2:172.21.15.120:3300/0 conn(0x7ffa100d3da0 0x7ffa100b7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:13.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.693+0000 7ffa14f4b640 1 -- 172.21.15.73:0/61502262 >> v1:172.21.15.73:6789/0 conn(0x7ffa100d1f30 legacy=0x7ffa100d3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:13.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.693+0000 7ffa14f4b640 1 -- 172.21.15.73:0/61502262 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa100d01d0 con 0x7ffa1015f3c0 2024-04-26T20:53:13.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.693+0000 7ffa0f7fe640 1 -- 172.21.15.73:0/61502262 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2735461587 0 0) 0x7ffa1015f7a0 con 0x7ffa100d1f30 2024-04-26T20:53:13.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.693+0000 7ffa1574c640 1 --2- 172.21.15.73:0/61502262 >> v2:172.21.15.120:3300/0 conn(0x7ffa100d3da0 0x7ffa100b7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:53:13.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.693+0000 7ffa14f4b640 1 --2- 172.21.15.73:0/61502262 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa1015f3c0 0x7ffa100d19f0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7ffa100d2be0 tx=0x7ffa0402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4de241ffcb2e6a7d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:13.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.693+0000 7ffa0f7fe640 1 -- 172.21.15.73:0/61502262 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa04037070 con 0x7ffa1015f3c0 2024-04-26T20:53:13.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.694+0000 7ffa0f7fe640 1 -- 172.21.15.73:0/61502262 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa04032440 con 0x7ffa1015f3c0 2024-04-26T20:53:13.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.694+0000 7ffa0f7fe640 1 -- 172.21.15.73:0/61502262 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa04032810 con 0x7ffa1015f3c0 2024-04-26T20:53:13.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.694+0000 7ffa171af640 1 -- 172.21.15.73:0/61502262 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa1015f3c0 msgr2=0x7ffa100d19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:13.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.694+0000 7ffa171af640 1 --2- 172.21.15.73:0/61502262 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa1015f3c0 0x7ffa100d19f0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7ffa100d2be0 tx=0x7ffa0402a1a0 comp rx=0 tx=0).stop 2024-04-26T20:53:13.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.694+0000 7ffa171af640 1 -- 172.21.15.73:0/61502262 shutdown_connections 2024-04-26T20:53:13.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.694+0000 7ffa171af640 1 --2- 172.21.15.73:0/61502262 >> v2:172.21.15.120:3300/0 conn(0x7ffa100d3da0 0x7ffa100b7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:13.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.694+0000 7ffa171af640 1 --2- 172.21.15.73:0/61502262 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa1015f3c0 0x7ffa100d19f0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:13.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.694+0000 7ffa171af640 1 -- 172.21.15.73:0/61502262 >> 172.21.15.73:0/61502262 conn(0x7ffa1009e380 msgr2=0x7ffa100d50b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:13.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.694+0000 7ffa171af640 1 -- 172.21.15.73:0/61502262 shutdown_connections 2024-04-26T20:53:13.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.694+0000 7ffa171af640 1 -- 172.21.15.73:0/61502262 wait complete. 2024-04-26T20:53:13.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.695+0000 7ffa171af640 1 Processor -- start 2024-04-26T20:53:13.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.695+0000 7ffa171af640 1 -- start start 2024-04-26T20:53:13.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.696+0000 7ffa171af640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa100d1f30 0x7ffa100c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:13.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.696+0000 7ffa171af640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa100d3da0 0x7ffa100cb160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:13.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.696+0000 7ffa171af640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ffa1015f3c0 0x7ffa100cdf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:13.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.696+0000 7ffa171af640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ffa100d0a70 con 0x7ffa100d3da0 2024-04-26T20:53:13.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.696+0000 7ffa171af640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7ffa100d0790 con 0x7ffa100d1f30 2024-04-26T20:53:13.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.696+0000 7ffa171af640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7ffa100cfba0 con 0x7ffa1015f3c0 2024-04-26T20:53:13.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.696+0000 7ffa0ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa100d3da0 0x7ffa100cb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:13.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.696+0000 7ffa0ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa100d3da0 0x7ffa100cb160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56910/0 (socket says 172.21.15.73:56910) 2024-04-26T20:53:13.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.696+0000 7ffa0ffff640 1 -- 172.21.15.73:0/1966577135 learned_addr learned my addr 172.21.15.73:0/1966577135 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:13.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.696+0000 7ffa14f4b640 1 --2- 172.21.15.73:0/1966577135 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa100d1f30 0x7ffa100c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:13.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.696+0000 7ffa1574c640 1 --2- 172.21.15.73:0/1966577135 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ffa1015f3c0 0x7ffa100cdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:13.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.697+0000 7ffa0ffff640 1 -- 172.21.15.73:0/1966577135 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ffa1015f3c0 msgr2=0x7ffa100cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:13.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.697+0000 7ffa0ffff640 1 --2- 172.21.15.73:0/1966577135 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ffa1015f3c0 0x7ffa100cdf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:13.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.697+0000 7ffa0ffff640 1 -- 172.21.15.73:0/1966577135 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa100d1f30 msgr2=0x7ffa100c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:13.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.697+0000 7ffa0ffff640 1 --2- 172.21.15.73:0/1966577135 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7ffa100d1f30 0x7ffa100c7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:13.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.697+0000 7ffa0ffff640 1 -- 172.21.15.73:0/1966577135 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa100c94a0 con 0x7ffa100d3da0 2024-04-26T20:53:13.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.697+0000 7ffa1574c640 1 --2- 172.21.15.73:0/1966577135 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7ffa1015f3c0 0x7ffa100cdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:53:13.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.697+0000 7ffa0ffff640 1 --2- 172.21.15.73:0/1966577135 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa100d3da0 0x7ffa100cb160 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc00c460 tx=0x7ff9fc00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:13.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.697+0000 7ffa0d7fa640 1 -- 172.21.15.73:0/1966577135 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc0074e0 con 0x7ffa100d3da0 2024-04-26T20:53:13.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.697+0000 7ffa0d7fa640 1 -- 172.21.15.73:0/1966577135 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9fc0047e0 con 0x7ffa100d3da0 2024-04-26T20:53:13.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.697+0000 7ffa171af640 1 -- 172.21.15.73:0/1966577135 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa100c9780 con 0x7ffa100d3da0 2024-04-26T20:53:13.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.697+0000 7ffa0d7fa640 1 -- 172.21.15.73:0/1966577135 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc004bd0 con 0x7ffa100d3da0 2024-04-26T20:53:13.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.697+0000 7ffa171af640 1 -- 172.21.15.73:0/1966577135 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa100c9d40 con 0x7ffa100d3da0 2024-04-26T20:53:13.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.698+0000 7ffa171af640 1 -- 172.21.15.73:0/1966577135 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffa10069600 con 0x7ffa100d3da0 2024-04-26T20:53:13.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.698+0000 7ffa0d7fa640 1 -- 172.21.15.73:0/1966577135 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff9fc011070 con 0x7ffa100d3da0 2024-04-26T20:53:13.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.699+0000 7ffa0d7fa640 1 --2- 172.21.15.73:0/1966577135 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff9f406c8f0 0x7ff9f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:13.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.699+0000 7ffa0d7fa640 1 -- 172.21.15.73:0/1966577135 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5421+0+0 (secure 0 0 0) 0x7ff9fc08b590 con 0x7ffa100d3da0 2024-04-26T20:53:13.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.699+0000 7ffa14f4b640 1 --2- 172.21.15.73:0/1966577135 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff9f406c8f0 0x7ff9f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:13.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.699+0000 7ffa0d7fa640 1 --2- 172.21.15.73:0/1966577135 >> [v2:172.21.15.73:6800/2542681064,v1:172.21.15.73:6801/2542681064] conn(0x7ff9f4072e30 0x7ff9f4075270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:13.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.699+0000 7ffa0d7fa640 1 -- 172.21.15.73:0/1966577135 --> [v2:172.21.15.73:6800/2542681064,v1:172.21.15.73:6801/2542681064] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff9f4075940 con 0x7ff9f4072e30 2024-04-26T20:53:13.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.699+0000 7ffa0d7fa640 1 -- 172.21.15.73:0/1966577135 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7ff9fc092050 con 0x7ffa100d3da0 2024-04-26T20:53:13.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.699+0000 7ffa1574c640 1 --2- 172.21.15.73:0/1966577135 >> [v2:172.21.15.73:6800/2542681064,v1:172.21.15.73:6801/2542681064] conn(0x7ff9f4072e30 0x7ff9f4075270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:13.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.699+0000 7ffa14f4b640 1 --2- 172.21.15.73:0/1966577135 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff9f406c8f0 0x7ff9f406edb0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ffa04034000 tx=0x7ffa0402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:13.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.699+0000 7ffa1574c640 1 --2- 172.21.15.73:0/1966577135 >> [v2:172.21.15.73:6800/2542681064,v1:172.21.15.73:6801/2542681064] conn(0x7ff9f4072e30 0x7ff9f4075270 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:13.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.700+0000 7ffa0d7fa640 1 -- 172.21.15.73:0/1966577135 <== osd.2 v2:172.21.15.73:6800/2542681064 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff9f4075940 con 0x7ff9f4072e30 2024-04-26T20:53:13.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.713+0000 7ffa171af640 1 -- 172.21.15.73:0/1966577135 --> [v2:172.21.15.73:6800/2542681064,v1:172.21.15.73:6801/2542681064] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ffa1008f130 con 0x7ff9f4072e30 2024-04-26T20:53:13.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.713+0000 7ffa0d7fa640 1 -- 172.21.15.73:0/1966577135 <== osd.2 v2:172.21.15.73:6800/2542681064 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7ffa1008f130 con 0x7ff9f4072e30 2024-04-26T20:53:13.714 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-26T20:53:13.714 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T20:53:13.714 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-26T20:53:13.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.713+0000 7ffa171af640 1 -- 172.21.15.73:0/1966577135 >> [v2:172.21.15.73:6800/2542681064,v1:172.21.15.73:6801/2542681064] conn(0x7ff9f4072e30 msgr2=0x7ff9f4075270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:13.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.713+0000 7ffa171af640 1 --2- 172.21.15.73:0/1966577135 >> [v2:172.21.15.73:6800/2542681064,v1:172.21.15.73:6801/2542681064] conn(0x7ff9f4072e30 0x7ff9f4075270 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:13.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.713+0000 7ffa171af640 1 -- 172.21.15.73:0/1966577135 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff9f406c8f0 msgr2=0x7ff9f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:13.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.713+0000 7ffa171af640 1 --2- 172.21.15.73:0/1966577135 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7ff9f406c8f0 0x7ff9f406edb0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ffa04034000 tx=0x7ffa0402e000 comp rx=0 tx=0).stop 2024-04-26T20:53:13.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.713+0000 7ffa171af640 1 -- 172.21.15.73:0/1966577135 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa100d3da0 msgr2=0x7ffa100cb160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:13.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.713+0000 7ffa171af640 1 --2- 172.21.15.73:0/1966577135 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa100d3da0 0x7ffa100cb160 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc00c460 tx=0x7ff9fc00c920 comp rx=0 tx=0).stop 2024-04-26T20:53:13.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.713+0000 7ffa1574c640 1 -- 172.21.15.73:0/1966577135 reap_dead start 2024-04-26T20:53:13.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.714+0000 7ffa171af640 1 -- 172.21.15.73:0/1966577135 shutdown_connections 2024-04-26T20:53:13.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.714+0000 7ffa171af640 1 -- 172.21.15.73:0/1966577135 >> 172.21.15.73:0/1966577135 conn(0x7ffa1009e380 msgr2=0x7ffa100b7f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:13.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.714+0000 7ffa171af640 1 -- 172.21.15.73:0/1966577135 shutdown_connections 2024-04-26T20:53:13.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.714+0000 7ffa171af640 1 -- 172.21.15.73:0/1966577135 wait complete. 2024-04-26T20:53:13.725 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-26T20:53:13.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.821+0000 7fdd37c08640 1 Processor -- start 2024-04-26T20:53:13.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.822+0000 7fdd37c08640 1 -- start start 2024-04-26T20:53:13.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.822+0000 7fdd37c08640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fdd300cc5b0 0x7fdd30072240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:13.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.822+0000 7fdd37c08640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdd30072780 0x7fdd30166b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:13.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.822+0000 7fdd37c08640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd3006e240 con 0x7fdd30071e60 2024-04-26T20:53:13.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.822+0000 7fdd37c08640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fdd300701f0 con 0x7fdd300cc5b0 2024-04-26T20:53:13.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.822+0000 7fdd37c08640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fdd3006fc30 con 0x7fdd30072780 2024-04-26T20:53:13.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.823+0000 7fdd361a5640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdd30072780 0x7fdd30166b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:13.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.823+0000 7fdd361a5640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdd30072780 0x7fdd30166b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.120:3301/0 says I am v2:172.21.15.73:44648/0 (socket says 172.21.15.73:44648) 2024-04-26T20:53:13.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.823+0000 7fdd351a3640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fdd300cc5b0 0x7fdd30072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:13.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.823+0000 7fdd361a5640 1 -- 172.21.15.73:0/3322674227 learned_addr learned my addr 172.21.15.73:0/3322674227 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:13.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.823+0000 7fdd351a3640 1 -- 172.21.15.73:0/3322674227 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdd30072780 msgr2=0x7fdd30166b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:13.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.823+0000 7fdd351a3640 1 --2- 172.21.15.73:0/3322674227 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdd30072780 0x7fdd30166b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:13.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.823+0000 7fdd351a3640 1 -- 172.21.15.73:0/3322674227 >> v1:172.21.15.120:6789/0 conn(0x7fdd30071e60 legacy=0x7fdd3015df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:13.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.823+0000 7fdd351a3640 1 -- 172.21.15.73:0/3322674227 --> v2:172.21.15.120:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd3006ff10 con 0x7fdd300cc5b0 2024-04-26T20:53:13.825 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.823+0000 7fdd361a5640 1 --2- 172.21.15.73:0/3322674227 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdd30072780 0x7fdd30166b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:53:13.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.824+0000 7fdd351a3640 1 --2- 172.21.15.73:0/3322674227 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdd300cc5b0 0x7fdd30072240 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fdd240097a0 tx=0x7fdd2402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c42f08e4c748e76 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:13.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.824+0000 7fdd349a2640 1 -- 172.21.15.73:0/3322674227 <== mon.1 v2:172.21.15.120:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd24037070 con 0x7fdd300cc5b0 2024-04-26T20:53:13.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.824+0000 7fdd349a2640 1 -- 172.21.15.73:0/3322674227 <== mon.1 v2:172.21.15.120:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd2402ae50 con 0x7fdd300cc5b0 2024-04-26T20:53:13.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.824+0000 7fdd349a2640 1 -- 172.21.15.73:0/3322674227 <== mon.1 v2:172.21.15.120:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd24032660 con 0x7fdd300cc5b0 2024-04-26T20:53:13.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.824+0000 7fdd37c08640 1 -- 172.21.15.73:0/3322674227 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdd300cc5b0 msgr2=0x7fdd30072240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:13.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.824+0000 7fdd37c08640 1 --2- 172.21.15.73:0/3322674227 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdd300cc5b0 0x7fdd30072240 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fdd240097a0 tx=0x7fdd2402a100 comp rx=0 tx=0).stop 2024-04-26T20:53:13.826 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.824+0000 7fdd37c08640 1 -- 172.21.15.73:0/3322674227 shutdown_connections 2024-04-26T20:53:13.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.824+0000 7fdd37c08640 1 --2- 172.21.15.73:0/3322674227 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdd30072780 0x7fdd30166b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:13.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.824+0000 7fdd37c08640 1 --2- 172.21.15.73:0/3322674227 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdd300cc5b0 0x7fdd30072240 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:13.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.824+0000 7fdd37c08640 1 -- 172.21.15.73:0/3322674227 >> 172.21.15.73:0/3322674227 conn(0x7fdd300d0be0 msgr2=0x7fdd300d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:13.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.825+0000 7fdd37c08640 1 -- 172.21.15.73:0/3322674227 shutdown_connections 2024-04-26T20:53:13.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.825+0000 7fdd37c08640 1 -- 172.21.15.73:0/3322674227 wait complete. 2024-04-26T20:53:13.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.826+0000 7fdd37c08640 1 Processor -- start 2024-04-26T20:53:13.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.826+0000 7fdd37c08640 1 -- start start 2024-04-26T20:53:13.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.826+0000 7fdd37c08640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdd30071e60 0x7fdd301f6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:13.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.826+0000 7fdd37c08640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdd30072780 0x7fdd3014e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:13.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.826+0000 7fdd37c08640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd300cc5b0 0x7fdd300c6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:13.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.826+0000 7fdd37c08640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdd3006f8e0 con 0x7fdd300cc5b0 2024-04-26T20:53:13.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.826+0000 7fdd37c08640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fdd30072c10 con 0x7fdd30071e60 2024-04-26T20:53:13.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.826+0000 7fdd37c08640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fdd3006ddf0 con 0x7fdd30072780 2024-04-26T20:53:13.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.827+0000 7fdd361a5640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd300cc5b0 0x7fdd300c6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:13.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.827+0000 7fdd361a5640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd300cc5b0 0x7fdd300c6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56914/0 (socket says 172.21.15.73:56914) 2024-04-26T20:53:13.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.827+0000 7fdd361a5640 1 -- 172.21.15.73:0/3906373007 learned_addr learned my addr 172.21.15.73:0/3906373007 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:13.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.827+0000 7fdd351a3640 1 --2- 172.21.15.73:0/3906373007 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdd30072780 0x7fdd3014e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:13.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.827+0000 7fdd359a4640 1 --2- 172.21.15.73:0/3906373007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdd30071e60 0x7fdd301f6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:13.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.827+0000 7fdd361a5640 1 -- 172.21.15.73:0/3906373007 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdd30072780 msgr2=0x7fdd3014e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:13.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.827+0000 7fdd361a5640 1 --2- 172.21.15.73:0/3906373007 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdd30072780 0x7fdd3014e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:13.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.827+0000 7fdd361a5640 1 -- 172.21.15.73:0/3906373007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdd30071e60 msgr2=0x7fdd301f6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:13.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.827+0000 7fdd361a5640 1 --2- 172.21.15.73:0/3906373007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdd30071e60 0x7fdd301f6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:13.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.827+0000 7fdd361a5640 1 -- 172.21.15.73:0/3906373007 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd301f7050 con 0x7fdd300cc5b0 2024-04-26T20:53:13.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.827+0000 7fdd359a4640 1 --2- 172.21.15.73:0/3906373007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdd30071e60 0x7fdd301f6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:53:13.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.827+0000 7fdd361a5640 1 --2- 172.21.15.73:0/3906373007 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd300cc5b0 0x7fdd300c6a80 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fdd2000e930 tx=0x7fdd2000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:13.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.827+0000 7fdd2e7fc640 1 -- 172.21.15.73:0/3906373007 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd20017070 con 0x7fdd300cc5b0 2024-04-26T20:53:13.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.827+0000 7fdd2e7fc640 1 -- 172.21.15.73:0/3906373007 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd2000ca00 con 0x7fdd300cc5b0 2024-04-26T20:53:13.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.827+0000 7fdd37c08640 1 -- 172.21.15.73:0/3906373007 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd300c4ce0 con 0x7fdd300cc5b0 2024-04-26T20:53:13.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.827+0000 7fdd2e7fc640 1 -- 172.21.15.73:0/3906373007 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd2000cdd0 con 0x7fdd300cc5b0 2024-04-26T20:53:13.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.827+0000 7fdd37c08640 1 -- 172.21.15.73:0/3906373007 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd300c52a0 con 0x7fdd300cc5b0 2024-04-26T20:53:13.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.828+0000 7fdd37c08640 1 -- 172.21.15.73:0/3906373007 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd300694f0 con 0x7fdd300cc5b0 2024-04-26T20:53:13.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.829+0000 7fdd2e7fc640 1 -- 172.21.15.73:0/3906373007 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdd20013470 con 0x7fdd300cc5b0 2024-04-26T20:53:13.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.829+0000 7fdd2e7fc640 1 --2- 172.21.15.73:0/3906373007 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fdd1c06c8f0 0x7fdd1c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:13.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.829+0000 7fdd359a4640 1 --2- 172.21.15.73:0/3906373007 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fdd1c06c8f0 0x7fdd1c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:13.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.829+0000 7fdd2e7fc640 1 -- 172.21.15.73:0/3906373007 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5421+0+0 (secure 0 0 0) 0x7fdd2008b430 con 0x7fdd300cc5b0 2024-04-26T20:53:13.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.830+0000 7fdd359a4640 1 --2- 172.21.15.73:0/3906373007 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fdd1c06c8f0 0x7fdd1c06edb0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fdd300c8520 tx=0x7fdd28009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:13.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:13.833+0000 7fdd2e7fc640 1 -- 172.21.15.73:0/3906373007 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdd301f7050 con 0x7fdd300cc5b0 2024-04-26T20:53:14.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.013+0000 7fdd37c08640 1 -- 172.21.15.73:0/3906373007 --> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7fdd30069d40 con 0x7fdd1c06c8f0 2024-04-26T20:53:14.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.013+0000 7fdd2e7fc640 1 -- 172.21.15.73:0/3906373007 <== mgr.4107 v2:172.21.15.73:6832/666905549 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fdd30069d40 con 0x7fdd1c06c8f0 2024-04-26T20:53:14.014 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 2 to deep-scrub 2024-04-26T20:53:14.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.015+0000 7fdd37c08640 1 -- 172.21.15.73:0/3906373007 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fdd1c06c8f0 msgr2=0x7fdd1c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:14.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.015+0000 7fdd37c08640 1 --2- 172.21.15.73:0/3906373007 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fdd1c06c8f0 0x7fdd1c06edb0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fdd300c8520 tx=0x7fdd28009040 comp rx=0 tx=0).stop 2024-04-26T20:53:14.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.015+0000 7fdd37c08640 1 -- 172.21.15.73:0/3906373007 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd300cc5b0 msgr2=0x7fdd300c6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:14.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.015+0000 7fdd37c08640 1 --2- 172.21.15.73:0/3906373007 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd300cc5b0 0x7fdd300c6a80 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fdd2000e930 tx=0x7fdd2000edf0 comp rx=0 tx=0).stop 2024-04-26T20:53:14.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.015+0000 7fdd37c08640 1 -- 172.21.15.73:0/3906373007 shutdown_connections 2024-04-26T20:53:14.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.015+0000 7fdd37c08640 1 --2- 172.21.15.73:0/3906373007 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd300cc5b0 0x7fdd300c6a80 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:14.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.015+0000 7fdd37c08640 1 --2- 172.21.15.73:0/3906373007 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fdd1c06c8f0 0x7fdd1c06edb0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:14.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.015+0000 7fdd37c08640 1 --2- 172.21.15.73:0/3906373007 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fdd30072780 0x7fdd3014e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:14.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.015+0000 7fdd37c08640 1 --2- 172.21.15.73:0/3906373007 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fdd30071e60 0x7fdd301f6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:14.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.015+0000 7fdd37c08640 1 -- 172.21.15.73:0/3906373007 >> 172.21.15.73:0/3906373007 conn(0x7fdd300d0be0 msgr2=0x7fdd3015eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:14.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.016+0000 7fdd37c08640 1 -- 172.21.15.73:0/3906373007 shutdown_connections 2024-04-26T20:53:14.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.016+0000 7fdd37c08640 1 -- 172.21.15.73:0/3906373007 wait complete. 2024-04-26T20:53:14.027 INFO:tasks.ceph:Scrubbing osd.3 2024-04-26T20:53:14.027 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-26T20:53:14.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.127+0000 7f1ab9623640 1 Processor -- start 2024-04-26T20:53:14.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.127+0000 7f1ab9623640 1 -- start start 2024-04-26T20:53:14.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.128+0000 7f1ab9623640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1ab415f7c0 0x7f1ab40d3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:14.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.128+0000 7f1ab9623640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ab4160930 con 0x7f1ab415f3e0 2024-04-26T20:53:14.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.128+0000 7f1ab9623640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ab415fd20 con 0x7f1ab40d3bd0 2024-04-26T20:53:14.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.128+0000 7f1ab9623640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f1ab40d4040 con 0x7f1ab415f7c0 2024-04-26T20:53:14.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.128+0000 7f1ab27fc640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1ab415f7c0 0x7f1ab40d3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:14.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.128+0000 7f1ab27fc640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1ab415f7c0 0x7f1ab40d3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56920/0 (socket says 172.21.15.73:56920) 2024-04-26T20:53:14.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.128+0000 7f1ab2ffd640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f1ab415f3e0 0x7f1ab40d2ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51796/0 (socket says 172.21.15.73:51796) 2024-04-26T20:53:14.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.128+0000 7f1ab27fc640 1 -- 172.21.15.73:0/2780387538 learned_addr learned my addr 172.21.15.73:0/2780387538 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:14.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.128+0000 7f1ab27fc640 1 -- 172.21.15.73:0/2780387538 >> v1:172.21.15.120:6789/0 conn(0x7f1ab40d3bd0 legacy=0x7f1ab41620a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:14.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.128+0000 7f1ab27fc640 1 -- 172.21.15.73:0/2780387538 >> v1:172.21.15.73:6789/0 conn(0x7f1ab415f3e0 legacy=0x7f1ab40d2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:14.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.128+0000 7f1ab27fc640 1 -- 172.21.15.73:0/2780387538 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab4160000 con 0x7f1ab415f7c0 2024-04-26T20:53:14.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.128+0000 7f1ab1ffb640 1 -- 172.21.15.73:0/2780387538 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3973912983 0 0) 0x7f1ab4160930 con 0x7f1ab415f3e0 2024-04-26T20:53:14.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.129+0000 7f1ab27fc640 1 --2- 172.21.15.73:0/2780387538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ab415f7c0 0x7f1ab40d3690 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c009870 tx=0x7f1a9c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7424929549969f18 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:14.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.129+0000 7f1ab1ffb640 1 -- 172.21.15.73:0/2780387538 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c037070 con 0x7f1ab415f7c0 2024-04-26T20:53:14.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.129+0000 7f1ab1ffb640 1 -- 172.21.15.73:0/2780387538 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a9c032440 con 0x7f1ab415f7c0 2024-04-26T20:53:14.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.129+0000 7f1ab1ffb640 1 -- 172.21.15.73:0/2780387538 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c032810 con 0x7f1ab415f7c0 2024-04-26T20:53:14.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.129+0000 7f1ab9623640 1 -- 172.21.15.73:0/2780387538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ab415f7c0 msgr2=0x7f1ab40d3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:14.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.129+0000 7f1ab9623640 1 --2- 172.21.15.73:0/2780387538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ab415f7c0 0x7f1ab40d3690 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c009870 tx=0x7f1a9c02a210 comp rx=0 tx=0).stop 2024-04-26T20:53:14.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.130+0000 7f1ab9623640 1 -- 172.21.15.73:0/2780387538 shutdown_connections 2024-04-26T20:53:14.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.130+0000 7f1ab9623640 1 --2- 172.21.15.73:0/2780387538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ab415f7c0 0x7f1ab40d3690 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:14.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.130+0000 7f1ab9623640 1 -- 172.21.15.73:0/2780387538 >> 172.21.15.73:0/2780387538 conn(0x7f1ab409e3c0 msgr2=0x7f1ab40d50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:14.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.130+0000 7f1ab9623640 1 -- 172.21.15.73:0/2780387538 shutdown_connections 2024-04-26T20:53:14.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.130+0000 7f1ab9623640 1 -- 172.21.15.73:0/2780387538 wait complete. 2024-04-26T20:53:14.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.131+0000 7f1ab9623640 1 Processor -- start 2024-04-26T20:53:14.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.131+0000 7f1ab9623640 1 -- start start 2024-04-26T20:53:14.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.132+0000 7f1ab9623640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ab40d3bd0 0x7f1ab415bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:14.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.132+0000 7f1ab9623640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ab415f3e0 0x7f1ab40c8be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:14.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.132+0000 7f1ab9623640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1ab415f7c0 0x7f1ab40c9120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:14.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.132+0000 7f1ab9623640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab4160d40 con 0x7f1ab415f3e0 2024-04-26T20:53:14.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.132+0000 7f1ab9623640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab41605c0 con 0x7f1ab40d3bd0 2024-04-26T20:53:14.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.132+0000 7f1ab9623640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f1ab40d1040 con 0x7f1ab415f7c0 2024-04-26T20:53:14.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.132+0000 7f1ab27fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ab415f3e0 0x7f1ab40c8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:14.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.132+0000 7f1ab27fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ab415f3e0 0x7f1ab40c8be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56922/0 (socket says 172.21.15.73:56922) 2024-04-26T20:53:14.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.132+0000 7f1ab27fc640 1 -- 172.21.15.73:0/2027038299 learned_addr learned my addr 172.21.15.73:0/2027038299 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:14.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.132+0000 7f1ab37fe640 1 --2- 172.21.15.73:0/2027038299 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1ab415f7c0 0x7f1ab40c9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:14.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.132+0000 7f1ab2ffd640 1 --2- 172.21.15.73:0/2027038299 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ab40d3bd0 0x7f1ab415bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:14.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.132+0000 7f1ab27fc640 1 -- 172.21.15.73:0/2027038299 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1ab415f7c0 msgr2=0x7f1ab40c9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:14.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.132+0000 7f1ab27fc640 1 --2- 172.21.15.73:0/2027038299 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f1ab415f7c0 0x7f1ab40c9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:14.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.132+0000 7f1ab27fc640 1 -- 172.21.15.73:0/2027038299 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ab40d3bd0 msgr2=0x7f1ab415bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:14.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.132+0000 7f1ab27fc640 1 --2- 172.21.15.73:0/2027038299 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ab40d3bd0 0x7f1ab415bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:14.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.132+0000 7f1ab27fc640 1 -- 172.21.15.73:0/2027038299 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab415ccc0 con 0x7f1ab415f3e0 2024-04-26T20:53:14.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.132+0000 7f1ab2ffd640 1 --2- 172.21.15.73:0/2027038299 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f1ab40d3bd0 0x7f1ab415bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:53:14.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.132+0000 7f1ab27fc640 1 --2- 172.21.15.73:0/2027038299 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ab415f3e0 0x7f1ab40c8be0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c009870 tx=0x7f1a9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:14.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.132+0000 7f1aab7fe640 1 -- 172.21.15.73:0/2027038299 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c037040 con 0x7f1ab415f3e0 2024-04-26T20:53:14.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.132+0000 7f1aab7fe640 1 -- 172.21.15.73:0/2027038299 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a9c004250 con 0x7f1ab415f3e0 2024-04-26T20:53:14.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.132+0000 7f1ab9623640 1 -- 172.21.15.73:0/2027038299 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ab40cb850 con 0x7f1ab415f3e0 2024-04-26T20:53:14.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.132+0000 7f1aab7fe640 1 -- 172.21.15.73:0/2027038299 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c03c670 con 0x7f1ab415f3e0 2024-04-26T20:53:14.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.132+0000 7f1ab9623640 1 -- 172.21.15.73:0/2027038299 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ab40cbe10 con 0x7f1ab415f3e0 2024-04-26T20:53:14.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.133+0000 7f1ab9623640 1 -- 172.21.15.73:0/2027038299 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1ab4069600 con 0x7f1ab415f3e0 2024-04-26T20:53:14.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.134+0000 7f1aab7fe640 1 -- 172.21.15.73:0/2027038299 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1a9c03a070 con 0x7f1ab415f3e0 2024-04-26T20:53:14.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.134+0000 7f1aab7fe640 1 --2- 172.21.15.73:0/2027038299 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1a9406c8f0 0x7f1a9406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:14.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.134+0000 7f1aab7fe640 1 -- 172.21.15.73:0/2027038299 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5421+0+0 (secure 0 0 0) 0x7f1a9c0ac300 con 0x7f1ab415f3e0 2024-04-26T20:53:14.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.134+0000 7f1ab2ffd640 1 --2- 172.21.15.73:0/2027038299 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1a9406c8f0 0x7f1a9406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:14.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.134+0000 7f1aab7fe640 1 --2- 172.21.15.73:0/2027038299 >> [v2:172.21.15.73:6820/1604042718,v1:172.21.15.73:6822/1604042718] conn(0x7f1a94072e30 0x7f1a94075270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:14.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.134+0000 7f1aab7fe640 1 -- 172.21.15.73:0/2027038299 --> [v2:172.21.15.73:6820/1604042718,v1:172.21.15.73:6822/1604042718] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1a9c032440 con 0x7f1a94072e30 2024-04-26T20:53:14.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.134+0000 7f1aab7fe640 1 -- 172.21.15.73:0/2027038299 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f1a9c0b3050 con 0x7f1ab415f3e0 2024-04-26T20:53:14.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.135+0000 7f1ab37fe640 1 --2- 172.21.15.73:0/2027038299 >> [v2:172.21.15.73:6820/1604042718,v1:172.21.15.73:6822/1604042718] conn(0x7f1a94072e30 0x7f1a94075270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:14.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.135+0000 7f1ab2ffd640 1 --2- 172.21.15.73:0/2027038299 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1a9406c8f0 0x7f1a9406edb0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4000f60 tx=0x7f1aa4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:14.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.135+0000 7f1ab37fe640 1 --2- 172.21.15.73:0/2027038299 >> [v2:172.21.15.73:6820/1604042718,v1:172.21.15.73:6822/1604042718] conn(0x7f1a94072e30 0x7f1a94075270 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:14.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.135+0000 7f1aab7fe640 1 -- 172.21.15.73:0/2027038299 <== osd.3 v2:172.21.15.73:6820/1604042718 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1a9c032440 con 0x7f1a94072e30 2024-04-26T20:53:14.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.148+0000 7f1ab9623640 1 -- 172.21.15.73:0/2027038299 --> [v2:172.21.15.73:6820/1604042718,v1:172.21.15.73:6822/1604042718] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f1ab40d57e0 con 0x7f1a94072e30 2024-04-26T20:53:14.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.149+0000 7f1aab7fe640 1 -- 172.21.15.73:0/2027038299 <== osd.3 v2:172.21.15.73:6820/1604042718 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f1ab40d57e0 con 0x7f1a94072e30 2024-04-26T20:53:14.150 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-26T20:53:14.150 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T20:53:14.150 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-26T20:53:14.151 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.149+0000 7f1ab9623640 1 -- 172.21.15.73:0/2027038299 >> [v2:172.21.15.73:6820/1604042718,v1:172.21.15.73:6822/1604042718] conn(0x7f1a94072e30 msgr2=0x7f1a94075270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:14.151 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.149+0000 7f1ab9623640 1 --2- 172.21.15.73:0/2027038299 >> [v2:172.21.15.73:6820/1604042718,v1:172.21.15.73:6822/1604042718] conn(0x7f1a94072e30 0x7f1a94075270 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:14.151 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.149+0000 7f1ab9623640 1 -- 172.21.15.73:0/2027038299 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1a9406c8f0 msgr2=0x7f1a9406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:14.151 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.149+0000 7f1ab9623640 1 --2- 172.21.15.73:0/2027038299 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f1a9406c8f0 0x7f1a9406edb0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4000f60 tx=0x7f1aa4009040 comp rx=0 tx=0).stop 2024-04-26T20:53:14.151 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.149+0000 7f1ab9623640 1 -- 172.21.15.73:0/2027038299 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ab415f3e0 msgr2=0x7f1ab40c8be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:14.151 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.149+0000 7f1ab9623640 1 --2- 172.21.15.73:0/2027038299 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ab415f3e0 0x7f1ab40c8be0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c009870 tx=0x7f1a9c0046c0 comp rx=0 tx=0).stop 2024-04-26T20:53:14.151 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.149+0000 7f1ab37fe640 1 -- 172.21.15.73:0/2027038299 reap_dead start 2024-04-26T20:53:14.151 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.149+0000 7f1ab9623640 1 -- 172.21.15.73:0/2027038299 shutdown_connections 2024-04-26T20:53:14.151 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.149+0000 7f1ab9623640 1 -- 172.21.15.73:0/2027038299 >> 172.21.15.73:0/2027038299 conn(0x7f1ab409e3c0 msgr2=0x7f1ab40d1250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:14.152 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.149+0000 7f1ab9623640 1 -- 172.21.15.73:0/2027038299 shutdown_connections 2024-04-26T20:53:14.152 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.149+0000 7f1ab9623640 1 -- 172.21.15.73:0/2027038299 wait complete. 2024-04-26T20:53:14.160 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-26T20:53:14.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.257+0000 7f305fb78640 1 Processor -- start 2024-04-26T20:53:14.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.257+0000 7f305fb78640 1 -- start start 2024-04-26T20:53:14.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.258+0000 7f305fb78640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f30581608f0 0x7f30580d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:14.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.258+0000 7f305fb78640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f30580d0a90 0x7f30580cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:14.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.258+0000 7f305fb78640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f30580cfdd0 0x7f3058160e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:14.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.258+0000 7f305fb78640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f30580cea30 con 0x7f30581608f0 2024-04-26T20:53:14.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.258+0000 7f305fb78640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f30580cde20 con 0x7f30580d0a90 2024-04-26T20:53:14.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.258+0000 7f305fb78640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f30580ce100 con 0x7f30580cfdd0 2024-04-26T20:53:14.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.258+0000 7f305d914640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f30581608f0 0x7f30580d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:14.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.259+0000 7f305d914640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f30581608f0 0x7f30580d1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56924/0 (socket says 172.21.15.73:56924) 2024-04-26T20:53:14.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.259+0000 7f305d914640 1 -- 172.21.15.73:0/3221142870 learned_addr learned my addr 172.21.15.73:0/3221142870 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:14.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.259+0000 7f305e115640 1 --2- 172.21.15.73:0/3221142870 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f30580cfdd0 0x7f3058160e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:14.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.259+0000 7f305d113640 1 --2- 172.21.15.73:0/3221142870 >> v2:172.21.15.120:3300/0 conn(0x7f30580d0a90 0x7f30580cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:14.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.259+0000 7f305d914640 1 -- 172.21.15.73:0/3221142870 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f30580cfdd0 msgr2=0x7f3058160e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:14.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.259+0000 7f305d914640 1 --2- 172.21.15.73:0/3221142870 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f30580cfdd0 0x7f3058160e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:14.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.259+0000 7f305d914640 1 -- 172.21.15.73:0/3221142870 >> v2:172.21.15.120:3300/0 conn(0x7f30580d0a90 msgr2=0x7f30580cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:14.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.259+0000 7f305d914640 1 --2- 172.21.15.73:0/3221142870 >> v2:172.21.15.120:3300/0 conn(0x7f30580d0a90 0x7f30580cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:14.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.259+0000 7f305d914640 1 -- 172.21.15.73:0/3221142870 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30580ce3e0 con 0x7f30581608f0 2024-04-26T20:53:14.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.259+0000 7f305d113640 1 --2- 172.21.15.73:0/3221142870 >> v2:172.21.15.120:3300/0 conn(0x7f30580d0a90 0x7f30580cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:53:14.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.259+0000 7f305d914640 1 --2- 172.21.15.73:0/3221142870 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30581608f0 0x7f30580d1bf0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f305806c610 tx=0x7f305402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54cdfb78eb3af990 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:14.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.260+0000 7f305c912640 1 -- 172.21.15.73:0/3221142870 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3054036070 con 0x7f30581608f0 2024-04-26T20:53:14.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.260+0000 7f305c912640 1 -- 172.21.15.73:0/3221142870 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3054031600 con 0x7f30581608f0 2024-04-26T20:53:14.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.260+0000 7f305c912640 1 -- 172.21.15.73:0/3221142870 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30540319d0 con 0x7f30581608f0 2024-04-26T20:53:14.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.260+0000 7f305fb78640 1 -- 172.21.15.73:0/3221142870 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30581608f0 msgr2=0x7f30580d1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:14.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.260+0000 7f305fb78640 1 --2- 172.21.15.73:0/3221142870 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30581608f0 0x7f30580d1bf0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f305806c610 tx=0x7f305402a6a0 comp rx=0 tx=0).stop 2024-04-26T20:53:14.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.260+0000 7f305fb78640 1 -- 172.21.15.73:0/3221142870 shutdown_connections 2024-04-26T20:53:14.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.260+0000 7f305fb78640 1 --2- 172.21.15.73:0/3221142870 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f30580cfdd0 0x7f3058160e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:14.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.260+0000 7f305fb78640 1 --2- 172.21.15.73:0/3221142870 >> v2:172.21.15.120:3300/0 conn(0x7f30580d0a90 0x7f30580cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:14.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.260+0000 7f305fb78640 1 --2- 172.21.15.73:0/3221142870 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30581608f0 0x7f30580d1bf0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:14.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.260+0000 7f305fb78640 1 -- 172.21.15.73:0/3221142870 >> 172.21.15.73:0/3221142870 conn(0x7f30580d2ec0 msgr2=0x7f30580d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:14.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.260+0000 7f305fb78640 1 -- 172.21.15.73:0/3221142870 shutdown_connections 2024-04-26T20:53:14.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.260+0000 7f305fb78640 1 -- 172.21.15.73:0/3221142870 wait complete. 2024-04-26T20:53:14.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.261+0000 7f305fb78640 1 Processor -- start 2024-04-26T20:53:14.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.261+0000 7f305fb78640 1 -- start start 2024-04-26T20:53:14.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.262+0000 7f305fb78640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f30580cfdd0 0x7f3058149990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:14.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.262+0000 7f305fb78640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30580d0a90 0x7f30580c89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:14.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.262+0000 7f305fb78640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f30581608f0 0x7f30580c8f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:14.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.262+0000 7f305fb78640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f30580ceec0 con 0x7f30580d0a90 2024-04-26T20:53:14.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.262+0000 7f305fb78640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f30580d0ef0 con 0x7f30580cfdd0 2024-04-26T20:53:14.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.262+0000 7f305fb78640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f30580cda40 con 0x7f30581608f0 2024-04-26T20:53:14.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.262+0000 7f305d113640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30580d0a90 0x7f30580c89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:14.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.262+0000 7f305d113640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30580d0a90 0x7f30580c89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56928/0 (socket says 172.21.15.73:56928) 2024-04-26T20:53:14.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.262+0000 7f305d113640 1 -- 172.21.15.73:0/2362581500 learned_addr learned my addr 172.21.15.73:0/2362581500 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:14.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.262+0000 7f305e115640 1 --2- 172.21.15.73:0/2362581500 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f30581608f0 0x7f30580c8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:14.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.262+0000 7f305d914640 1 --2- 172.21.15.73:0/2362581500 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f30580cfdd0 0x7f3058149990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:14.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.262+0000 7f305d113640 1 -- 172.21.15.73:0/2362581500 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f30581608f0 msgr2=0x7f30580c8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:14.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.263+0000 7f305d113640 1 --2- 172.21.15.73:0/2362581500 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f30581608f0 0x7f30580c8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:14.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.263+0000 7f305d113640 1 -- 172.21.15.73:0/2362581500 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f30580cfdd0 msgr2=0x7f3058149990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:14.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.263+0000 7f305d113640 1 --2- 172.21.15.73:0/2362581500 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f30580cfdd0 0x7f3058149990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:14.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.263+0000 7f305d113640 1 -- 172.21.15.73:0/2362581500 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30580cad30 con 0x7f30580d0a90 2024-04-26T20:53:14.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.263+0000 7f305d914640 1 --2- 172.21.15.73:0/2362581500 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f30580cfdd0 0x7f3058149990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:53:14.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.263+0000 7f305d113640 1 --2- 172.21.15.73:0/2362581500 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30580d0a90 0x7f30580c89f0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f304800c7f0 tx=0x7f304800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:14.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.263+0000 7f304e7fc640 1 -- 172.21.15.73:0/2362581500 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3048015070 con 0x7f30580d0a90 2024-04-26T20:53:14.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.263+0000 7f304e7fc640 1 -- 172.21.15.73:0/2362581500 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3048007a30 con 0x7f30580d0a90 2024-04-26T20:53:14.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.263+0000 7f305fb78640 1 -- 172.21.15.73:0/2362581500 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f305814a380 con 0x7f30580d0a90 2024-04-26T20:53:14.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.263+0000 7f304e7fc640 1 -- 172.21.15.73:0/2362581500 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3048007e00 con 0x7f30580d0a90 2024-04-26T20:53:14.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.263+0000 7f305fb78640 1 -- 172.21.15.73:0/2362581500 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f305814a940 con 0x7f30580d0a90 2024-04-26T20:53:14.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.264+0000 7f305fb78640 1 -- 172.21.15.73:0/2362581500 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30580694f0 con 0x7f30580d0a90 2024-04-26T20:53:14.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.264+0000 7f304e7fc640 1 -- 172.21.15.73:0/2362581500 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3048018070 con 0x7f30580d0a90 2024-04-26T20:53:14.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.265+0000 7f304e7fc640 1 --2- 172.21.15.73:0/2362581500 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f303c06c8f0 0x7f303c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:14.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.265+0000 7f304e7fc640 1 -- 172.21.15.73:0/2362581500 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5421+0+0 (secure 0 0 0) 0x7f304808bad0 con 0x7f30580d0a90 2024-04-26T20:53:14.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.265+0000 7f305d914640 1 --2- 172.21.15.73:0/2362581500 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f303c06c8f0 0x7f303c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:14.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.265+0000 7f305d914640 1 --2- 172.21.15.73:0/2362581500 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f303c06c8f0 0x7f303c06edb0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3054033000 tx=0x7f3054009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:14.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.269+0000 7f304e7fc640 1 -- 172.21.15.73:0/2362581500 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3048059f90 con 0x7f30580d0a90 2024-04-26T20:53:14.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.442+0000 7f305fb78640 1 -- 172.21.15.73:0/2362581500 --> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f30580d0470 con 0x7f303c06c8f0 2024-04-26T20:53:14.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.443+0000 7f304e7fc640 1 -- 172.21.15.73:0/2362581500 <== mgr.4107 v2:172.21.15.73:6832/666905549 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f30580d0470 con 0x7f303c06c8f0 2024-04-26T20:53:14.444 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 3 to deep-scrub 2024-04-26T20:53:14.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.444+0000 7f305fb78640 1 -- 172.21.15.73:0/2362581500 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f303c06c8f0 msgr2=0x7f303c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:14.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.445+0000 7f305fb78640 1 --2- 172.21.15.73:0/2362581500 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f303c06c8f0 0x7f303c06edb0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3054033000 tx=0x7f3054009000 comp rx=0 tx=0).stop 2024-04-26T20:53:14.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.445+0000 7f305fb78640 1 -- 172.21.15.73:0/2362581500 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30580d0a90 msgr2=0x7f30580c89f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:14.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.445+0000 7f305fb78640 1 --2- 172.21.15.73:0/2362581500 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30580d0a90 0x7f30580c89f0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f304800c7f0 tx=0x7f304800cbf0 comp rx=0 tx=0).stop 2024-04-26T20:53:14.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.445+0000 7f305fb78640 1 -- 172.21.15.73:0/2362581500 shutdown_connections 2024-04-26T20:53:14.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.445+0000 7f305fb78640 1 --2- 172.21.15.73:0/2362581500 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30580d0a90 0x7f30580c89f0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:14.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.445+0000 7f305fb78640 1 --2- 172.21.15.73:0/2362581500 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f303c06c8f0 0x7f303c06edb0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:14.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.445+0000 7f305fb78640 1 --2- 172.21.15.73:0/2362581500 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f30581608f0 0x7f30580c8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:14.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.445+0000 7f305fb78640 1 --2- 172.21.15.73:0/2362581500 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f30580cfdd0 0x7f3058149990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:14.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.445+0000 7f305fb78640 1 -- 172.21.15.73:0/2362581500 >> 172.21.15.73:0/2362581500 conn(0x7f30580d2ec0 msgr2=0x7f30580b5ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:14.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.445+0000 7f305fb78640 1 -- 172.21.15.73:0/2362581500 shutdown_connections 2024-04-26T20:53:14.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.445+0000 7f305fb78640 1 -- 172.21.15.73:0/2362581500 wait complete. 2024-04-26T20:53:14.457 INFO:tasks.ceph:Scrubbing osd.4 2024-04-26T20:53:14.457 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-26T20:53:14.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.557+0000 7f73eb0a0640 1 Processor -- start 2024-04-26T20:53:14.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.557+0000 7f73eb0a0640 1 -- start start 2024-04-26T20:53:14.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.558+0000 7f73eb0a0640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f73e406c950 0x7f73e40b0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:14.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.558+0000 7f73eb0a0640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73e4148d80 con 0x7f73e40b04b0 2024-04-26T20:53:14.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.558+0000 7f73eb0a0640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73e40b0f10 con 0x7f73e41489a0 2024-04-26T20:53:14.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.558+0000 7f73eb0a0640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f73e40b11d0 con 0x7f73e406c950 2024-04-26T20:53:14.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.558+0000 7f73e963d640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f73e406c950 0x7f73e40b0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:14.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.558+0000 7f73e963d640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f73e406c950 0x7f73e40b0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56938/0 (socket says 172.21.15.73:56938) 2024-04-26T20:53:14.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.558+0000 7f73e963d640 1 -- 172.21.15.73:0/1205450345 learned_addr learned my addr 172.21.15.73:0/1205450345 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:14.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.558+0000 7f73e3fff640 1 --1- 172.21.15.73:0/1205450345 >> v1:172.21.15.73:6789/0 conn(0x7f73e40b04b0 0x7f73e406c240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51798/0 (socket says 172.21.15.73:51798) 2024-04-26T20:53:14.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.558+0000 7f73e963d640 1 -- 172.21.15.73:0/1205450345 >> v1:172.21.15.120:6789/0 conn(0x7f73e41489a0 legacy=0x7f73e40710e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:14.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.558+0000 7f73e963d640 1 -- 172.21.15.73:0/1205450345 >> v1:172.21.15.73:6789/0 conn(0x7f73e40b04b0 legacy=0x7f73e406c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:14.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.558+0000 7f73e37fe640 1 -- 172.21.15.73:0/1205450345 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1741579361 0 0) 0x7f73e4148d80 con 0x7f73e40b04b0 2024-04-26T20:53:14.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.559+0000 7f73e963d640 1 -- 172.21.15.73:0/1205450345 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73e4070600 con 0x7f73e406c950 2024-04-26T20:53:14.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.559+0000 7f73e963d640 1 --2- 172.21.15.73:0/1205450345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73e406c950 0x7f73e40b0920 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f73e40b1350 tx=0x7f73dc02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89f7af16a0337911 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:14.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.559+0000 7f73e37fe640 1 -- 172.21.15.73:0/1205450345 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73dc036070 con 0x7f73e406c950 2024-04-26T20:53:14.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.559+0000 7f73e37fe640 1 -- 172.21.15.73:0/1205450345 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73dc031600 con 0x7f73e406c950 2024-04-26T20:53:14.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.559+0000 7f73e37fe640 1 -- 172.21.15.73:0/1205450345 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73dc0319d0 con 0x7f73e406c950 2024-04-26T20:53:14.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.560+0000 7f73eb0a0640 1 -- 172.21.15.73:0/1205450345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73e406c950 msgr2=0x7f73e40b0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:14.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.560+0000 7f73eb0a0640 1 --2- 172.21.15.73:0/1205450345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73e406c950 0x7f73e40b0920 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f73e40b1350 tx=0x7f73dc02a6c0 comp rx=0 tx=0).stop 2024-04-26T20:53:14.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.560+0000 7f73eb0a0640 1 -- 172.21.15.73:0/1205450345 shutdown_connections 2024-04-26T20:53:14.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.560+0000 7f73eb0a0640 1 --2- 172.21.15.73:0/1205450345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73e406c950 0x7f73e40b0920 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:14.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.560+0000 7f73eb0a0640 1 -- 172.21.15.73:0/1205450345 >> 172.21.15.73:0/1205450345 conn(0x7f73e40b3be0 msgr2=0x7f73e40b3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:14.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.560+0000 7f73eb0a0640 1 -- 172.21.15.73:0/1205450345 shutdown_connections 2024-04-26T20:53:14.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.560+0000 7f73eb0a0640 1 -- 172.21.15.73:0/1205450345 wait complete. 2024-04-26T20:53:14.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.561+0000 7f73eb0a0640 1 Processor -- start 2024-04-26T20:53:14.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.561+0000 7f73eb0a0640 1 -- start start 2024-04-26T20:53:14.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.562+0000 7f73eb0a0640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f73e406c950 0x7f73e40aec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:14.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.562+0000 7f73eb0a0640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73e40b04b0 0x7f73e40af190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:14.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.562+0000 7f73eb0a0640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73e41489a0 0x7f73e4146530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:14.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.562+0000 7f73eb0a0640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f73e41604d0 con 0x7f73e41489a0 2024-04-26T20:53:14.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.562+0000 7f73eb0a0640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f73e4070bc0 con 0x7f73e40b04b0 2024-04-26T20:53:14.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.562+0000 7f73eb0a0640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f73e40ae8c0 con 0x7f73e406c950 2024-04-26T20:53:14.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.562+0000 7f73e963d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73e41489a0 0x7f73e4146530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:14.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.562+0000 7f73e963d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73e41489a0 0x7f73e4146530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56954/0 (socket says 172.21.15.73:56954) 2024-04-26T20:53:14.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.562+0000 7f73e963d640 1 -- 172.21.15.73:0/26157251 learned_addr learned my addr 172.21.15.73:0/26157251 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:14.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.562+0000 7f73e8e3c640 1 --2- 172.21.15.73:0/26157251 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f73e406c950 0x7f73e40aec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:14.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.562+0000 7f73e3fff640 1 --2- 172.21.15.73:0/26157251 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73e40b04b0 0x7f73e40af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:14.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.562+0000 7f73e963d640 1 -- 172.21.15.73:0/26157251 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f73e406c950 msgr2=0x7f73e40aec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:14.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.562+0000 7f73e963d640 1 --2- 172.21.15.73:0/26157251 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f73e406c950 0x7f73e40aec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:14.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.562+0000 7f73e963d640 1 -- 172.21.15.73:0/26157251 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73e40b04b0 msgr2=0x7f73e40af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:14.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.562+0000 7f73e963d640 1 --2- 172.21.15.73:0/26157251 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f73e40b04b0 0x7f73e40af190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:14.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.562+0000 7f73e963d640 1 -- 172.21.15.73:0/26157251 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73e413f990 con 0x7f73e41489a0 2024-04-26T20:53:14.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.562+0000 7f73e8e3c640 1 --2- 172.21.15.73:0/26157251 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f73e406c950 0x7f73e40aec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:53:14.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.562+0000 7f73e963d640 1 --2- 172.21.15.73:0/26157251 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73e41489a0 0x7f73e4146530 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f73dc009f40 tx=0x7f73dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:14.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.563+0000 7f73e17fa640 1 -- 172.21.15.73:0/26157251 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73dc041020 con 0x7f73e41489a0 2024-04-26T20:53:14.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.563+0000 7f73e17fa640 1 -- 172.21.15.73:0/26157251 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73dc007780 con 0x7f73e41489a0 2024-04-26T20:53:14.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.563+0000 7f73e17fa640 1 -- 172.21.15.73:0/26157251 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73dc036040 con 0x7f73e41489a0 2024-04-26T20:53:14.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.563+0000 7f73eb0a0640 1 -- 172.21.15.73:0/26157251 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73e413fc70 con 0x7f73e41489a0 2024-04-26T20:53:14.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.563+0000 7f73eb0a0640 1 -- 172.21.15.73:0/26157251 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73e4140230 con 0x7f73e41489a0 2024-04-26T20:53:14.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.563+0000 7f73eb0a0640 1 -- 172.21.15.73:0/26157251 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f73e4069600 con 0x7f73e41489a0 2024-04-26T20:53:14.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.564+0000 7f73e17fa640 1 -- 172.21.15.73:0/26157251 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f73dc044020 con 0x7f73e41489a0 2024-04-26T20:53:14.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.564+0000 7f73e17fa640 1 --2- 172.21.15.73:0/26157251 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f73d006c8f0 0x7f73d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:14.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.565+0000 7f73e17fa640 1 -- 172.21.15.73:0/26157251 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5421+0+0 (secure 0 0 0) 0x7f73dc0ac7a0 con 0x7f73e41489a0 2024-04-26T20:53:14.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.565+0000 7f73e8e3c640 1 --2- 172.21.15.73:0/26157251 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f73d006c8f0 0x7f73d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:14.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.565+0000 7f73e17fa640 1 --2- 172.21.15.73:0/26157251 >> [v2:172.21.15.120:6817/3591055561,v1:172.21.15.120:6819/3591055561] conn(0x7f73d0072e30 0x7f73d0075270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:14.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.565+0000 7f73e17fa640 1 -- 172.21.15.73:0/26157251 --> [v2:172.21.15.120:6817/3591055561,v1:172.21.15.120:6819/3591055561] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f73dc03de50 con 0x7f73d0072e30 2024-04-26T20:53:14.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.565+0000 7f73e17fa640 1 -- 172.21.15.73:0/26157251 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f73dc0b12c0 con 0x7f73e41489a0 2024-04-26T20:53:14.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.565+0000 7f73e8e3c640 1 --2- 172.21.15.73:0/26157251 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f73d006c8f0 0x7f73d006edb0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f73d400a830 tx=0x7f73d4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:14.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.565+0000 7f73e3fff640 1 --2- 172.21.15.73:0/26157251 >> [v2:172.21.15.120:6817/3591055561,v1:172.21.15.120:6819/3591055561] conn(0x7f73d0072e30 0x7f73d0075270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:14.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.565+0000 7f73e3fff640 1 --2- 172.21.15.73:0/26157251 >> [v2:172.21.15.120:6817/3591055561,v1:172.21.15.120:6819/3591055561] conn(0x7f73d0072e30 0x7f73d0075270 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:14.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.566+0000 7f73e17fa640 1 -- 172.21.15.73:0/26157251 <== osd.4 v2:172.21.15.120:6817/3591055561 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f73dc03de50 con 0x7f73d0072e30 2024-04-26T20:53:14.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.579+0000 7f73eb0a0640 1 -- 172.21.15.73:0/26157251 --> [v2:172.21.15.120:6817/3591055561,v1:172.21.15.120:6819/3591055561] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f73e40b49e0 con 0x7f73d0072e30 2024-04-26T20:53:14.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.580+0000 7f73e17fa640 1 -- 172.21.15.73:0/26157251 <== osd.4 v2:172.21.15.120:6817/3591055561 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f73e40b49e0 con 0x7f73d0072e30 2024-04-26T20:53:14.581 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-26T20:53:14.581 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T20:53:14.581 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-26T20:53:14.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.580+0000 7f73eb0a0640 1 -- 172.21.15.73:0/26157251 >> [v2:172.21.15.120:6817/3591055561,v1:172.21.15.120:6819/3591055561] conn(0x7f73d0072e30 msgr2=0x7f73d0075270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:14.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.580+0000 7f73eb0a0640 1 --2- 172.21.15.73:0/26157251 >> [v2:172.21.15.120:6817/3591055561,v1:172.21.15.120:6819/3591055561] conn(0x7f73d0072e30 0x7f73d0075270 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:14.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.580+0000 7f73eb0a0640 1 -- 172.21.15.73:0/26157251 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f73d006c8f0 msgr2=0x7f73d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:14.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.580+0000 7f73eb0a0640 1 --2- 172.21.15.73:0/26157251 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f73d006c8f0 0x7f73d006edb0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f73d400a830 tx=0x7f73d4008040 comp rx=0 tx=0).stop 2024-04-26T20:53:14.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.580+0000 7f73eb0a0640 1 -- 172.21.15.73:0/26157251 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73e41489a0 msgr2=0x7f73e4146530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:14.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.580+0000 7f73eb0a0640 1 --2- 172.21.15.73:0/26157251 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73e41489a0 0x7f73e4146530 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f73dc009f40 tx=0x7f73dc004a40 comp rx=0 tx=0).stop 2024-04-26T20:53:14.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.580+0000 7f73e963d640 1 -- 172.21.15.73:0/26157251 reap_dead start 2024-04-26T20:53:14.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.580+0000 7f73eb0a0640 1 -- 172.21.15.73:0/26157251 shutdown_connections 2024-04-26T20:53:14.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.580+0000 7f73eb0a0640 1 -- 172.21.15.73:0/26157251 >> 172.21.15.73:0/26157251 conn(0x7f73e40b3be0 msgr2=0x7f73e4144ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:14.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.580+0000 7f73eb0a0640 1 -- 172.21.15.73:0/26157251 shutdown_connections 2024-04-26T20:53:14.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.580+0000 7f73eb0a0640 1 -- 172.21.15.73:0/26157251 wait complete. 2024-04-26T20:53:14.591 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-26T20:53:14.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.689+0000 7f537deda640 1 Processor -- start 2024-04-26T20:53:14.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.689+0000 7f537deda640 1 -- start start 2024-04-26T20:53:14.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.690+0000 7f537deda640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f537815bd40 0x7f53780b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:14.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.690+0000 7f537deda640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f53780b52b0 0x7f5378072700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:14.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.690+0000 7f537deda640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f5378072c40 0x7f537815f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:14.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.690+0000 7f537deda640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f5378073060 con 0x7f53780b52b0 2024-04-26T20:53:14.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.690+0000 7f537deda640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f537806e740 con 0x7f5378072c40 2024-04-26T20:53:14.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.690+0000 7f537deda640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f53780b58e0 con 0x7f537815bd40 2024-04-26T20:53:14.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.690+0000 7f536ffff640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f53780b52b0 0x7f5378072700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:14.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.690+0000 7f536ffff640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f53780b52b0 0x7f5378072700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56968/0 (socket says 172.21.15.73:56968) 2024-04-26T20:53:14.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.690+0000 7f536ffff640 1 -- 172.21.15.73:0/826298445 learned_addr learned my addr 172.21.15.73:0/826298445 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:14.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.690+0000 7f53777fe640 1 --2- 172.21.15.73:0/826298445 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f537815bd40 0x7f53780b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:14.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.690+0000 7f5377fff640 1 --2- 172.21.15.73:0/826298445 >> v2:172.21.15.120:3300/0 conn(0x7f5378072c40 0x7f537815f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:14.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.690+0000 7f536ffff640 1 -- 172.21.15.73:0/826298445 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f537815bd40 msgr2=0x7f53780b4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:14.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.690+0000 7f536ffff640 1 --2- 172.21.15.73:0/826298445 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f537815bd40 0x7f53780b4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:14.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.690+0000 7f536ffff640 1 -- 172.21.15.73:0/826298445 >> v2:172.21.15.120:3300/0 conn(0x7f5378072c40 msgr2=0x7f537815f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:14.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.690+0000 7f536ffff640 1 --2- 172.21.15.73:0/826298445 >> v2:172.21.15.120:3300/0 conn(0x7f5378072c40 0x7f537815f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:14.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.690+0000 7f536ffff640 1 -- 172.21.15.73:0/826298445 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53780b5bc0 con 0x7f53780b52b0 2024-04-26T20:53:14.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.690+0000 7f5377fff640 1 --2- 172.21.15.73:0/826298445 >> v2:172.21.15.120:3300/0 conn(0x7f5378072c40 0x7f537815f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:53:14.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.691+0000 7f536ffff640 1 --2- 172.21.15.73:0/826298445 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53780b52b0 0x7f5378072700 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f537806e400 tx=0x7f536002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db03c5823d2c921 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:14.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.691+0000 7f5376ffd640 1 -- 172.21.15.73:0/826298445 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5360037070 con 0x7f53780b52b0 2024-04-26T20:53:14.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.691+0000 7f5376ffd640 1 -- 172.21.15.73:0/826298445 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5360032440 con 0x7f53780b52b0 2024-04-26T20:53:14.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.691+0000 7f5376ffd640 1 -- 172.21.15.73:0/826298445 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5360032810 con 0x7f53780b52b0 2024-04-26T20:53:14.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.691+0000 7f537deda640 1 -- 172.21.15.73:0/826298445 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53780b52b0 msgr2=0x7f5378072700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:14.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.691+0000 7f537deda640 1 --2- 172.21.15.73:0/826298445 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53780b52b0 0x7f5378072700 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f537806e400 tx=0x7f536002a140 comp rx=0 tx=0).stop 2024-04-26T20:53:14.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.691+0000 7f537deda640 1 -- 172.21.15.73:0/826298445 shutdown_connections 2024-04-26T20:53:14.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.691+0000 7f537deda640 1 --2- 172.21.15.73:0/826298445 >> v2:172.21.15.120:3300/0 conn(0x7f5378072c40 0x7f537815f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:14.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.691+0000 7f537deda640 1 --2- 172.21.15.73:0/826298445 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53780b52b0 0x7f5378072700 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:14.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.691+0000 7f537deda640 1 --2- 172.21.15.73:0/826298445 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f537815bd40 0x7f53780b4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:14.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.691+0000 7f537deda640 1 -- 172.21.15.73:0/826298445 >> 172.21.15.73:0/826298445 conn(0x7f53780b3110 msgr2=0x7f53780b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:14.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.691+0000 7f537deda640 1 -- 172.21.15.73:0/826298445 shutdown_connections 2024-04-26T20:53:14.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.691+0000 7f537deda640 1 -- 172.21.15.73:0/826298445 wait complete. 2024-04-26T20:53:14.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.693+0000 7f537deda640 1 Processor -- start 2024-04-26T20:53:14.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.693+0000 7f537deda640 1 -- start start 2024-04-26T20:53:14.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.693+0000 7f537deda640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5378072c40 0x7f53781593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:14.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.693+0000 7f537deda640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f53780b52b0 0x7f537806c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:14.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.693+0000 7f537deda640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f537815bd40 0x7f537806c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:14.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.693+0000 7f537deda640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f537815c120 con 0x7f537815bd40 2024-04-26T20:53:14.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.693+0000 7f537deda640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f537815c730 con 0x7f5378072c40 2024-04-26T20:53:14.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.693+0000 7f537deda640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f537815c2a0 con 0x7f53780b52b0 2024-04-26T20:53:14.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.693+0000 7f5377fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f537815bd40 0x7f537806c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:14.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.693+0000 7f5377fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f537815bd40 0x7f537806c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56974/0 (socket says 172.21.15.73:56974) 2024-04-26T20:53:14.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.693+0000 7f5377fff640 1 -- 172.21.15.73:0/2021606695 learned_addr learned my addr 172.21.15.73:0/2021606695 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:14.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.694+0000 7f53777fe640 1 --2- 172.21.15.73:0/2021606695 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5378072c40 0x7f53781593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:14.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.694+0000 7f536ffff640 1 --2- 172.21.15.73:0/2021606695 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f53780b52b0 0x7f537806c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:14.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.694+0000 7f5377fff640 1 -- 172.21.15.73:0/2021606695 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f53780b52b0 msgr2=0x7f537806c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:14.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.694+0000 7f5377fff640 1 --2- 172.21.15.73:0/2021606695 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f53780b52b0 0x7f537806c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:14.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.694+0000 7f5377fff640 1 -- 172.21.15.73:0/2021606695 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5378072c40 msgr2=0x7f53781593e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:14.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.694+0000 7f5377fff640 1 --2- 172.21.15.73:0/2021606695 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5378072c40 0x7f53781593e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:14.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.694+0000 7f5377fff640 1 -- 172.21.15.73:0/2021606695 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5378157650 con 0x7f537815bd40 2024-04-26T20:53:14.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.694+0000 7f536ffff640 1 --2- 172.21.15.73:0/2021606695 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f53780b52b0 0x7f537806c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:53:14.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.694+0000 7f5377fff640 1 --2- 172.21.15.73:0/2021606695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f537815bd40 0x7f537806c780 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f537000cf20 tx=0x7f537000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:14.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.694+0000 7f5374ff9640 1 -- 172.21.15.73:0/2021606695 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f537001a070 con 0x7f537815bd40 2024-04-26T20:53:14.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.694+0000 7f5374ff9640 1 -- 172.21.15.73:0/2021606695 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f537000d840 con 0x7f537815bd40 2024-04-26T20:53:14.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.694+0000 7f537deda640 1 -- 172.21.15.73:0/2021606695 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53781570d0 con 0x7f537815bd40 2024-04-26T20:53:14.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.694+0000 7f5374ff9640 1 -- 172.21.15.73:0/2021606695 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f537000dc30 con 0x7f537815bd40 2024-04-26T20:53:14.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.694+0000 7f537deda640 1 -- 172.21.15.73:0/2021606695 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53781546a0 con 0x7f537815bd40 2024-04-26T20:53:14.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.695+0000 7f537deda640 1 -- 172.21.15.73:0/2021606695 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5378068fc0 con 0x7f537815bd40 2024-04-26T20:53:14.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.696+0000 7f5374ff9640 1 -- 172.21.15.73:0/2021606695 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f537001d070 con 0x7f537815bd40 2024-04-26T20:53:14.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.696+0000 7f5374ff9640 1 --2- 172.21.15.73:0/2021606695 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f535806c8f0 0x7f535806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:14.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.696+0000 7f5374ff9640 1 -- 172.21.15.73:0/2021606695 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5421+0+0 (secure 0 0 0) 0x7f537008c200 con 0x7f537815bd40 2024-04-26T20:53:14.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.696+0000 7f53777fe640 1 --2- 172.21.15.73:0/2021606695 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f535806c8f0 0x7f535806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:14.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.696+0000 7f53777fe640 1 --2- 172.21.15.73:0/2021606695 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f535806c8f0 0x7f535806edb0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5368006fd0 tx=0x7f5368008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:14.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.700+0000 7f5374ff9640 1 -- 172.21.15.73:0/2021606695 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f537005a750 con 0x7f537815bd40 2024-04-26T20:53:14.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.876+0000 7f537deda640 1 -- 172.21.15.73:0/2021606695 --> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f53780694f0 con 0x7f535806c8f0 2024-04-26T20:53:14.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.876+0000 7f5374ff9640 1 -- 172.21.15.73:0/2021606695 <== mgr.4107 v2:172.21.15.73:6832/666905549 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f53780694f0 con 0x7f535806c8f0 2024-04-26T20:53:14.877 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 4 to deep-scrub 2024-04-26T20:53:14.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.878+0000 7f537deda640 1 -- 172.21.15.73:0/2021606695 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f535806c8f0 msgr2=0x7f535806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:14.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.878+0000 7f537deda640 1 --2- 172.21.15.73:0/2021606695 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f535806c8f0 0x7f535806edb0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5368006fd0 tx=0x7f5368008040 comp rx=0 tx=0).stop 2024-04-26T20:53:14.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.878+0000 7f537deda640 1 -- 172.21.15.73:0/2021606695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f537815bd40 msgr2=0x7f537806c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:14.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.878+0000 7f537deda640 1 --2- 172.21.15.73:0/2021606695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f537815bd40 0x7f537806c780 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f537000cf20 tx=0x7f537000a710 comp rx=0 tx=0).stop 2024-04-26T20:53:14.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.878+0000 7f537deda640 1 -- 172.21.15.73:0/2021606695 shutdown_connections 2024-04-26T20:53:14.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.878+0000 7f537deda640 1 --2- 172.21.15.73:0/2021606695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f537815bd40 0x7f537806c780 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:14.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.878+0000 7f537deda640 1 --2- 172.21.15.73:0/2021606695 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f535806c8f0 0x7f535806edb0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:14.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.878+0000 7f537deda640 1 --2- 172.21.15.73:0/2021606695 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f53780b52b0 0x7f537806c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:14.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.878+0000 7f537deda640 1 --2- 172.21.15.73:0/2021606695 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5378072c40 0x7f53781593e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:14.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.878+0000 7f537deda640 1 -- 172.21.15.73:0/2021606695 >> 172.21.15.73:0/2021606695 conn(0x7f53780b3110 msgr2=0x7f5378075880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:14.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.879+0000 7f537deda640 1 -- 172.21.15.73:0/2021606695 shutdown_connections 2024-04-26T20:53:14.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.879+0000 7f537deda640 1 -- 172.21.15.73:0/2021606695 wait complete. 2024-04-26T20:53:14.890 INFO:tasks.ceph:Scrubbing osd.5 2024-04-26T20:53:14.890 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-26T20:53:14.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.990+0000 7f4d4f672640 1 Processor -- start 2024-04-26T20:53:14.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.990+0000 7f4d4f672640 1 -- start start 2024-04-26T20:53:14.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.991+0000 7f4d4f672640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4d481489a0 0x7f4d480710e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:14.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.991+0000 7f4d4f672640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4d480b04b0 0x7f4d480b08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:14.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.991+0000 7f4d4f672640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d48148d80 con 0x7f4d4806c240 2024-04-26T20:53:14.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.991+0000 7f4d4f672640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f4d480b11d0 con 0x7f4d481489a0 2024-04-26T20:53:14.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.991+0000 7f4d4f672640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f4d480b0f10 con 0x7f4d480b04b0 2024-04-26T20:53:14.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.991+0000 7f4d4d40e640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4d481489a0 0x7f4d480710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:14.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.991+0000 7f4d4d40e640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4d481489a0 0x7f4d480710e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56988/0 (socket says 172.21.15.73:56988) 2024-04-26T20:53:14.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.991+0000 7f4d4d40e640 1 -- 172.21.15.73:0/1798174147 learned_addr learned my addr 172.21.15.73:0/1798174147 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:14.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.991+0000 7f4d4cc0d640 1 --2- 172.21.15.73:0/1798174147 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4d480b04b0 0x7f4d480b08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:14.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.991+0000 7f4d4d40e640 1 -- 172.21.15.73:0/1798174147 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4d480b04b0 msgr2=0x7f4d480b08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:14.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.991+0000 7f4d4d40e640 1 --2- 172.21.15.73:0/1798174147 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4d480b04b0 0x7f4d480b08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:14.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.991+0000 7f4d4d40e640 1 -- 172.21.15.73:0/1798174147 >> v1:172.21.15.120:6789/0 conn(0x7f4d4806c240 legacy=0x7f4d480ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:14.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.991+0000 7f4d4d40e640 1 -- 172.21.15.73:0/1798174147 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d48070600 con 0x7f4d481489a0 2024-04-26T20:53:14.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.991+0000 7f4d4cc0d640 1 --2- 172.21.15.73:0/1798174147 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4d480b04b0 0x7f4d480b08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:53:14.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.992+0000 7f4d4d40e640 1 --2- 172.21.15.73:0/1798174147 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d481489a0 0x7f4d480710e0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f4d44009870 tx=0x7f4d4402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d285667f1e08136 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:14.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.992+0000 7f4d3ffff640 1 -- 172.21.15.73:0/1798174147 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d44037070 con 0x7f4d481489a0 2024-04-26T20:53:14.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.992+0000 7f4d3ffff640 1 -- 172.21.15.73:0/1798174147 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d44032440 con 0x7f4d481489a0 2024-04-26T20:53:14.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.992+0000 7f4d3ffff640 1 -- 172.21.15.73:0/1798174147 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d44032810 con 0x7f4d481489a0 2024-04-26T20:53:14.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.993+0000 7f4d4f672640 1 -- 172.21.15.73:0/1798174147 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d481489a0 msgr2=0x7f4d480710e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:14.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.993+0000 7f4d4f672640 1 --2- 172.21.15.73:0/1798174147 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d481489a0 0x7f4d480710e0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f4d44009870 tx=0x7f4d4402a210 comp rx=0 tx=0).stop 2024-04-26T20:53:14.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.993+0000 7f4d4f672640 1 -- 172.21.15.73:0/1798174147 shutdown_connections 2024-04-26T20:53:14.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.993+0000 7f4d4f672640 1 --2- 172.21.15.73:0/1798174147 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4d480b04b0 0x7f4d480b08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:14.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.993+0000 7f4d4f672640 1 --2- 172.21.15.73:0/1798174147 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d481489a0 0x7f4d480710e0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:14.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.993+0000 7f4d4f672640 1 -- 172.21.15.73:0/1798174147 >> 172.21.15.73:0/1798174147 conn(0x7f4d480b3be0 msgr2=0x7f4d480b3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:14.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.993+0000 7f4d4f672640 1 -- 172.21.15.73:0/1798174147 shutdown_connections 2024-04-26T20:53:14.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.993+0000 7f4d4f672640 1 -- 172.21.15.73:0/1798174147 wait complete. 2024-04-26T20:53:14.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.994+0000 7f4d4f672640 1 Processor -- start 2024-04-26T20:53:14.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.994+0000 7f4d4f672640 1 -- start start 2024-04-26T20:53:14.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.994+0000 7f4d4f672640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d4806c240 0x7f4d480cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:14.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.994+0000 7f4d4f672640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d480b04b0 0x7f4d481413c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:14.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.994+0000 7f4d4f672640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4d481489a0 0x7f4d480b6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:14.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.994+0000 7f4d4f672640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4d48071620 con 0x7f4d480b04b0 2024-04-26T20:53:14.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.994+0000 7f4d4f672640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f4d48070bc0 con 0x7f4d4806c240 2024-04-26T20:53:14.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.995+0000 7f4d4f672640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f4d481604d0 con 0x7f4d481489a0 2024-04-26T20:53:14.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.995+0000 7f4d4cc0d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d480b04b0 0x7f4d481413c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:14.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.995+0000 7f4d4cc0d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d480b04b0 0x7f4d481413c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56994/0 (socket says 172.21.15.73:56994) 2024-04-26T20:53:14.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.995+0000 7f4d4cc0d640 1 -- 172.21.15.73:0/1907542594 learned_addr learned my addr 172.21.15.73:0/1907542594 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:14.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.995+0000 7f4d4d40e640 1 --2- 172.21.15.73:0/1907542594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d4806c240 0x7f4d480cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:14.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.995+0000 7f4d4dc0f640 1 --2- 172.21.15.73:0/1907542594 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4d481489a0 0x7f4d480b6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:14.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.995+0000 7f4d4cc0d640 1 -- 172.21.15.73:0/1907542594 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4d481489a0 msgr2=0x7f4d480b6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:14.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.995+0000 7f4d4cc0d640 1 --2- 172.21.15.73:0/1907542594 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f4d481489a0 0x7f4d480b6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:15.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.995+0000 7f4d4cc0d640 1 -- 172.21.15.73:0/1907542594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d4806c240 msgr2=0x7f4d480cfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:15.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.995+0000 7f4d4cc0d640 1 --2- 172.21.15.73:0/1907542594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d4806c240 0x7f4d480cfe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:15.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.995+0000 7f4d4cc0d640 1 -- 172.21.15.73:0/1907542594 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d481f5100 con 0x7f4d480b04b0 2024-04-26T20:53:15.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.995+0000 7f4d4d40e640 1 --2- 172.21.15.73:0/1907542594 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f4d4806c240 0x7f4d480cfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:53:15.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.995+0000 7f4d4cc0d640 1 --2- 172.21.15.73:0/1907542594 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d480b04b0 0x7f4d481413c0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f4d3800c7f0 tx=0x7f4d3800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:15.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.995+0000 7f4d3dffb640 1 -- 172.21.15.73:0/1907542594 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d38015070 con 0x7f4d480b04b0 2024-04-26T20:53:15.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.995+0000 7f4d3dffb640 1 -- 172.21.15.73:0/1907542594 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d38007820 con 0x7f4d480b04b0 2024-04-26T20:53:15.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.995+0000 7f4d4f672640 1 -- 172.21.15.73:0/1907542594 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d481f53e0 con 0x7f4d480b04b0 2024-04-26T20:53:15.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.995+0000 7f4d3dffb640 1 -- 172.21.15.73:0/1907542594 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d38007bd0 con 0x7f4d480b04b0 2024-04-26T20:53:15.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.995+0000 7f4d4f672640 1 -- 172.21.15.73:0/1907542594 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d481f59a0 con 0x7f4d480b04b0 2024-04-26T20:53:15.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.996+0000 7f4d4f672640 1 -- 172.21.15.73:0/1907542594 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4d480d0350 con 0x7f4d480b04b0 2024-04-26T20:53:15.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.997+0000 7f4d3dffb640 1 -- 172.21.15.73:0/1907542594 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4d38018070 con 0x7f4d480b04b0 2024-04-26T20:53:15.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.997+0000 7f4d3dffb640 1 --2- 172.21.15.73:0/1907542594 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f4d2c06c8f0 0x7f4d2c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:15.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.997+0000 7f4d3dffb640 1 -- 172.21.15.73:0/1907542594 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5421+0+0 (secure 0 0 0) 0x7f4d3808bb10 con 0x7f4d480b04b0 2024-04-26T20:53:15.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.997+0000 7f4d4d40e640 1 --2- 172.21.15.73:0/1907542594 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f4d2c06c8f0 0x7f4d2c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:15.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.997+0000 7f4d3dffb640 1 --2- 172.21.15.73:0/1907542594 >> [v2:172.21.15.120:6808/819450433,v1:172.21.15.120:6809/819450433] conn(0x7f4d2c072e30 0x7f4d2c075270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:15.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.997+0000 7f4d3dffb640 1 -- 172.21.15.73:0/1907542594 --> [v2:172.21.15.120:6808/819450433,v1:172.21.15.120:6809/819450433] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4d3801de50 con 0x7f4d2c072e30 2024-04-26T20:53:15.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.997+0000 7f4d3dffb640 1 -- 172.21.15.73:0/1907542594 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f4d380902c0 con 0x7f4d480b04b0 2024-04-26T20:53:15.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.998+0000 7f4d4d40e640 1 --2- 172.21.15.73:0/1907542594 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f4d2c06c8f0 0x7f4d2c06edb0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4d44034000 tx=0x7f4d4402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:15.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.998+0000 7f4d4dc0f640 1 --2- 172.21.15.73:0/1907542594 >> [v2:172.21.15.120:6808/819450433,v1:172.21.15.120:6809/819450433] conn(0x7f4d2c072e30 0x7f4d2c075270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:15.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.998+0000 7f4d4dc0f640 1 --2- 172.21.15.73:0/1907542594 >> [v2:172.21.15.120:6808/819450433,v1:172.21.15.120:6809/819450433] conn(0x7f4d2c072e30 0x7f4d2c075270 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:15.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:14.999+0000 7f4d3dffb640 1 -- 172.21.15.73:0/1907542594 <== osd.5 v2:172.21.15.120:6808/819450433 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4d3801de50 con 0x7f4d2c072e30 2024-04-26T20:53:15.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.012+0000 7f4d4f672640 1 -- 172.21.15.73:0/1907542594 --> [v2:172.21.15.120:6808/819450433,v1:172.21.15.120:6809/819450433] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f4d48069600 con 0x7f4d2c072e30 2024-04-26T20:53:15.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.013+0000 7f4d3dffb640 1 -- 172.21.15.73:0/1907542594 <== osd.5 v2:172.21.15.120:6808/819450433 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f4d48069600 con 0x7f4d2c072e30 2024-04-26T20:53:15.014 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-26T20:53:15.014 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T20:53:15.014 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-26T20:53:15.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.013+0000 7f4d4f672640 1 -- 172.21.15.73:0/1907542594 >> [v2:172.21.15.120:6808/819450433,v1:172.21.15.120:6809/819450433] conn(0x7f4d2c072e30 msgr2=0x7f4d2c075270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:15.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.013+0000 7f4d4f672640 1 --2- 172.21.15.73:0/1907542594 >> [v2:172.21.15.120:6808/819450433,v1:172.21.15.120:6809/819450433] conn(0x7f4d2c072e30 0x7f4d2c075270 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:15.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.013+0000 7f4d4f672640 1 -- 172.21.15.73:0/1907542594 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f4d2c06c8f0 msgr2=0x7f4d2c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:15.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.013+0000 7f4d4f672640 1 --2- 172.21.15.73:0/1907542594 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f4d2c06c8f0 0x7f4d2c06edb0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4d44034000 tx=0x7f4d4402e000 comp rx=0 tx=0).stop 2024-04-26T20:53:15.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.013+0000 7f4d4f672640 1 -- 172.21.15.73:0/1907542594 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d480b04b0 msgr2=0x7f4d481413c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:15.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.013+0000 7f4d4f672640 1 --2- 172.21.15.73:0/1907542594 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d480b04b0 0x7f4d481413c0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f4d3800c7f0 tx=0x7f4d3800ccb0 comp rx=0 tx=0).stop 2024-04-26T20:53:15.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.013+0000 7f4d4dc0f640 1 -- 172.21.15.73:0/1907542594 reap_dead start 2024-04-26T20:53:15.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.013+0000 7f4d4f672640 1 -- 172.21.15.73:0/1907542594 shutdown_connections 2024-04-26T20:53:15.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.013+0000 7f4d4f672640 1 -- 172.21.15.73:0/1907542594 >> 172.21.15.73:0/1907542594 conn(0x7f4d480b3be0 msgr2=0x7f4d4815fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:15.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.013+0000 7f4d4f672640 1 -- 172.21.15.73:0/1907542594 shutdown_connections 2024-04-26T20:53:15.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.013+0000 7f4d4f672640 1 -- 172.21.15.73:0/1907542594 wait complete. 2024-04-26T20:53:15.025 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-26T20:53:15.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.121+0000 7efe0c3ec640 1 Processor -- start 2024-04-26T20:53:15.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.121+0000 7efe0c3ec640 1 -- start start 2024-04-26T20:53:15.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.122+0000 7efe0c3ec640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efe040b2340 0x7efe040cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:15.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.122+0000 7efe0c3ec640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe040d10d0 con 0x7efe04162810 2024-04-26T20:53:15.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.122+0000 7efe0c3ec640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe040d1370 con 0x7efe040cd930 2024-04-26T20:53:15.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.122+0000 7efe0c3ec640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7efe040d0750 con 0x7efe040b2340 2024-04-26T20:53:15.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.122+0000 7efe0a188640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7efe04162810 0x7efe04070350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51810/0 (socket says 172.21.15.73:51810) 2024-04-26T20:53:15.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.122+0000 7efe0a188640 1 -- 172.21.15.73:0/1956029528 learned_addr learned my addr 172.21.15.73:0/1956029528 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:53:15.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.122+0000 7efe09987640 1 --2- 172.21.15.73:0/1956029528 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efe040b2340 0x7efe040cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:15.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.122+0000 7efe09186640 1 -- 172.21.15.73:0/1956029528 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 345488104 0 0) 0x7efe040d10d0 con 0x7efe04162810 2024-04-26T20:53:15.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.122+0000 7efe09186640 1 -- 172.21.15.73:0/1956029528 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efdf0003520 con 0x7efe04162810 2024-04-26T20:53:15.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.122+0000 7efe09186640 1 -- 172.21.15.73:0/1956029528 <== mon.1 v1:172.21.15.120:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3142005071 0 0) 0x7efe040d1370 con 0x7efe040cd930 2024-04-26T20:53:15.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.122+0000 7efe09186640 1 -- 172.21.15.73:0/1956029528 --> v1:172.21.15.120:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe040d10d0 con 0x7efe040cd930 2024-04-26T20:53:15.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.122+0000 7efe09186640 1 -- 172.21.15.73:0/1956029528 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4285092952 0 0) 0x7efdf0003520 con 0x7efe04162810 2024-04-26T20:53:15.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.122+0000 7efe09186640 1 -- 172.21.15.73:0/1956029528 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efe040b2340 msgr2=0x7efe040cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:15.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.122+0000 7efe09186640 1 --2- 172.21.15.73:0/1956029528 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efe040b2340 0x7efe040cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:15.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.122+0000 7efe09186640 1 -- 172.21.15.73:0/1956029528 >> v1:172.21.15.120:6789/0 conn(0x7efe040cd930 legacy=0x7efe040d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:15.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.122+0000 7efe09186640 1 -- 172.21.15.73:0/1956029528 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe040d01b0 con 0x7efe04162810 2024-04-26T20:53:15.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.122+0000 7efe09186640 1 -- 172.21.15.73:0/1956029528 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7efe00001670 con 0x7efe04162810 2024-04-26T20:53:15.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.122+0000 7efe09186640 1 -- 172.21.15.73:0/1956029528 >> v1:172.21.15.73:6789/0 conn(0x7efe04162810 legacy=0x7efe04070350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:15.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.122+0000 7efe09186640 1 --2- 172.21.15.73:0/1956029528 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efe040b2340 0x7efdf0003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:15.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.123+0000 7efe09186640 1 --2- 172.21.15.73:0/1956029528 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efdf00040f0 0x7efdf0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:15.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.123+0000 7efe09186640 1 --2- 172.21.15.73:0/1956029528 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efdf0004d40 0x7efdf00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:15.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.123+0000 7efe09186640 1 -- 172.21.15.73:0/1956029528 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efe040d0750 con 0x7efdf00040f0 2024-04-26T20:53:15.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.123+0000 7efe09186640 1 -- 172.21.15.73:0/1956029528 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efdf000baa0 con 0x7efdf0004d40 2024-04-26T20:53:15.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.123+0000 7efe09186640 1 -- 172.21.15.73:0/1956029528 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7efdf000bca0 con 0x7efe040b2340 2024-04-26T20:53:15.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.123+0000 7efe09987640 1 --2- 172.21.15.73:0/1956029528 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efdf00040f0 0x7efdf0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:15.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.123+0000 7efe0a188640 1 --2- 172.21.15.73:0/1956029528 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efe040b2340 0x7efdf0003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:15.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.123+0000 7efe0a989640 1 --2- 172.21.15.73:0/1956029528 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efdf0004d40 0x7efdf00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:15.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.123+0000 7efe09987640 1 -- 172.21.15.73:0/1956029528 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efe040b2340 msgr2=0x7efdf0003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:15.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.123+0000 7efe09987640 1 --2- 172.21.15.73:0/1956029528 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efe040b2340 0x7efdf0003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:15.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.123+0000 7efe09987640 1 -- 172.21.15.73:0/1956029528 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efdf0004d40 msgr2=0x7efdf00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:15.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.123+0000 7efe09987640 1 --2- 172.21.15.73:0/1956029528 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efdf0004d40 0x7efdf00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:15.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.123+0000 7efe09987640 1 -- 172.21.15.73:0/1956029528 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdf0003520 con 0x7efdf00040f0 2024-04-26T20:53:15.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.123+0000 7efe0a989640 1 --2- 172.21.15.73:0/1956029528 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efdf0004d40 0x7efdf00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:53:15.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.124+0000 7efe09987640 1 --2- 172.21.15.73:0/1956029528 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efdf00040f0 0x7efdf0004800 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7efdf4009ac0 tx=0x7efdf0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dee2a4aee7cc9cb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:15.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.124+0000 7efe09186640 1 -- 172.21.15.73:0/1956029528 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf4036070 con 0x7efdf00040f0 2024-04-26T20:53:15.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.124+0000 7efe09186640 1 -- 172.21.15.73:0/1956029528 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdf4031400 con 0x7efdf00040f0 2024-04-26T20:53:15.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.124+0000 7efe09186640 1 -- 172.21.15.73:0/1956029528 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf4009690 con 0x7efdf00040f0 2024-04-26T20:53:15.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.124+0000 7efe0c3ec640 1 -- 172.21.15.73:0/1956029528 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efdf00040f0 msgr2=0x7efdf0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:15.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.124+0000 7efe0c3ec640 1 --2- 172.21.15.73:0/1956029528 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efdf00040f0 0x7efdf0004800 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7efdf4009ac0 tx=0x7efdf0005830 comp rx=0 tx=0).stop 2024-04-26T20:53:15.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.124+0000 7efe0a989640 1 -- 172.21.15.73:0/1956029528 reap_dead start 2024-04-26T20:53:15.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.124+0000 7efe0c3ec640 1 -- 172.21.15.73:0/1956029528 shutdown_connections 2024-04-26T20:53:15.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.124+0000 7efe0c3ec640 1 -- 172.21.15.73:0/1956029528 >> 172.21.15.73:0/1956029528 conn(0x7efe040d2da0 msgr2=0x7efe040d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:15.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.124+0000 7efe0c3ec640 1 -- 172.21.15.73:0/1956029528 shutdown_connections 2024-04-26T20:53:15.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.124+0000 7efe0c3ec640 1 -- 172.21.15.73:0/1956029528 wait complete. 2024-04-26T20:53:15.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.125+0000 7efe0c3ec640 1 Processor -- start 2024-04-26T20:53:15.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.125+0000 7efe0c3ec640 1 -- start start 2024-04-26T20:53:15.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.126+0000 7efe0c3ec640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efdf00040f0 0x7efe040cb290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:15.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.126+0000 7efe0c3ec640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efe040c7e70 0x7efe040c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:15.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.126+0000 7efe0c3ec640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe0415d010 0x7efe0415d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:15.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.126+0000 7efe0c3ec640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efe040d2240 con 0x7efdf00040f0 2024-04-26T20:53:15.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.126+0000 7efe0c3ec640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7efe040d16a0 con 0x7efe0415d010 2024-04-26T20:53:15.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.126+0000 7efe0c3ec640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7efe040d23c0 con 0x7efe040c7e70 2024-04-26T20:53:15.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.126+0000 7efe0a188640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efdf00040f0 0x7efe040cb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:15.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.126+0000 7efe0a188640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efdf00040f0 0x7efe040cb290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:57024/0 (socket says 172.21.15.73:57024) 2024-04-26T20:53:15.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.126+0000 7efe0a188640 1 -- 172.21.15.73:0/256946707 learned_addr learned my addr 172.21.15.73:0/256946707 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:15.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.126+0000 7efe09987640 1 --2- 172.21.15.73:0/256946707 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efe040c7e70 0x7efe040c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:15.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.126+0000 7efe0a989640 1 --2- 172.21.15.73:0/256946707 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe0415d010 0x7efe0415d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:15.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.126+0000 7efe0a188640 1 -- 172.21.15.73:0/256946707 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efe040c7e70 msgr2=0x7efe040c82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:15.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.126+0000 7efe0a188640 1 --2- 172.21.15.73:0/256946707 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efe040c7e70 0x7efe040c82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:15.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.126+0000 7efe0a188640 1 -- 172.21.15.73:0/256946707 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe0415d010 msgr2=0x7efe0415d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:15.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.126+0000 7efe0a188640 1 --2- 172.21.15.73:0/256946707 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe0415d010 0x7efe0415d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:15.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.126+0000 7efe0a188640 1 -- 172.21.15.73:0/256946707 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe040c89e0 con 0x7efdf00040f0 2024-04-26T20:53:15.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.126+0000 7efe09987640 1 --2- 172.21.15.73:0/256946707 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efe040c7e70 0x7efe040c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:53:15.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.126+0000 7efe0a989640 1 --2- 172.21.15.73:0/256946707 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe0415d010 0x7efe0415d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:53:15.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.127+0000 7efe0a188640 1 --2- 172.21.15.73:0/256946707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efdf00040f0 0x7efe040cb290 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7efe000073a0 tx=0x7efe00007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:15.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.127+0000 7efdfaffd640 1 -- 172.21.15.73:0/256946707 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0001a070 con 0x7efdf00040f0 2024-04-26T20:53:15.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.127+0000 7efdfaffd640 1 -- 172.21.15.73:0/256946707 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe0000f8e0 con 0x7efdf00040f0 2024-04-26T20:53:15.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.127+0000 7efe0c3ec640 1 -- 172.21.15.73:0/256946707 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe040c8cc0 con 0x7efdf00040f0 2024-04-26T20:53:15.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.127+0000 7efdfaffd640 1 -- 172.21.15.73:0/256946707 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0000fc90 con 0x7efdf00040f0 2024-04-26T20:53:15.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.127+0000 7efe0c3ec640 1 -- 172.21.15.73:0/256946707 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe040c9280 con 0x7efdf00040f0 2024-04-26T20:53:15.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.128+0000 7efe0c3ec640 1 -- 172.21.15.73:0/256946707 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe040694f0 con 0x7efdf00040f0 2024-04-26T20:53:15.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.128+0000 7efdfaffd640 1 -- 172.21.15.73:0/256946707 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efe0001d070 con 0x7efdf00040f0 2024-04-26T20:53:15.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.128+0000 7efdfaffd640 1 --2- 172.21.15.73:0/256946707 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7efde806c8f0 0x7efde806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:15.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.129+0000 7efdfaffd640 1 -- 172.21.15.73:0/256946707 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5421+0+0 (secure 0 0 0) 0x7efe0008b3a0 con 0x7efdf00040f0 2024-04-26T20:53:15.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.129+0000 7efe09987640 1 --2- 172.21.15.73:0/256946707 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7efde806c8f0 0x7efde806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:15.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.129+0000 7efe09987640 1 --2- 172.21.15.73:0/256946707 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7efde806c8f0 0x7efde806edb0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7efdf4033000 tx=0x7efdf4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:15.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.133+0000 7efdfaffd640 1 -- 172.21.15.73:0/256946707 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efe00059980 con 0x7efdf00040f0 2024-04-26T20:53:15.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.309+0000 7efe0c3ec640 1 -- 172.21.15.73:0/256946707 --> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7efe040d1c60 con 0x7efde806c8f0 2024-04-26T20:53:15.310 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.309+0000 7efdfaffd640 1 -- 172.21.15.73:0/256946707 <== mgr.4107 v2:172.21.15.73:6832/666905549 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7efe040d1c60 con 0x7efde806c8f0 2024-04-26T20:53:15.310 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 5 to deep-scrub 2024-04-26T20:53:15.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.311+0000 7efe0c3ec640 1 -- 172.21.15.73:0/256946707 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7efde806c8f0 msgr2=0x7efde806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:15.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.311+0000 7efe0c3ec640 1 --2- 172.21.15.73:0/256946707 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7efde806c8f0 0x7efde806edb0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7efdf4033000 tx=0x7efdf4008030 comp rx=0 tx=0).stop 2024-04-26T20:53:15.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.311+0000 7efe0c3ec640 1 -- 172.21.15.73:0/256946707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efdf00040f0 msgr2=0x7efe040cb290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:15.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.311+0000 7efe0c3ec640 1 --2- 172.21.15.73:0/256946707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efdf00040f0 0x7efe040cb290 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7efe000073a0 tx=0x7efe00007860 comp rx=0 tx=0).stop 2024-04-26T20:53:15.312 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.311+0000 7efe0c3ec640 1 -- 172.21.15.73:0/256946707 shutdown_connections 2024-04-26T20:53:15.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.311+0000 7efe0c3ec640 1 --2- 172.21.15.73:0/256946707 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7efde806c8f0 0x7efde806edb0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:15.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.311+0000 7efe0c3ec640 1 --2- 172.21.15.73:0/256946707 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7efe0415d010 0x7efe0415d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:15.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.311+0000 7efe0c3ec640 1 --2- 172.21.15.73:0/256946707 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7efe040c7e70 0x7efe040c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:15.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.311+0000 7efe0c3ec640 1 --2- 172.21.15.73:0/256946707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efdf00040f0 0x7efe040cb290 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:15.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.311+0000 7efe0c3ec640 1 -- 172.21.15.73:0/256946707 >> 172.21.15.73:0/256946707 conn(0x7efe040d2da0 msgr2=0x7efe040cd1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:15.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.311+0000 7efe0c3ec640 1 -- 172.21.15.73:0/256946707 shutdown_connections 2024-04-26T20:53:15.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.311+0000 7efe0c3ec640 1 -- 172.21.15.73:0/256946707 wait complete. 2024-04-26T20:53:15.322 INFO:tasks.ceph:Scrubbing osd.6 2024-04-26T20:53:15.323 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-26T20:53:15.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.422+0000 7fedab36a640 1 Processor -- start 2024-04-26T20:53:15.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.422+0000 7fedab36a640 1 -- start start 2024-04-26T20:53:15.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.423+0000 7fedab36a640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7feda406eba0 0x7feda4160380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:15.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.423+0000 7fedab36a640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7feda41404c0 0x7feda4140930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:15.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.423+0000 7fedab36a640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feda413f100 con 0x7feda415cea0 2024-04-26T20:53:15.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.423+0000 7fedab36a640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7feda415d280 con 0x7feda406eba0 2024-04-26T20:53:15.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.423+0000 7fedab36a640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7feda4141840 con 0x7feda41404c0 2024-04-26T20:53:15.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.423+0000 7feda8905640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7feda406eba0 0x7feda4160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:15.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.423+0000 7feda9106640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7feda415cea0 0x7feda406e490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51822/0 (socket says 172.21.15.73:51822) 2024-04-26T20:53:15.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.423+0000 7feda8905640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7feda406eba0 0x7feda4160380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:57036/0 (socket says 172.21.15.73:57036) 2024-04-26T20:53:15.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.423+0000 7feda9106640 1 -- 172.21.15.73:0/1961393434 learned_addr learned my addr 172.21.15.73:0/1961393434 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:53:15.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.423+0000 7feda9907640 1 --2- 172.21.15.73:0/1961393434 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7feda41404c0 0x7feda4140930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:15.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.423+0000 7feda8905640 1 -- 172.21.15.73:0/1961393434 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7feda41404c0 msgr2=0x7feda4140930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:15.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.423+0000 7feda8905640 1 --2- 172.21.15.73:0/1961393434 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7feda41404c0 0x7feda4140930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:15.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.423+0000 7feda8905640 1 -- 172.21.15.73:0/1961393434 >> v1:172.21.15.73:6789/0 conn(0x7feda415cea0 legacy=0x7feda406e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:15.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.423+0000 7feda8905640 1 -- 172.21.15.73:0/1961393434 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feda41423c0 con 0x7feda406eba0 2024-04-26T20:53:15.425 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.423+0000 7feda3fff640 1 -- 172.21.15.73:0/1961393434 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2390847264 0 0) 0x7feda413f100 con 0x7feda415cea0 2024-04-26T20:53:15.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.424+0000 7feda8905640 1 --2- 172.21.15.73:0/1961393434 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feda406eba0 0x7feda4160380 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fed980097a0 tx=0x7fed9802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58428b511088198b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:15.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.424+0000 7feda3fff640 1 -- 172.21.15.73:0/1961393434 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed98037070 con 0x7feda406eba0 2024-04-26T20:53:15.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.424+0000 7feda3fff640 1 -- 172.21.15.73:0/1961393434 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed98032440 con 0x7feda406eba0 2024-04-26T20:53:15.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.424+0000 7feda3fff640 1 -- 172.21.15.73:0/1961393434 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed98032810 con 0x7feda406eba0 2024-04-26T20:53:15.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.425+0000 7fedab36a640 1 -- 172.21.15.73:0/1961393434 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feda406eba0 msgr2=0x7feda4160380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:15.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.425+0000 7fedab36a640 1 --2- 172.21.15.73:0/1961393434 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feda406eba0 0x7feda4160380 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fed980097a0 tx=0x7fed9802a180 comp rx=0 tx=0).stop 2024-04-26T20:53:15.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.425+0000 7fedab36a640 1 -- 172.21.15.73:0/1961393434 shutdown_connections 2024-04-26T20:53:15.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.425+0000 7fedab36a640 1 --2- 172.21.15.73:0/1961393434 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7feda41404c0 0x7feda4140930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:15.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.425+0000 7fedab36a640 1 --2- 172.21.15.73:0/1961393434 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feda406eba0 0x7feda4160380 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:15.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.425+0000 7fedab36a640 1 -- 172.21.15.73:0/1961393434 >> 172.21.15.73:0/1961393434 conn(0x7feda406f310 msgr2=0x7feda406f720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:15.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.425+0000 7fedab36a640 1 -- 172.21.15.73:0/1961393434 shutdown_connections 2024-04-26T20:53:15.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.425+0000 7fedab36a640 1 -- 172.21.15.73:0/1961393434 wait complete. 2024-04-26T20:53:15.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.426+0000 7fedab36a640 1 Processor -- start 2024-04-26T20:53:15.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.426+0000 7fedab36a640 1 -- start start 2024-04-26T20:53:15.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.427+0000 7fedab36a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feda406eba0 0x7feda4153e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:15.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.427+0000 7fedab36a640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7feda41404c0 0x7feda4154390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:15.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.427+0000 7fedab36a640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feda415cea0 0x7feda413d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:15.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.427+0000 7fedab36a640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7feda41420e0 con 0x7feda406eba0 2024-04-26T20:53:15.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.427+0000 7fedab36a640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7feda4141e00 con 0x7feda415cea0 2024-04-26T20:53:15.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.427+0000 7fedab36a640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7feda415c610 con 0x7feda41404c0 2024-04-26T20:53:15.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.427+0000 7feda9106640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feda406eba0 0x7feda4153e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:15.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.427+0000 7feda9106640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feda406eba0 0x7feda4153e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:57046/0 (socket says 172.21.15.73:57046) 2024-04-26T20:53:15.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.427+0000 7feda9106640 1 -- 172.21.15.73:0/1480729490 learned_addr learned my addr 172.21.15.73:0/1480729490 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:15.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.427+0000 7feda9106640 1 -- 172.21.15.73:0/1480729490 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7feda41404c0 msgr2=0x7feda4154390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:15.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.427+0000 7feda9106640 1 --2- 172.21.15.73:0/1480729490 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7feda41404c0 0x7feda4154390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:15.431 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.427+0000 7feda9106640 1 -- 172.21.15.73:0/1480729490 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feda415cea0 msgr2=0x7feda413d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:15.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.427+0000 7feda9106640 1 --2- 172.21.15.73:0/1480729490 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7feda415cea0 0x7feda413d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:15.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.427+0000 7feda9106640 1 -- 172.21.15.73:0/1480729490 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feda413df60 con 0x7feda406eba0 2024-04-26T20:53:15.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.427+0000 7feda9106640 1 --2- 172.21.15.73:0/1480729490 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feda406eba0 0x7feda4153e50 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fed9400c460 tx=0x7fed9400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:15.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.427+0000 7feda1ffb640 1 -- 172.21.15.73:0/1480729490 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed94018070 con 0x7feda406eba0 2024-04-26T20:53:15.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.427+0000 7feda1ffb640 1 -- 172.21.15.73:0/1480729490 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed94007500 con 0x7feda406eba0 2024-04-26T20:53:15.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.427+0000 7fedab36a640 1 -- 172.21.15.73:0/1480729490 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feda413bef0 con 0x7feda406eba0 2024-04-26T20:53:15.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.427+0000 7feda1ffb640 1 -- 172.21.15.73:0/1480729490 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed94004a20 con 0x7feda406eba0 2024-04-26T20:53:15.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.427+0000 7fedab36a640 1 -- 172.21.15.73:0/1480729490 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feda413c4b0 con 0x7feda406eba0 2024-04-26T20:53:15.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.428+0000 7fedab36a640 1 -- 172.21.15.73:0/1480729490 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7feda41f0e20 con 0x7feda406eba0 2024-04-26T20:53:15.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.429+0000 7feda1ffb640 1 -- 172.21.15.73:0/1480729490 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fed9401b070 con 0x7feda406eba0 2024-04-26T20:53:15.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.429+0000 7feda1ffb640 1 --2- 172.21.15.73:0/1480729490 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fed9006c920 0x7fed9006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:15.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.429+0000 7feda1ffb640 1 -- 172.21.15.73:0/1480729490 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5421+0+0 (secure 0 0 0) 0x7fed9408ac50 con 0x7feda406eba0 2024-04-26T20:53:15.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.429+0000 7feda8905640 1 --2- 172.21.15.73:0/1480729490 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fed9006c920 0x7fed9006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:15.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.429+0000 7feda1ffb640 1 --2- 172.21.15.73:0/1480729490 >> [v2:172.21.15.120:6800/3946210727,v1:172.21.15.120:6801/3946210727] conn(0x7fed90072e60 0x7fed900752a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:15.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.429+0000 7feda1ffb640 1 -- 172.21.15.73:0/1480729490 --> [v2:172.21.15.120:6800/3946210727,v1:172.21.15.120:6801/3946210727] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fed90075970 con 0x7fed90072e60 2024-04-26T20:53:15.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.429+0000 7feda1ffb640 1 -- 172.21.15.73:0/1480729490 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fed9408d350 con 0x7feda406eba0 2024-04-26T20:53:15.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.429+0000 7feda9907640 1 --2- 172.21.15.73:0/1480729490 >> [v2:172.21.15.120:6800/3946210727,v1:172.21.15.120:6801/3946210727] conn(0x7fed90072e60 0x7fed900752a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:15.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.430+0000 7feda8905640 1 --2- 172.21.15.73:0/1480729490 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fed9006c920 0x7fed9006ede0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fed98034000 tx=0x7fed9802c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:15.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.430+0000 7feda9907640 1 --2- 172.21.15.73:0/1480729490 >> [v2:172.21.15.120:6800/3946210727,v1:172.21.15.120:6801/3946210727] conn(0x7fed90072e60 0x7fed900752a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:15.433 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.431+0000 7feda1ffb640 1 -- 172.21.15.73:0/1480729490 <== osd.6 v2:172.21.15.120:6800/3946210727 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fed90075970 con 0x7fed90072e60 2024-04-26T20:53:15.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.443+0000 7fedab36a640 1 -- 172.21.15.73:0/1480729490 --> [v2:172.21.15.120:6800/3946210727,v1:172.21.15.120:6801/3946210727] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7feda4069600 con 0x7fed90072e60 2024-04-26T20:53:15.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.444+0000 7feda1ffb640 1 -- 172.21.15.73:0/1480729490 <== osd.6 v2:172.21.15.120:6800/3946210727 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7feda4069600 con 0x7fed90072e60 2024-04-26T20:53:15.445 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-26T20:53:15.445 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T20:53:15.445 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-26T20:53:15.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.444+0000 7fedab36a640 1 -- 172.21.15.73:0/1480729490 >> [v2:172.21.15.120:6800/3946210727,v1:172.21.15.120:6801/3946210727] conn(0x7fed90072e60 msgr2=0x7fed900752a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:15.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.444+0000 7fedab36a640 1 --2- 172.21.15.73:0/1480729490 >> [v2:172.21.15.120:6800/3946210727,v1:172.21.15.120:6801/3946210727] conn(0x7fed90072e60 0x7fed900752a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:15.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.444+0000 7fedab36a640 1 -- 172.21.15.73:0/1480729490 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fed9006c920 msgr2=0x7fed9006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:15.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.444+0000 7fedab36a640 1 --2- 172.21.15.73:0/1480729490 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fed9006c920 0x7fed9006ede0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fed98034000 tx=0x7fed9802c000 comp rx=0 tx=0).stop 2024-04-26T20:53:15.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.444+0000 7fedab36a640 1 -- 172.21.15.73:0/1480729490 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feda406eba0 msgr2=0x7feda4153e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:15.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.444+0000 7fedab36a640 1 --2- 172.21.15.73:0/1480729490 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feda406eba0 0x7feda4153e50 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fed9400c460 tx=0x7fed9400c920 comp rx=0 tx=0).stop 2024-04-26T20:53:15.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.444+0000 7feda9907640 1 -- 172.21.15.73:0/1480729490 reap_dead start 2024-04-26T20:53:15.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.444+0000 7fedab36a640 1 -- 172.21.15.73:0/1480729490 shutdown_connections 2024-04-26T20:53:15.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.444+0000 7fedab36a640 1 -- 172.21.15.73:0/1480729490 >> 172.21.15.73:0/1480729490 conn(0x7feda406f310 msgr2=0x7feda4071fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:15.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.444+0000 7fedab36a640 1 -- 172.21.15.73:0/1480729490 shutdown_connections 2024-04-26T20:53:15.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.444+0000 7fedab36a640 1 -- 172.21.15.73:0/1480729490 wait complete. 2024-04-26T20:53:15.455 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-26T20:53:15.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.552+0000 7fb744446640 1 Processor -- start 2024-04-26T20:53:15.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.552+0000 7fb744446640 1 -- start start 2024-04-26T20:53:15.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.553+0000 7fb744446640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb73c0b0b70 0x7fb73c0b0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:15.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.553+0000 7fb744446640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fb73c0aed90 0x7fb73c0af240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:15.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.553+0000 7fb744446640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb73c0b1490 con 0x7fb73c0b4d40 2024-04-26T20:53:15.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.553+0000 7fb744446640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fb73c073bc0 con 0x7fb73c0aed90 2024-04-26T20:53:15.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.553+0000 7fb744446640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fb73c072fb0 con 0x7fb73c0b0b70 2024-04-26T20:53:15.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.554+0000 7fb7419e1640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fb73c0b4d40 0x7fb73c0b6710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51826/0 (socket says 172.21.15.73:51826) 2024-04-26T20:53:15.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.554+0000 7fb7419e1640 1 -- 172.21.15.73:0/4143784821 learned_addr learned my addr 172.21.15.73:0/4143784821 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:53:15.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.554+0000 7fb7411e0640 1 -- 172.21.15.73:0/4143784821 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3527897978 0 0) 0x7fb73c0b1490 con 0x7fb73c0b4d40 2024-04-26T20:53:15.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.554+0000 7fb7411e0640 1 -- 172.21.15.73:0/4143784821 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb728003520 con 0x7fb73c0b4d40 2024-04-26T20:53:15.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.554+0000 7fb7429e3640 1 --2- 172.21.15.73:0/4143784821 >> v2:172.21.15.120:3300/0 conn(0x7fb73c0aed90 0x7fb73c0af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:15.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.554+0000 7fb7421e2640 1 --2- 172.21.15.73:0/4143784821 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb73c0b0b70 0x7fb73c0b0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:15.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.554+0000 7fb7411e0640 1 -- 172.21.15.73:0/4143784821 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2153686382 0 0) 0x7fb728003520 con 0x7fb73c0b4d40 2024-04-26T20:53:15.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.554+0000 7fb7411e0640 1 -- 172.21.15.73:0/4143784821 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb73c0b0b70 msgr2=0x7fb73c0b0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:15.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.554+0000 7fb7411e0640 1 --2- 172.21.15.73:0/4143784821 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb73c0b0b70 0x7fb73c0b0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:15.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.554+0000 7fb7411e0640 1 -- 172.21.15.73:0/4143784821 >> v2:172.21.15.120:3300/0 conn(0x7fb73c0aed90 msgr2=0x7fb73c0af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:15.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.554+0000 7fb7411e0640 1 --2- 172.21.15.73:0/4143784821 >> v2:172.21.15.120:3300/0 conn(0x7fb73c0aed90 0x7fb73c0af240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:15.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.554+0000 7fb7411e0640 1 -- 172.21.15.73:0/4143784821 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb73c073290 con 0x7fb73c0b4d40 2024-04-26T20:53:15.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.554+0000 7fb7411e0640 1 -- 172.21.15.73:0/4143784821 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7fb72c001670 con 0x7fb73c0b4d40 2024-04-26T20:53:15.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.554+0000 7fb7411e0640 1 -- 172.21.15.73:0/4143784821 >> v1:172.21.15.73:6789/0 conn(0x7fb73c0b4d40 legacy=0x7fb73c0b6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:15.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.554+0000 7fb7411e0640 1 --2- 172.21.15.73:0/4143784821 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb7280039b0 0x7fb728003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:15.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.554+0000 7fb7411e0640 1 --2- 172.21.15.73:0/4143784821 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb728004800 0x7fb728004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:15.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.554+0000 7fb7411e0640 1 --2- 172.21.15.73:0/4143784821 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb73c0b0b70 0x7fb7280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:15.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.554+0000 7fb7411e0640 1 -- 172.21.15.73:0/4143784821 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb73c073bc0 con 0x7fb728004800 2024-04-26T20:53:15.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.554+0000 7fb7411e0640 1 -- 172.21.15.73:0/4143784821 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb73c072fb0 con 0x7fb7280039b0 2024-04-26T20:53:15.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.554+0000 7fb7411e0640 1 -- 172.21.15.73:0/4143784821 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fb72800b9c0 con 0x7fb73c0b0b70 2024-04-26T20:53:15.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.554+0000 7fb7419e1640 1 --2- 172.21.15.73:0/4143784821 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb728004800 0x7fb728004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:15.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.554+0000 7fb7429e3640 1 --2- 172.21.15.73:0/4143784821 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb73c0b0b70 0x7fb7280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:15.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.554+0000 7fb7421e2640 1 --2- 172.21.15.73:0/4143784821 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb7280039b0 0x7fb728003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:15.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.555+0000 7fb7419e1640 1 -- 172.21.15.73:0/4143784821 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb73c0b0b70 msgr2=0x7fb7280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:15.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.555+0000 7fb7419e1640 1 --2- 172.21.15.73:0/4143784821 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb73c0b0b70 0x7fb7280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:15.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.555+0000 7fb7419e1640 1 -- 172.21.15.73:0/4143784821 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb7280039b0 msgr2=0x7fb728003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:15.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.555+0000 7fb7419e1640 1 --2- 172.21.15.73:0/4143784821 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb7280039b0 0x7fb728003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:15.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.555+0000 7fb7419e1640 1 -- 172.21.15.73:0/4143784821 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb728003520 con 0x7fb728004800 2024-04-26T20:53:15.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.555+0000 7fb7419e1640 1 --2- 172.21.15.73:0/4143784821 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb728004800 0x7fb728004c80 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fb72c007fd0 tx=0x7fb728005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e82e42214939c0d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:15.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.555+0000 7fb7411e0640 1 -- 172.21.15.73:0/4143784821 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb72c007d50 con 0x7fb728004800 2024-04-26T20:53:15.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.555+0000 7fb7411e0640 1 -- 172.21.15.73:0/4143784821 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb72c005ad0 con 0x7fb728004800 2024-04-26T20:53:15.559 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.555+0000 7fb7411e0640 1 -- 172.21.15.73:0/4143784821 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb72c007070 con 0x7fb728004800 2024-04-26T20:53:15.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.556+0000 7fb744446640 1 -- 172.21.15.73:0/4143784821 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb728004800 msgr2=0x7fb728004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:15.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.556+0000 7fb744446640 1 --2- 172.21.15.73:0/4143784821 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb728004800 0x7fb728004c80 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fb72c007fd0 tx=0x7fb728005830 comp rx=0 tx=0).stop 2024-04-26T20:53:15.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.556+0000 7fb7429e3640 1 -- 172.21.15.73:0/4143784821 reap_dead start 2024-04-26T20:53:15.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.556+0000 7fb744446640 1 -- 172.21.15.73:0/4143784821 shutdown_connections 2024-04-26T20:53:15.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.556+0000 7fb744446640 1 -- 172.21.15.73:0/4143784821 >> 172.21.15.73:0/4143784821 conn(0x7fb73c0b39b0 msgr2=0x7fb73c0b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:15.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.556+0000 7fb744446640 1 -- 172.21.15.73:0/4143784821 shutdown_connections 2024-04-26T20:53:15.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.556+0000 7fb744446640 1 -- 172.21.15.73:0/4143784821 wait complete. 2024-04-26T20:53:15.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.557+0000 7fb744446640 1 Processor -- start 2024-04-26T20:53:15.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.557+0000 7fb744446640 1 -- start start 2024-04-26T20:53:15.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.558+0000 7fb744446640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb728004800 0x7fb73c1488c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:15.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.558+0000 7fb744446640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb73c15b950 0x7fb73c14b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:15.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.558+0000 7fb744446640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb73c0b2ca0 0x7fb73c0b3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:15.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.558+0000 7fb744446640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb73c0b6e20 con 0x7fb728004800 2024-04-26T20:53:15.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.558+0000 7fb744446640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fb73c073850 con 0x7fb73c0b2ca0 2024-04-26T20:53:15.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.558+0000 7fb744446640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fb73c160360 con 0x7fb73c15b950 2024-04-26T20:53:15.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.558+0000 7fb7421e2640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb728004800 0x7fb73c1488c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:15.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.558+0000 7fb7421e2640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb728004800 0x7fb73c1488c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:57054/0 (socket says 172.21.15.73:57054) 2024-04-26T20:53:15.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.558+0000 7fb7421e2640 1 -- 172.21.15.73:0/3150217956 learned_addr learned my addr 172.21.15.73:0/3150217956 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:15.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.558+0000 7fb7429e3640 1 --2- 172.21.15.73:0/3150217956 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb73c0b2ca0 0x7fb73c0b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:15.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.558+0000 7fb7419e1640 1 --2- 172.21.15.73:0/3150217956 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb73c15b950 0x7fb73c14b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:15.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.558+0000 7fb7421e2640 1 -- 172.21.15.73:0/3150217956 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb73c15b950 msgr2=0x7fb73c14b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:15.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.558+0000 7fb7421e2640 1 --2- 172.21.15.73:0/3150217956 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb73c15b950 0x7fb73c14b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:15.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.558+0000 7fb7421e2640 1 -- 172.21.15.73:0/3150217956 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb73c0b2ca0 msgr2=0x7fb73c0b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:15.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.558+0000 7fb7421e2640 1 --2- 172.21.15.73:0/3150217956 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb73c0b2ca0 0x7fb73c0b3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:15.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.558+0000 7fb7421e2640 1 -- 172.21.15.73:0/3150217956 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb73c14c200 con 0x7fb728004800 2024-04-26T20:53:15.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.558+0000 7fb7429e3640 1 --2- 172.21.15.73:0/3150217956 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb73c0b2ca0 0x7fb73c0b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:53:15.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.558+0000 7fb7421e2640 1 --2- 172.21.15.73:0/3150217956 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb728004800 0x7fb73c1488c0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fb738011910 tx=0x7fb738011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:15.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.559+0000 7fb732ffd640 1 -- 172.21.15.73:0/3150217956 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb73800faf0 con 0x7fb728004800 2024-04-26T20:53:15.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.559+0000 7fb732ffd640 1 -- 172.21.15.73:0/3150217956 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb73800fc90 con 0x7fb728004800 2024-04-26T20:53:15.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.559+0000 7fb744446640 1 -- 172.21.15.73:0/3150217956 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb73c14bc40 con 0x7fb728004800 2024-04-26T20:53:15.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.559+0000 7fb732ffd640 1 -- 172.21.15.73:0/3150217956 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb738014440 con 0x7fb728004800 2024-04-26T20:53:15.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.559+0000 7fb744446640 1 -- 172.21.15.73:0/3150217956 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb73c149320 con 0x7fb728004800 2024-04-26T20:53:15.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.559+0000 7fb744446640 1 -- 172.21.15.73:0/3150217956 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb73c0694f0 con 0x7fb728004800 2024-04-26T20:53:15.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.560+0000 7fb732ffd640 1 -- 172.21.15.73:0/3150217956 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb738012040 con 0x7fb728004800 2024-04-26T20:53:15.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.560+0000 7fb732ffd640 1 --2- 172.21.15.73:0/3150217956 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb72006c8f0 0x7fb72006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:15.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.560+0000 7fb732ffd640 1 -- 172.21.15.73:0/3150217956 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5421+0+0 (secure 0 0 0) 0x7fb73808bff0 con 0x7fb728004800 2024-04-26T20:53:15.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.560+0000 7fb7419e1640 1 --2- 172.21.15.73:0/3150217956 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb72006c8f0 0x7fb72006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:15.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.561+0000 7fb7419e1640 1 --2- 172.21.15.73:0/3150217956 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb72006c8f0 0x7fb72006edb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb72c033000 tx=0x7fb72c009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:15.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.565+0000 7fb732ffd640 1 -- 172.21.15.73:0/3150217956 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb738019030 con 0x7fb728004800 2024-04-26T20:53:15.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.745+0000 7fb744446640 1 -- 172.21.15.73:0/3150217956 --> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7fb73c162620 con 0x7fb72006c8f0 2024-04-26T20:53:15.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.746+0000 7fb732ffd640 1 -- 172.21.15.73:0/3150217956 <== mgr.4107 v2:172.21.15.73:6832/666905549 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb73c162620 con 0x7fb72006c8f0 2024-04-26T20:53:15.747 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 6 to deep-scrub 2024-04-26T20:53:15.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.748+0000 7fb744446640 1 -- 172.21.15.73:0/3150217956 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb72006c8f0 msgr2=0x7fb72006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:15.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.748+0000 7fb744446640 1 --2- 172.21.15.73:0/3150217956 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb72006c8f0 0x7fb72006edb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb72c033000 tx=0x7fb72c009030 comp rx=0 tx=0).stop 2024-04-26T20:53:15.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.748+0000 7fb744446640 1 -- 172.21.15.73:0/3150217956 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb728004800 msgr2=0x7fb73c1488c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:15.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.748+0000 7fb744446640 1 --2- 172.21.15.73:0/3150217956 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb728004800 0x7fb73c1488c0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fb738011910 tx=0x7fb738011dd0 comp rx=0 tx=0).stop 2024-04-26T20:53:15.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.748+0000 7fb744446640 1 -- 172.21.15.73:0/3150217956 shutdown_connections 2024-04-26T20:53:15.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.748+0000 7fb744446640 1 --2- 172.21.15.73:0/3150217956 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fb72006c8f0 0x7fb72006edb0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:15.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.748+0000 7fb744446640 1 --2- 172.21.15.73:0/3150217956 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fb73c0b2ca0 0x7fb73c0b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:15.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.748+0000 7fb744446640 1 --2- 172.21.15.73:0/3150217956 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fb73c15b950 0x7fb73c14b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:15.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.748+0000 7fb744446640 1 --2- 172.21.15.73:0/3150217956 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb728004800 0x7fb73c1488c0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:15.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.748+0000 7fb744446640 1 -- 172.21.15.73:0/3150217956 >> 172.21.15.73:0/3150217956 conn(0x7fb73c0b39b0 msgr2=0x7fb73c0b1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:15.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.748+0000 7fb744446640 1 -- 172.21.15.73:0/3150217956 shutdown_connections 2024-04-26T20:53:15.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.748+0000 7fb744446640 1 -- 172.21.15.73:0/3150217956 wait complete. 2024-04-26T20:53:15.759 INFO:tasks.ceph:Scrubbing osd.7 2024-04-26T20:53:15.760 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-26T20:53:15.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.858+0000 7f33b399d640 1 Processor -- start 2024-04-26T20:53:15.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.859+0000 7f33b399d640 1 -- start start 2024-04-26T20:53:15.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.859+0000 7f33b399d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f33ac15f3e0 0x7f33ac0d2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:15.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.859+0000 7f33b399d640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f33ac0d3da0 0x7f33ac1620a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:15.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.859+0000 7f33b399d640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33ac160930 con 0x7f33ac0d31e0 2024-04-26T20:53:15.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.859+0000 7f33b399d640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f33ac15fd20 con 0x7f33ac0d3da0 2024-04-26T20:53:15.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.859+0000 7f33b399d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f33ac160000 con 0x7f33ac15f3e0 2024-04-26T20:53:15.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.860+0000 7f33b1f3a640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f33ac0d3da0 0x7f33ac1620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:15.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.860+0000 7f33b0f38640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f33ac0d31e0 0x7f33ac0d3690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51834/0 (socket says 172.21.15.73:51834) 2024-04-26T20:53:15.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.860+0000 7f33b1f3a640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f33ac0d3da0 0x7f33ac1620a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:57060/0 (socket says 172.21.15.73:57060) 2024-04-26T20:53:15.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.860+0000 7f33b0f38640 1 -- 172.21.15.73:0/1309219921 learned_addr learned my addr 172.21.15.73:0/1309219921 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:53:15.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.860+0000 7f33b1f3a640 1 -- 172.21.15.73:0/1309219921 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f33ac15f3e0 msgr2=0x7f33ac0d2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:15.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.860+0000 7f33b1739640 1 --2- 172.21.15.73:0/1309219921 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f33ac15f3e0 0x7f33ac0d2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:15.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.860+0000 7f33a3fff640 1 -- 172.21.15.73:0/1309219921 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2714491557 0 0) 0x7f33ac160930 con 0x7f33ac0d31e0 2024-04-26T20:53:15.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.860+0000 7f33b1f3a640 1 --2- 172.21.15.73:0/1309219921 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f33ac15f3e0 0x7f33ac0d2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:15.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.860+0000 7f33b1f3a640 1 -- 172.21.15.73:0/1309219921 >> v1:172.21.15.73:6789/0 conn(0x7f33ac0d31e0 legacy=0x7f33ac0d3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:15.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.860+0000 7f33b1f3a640 1 -- 172.21.15.73:0/1309219921 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33ac1602e0 con 0x7f33ac0d3da0 2024-04-26T20:53:15.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.860+0000 7f33b1739640 1 --2- 172.21.15.73:0/1309219921 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f33ac15f3e0 0x7f33ac0d2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:53:15.862 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.861+0000 7f33b1f3a640 1 --2- 172.21.15.73:0/1309219921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33ac0d3da0 0x7f33ac1620a0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f33a4009d90 tx=0x7f33a402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1e15cf8bdbc1539 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:15.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.861+0000 7f33a3fff640 1 -- 172.21.15.73:0/1309219921 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a4031600 con 0x7f33ac0d3da0 2024-04-26T20:53:15.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.861+0000 7f33a3fff640 1 -- 172.21.15.73:0/1309219921 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33a40317a0 con 0x7f33ac0d3da0 2024-04-26T20:53:15.863 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.861+0000 7f33a3fff640 1 -- 172.21.15.73:0/1309219921 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a4031b70 con 0x7f33ac0d3da0 2024-04-26T20:53:15.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.861+0000 7f33b399d640 1 -- 172.21.15.73:0/1309219921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33ac0d3da0 msgr2=0x7f33ac1620a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:15.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.861+0000 7f33b399d640 1 --2- 172.21.15.73:0/1309219921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33ac0d3da0 0x7f33ac1620a0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f33a4009d90 tx=0x7f33a402a5d0 comp rx=0 tx=0).stop 2024-04-26T20:53:15.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.861+0000 7f33b399d640 1 -- 172.21.15.73:0/1309219921 shutdown_connections 2024-04-26T20:53:15.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.861+0000 7f33b399d640 1 --2- 172.21.15.73:0/1309219921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33ac0d3da0 0x7f33ac1620a0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:15.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.861+0000 7f33b399d640 1 --2- 172.21.15.73:0/1309219921 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f33ac15f3e0 0x7f33ac0d2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:15.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.861+0000 7f33b399d640 1 -- 172.21.15.73:0/1309219921 >> 172.21.15.73:0/1309219921 conn(0x7f33ac09e3c0 msgr2=0x7f33ac0d50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:15.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.861+0000 7f33b399d640 1 -- 172.21.15.73:0/1309219921 shutdown_connections 2024-04-26T20:53:15.865 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.861+0000 7f33b399d640 1 -- 172.21.15.73:0/1309219921 wait complete. 2024-04-26T20:53:15.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.862+0000 7f33b399d640 1 Processor -- start 2024-04-26T20:53:15.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.863+0000 7f33b399d640 1 -- start start 2024-04-26T20:53:15.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.863+0000 7f33b399d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33ac0d31e0 0x7f33ac156680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:15.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.863+0000 7f33b399d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33ac0d3da0 0x7f33ac15d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:15.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.863+0000 7f33b399d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f33ac15f3e0 0x7f33ac0a1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:15.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.863+0000 7f33b399d640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f33ac15f7c0 con 0x7f33ac0d3da0 2024-04-26T20:53:15.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.863+0000 7f33b399d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f33ac160d40 con 0x7f33ac0d31e0 2024-04-26T20:53:15.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.863+0000 7f33b399d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f33ac15f940 con 0x7f33ac15f3e0 2024-04-26T20:53:15.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.863+0000 7f33b0f38640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33ac0d3da0 0x7f33ac15d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:15.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.863+0000 7f33b0f38640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33ac0d3da0 0x7f33ac15d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:57076/0 (socket says 172.21.15.73:57076) 2024-04-26T20:53:15.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.863+0000 7f33b0f38640 1 -- 172.21.15.73:0/1450212301 learned_addr learned my addr 172.21.15.73:0/1450212301 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:15.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.864+0000 7f33b0f38640 1 -- 172.21.15.73:0/1450212301 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f33ac15f3e0 msgr2=0x7f33ac0a1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:15.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.864+0000 7f33b0f38640 1 --2- 172.21.15.73:0/1450212301 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f33ac15f3e0 0x7f33ac0a1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:15.866 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.864+0000 7f33b0f38640 1 -- 172.21.15.73:0/1450212301 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33ac0d31e0 msgr2=0x7f33ac156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:15.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.864+0000 7f33b0f38640 1 --2- 172.21.15.73:0/1450212301 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f33ac0d31e0 0x7f33ac156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:15.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.864+0000 7f33b0f38640 1 -- 172.21.15.73:0/1450212301 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33ac09f0e0 con 0x7f33ac0d3da0 2024-04-26T20:53:15.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.864+0000 7f33b0f38640 1 --2- 172.21.15.73:0/1450212301 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33ac0d3da0 0x7f33ac15d360 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f339c00c4e0 tx=0x7f339c00c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:15.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.864+0000 7f33a1ffb640 1 -- 172.21.15.73:0/1450212301 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339c018070 con 0x7f33ac0d3da0 2024-04-26T20:53:15.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.864+0000 7f33a1ffb640 1 -- 172.21.15.73:0/1450212301 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f339c007500 con 0x7f33ac0d3da0 2024-04-26T20:53:15.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.864+0000 7f33b399d640 1 -- 172.21.15.73:0/1450212301 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33ac0cf0e0 con 0x7f33ac0d3da0 2024-04-26T20:53:15.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.864+0000 7f33a1ffb640 1 -- 172.21.15.73:0/1450212301 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339c004a20 con 0x7f33ac0d3da0 2024-04-26T20:53:15.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.864+0000 7f33b399d640 1 -- 172.21.15.73:0/1450212301 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33ac0cd030 con 0x7f33ac0d3da0 2024-04-26T20:53:15.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.865+0000 7f33b399d640 1 -- 172.21.15.73:0/1450212301 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f33ac0ccb80 con 0x7f33ac0d3da0 2024-04-26T20:53:15.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.865+0000 7f33a1ffb640 1 -- 172.21.15.73:0/1450212301 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f339c01b070 con 0x7f33ac0d3da0 2024-04-26T20:53:15.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.866+0000 7f33a1ffb640 1 --2- 172.21.15.73:0/1450212301 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f339006c8f0 0x7f339006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:15.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.866+0000 7f33a1ffb640 1 -- 172.21.15.73:0/1450212301 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5421+0+0 (secure 0 0 0) 0x7f339c08acd0 con 0x7f33ac0d3da0 2024-04-26T20:53:15.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.866+0000 7f33b1739640 1 --2- 172.21.15.73:0/1450212301 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f339006c8f0 0x7f339006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:15.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.866+0000 7f33a1ffb640 1 --2- 172.21.15.73:0/1450212301 >> [v2:172.21.15.120:6816/2915574674,v1:172.21.15.120:6818/2915574674] conn(0x7f3390072e30 0x7f3390075270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:15.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.866+0000 7f33a1ffb640 1 -- 172.21.15.73:0/1450212301 --> [v2:172.21.15.120:6816/2915574674,v1:172.21.15.120:6818/2915574674] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f339c01ee50 con 0x7f3390072e30 2024-04-26T20:53:15.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.866+0000 7f33a1ffb640 1 -- 172.21.15.73:0/1450212301 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f339c08d3d0 con 0x7f33ac0d3da0 2024-04-26T20:53:15.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.866+0000 7f33b1739640 1 --2- 172.21.15.73:0/1450212301 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f339006c8f0 0x7f339006edb0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f33a8006fd0 tx=0x7f33a8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:15.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.866+0000 7f33b1f3a640 1 --2- 172.21.15.73:0/1450212301 >> [v2:172.21.15.120:6816/2915574674,v1:172.21.15.120:6818/2915574674] conn(0x7f3390072e30 0x7f3390075270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:15.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.867+0000 7f33b1f3a640 1 --2- 172.21.15.73:0/1450212301 >> [v2:172.21.15.120:6816/2915574674,v1:172.21.15.120:6818/2915574674] conn(0x7f3390072e30 0x7f3390075270 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:15.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.867+0000 7f33a1ffb640 1 -- 172.21.15.73:0/1450212301 <== osd.7 v2:172.21.15.120:6816/2915574674 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f339c01ee50 con 0x7f3390072e30 2024-04-26T20:53:15.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.880+0000 7f33b399d640 1 -- 172.21.15.73:0/1450212301 --> [v2:172.21.15.120:6816/2915574674,v1:172.21.15.120:6818/2915574674] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f33ac0cd480 con 0x7f3390072e30 2024-04-26T20:53:15.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.881+0000 7f33a1ffb640 1 -- 172.21.15.73:0/1450212301 <== osd.7 v2:172.21.15.120:6816/2915574674 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f33ac0cd480 con 0x7f3390072e30 2024-04-26T20:53:15.882 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-26T20:53:15.882 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T20:53:15.882 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-26T20:53:15.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.881+0000 7f33b399d640 1 -- 172.21.15.73:0/1450212301 >> [v2:172.21.15.120:6816/2915574674,v1:172.21.15.120:6818/2915574674] conn(0x7f3390072e30 msgr2=0x7f3390075270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:15.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.881+0000 7f33b399d640 1 --2- 172.21.15.73:0/1450212301 >> [v2:172.21.15.120:6816/2915574674,v1:172.21.15.120:6818/2915574674] conn(0x7f3390072e30 0x7f3390075270 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:15.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.881+0000 7f33b399d640 1 -- 172.21.15.73:0/1450212301 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f339006c8f0 msgr2=0x7f339006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:15.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.881+0000 7f33b399d640 1 --2- 172.21.15.73:0/1450212301 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f339006c8f0 0x7f339006edb0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f33a8006fd0 tx=0x7f33a8008040 comp rx=0 tx=0).stop 2024-04-26T20:53:15.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.881+0000 7f33b399d640 1 -- 172.21.15.73:0/1450212301 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33ac0d3da0 msgr2=0x7f33ac15d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:15.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.881+0000 7f33b399d640 1 --2- 172.21.15.73:0/1450212301 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33ac0d3da0 0x7f33ac15d360 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f339c00c4e0 tx=0x7f339c00c9a0 comp rx=0 tx=0).stop 2024-04-26T20:53:15.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.881+0000 7f33b1f3a640 1 -- 172.21.15.73:0/1450212301 reap_dead start 2024-04-26T20:53:15.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.881+0000 7f33b399d640 1 -- 172.21.15.73:0/1450212301 shutdown_connections 2024-04-26T20:53:15.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.881+0000 7f33b399d640 1 -- 172.21.15.73:0/1450212301 >> 172.21.15.73:0/1450212301 conn(0x7f33ac09e3c0 msgr2=0x7f33ac0b7d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:15.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.881+0000 7f33b399d640 1 -- 172.21.15.73:0/1450212301 shutdown_connections 2024-04-26T20:53:15.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.881+0000 7f33b399d640 1 -- 172.21.15.73:0/1450212301 wait complete. 2024-04-26T20:53:15.892 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-26T20:53:15.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.989+0000 7f0394711640 1 Processor -- start 2024-04-26T20:53:15.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.990+0000 7f0394711640 1 -- start start 2024-04-26T20:53:15.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.990+0000 7f0394711640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f038c162880 0x7f038c0d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:15.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.990+0000 7f0394711640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f038c0d3bc0 0x7f038c0b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:15.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.990+0000 7f0394711640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f038c0b7e50 con 0x7f038c0b5f20 2024-04-26T20:53:15.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.990+0000 7f0394711640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f038c0d0580 con 0x7f038c162880 2024-04-26T20:53:15.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.990+0000 7f0394711640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f038c0d1800 con 0x7f038c0d3bc0 2024-04-26T20:53:15.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.991+0000 7f03924ad640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f038c162880 0x7f038c0d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:15.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.991+0000 7f0391cac640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f038c0b5f20 0x7f038c0d34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51840/0 (socket says 172.21.15.73:51840) 2024-04-26T20:53:15.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.991+0000 7f03924ad640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f038c162880 0x7f038c0d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:57082/0 (socket says 172.21.15.73:57082) 2024-04-26T20:53:15.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.991+0000 7f03924ad640 1 -- 172.21.15.73:0/2369749025 learned_addr learned my addr 172.21.15.73:0/2369749025 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:15.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.991+0000 7f0392cae640 1 --2- 172.21.15.73:0/2369749025 >> v2:172.21.15.120:3300/0 conn(0x7f038c0d3bc0 0x7f038c0b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:15.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.991+0000 7f03924ad640 1 -- 172.21.15.73:0/2369749025 >> v2:172.21.15.120:3300/0 conn(0x7f038c0d3bc0 msgr2=0x7f038c0b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:15.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.991+0000 7f03924ad640 1 --2- 172.21.15.73:0/2369749025 >> v2:172.21.15.120:3300/0 conn(0x7f038c0d3bc0 0x7f038c0b8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:15.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.991+0000 7f03924ad640 1 -- 172.21.15.73:0/2369749025 >> v1:172.21.15.73:6789/0 conn(0x7f038c0b5f20 legacy=0x7f038c0d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:15.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.991+0000 7f03924ad640 1 -- 172.21.15.73:0/2369749025 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f038c0d07d0 con 0x7f038c162880 2024-04-26T20:53:15.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.991+0000 7f0392cae640 1 --2- 172.21.15.73:0/2369749025 >> v2:172.21.15.120:3300/0 conn(0x7f038c0d3bc0 0x7f038c0b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:53:15.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.992+0000 7f03924ad640 1 --2- 172.21.15.73:0/2369749025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f038c162880 0x7f038c0d2cb0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f038c0d11d0 tx=0x7f037c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81748fe211548da9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:15.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.992+0000 7f03914ab640 1 -- 172.21.15.73:0/2369749025 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f037c037070 con 0x7f038c162880 2024-04-26T20:53:15.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.992+0000 7f03914ab640 1 -- 172.21.15.73:0/2369749025 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f037c032440 con 0x7f038c162880 2024-04-26T20:53:15.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.992+0000 7f03914ab640 1 -- 172.21.15.73:0/2369749025 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f037c032810 con 0x7f038c162880 2024-04-26T20:53:15.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.992+0000 7f0394711640 1 -- 172.21.15.73:0/2369749025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f038c162880 msgr2=0x7f038c0d2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:15.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.992+0000 7f0394711640 1 --2- 172.21.15.73:0/2369749025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f038c162880 0x7f038c0d2cb0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f038c0d11d0 tx=0x7f037c02a270 comp rx=0 tx=0).stop 2024-04-26T20:53:15.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.992+0000 7f0394711640 1 -- 172.21.15.73:0/2369749025 shutdown_connections 2024-04-26T20:53:15.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.992+0000 7f0394711640 1 --2- 172.21.15.73:0/2369749025 >> v2:172.21.15.120:3300/0 conn(0x7f038c0d3bc0 0x7f038c0b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:15.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.992+0000 7f0394711640 1 --2- 172.21.15.73:0/2369749025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f038c162880 0x7f038c0d2cb0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:15.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.992+0000 7f0394711640 1 -- 172.21.15.73:0/2369749025 >> 172.21.15.73:0/2369749025 conn(0x7f038c0d4f60 msgr2=0x7f038c0d5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:15.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.992+0000 7f0394711640 1 -- 172.21.15.73:0/2369749025 shutdown_connections 2024-04-26T20:53:15.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.992+0000 7f0394711640 1 -- 172.21.15.73:0/2369749025 wait complete. 2024-04-26T20:53:15.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.993+0000 7f0394711640 1 Processor -- start 2024-04-26T20:53:15.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.994+0000 7f0394711640 1 -- start start 2024-04-26T20:53:15.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.994+0000 7f0394711640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f038c0b5f20 0x7f038c0b2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:15.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.994+0000 7f0394711640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f038c0d3bc0 0x7f038c153430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:15.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.994+0000 7f0394711640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f038c162880 0x7f038c0cdf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:15.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.994+0000 7f0394711640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f038c162c60 con 0x7f038c0b5f20 2024-04-26T20:53:15.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.994+0000 7f0394711640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f038c0d0d90 con 0x7f038c0d3bc0 2024-04-26T20:53:15.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.994+0000 7f0394711640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f038c162de0 con 0x7f038c162880 2024-04-26T20:53:15.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.994+0000 7f03924ad640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f038c0b5f20 0x7f038c0b2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:15.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.994+0000 7f03924ad640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f038c0b5f20 0x7f038c0b2f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:57092/0 (socket says 172.21.15.73:57092) 2024-04-26T20:53:15.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.994+0000 7f03924ad640 1 -- 172.21.15.73:0/1146905662 learned_addr learned my addr 172.21.15.73:0/1146905662 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:15.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.995+0000 7f03924ad640 1 -- 172.21.15.73:0/1146905662 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f038c162880 msgr2=0x7f038c0cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:15.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.995+0000 7f03924ad640 1 --2- 172.21.15.73:0/1146905662 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f038c162880 0x7f038c0cdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:15.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.995+0000 7f03924ad640 1 -- 172.21.15.73:0/1146905662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f038c0d3bc0 msgr2=0x7f038c153430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:15.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.995+0000 7f03924ad640 1 --2- 172.21.15.73:0/1146905662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f038c0d3bc0 0x7f038c153430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:15.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.995+0000 7f03924ad640 1 -- 172.21.15.73:0/1146905662 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f038c0ce4d0 con 0x7f038c0b5f20 2024-04-26T20:53:15.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.995+0000 7f03924ad640 1 --2- 172.21.15.73:0/1146905662 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f038c0b5f20 0x7f038c0b2f30 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f037c000c00 tx=0x7f037c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:15.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.995+0000 7f0382ffd640 1 -- 172.21.15.73:0/1146905662 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f037c037040 con 0x7f038c0b5f20 2024-04-26T20:53:15.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.995+0000 7f0382ffd640 1 -- 172.21.15.73:0/1146905662 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f037c042070 con 0x7f038c0b5f20 2024-04-26T20:53:15.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.995+0000 7f0394711640 1 -- 172.21.15.73:0/1146905662 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f038c0b35c0 con 0x7f038c0b5f20 2024-04-26T20:53:15.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.995+0000 7f0382ffd640 1 -- 172.21.15.73:0/1146905662 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f037c030bb0 con 0x7f038c0b5f20 2024-04-26T20:53:15.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.995+0000 7f0394711640 1 -- 172.21.15.73:0/1146905662 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f038c0b1da0 con 0x7f038c0b5f20 2024-04-26T20:53:15.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.996+0000 7f0394711640 1 -- 172.21.15.73:0/1146905662 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f038c0694f0 con 0x7f038c0b5f20 2024-04-26T20:53:15.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.996+0000 7f0382ffd640 1 -- 172.21.15.73:0/1146905662 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f037c047460 con 0x7f038c0b5f20 2024-04-26T20:53:15.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.997+0000 7f0382ffd640 1 --2- 172.21.15.73:0/1146905662 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f037006c920 0x7f037006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:15.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.997+0000 7f0382ffd640 1 -- 172.21.15.73:0/1146905662 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5421+0+0 (secure 0 0 0) 0x7f037c0ac580 con 0x7f038c0b5f20 2024-04-26T20:53:15.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.997+0000 7f0391cac640 1 --2- 172.21.15.73:0/1146905662 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f037006c920 0x7f037006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:16.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:15.997+0000 7f0391cac640 1 --2- 172.21.15.73:0/1146905662 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f037006c920 0x7f037006ede0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0388000f60 tx=0x7f0388007a00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:16.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.001+0000 7f0382ffd640 1 -- 172.21.15.73:0/1146905662 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f037c045360 con 0x7f038c0b5f20 2024-04-26T20:53:16.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.178+0000 7f0394711640 1 -- 172.21.15.73:0/1146905662 --> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f038c069c20 con 0x7f037006c920 2024-04-26T20:53:16.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.179+0000 7f0382ffd640 1 -- 172.21.15.73:0/1146905662 <== mgr.4107 v2:172.21.15.73:6832/666905549 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f038c069c20 con 0x7f037006c920 2024-04-26T20:53:16.180 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 7 to deep-scrub 2024-04-26T20:53:16.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.181+0000 7f0394711640 1 -- 172.21.15.73:0/1146905662 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f037006c920 msgr2=0x7f037006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:16.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.181+0000 7f0394711640 1 --2- 172.21.15.73:0/1146905662 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f037006c920 0x7f037006ede0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0388000f60 tx=0x7f0388007a00 comp rx=0 tx=0).stop 2024-04-26T20:53:16.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.181+0000 7f0394711640 1 -- 172.21.15.73:0/1146905662 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f038c0b5f20 msgr2=0x7f038c0b2f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:16.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.181+0000 7f0394711640 1 --2- 172.21.15.73:0/1146905662 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f038c0b5f20 0x7f038c0b2f30 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f037c000c00 tx=0x7f037c0046c0 comp rx=0 tx=0).stop 2024-04-26T20:53:16.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.181+0000 7f0394711640 1 -- 172.21.15.73:0/1146905662 shutdown_connections 2024-04-26T20:53:16.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.181+0000 7f0394711640 1 --2- 172.21.15.73:0/1146905662 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f037006c920 0x7f037006ede0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:16.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.181+0000 7f0394711640 1 --2- 172.21.15.73:0/1146905662 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f038c162880 0x7f038c0cdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:16.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.181+0000 7f0394711640 1 --2- 172.21.15.73:0/1146905662 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f038c0d3bc0 0x7f038c153430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:16.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.181+0000 7f0394711640 1 --2- 172.21.15.73:0/1146905662 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f038c0b5f20 0x7f038c0b2f30 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:16.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.181+0000 7f0394711640 1 -- 172.21.15.73:0/1146905662 >> 172.21.15.73:0/1146905662 conn(0x7f038c0d4f60 msgr2=0x7f038c15f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:16.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.181+0000 7f0394711640 1 -- 172.21.15.73:0/1146905662 shutdown_connections 2024-04-26T20:53:16.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.181+0000 7f0394711640 1 -- 172.21.15.73:0/1146905662 wait complete. 2024-04-26T20:53:16.193 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:53:16.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.287+0000 7f76b8b9d640 1 Processor -- start 2024-04-26T20:53:16.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.287+0000 7f76b8b9d640 1 -- start start 2024-04-26T20:53:16.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.288+0000 7f76b8b9d640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f76b415f2d0 0x7f76b41604e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:16.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.288+0000 7f76b8b9d640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f76b406fea0 0x7f76b40ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:16.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.288+0000 7f76b8b9d640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76b4072f70 con 0x7f76b4072700 2024-04-26T20:53:16.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.288+0000 7f76b8b9d640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f76b4074010 con 0x7f76b406fea0 2024-04-26T20:53:16.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.288+0000 7f76b8b9d640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f76b4073250 con 0x7f76b415f2d0 2024-04-26T20:53:16.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.288+0000 7f76b2d9d640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f76b406fea0 0x7f76b40ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:16.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.288+0000 7f76abfff640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f76b4072700 0x7f76b406f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51850/0 (socket says 172.21.15.73:51850) 2024-04-26T20:53:16.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.288+0000 7f76abfff640 1 -- 172.21.15.73:0/79963327 learned_addr learned my addr 172.21.15.73:0/79963327 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:53:16.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.289+0000 7f76b259c640 1 --2- 172.21.15.73:0/79963327 >> v2:172.21.15.120:3300/0 conn(0x7f76b415f2d0 0x7f76b41604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:16.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.289+0000 7f76b2d9d640 1 -- 172.21.15.73:0/79963327 >> v2:172.21.15.120:3300/0 conn(0x7f76b415f2d0 msgr2=0x7f76b41604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:16.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.289+0000 7f76b2d9d640 1 --2- 172.21.15.73:0/79963327 >> v2:172.21.15.120:3300/0 conn(0x7f76b415f2d0 0x7f76b41604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:16.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.289+0000 7f76b2d9d640 1 -- 172.21.15.73:0/79963327 >> v1:172.21.15.73:6789/0 conn(0x7f76b4072700 legacy=0x7f76b406f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:16.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.289+0000 7f76b2d9d640 1 -- 172.21.15.73:0/79963327 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76b4073530 con 0x7f76b406fea0 2024-04-26T20:53:16.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.289+0000 7f76b1d9b640 1 -- 172.21.15.73:0/79963327 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1372756851 0 0) 0x7f76b4072f70 con 0x7f76b4072700 2024-04-26T20:53:16.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.289+0000 7f76b2d9d640 1 --2- 172.21.15.73:0/79963327 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76b406fea0 0x7f76b40ce7b0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f76b40702c0 tx=0x7f76ac02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d88a995104883305 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:16.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.289+0000 7f76b1d9b640 1 -- 172.21.15.73:0/79963327 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ac007740 con 0x7f76b406fea0 2024-04-26T20:53:16.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.290+0000 7f76b1d9b640 1 -- 172.21.15.73:0/79963327 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76ac0078e0 con 0x7f76b406fea0 2024-04-26T20:53:16.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.290+0000 7f76b1d9b640 1 -- 172.21.15.73:0/79963327 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ac007cb0 con 0x7f76b406fea0 2024-04-26T20:53:16.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.290+0000 7f76b8b9d640 1 -- 172.21.15.73:0/79963327 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76b406fea0 msgr2=0x7f76b40ce7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:16.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.290+0000 7f76b8b9d640 1 --2- 172.21.15.73:0/79963327 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76b406fea0 0x7f76b40ce7b0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f76b40702c0 tx=0x7f76ac02a680 comp rx=0 tx=0).stop 2024-04-26T20:53:16.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.290+0000 7f76b8b9d640 1 -- 172.21.15.73:0/79963327 shutdown_connections 2024-04-26T20:53:16.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.290+0000 7f76b8b9d640 1 --2- 172.21.15.73:0/79963327 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76b406fea0 0x7f76b40ce7b0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:16.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.290+0000 7f76b8b9d640 1 --2- 172.21.15.73:0/79963327 >> v2:172.21.15.120:3300/0 conn(0x7f76b415f2d0 0x7f76b41604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:16.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.290+0000 7f76b8b9d640 1 -- 172.21.15.73:0/79963327 >> 172.21.15.73:0/79963327 conn(0x7f76b40d0be0 msgr2=0x7f76b40d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:16.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.290+0000 7f76b8b9d640 1 -- 172.21.15.73:0/79963327 shutdown_connections 2024-04-26T20:53:16.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.290+0000 7f76b8b9d640 1 -- 172.21.15.73:0/79963327 wait complete. 2024-04-26T20:53:16.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.291+0000 7f76b8b9d640 1 Processor -- start 2024-04-26T20:53:16.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.291+0000 7f76b8b9d640 1 -- start start 2024-04-26T20:53:16.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.292+0000 7f76b8b9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76b406fea0 0x7f76b40ca770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:16.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.292+0000 7f76b8b9d640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f76b4072700 0x7f76b4148bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:16.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.292+0000 7f76b8b9d640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f76b415f2d0 0x7f76b40c4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:16.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.292+0000 7f76b8b9d640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f76b415f6b0 con 0x7f76b406fea0 2024-04-26T20:53:16.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.292+0000 7f76b8b9d640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f76b4073af0 con 0x7f76b4072700 2024-04-26T20:53:16.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.292+0000 7f76b8b9d640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f76b4072b90 con 0x7f76b415f2d0 2024-04-26T20:53:16.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.292+0000 7f76b259c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76b406fea0 0x7f76b40ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:16.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.292+0000 7f76b259c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76b406fea0 0x7f76b40ca770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:57108/0 (socket says 172.21.15.73:57108) 2024-04-26T20:53:16.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.292+0000 7f76b259c640 1 -- 172.21.15.73:0/2040992416 learned_addr learned my addr 172.21.15.73:0/2040992416 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:16.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.292+0000 7f76b259c640 1 -- 172.21.15.73:0/2040992416 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f76b415f2d0 msgr2=0x7f76b40c4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:16.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.292+0000 7f76b259c640 1 --2- 172.21.15.73:0/2040992416 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f76b415f2d0 0x7f76b40c4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:16.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.292+0000 7f76b259c640 1 -- 172.21.15.73:0/2040992416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f76b4072700 msgr2=0x7f76b4148bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:16.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.292+0000 7f76b259c640 1 --2- 172.21.15.73:0/2040992416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f76b4072700 0x7f76b4148bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:16.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.292+0000 7f76b259c640 1 -- 172.21.15.73:0/2040992416 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76b40c4330 con 0x7f76b406fea0 2024-04-26T20:53:16.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.293+0000 7f76b259c640 1 --2- 172.21.15.73:0/2040992416 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76b406fea0 0x7f76b40ca770 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f76a400d830 tx=0x7f76a400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:16.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.293+0000 7f76aaffd640 1 -- 172.21.15.73:0/2040992416 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a400b6c0 con 0x7f76b406fea0 2024-04-26T20:53:16.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.293+0000 7f76aaffd640 1 -- 172.21.15.73:0/2040992416 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76a4019070 con 0x7f76b406fea0 2024-04-26T20:53:16.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.293+0000 7f76b8b9d640 1 -- 172.21.15.73:0/2040992416 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76b40c45d0 con 0x7f76b406fea0 2024-04-26T20:53:16.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.293+0000 7f76aaffd640 1 -- 172.21.15.73:0/2040992416 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a400ba80 con 0x7f76b406fea0 2024-04-26T20:53:16.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.293+0000 7f76b8b9d640 1 -- 172.21.15.73:0/2040992416 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76b40c8a20 con 0x7f76b406fea0 2024-04-26T20:53:16.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.294+0000 7f76b8b9d640 1 -- 172.21.15.73:0/2040992416 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76b406f720 con 0x7f76b406fea0 2024-04-26T20:53:16.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.294+0000 7f76aaffd640 1 -- 172.21.15.73:0/2040992416 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f76a401e460 con 0x7f76b406fea0 2024-04-26T20:53:16.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.295+0000 7f76aaffd640 1 --2- 172.21.15.73:0/2040992416 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f769406c8f0 0x7f769406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:16.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.295+0000 7f76aaffd640 1 -- 172.21.15.73:0/2040992416 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5421+0+0 (secure 0 0 0) 0x7f76a408c100 con 0x7f76b406fea0 2024-04-26T20:53:16.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.295+0000 7f76abfff640 1 --2- 172.21.15.73:0/2040992416 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f769406c8f0 0x7f769406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:16.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.295+0000 7f76abfff640 1 --2- 172.21.15.73:0/2040992416 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f769406c8f0 0x7f769406edb0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f769c000c00 tx=0x7f769c0079c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:16.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.299+0000 7f76aaffd640 1 -- 172.21.15.73:0/2040992416 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f76b406f720 con 0x7f76b406fea0 2024-04-26T20:53:16.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.472+0000 7f76b8b9d640 1 -- 172.21.15.73:0/2040992416 --> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f76b41490f0 con 0x7f769406c8f0 2024-04-26T20:53:16.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.473+0000 7f76aaffd640 1 -- 172.21.15.73:0/2040992416 <== mgr.4107 v2:172.21.15.73:6832/666905549 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13360 (secure 0 0 0) 0x7f769c0070d0 con 0x7f769406c8f0 2024-04-26T20:53:16.474 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:53:16.474 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-26T20:53:16.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.475+0000 7f76b8b9d640 1 -- 172.21.15.73:0/2040992416 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f769406c8f0 msgr2=0x7f769406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:16.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.475+0000 7f76b8b9d640 1 --2- 172.21.15.73:0/2040992416 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f769406c8f0 0x7f769406edb0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f769c000c00 tx=0x7f769c0079c0 comp rx=0 tx=0).stop 2024-04-26T20:53:16.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.475+0000 7f76b8b9d640 1 -- 172.21.15.73:0/2040992416 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76b406fea0 msgr2=0x7f76b40ca770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:16.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.475+0000 7f76b8b9d640 1 --2- 172.21.15.73:0/2040992416 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76b406fea0 0x7f76b40ca770 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f76a400d830 tx=0x7f76a400dcf0 comp rx=0 tx=0).stop 2024-04-26T20:53:16.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.475+0000 7f76b8b9d640 1 -- 172.21.15.73:0/2040992416 shutdown_connections 2024-04-26T20:53:16.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.475+0000 7f76b8b9d640 1 --2- 172.21.15.73:0/2040992416 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f769406c8f0 0x7f769406edb0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:16.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.475+0000 7f76b8b9d640 1 --2- 172.21.15.73:0/2040992416 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f76b415f2d0 0x7f76b40c4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:16.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.475+0000 7f76b8b9d640 1 --2- 172.21.15.73:0/2040992416 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f76b4072700 0x7f76b4148bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:16.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.475+0000 7f76b8b9d640 1 --2- 172.21.15.73:0/2040992416 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76b406fea0 0x7f76b40ca770 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:16.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.475+0000 7f76b8b9d640 1 -- 172.21.15.73:0/2040992416 >> 172.21.15.73:0/2040992416 conn(0x7f76b40d0be0 msgr2=0x7f76b415a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:16.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.475+0000 7f76b8b9d640 1 -- 172.21.15.73:0/2040992416 shutdown_connections 2024-04-26T20:53:16.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:16.475+0000 7f76b8b9d640 1 -- 172.21.15.73:0/2040992416 wait complete. 2024-04-26T20:53:16.487 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":140,"stamp":"2024-04-26T20:53:16.015282+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":222904,"kb_used_data":6728,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754751816,"statfs":{"total":773094113280,"available":772865859584,"internally_reserved":0,"allocated":6889472,"data_stored":3756576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-42,"num_objects":-1,"num_object_clones":0,"num_object_copies":-2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1,"num_whiteouts":0,"num_read":-1,"num_read_kb":0,"num_write":-2,"num_write_kb":-2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.405340"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":241,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T20:53:09.346035+0000","last_change":"2024-04-26T20:50:21.150132+0000","last_active":"2024-04-26T20:53:09.346035+0000","last_peered":"2024-04-26T20:53:09.346035+0000","last_clean":"2024-04-26T20:53:09.346035+0000","last_became_active":"2024-04-26T20:50:21.147439+0000","last_became_peered":"2024-04-26T20:50:21.147439+0000","last_unstale":"2024-04-26T20:53:09.346035+0000","last_undegraded":"2024-04-26T20:53:09.346035+0000","last_fullsized":"2024-04-26T20:53:09.346035+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:50:20.084499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:50:20.084499+0000","last_clean_scrub_stamp":"2024-04-26T20:50:20.084499+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:19:29.395477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574886,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28300,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343540,"statfs":{"total":96636764160,"available":96607784960,"internally_reserved":0,"allocated":1306624,"data_stored":912348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27716,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":712704,"data_stored":321980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27716,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":712704,"data_stored":321980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27724,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344116,"statfs":{"total":96636764160,"available":96608374784,"internally_reserved":0,"allocated":712704,"data_stored":321980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27716,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":712704,"data_stored":321980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27716,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":712704,"data_stored":321980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27716,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":712704,"data_stored":321980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574886,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28300,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343540,"statfs":{"total":96636764160,"available":96607784960,"internally_reserved":0,"allocated":1306624,"data_stored":912348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:53:16.490 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-26T20:50:20.084499+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=20, tm_min=50, tm_sec=20, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=20, tm_min=53, tm_sec=11, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T20:53:16.490 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-26T20:53:22.498 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:53:22.501 DEBUG:teuthology.orchestra.run.smithi120:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:53:36.491 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:53:36.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.598+0000 7fc1a84b8640 1 Processor -- start 2024-04-26T20:53:36.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.598+0000 7fc1a84b8640 1 -- start start 2024-04-26T20:53:36.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.599+0000 7fc1a84b8640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc1a00cfb50 0x7fc1a006c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:36.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.599+0000 7fc1a84b8640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7fc1a00b66e0 0x7fc1a006c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:36.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.599+0000 7fc1a84b8640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1a015d490 con 0x7fc1a0157660 2024-04-26T20:53:36.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.599+0000 7fc1a84b8640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7fc1a015e080 con 0x7fc1a00cfb50 2024-04-26T20:53:36.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.599+0000 7fc1a84b8640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7fc1a015d750 con 0x7fc1a00b66e0 2024-04-26T20:53:36.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.599+0000 7fc1a5a53640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc1a00cfb50 0x7fc1a006c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:36.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.599+0000 7fc1a5a53640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc1a00cfb50 0x7fc1a006c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40554/0 (socket says 172.21.15.73:40554) 2024-04-26T20:53:36.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.599+0000 7fc1a5a53640 1 -- 172.21.15.73:0/4251899738 learned_addr learned my addr 172.21.15.73:0/4251899738 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:36.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.599+0000 7fc1a6a55640 1 --2- 172.21.15.73:0/4251899738 >> v2:172.21.15.120:3300/0 conn(0x7fc1a00b66e0 0x7fc1a006c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:36.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.599+0000 7fc1a5a53640 1 -- 172.21.15.73:0/4251899738 >> v2:172.21.15.120:3300/0 conn(0x7fc1a00b66e0 msgr2=0x7fc1a006c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:36.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.599+0000 7fc1a5a53640 1 --2- 172.21.15.73:0/4251899738 >> v2:172.21.15.120:3300/0 conn(0x7fc1a00b66e0 0x7fc1a006c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:36.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.599+0000 7fc1a5a53640 1 -- 172.21.15.73:0/4251899738 >> v1:172.21.15.120:6789/0 conn(0x7fc1a0157660 legacy=0x7fc1a00b5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:36.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.599+0000 7fc1a5a53640 1 -- 172.21.15.73:0/4251899738 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1a015da30 con 0x7fc1a00cfb50 2024-04-26T20:53:36.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.600+0000 7fc1a5a53640 1 --2- 172.21.15.73:0/4251899738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1a00cfb50 0x7fc1a006c240 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fc1a015dff0 tx=0x7fc19002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b3d54dba2282db9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:36.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.600+0000 7fc1a5252640 1 -- 172.21.15.73:0/4251899738 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc190037070 con 0x7fc1a00cfb50 2024-04-26T20:53:36.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.600+0000 7fc1a5252640 1 -- 172.21.15.73:0/4251899738 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc190032440 con 0x7fc1a00cfb50 2024-04-26T20:53:36.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.600+0000 7fc1a5252640 1 -- 172.21.15.73:0/4251899738 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc190032810 con 0x7fc1a00cfb50 2024-04-26T20:53:36.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.601+0000 7fc1a84b8640 1 -- 172.21.15.73:0/4251899738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1a00cfb50 msgr2=0x7fc1a006c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:36.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.601+0000 7fc1a84b8640 1 --2- 172.21.15.73:0/4251899738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1a00cfb50 0x7fc1a006c240 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fc1a015dff0 tx=0x7fc19002a270 comp rx=0 tx=0).stop 2024-04-26T20:53:36.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.601+0000 7fc1a84b8640 1 -- 172.21.15.73:0/4251899738 shutdown_connections 2024-04-26T20:53:36.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.601+0000 7fc1a84b8640 1 --2- 172.21.15.73:0/4251899738 >> v2:172.21.15.120:3300/0 conn(0x7fc1a00b66e0 0x7fc1a006c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:36.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.601+0000 7fc1a84b8640 1 --2- 172.21.15.73:0/4251899738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1a00cfb50 0x7fc1a006c240 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:36.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.601+0000 7fc1a84b8640 1 -- 172.21.15.73:0/4251899738 >> 172.21.15.73:0/4251899738 conn(0x7fc1a00b3180 msgr2=0x7fc1a00b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:36.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.601+0000 7fc1a84b8640 1 -- 172.21.15.73:0/4251899738 shutdown_connections 2024-04-26T20:53:36.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.601+0000 7fc1a84b8640 1 -- 172.21.15.73:0/4251899738 wait complete. 2024-04-26T20:53:36.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.602+0000 7fc1a84b8640 1 Processor -- start 2024-04-26T20:53:36.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.602+0000 7fc1a84b8640 1 -- start start 2024-04-26T20:53:36.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.603+0000 7fc1a84b8640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1a00b66e0 0x7fc1a013cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:36.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.603+0000 7fc1a84b8640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1a0157660 0x7fc1a013af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:36.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.603+0000 7fc1a84b8640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc1a013b4d0 0x7fc1a014db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:36.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.603+0000 7fc1a84b8640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a0072860 con 0x7fc1a00b66e0 2024-04-26T20:53:36.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.603+0000 7fc1a84b8640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a00726e0 con 0x7fc1a0157660 2024-04-26T20:53:36.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.603+0000 7fc1a84b8640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7fc1a00729e0 con 0x7fc1a013b4d0 2024-04-26T20:53:36.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.603+0000 7fc1a6254640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1a00b66e0 0x7fc1a013cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:36.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.603+0000 7fc1a6254640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1a00b66e0 0x7fc1a013cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40564/0 (socket says 172.21.15.73:40564) 2024-04-26T20:53:36.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.603+0000 7fc1a6254640 1 -- 172.21.15.73:0/870167891 learned_addr learned my addr 172.21.15.73:0/870167891 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:36.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.603+0000 7fc1a5a53640 1 --2- 172.21.15.73:0/870167891 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1a0157660 0x7fc1a013af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:36.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.603+0000 7fc1a6a55640 1 --2- 172.21.15.73:0/870167891 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc1a013b4d0 0x7fc1a014db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:36.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.603+0000 7fc1a6254640 1 -- 172.21.15.73:0/870167891 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc1a013b4d0 msgr2=0x7fc1a014db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:36.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.603+0000 7fc1a6254640 1 --2- 172.21.15.73:0/870167891 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc1a013b4d0 0x7fc1a014db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:36.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.603+0000 7fc1a6254640 1 -- 172.21.15.73:0/870167891 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1a0157660 msgr2=0x7fc1a013af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:36.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.603+0000 7fc1a6254640 1 --2- 172.21.15.73:0/870167891 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1a0157660 0x7fc1a013af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:36.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.603+0000 7fc1a6254640 1 -- 172.21.15.73:0/870167891 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1a013d510 con 0x7fc1a00b66e0 2024-04-26T20:53:36.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.603+0000 7fc1a6254640 1 --2- 172.21.15.73:0/870167891 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1a00b66e0 0x7fc1a013cfd0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fc19c00c3c0 tx=0x7fc19c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:36.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.604+0000 7fc196ffd640 1 -- 172.21.15.73:0/870167891 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19c0052d0 con 0x7fc1a00b66e0 2024-04-26T20:53:36.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.604+0000 7fc196ffd640 1 -- 172.21.15.73:0/870167891 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc19c005490 con 0x7fc1a00b66e0 2024-04-26T20:53:36.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.604+0000 7fc1a84b8640 1 -- 172.21.15.73:0/870167891 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1a014ca80 con 0x7fc1a00b66e0 2024-04-26T20:53:36.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.604+0000 7fc196ffd640 1 -- 172.21.15.73:0/870167891 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19c007260 con 0x7fc1a00b66e0 2024-04-26T20:53:36.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.604+0000 7fc1a84b8640 1 -- 172.21.15.73:0/870167891 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1a014d040 con 0x7fc1a00b66e0 2024-04-26T20:53:36.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.605+0000 7fc1a84b8640 1 -- 172.21.15.73:0/870167891 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1a0068fc0 con 0x7fc1a00b66e0 2024-04-26T20:53:36.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.605+0000 7fc196ffd640 1 -- 172.21.15.73:0/870167891 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc19c007400 con 0x7fc1a00b66e0 2024-04-26T20:53:36.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.606+0000 7fc196ffd640 1 --2- 172.21.15.73:0/870167891 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc18406c8f0 0x7fc18406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:36.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.606+0000 7fc1a5a53640 1 --2- 172.21.15.73:0/870167891 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc18406c8f0 0x7fc18406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:36.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.606+0000 7fc196ffd640 1 -- 172.21.15.73:0/870167891 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5421+0+0 (secure 0 0 0) 0x7fc19c08af30 con 0x7fc1a00b66e0 2024-04-26T20:53:36.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.606+0000 7fc1a5a53640 1 --2- 172.21.15.73:0/870167891 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc18406c8f0 0x7fc18406edb0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc190034000 tx=0x7fc19002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:36.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.610+0000 7fc196ffd640 1 -- 172.21.15.73:0/870167891 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc19c059480 con 0x7fc1a00b66e0 2024-04-26T20:53:36.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.781+0000 7fc1a84b8640 1 -- 172.21.15.73:0/870167891 --> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc1a013b8e0 con 0x7fc18406c8f0 2024-04-26T20:53:36.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.782+0000 7fc196ffd640 1 -- 172.21.15.73:0/870167891 <== mgr.4107 v2:172.21.15.73:6832/666905549 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13359 (secure 0 0 0) 0x7fc1a013b8e0 con 0x7fc18406c8f0 2024-04-26T20:53:36.783 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:53:36.783 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-26T20:53:36.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.784+0000 7fc1a84b8640 1 -- 172.21.15.73:0/870167891 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc18406c8f0 msgr2=0x7fc18406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:36.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.784+0000 7fc1a84b8640 1 --2- 172.21.15.73:0/870167891 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc18406c8f0 0x7fc18406edb0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc190034000 tx=0x7fc19002e000 comp rx=0 tx=0).stop 2024-04-26T20:53:36.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.784+0000 7fc1a84b8640 1 -- 172.21.15.73:0/870167891 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1a00b66e0 msgr2=0x7fc1a013cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:36.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.784+0000 7fc1a84b8640 1 --2- 172.21.15.73:0/870167891 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1a00b66e0 0x7fc1a013cfd0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fc19c00c3c0 tx=0x7fc19c00c880 comp rx=0 tx=0).stop 2024-04-26T20:53:36.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.784+0000 7fc1a84b8640 1 -- 172.21.15.73:0/870167891 shutdown_connections 2024-04-26T20:53:36.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.784+0000 7fc1a84b8640 1 --2- 172.21.15.73:0/870167891 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7fc18406c8f0 0x7fc18406edb0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:36.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.784+0000 7fc1a84b8640 1 --2- 172.21.15.73:0/870167891 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7fc1a013b4d0 0x7fc1a014db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:36.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.784+0000 7fc1a84b8640 1 --2- 172.21.15.73:0/870167891 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7fc1a0157660 0x7fc1a013af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:36.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.784+0000 7fc1a84b8640 1 --2- 172.21.15.73:0/870167891 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1a00b66e0 0x7fc1a013cfd0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:36.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.784+0000 7fc1a84b8640 1 -- 172.21.15.73:0/870167891 >> 172.21.15.73:0/870167891 conn(0x7fc1a00b3180 msgr2=0x7fc1a00b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:36.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.784+0000 7fc1a84b8640 1 -- 172.21.15.73:0/870167891 shutdown_connections 2024-04-26T20:53:36.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.784+0000 7fc1a84b8640 1 -- 172.21.15.73:0/870167891 wait complete. 2024-04-26T20:53:36.795 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":150,"stamp":"2024-04-26T20:53:36.019428+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":222888,"kb_used_data":6728,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754751832,"statfs":{"total":773094113280,"available":772865875968,"internally_reserved":0,"allocated":6889472,"data_stored":3756576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002804"},"pg_stats":[{"pgid":"1.0","version":"59'119","reported_seq":252,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T20:53:16.868325+0000","last_change":"2024-04-26T20:53:16.868370+0000","last_active":"2024-04-26T20:53:16.868325+0000","last_peered":"2024-04-26T20:53:16.868325+0000","last_clean":"2024-04-26T20:53:16.868325+0000","last_became_active":"2024-04-26T20:50:21.147439+0000","last_became_peered":"2024-04-26T20:50:21.147439+0000","last_unstale":"2024-04-26T20:53:16.868325+0000","last_undegraded":"2024-04-26T20:53:16.868325+0000","last_fullsized":"2024-04-26T20:53:16.868325+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'119","last_scrub_stamp":"2024-04-26T20:53:16.868236+0000","last_deep_scrub":"59'119","last_deep_scrub_stamp":"2024-04-26T20:53:16.868236+0000","last_clean_scrub_stamp":"2024-04-26T20:53:16.868236+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:41:06.434626+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574890,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28292,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343548,"statfs":{"total":96636764160,"available":96607793152,"internally_reserved":0,"allocated":1306624,"data_stored":912348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574891,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27716,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":712704,"data_stored":321980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574891,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27716,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":712704,"data_stored":321980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574891,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27724,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344116,"statfs":{"total":96636764160,"available":96608374784,"internally_reserved":0,"allocated":712704,"data_stored":321980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574891,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27716,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":712704,"data_stored":321980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574890,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27716,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":712704,"data_stored":321980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574890,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27716,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":712704,"data_stored":321980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574890,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28292,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343548,"statfs":{"total":96636764160,"available":96607793152,"internally_reserved":0,"allocated":1306624,"data_stored":912348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:53:36.796 DEBUG:teuthology.orchestra.run.smithi073:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-26T20:53:36.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.892+0000 7f84ae32b640 1 Processor -- start 2024-04-26T20:53:36.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.892+0000 7f84ae32b640 1 -- start start 2024-04-26T20:53:36.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.893+0000 7f84ae32b640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f84a80b0380 0x7f84a80ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:36.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.893+0000 7f84ae32b640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f84a80b10b0 0x7f84a806c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:36.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.893+0000 7f84ae32b640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f84a806c780 0x7f84a8072f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:36.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.893+0000 7f84ae32b640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 -- 0x7f84a80cfd80 con 0x7f84a806c780 2024-04-26T20:53:36.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.893+0000 7f84ae32b640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f84a80d0020 con 0x7f84a80b0380 2024-04-26T20:53:36.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.893+0000 7f84ae32b640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f84a80d0300 con 0x7f84a80b10b0 2024-04-26T20:53:36.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.893+0000 7f84ac8c8640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f84a806c780 0x7f84a8072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:36.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.893+0000 7f84ac8c8640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f84a806c780 0x7f84a8072f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40576/0 (socket says 172.21.15.73:40576) 2024-04-26T20:53:36.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.893+0000 7f84ac8c8640 1 -- 172.21.15.73:0/1379735429 learned_addr learned my addr 172.21.15.73:0/1379735429 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:36.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.893+0000 7f84a7fff640 1 --2- 172.21.15.73:0/1379735429 >> v2:172.21.15.120:3300/0 conn(0x7f84a80b0380 0x7f84a80ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:36.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.893+0000 7f84a77fe640 1 --2- 172.21.15.73:0/1379735429 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f84a80b10b0 0x7f84a806c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:36.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.893+0000 7f84ac8c8640 1 -- 172.21.15.73:0/1379735429 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f84a80b10b0 msgr2=0x7f84a806c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:36.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.893+0000 7f84ac8c8640 1 --2- 172.21.15.73:0/1379735429 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f84a80b10b0 0x7f84a806c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:36.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.893+0000 7f84ac8c8640 1 -- 172.21.15.73:0/1379735429 >> v2:172.21.15.120:3300/0 conn(0x7f84a80b0380 msgr2=0x7f84a80ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:36.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.893+0000 7f84ac8c8640 1 --2- 172.21.15.73:0/1379735429 >> v2:172.21.15.120:3300/0 conn(0x7f84a80b0380 0x7f84a80ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:36.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.893+0000 7f84ac8c8640 1 -- 172.21.15.73:0/1379735429 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84a80d05e0 con 0x7f84a806c780 2024-04-26T20:53:36.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.894+0000 7f84ac8c8640 1 --2- 172.21.15.73:0/1379735429 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84a806c780 0x7f84a8072f70 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f84a806d290 tx=0x7f84a002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38fa8538aabb53 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:36.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.894+0000 7f84a6ffd640 1 -- 172.21.15.73:0/1379735429 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a0036070 con 0x7f84a806c780 2024-04-26T20:53:36.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.894+0000 7f84a6ffd640 1 -- 172.21.15.73:0/1379735429 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84a0031600 con 0x7f84a806c780 2024-04-26T20:53:36.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.894+0000 7f84a6ffd640 1 -- 172.21.15.73:0/1379735429 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a00319d0 con 0x7f84a806c780 2024-04-26T20:53:36.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.894+0000 7f84ae32b640 1 -- 172.21.15.73:0/1379735429 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84a806c780 msgr2=0x7f84a8072f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:36.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.894+0000 7f84ae32b640 1 --2- 172.21.15.73:0/1379735429 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84a806c780 0x7f84a8072f70 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f84a806d290 tx=0x7f84a002a610 comp rx=0 tx=0).stop 2024-04-26T20:53:36.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.894+0000 7f84ae32b640 1 -- 172.21.15.73:0/1379735429 shutdown_connections 2024-04-26T20:53:36.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.894+0000 7f84ae32b640 1 --2- 172.21.15.73:0/1379735429 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84a806c780 0x7f84a8072f70 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:36.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.894+0000 7f84ae32b640 1 --2- 172.21.15.73:0/1379735429 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f84a80b10b0 0x7f84a806c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:36.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.894+0000 7f84ae32b640 1 --2- 172.21.15.73:0/1379735429 >> v2:172.21.15.120:3300/0 conn(0x7f84a80b0380 0x7f84a80ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:36.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.894+0000 7f84ae32b640 1 -- 172.21.15.73:0/1379735429 >> 172.21.15.73:0/1379735429 conn(0x7f84a80d1cf0 msgr2=0x7f84a80d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:36.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.894+0000 7f84ae32b640 1 -- 172.21.15.73:0/1379735429 shutdown_connections 2024-04-26T20:53:36.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.894+0000 7f84ae32b640 1 -- 172.21.15.73:0/1379735429 wait complete. 2024-04-26T20:53:36.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.896+0000 7f84ae32b640 1 Processor -- start 2024-04-26T20:53:36.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.896+0000 7f84ae32b640 1 -- start start 2024-04-26T20:53:36.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.896+0000 7f84ae32b640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f84a806c780 0x7f84a80b6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:36.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.896+0000 7f84ae32b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84a80b0380 0x7f84a81610c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:36.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.896+0000 7f84ae32b640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f84a80b10b0 0x7f84a80b6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:36.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.896+0000 7f84ae32b640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f84a80d15c0 con 0x7f84a80b0380 2024-04-26T20:53:36.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.896+0000 7f84ae32b640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f84a80b0760 con 0x7f84a80b10b0 2024-04-26T20:53:36.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.896+0000 7f84ae32b640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f84a80d1740 con 0x7f84a806c780 2024-04-26T20:53:36.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.897+0000 7f84a77fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84a80b0380 0x7f84a81610c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:36.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.897+0000 7f84a77fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84a80b0380 0x7f84a81610c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40590/0 (socket says 172.21.15.73:40590) 2024-04-26T20:53:36.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.897+0000 7f84a77fe640 1 -- 172.21.15.73:0/2376472890 learned_addr learned my addr 172.21.15.73:0/2376472890 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:36.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.897+0000 7f84a77fe640 1 -- 172.21.15.73:0/2376472890 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f84a806c780 msgr2=0x7f84a80b6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:36.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.897+0000 7f84a77fe640 1 --2- 172.21.15.73:0/2376472890 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f84a806c780 0x7f84a80b6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:36.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.897+0000 7f84a77fe640 1 -- 172.21.15.73:0/2376472890 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f84a80b10b0 msgr2=0x7f84a80b6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:36.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.897+0000 7f84a77fe640 1 --2- 172.21.15.73:0/2376472890 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f84a80b10b0 0x7f84a80b6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:36.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.897+0000 7f84a77fe640 1 -- 172.21.15.73:0/2376472890 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84a80c84d0 con 0x7f84a80b0380 2024-04-26T20:53:36.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.897+0000 7f84a77fe640 1 --2- 172.21.15.73:0/2376472890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84a80b0380 0x7f84a81610c0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f849c00d5f0 tx=0x7f849c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:36.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.897+0000 7f84a4ff9640 1 -- 172.21.15.73:0/2376472890 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849c0125f0 con 0x7f84a80b0380 2024-04-26T20:53:36.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.897+0000 7f84a4ff9640 1 -- 172.21.15.73:0/2376472890 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f849c019070 con 0x7f84a80b0380 2024-04-26T20:53:36.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.897+0000 7f84a4ff9640 1 -- 172.21.15.73:0/2376472890 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849c0129d0 con 0x7f84a80b0380 2024-04-26T20:53:36.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.897+0000 7f84ae32b640 1 -- 172.21.15.73:0/2376472890 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84a80c5a40 con 0x7f84a80b0380 2024-04-26T20:53:36.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.897+0000 7f84ae32b640 1 -- 172.21.15.73:0/2376472890 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84a80c6000 con 0x7f84a80b0380 2024-04-26T20:53:36.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.898+0000 7f84ae32b640 1 -- 172.21.15.73:0/2376472890 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84a80c5850 con 0x7f84a80b0380 2024-04-26T20:53:36.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.898+0000 7f84a4ff9640 1 -- 172.21.15.73:0/2376472890 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f849c012b70 con 0x7f84a80b0380 2024-04-26T20:53:36.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.899+0000 7f84a4ff9640 1 --2- 172.21.15.73:0/2376472890 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f849406c8f0 0x7f849406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:36.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.899+0000 7f84a4ff9640 1 -- 172.21.15.73:0/2376472890 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5421+0+0 (secure 0 0 0) 0x7f849c08be80 con 0x7f84a80b0380 2024-04-26T20:53:36.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.899+0000 7f84a7fff640 1 --2- 172.21.15.73:0/2376472890 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f849406c8f0 0x7f849406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:36.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.899+0000 7f84a7fff640 1 --2- 172.21.15.73:0/2376472890 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f849406c8f0 0x7f849406edb0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f8498009870 tx=0x7f8498008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:36.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:36.903+0000 7f84a4ff9640 1 -- 172.21.15.73:0/2376472890 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f849c05a400 con 0x7f84a80b0380 2024-04-26T20:53:37.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.081+0000 7f84ae32b640 1 -- 172.21.15.73:0/2376472890 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f84a80694f0 con 0x7f84a80b0380 2024-04-26T20:53:37.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.085+0000 7f84a4ff9640 1 -- 172.21.15.73:0/2376472890 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v14)=0 v14) ==== 123+0+0 (secure 0 0 0) 0x7f849c059d00 con 0x7f84a80b0380 2024-04-26T20:53:37.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.085+0000 7f84a4ff9640 1 -- 172.21.15.73:0/2376472890 <== mon.0 v2:172.21.15.73:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f849c05e0b0 con 0x7f84a80b0380 2024-04-26T20:53:37.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.087+0000 7f84ae32b640 1 -- 172.21.15.73:0/2376472890 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f849406c8f0 msgr2=0x7f849406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:37.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.087+0000 7f84ae32b640 1 --2- 172.21.15.73:0/2376472890 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f849406c8f0 0x7f849406edb0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f8498009870 tx=0x7f8498008040 comp rx=0 tx=0).stop 2024-04-26T20:53:37.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.087+0000 7f84ae32b640 1 -- 172.21.15.73:0/2376472890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84a80b0380 msgr2=0x7f84a81610c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:37.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.087+0000 7f84ae32b640 1 --2- 172.21.15.73:0/2376472890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84a80b0380 0x7f84a81610c0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f849c00d5f0 tx=0x7f849c00dab0 comp rx=0 tx=0).stop 2024-04-26T20:53:37.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.087+0000 7f84ae32b640 1 -- 172.21.15.73:0/2376472890 shutdown_connections 2024-04-26T20:53:37.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.087+0000 7f84ae32b640 1 --2- 172.21.15.73:0/2376472890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84a80b0380 0x7f84a81610c0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:37.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.087+0000 7f84ae32b640 1 --2- 172.21.15.73:0/2376472890 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f849406c8f0 0x7f849406edb0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:37.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.087+0000 7f84ae32b640 1 --2- 172.21.15.73:0/2376472890 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f84a80b10b0 0x7f84a80b6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:37.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.087+0000 7f84ae32b640 1 --2- 172.21.15.73:0/2376472890 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f84a806c780 0x7f84a80b6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:37.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.087+0000 7f84ae32b640 1 -- 172.21.15.73:0/2376472890 >> 172.21.15.73:0/2376472890 conn(0x7f84a80d1cf0 msgr2=0x7f84a80cc4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:37.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.088+0000 7f84ae32b640 1 -- 172.21.15.73:0/2376472890 shutdown_connections 2024-04-26T20:53:37.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.088+0000 7f84ae32b640 1 -- 172.21.15.73:0/2376472890 wait complete. 2024-04-26T20:53:37.100 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T20:53:37.100 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:53:37.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.196+0000 7f5850146640 1 Processor -- start 2024-04-26T20:53:37.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.196+0000 7f5850146640 1 -- start start 2024-04-26T20:53:37.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.197+0000 7f5850146640 1 --2- >> v2:172.21.15.120:3300/0 conn(0x7f58480b4790 0x7f58480b4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:37.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.197+0000 7f5850146640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f584815dd30 con 0x7f584815a9f0 2024-04-26T20:53:37.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.197+0000 7f5850146640 1 -- --> v1:172.21.15.120:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f584815e010 con 0x7f58480ce370 2024-04-26T20:53:37.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.197+0000 7f5850146640 1 -- --> v2:172.21.15.120:3300/0 -- mon_getmap magic: 0 -- 0x7f58480726e0 con 0x7f58480b4790 2024-04-26T20:53:37.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.197+0000 7f584dee2640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f584815a9f0 0x7f584815d4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:52476/0 (socket says 172.21.15.73:52476) 2024-04-26T20:53:37.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.197+0000 7f584dee2640 1 -- 172.21.15.73:0/694939531 learned_addr learned my addr 172.21.15.73:0/694939531 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-04-26T20:53:37.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.197+0000 7f584cee0640 1 -- 172.21.15.73:0/694939531 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3445750402 0 0) 0x7f584815dd30 con 0x7f584815a9f0 2024-04-26T20:53:37.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.197+0000 7f584cee0640 1 -- 172.21.15.73:0/694939531 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5834003520 con 0x7f584815a9f0 2024-04-26T20:53:37.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.197+0000 7f584e6e3640 1 --2- 172.21.15.73:0/694939531 >> v2:172.21.15.120:3300/0 conn(0x7f58480b4790 0x7f58480b4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:37.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.197+0000 7f584cee0640 1 -- 172.21.15.73:0/694939531 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3603818721 0 0) 0x7f5834003520 con 0x7f584815a9f0 2024-04-26T20:53:37.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.197+0000 7f584cee0640 1 -- 172.21.15.73:0/694939531 >> v2:172.21.15.120:3300/0 conn(0x7f58480b4790 msgr2=0x7f58480b4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:37.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.197+0000 7f584cee0640 1 --2- 172.21.15.73:0/694939531 >> v2:172.21.15.120:3300/0 conn(0x7f58480b4790 0x7f58480b4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:37.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.197+0000 7f584cee0640 1 -- 172.21.15.73:0/694939531 >> v1:172.21.15.120:6789/0 conn(0x7f58480ce370 legacy=0x7f5848072e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:37.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.197+0000 7f584cee0640 1 -- 172.21.15.73:0/694939531 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f584815e2f0 con 0x7f584815a9f0 2024-04-26T20:53:37.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.197+0000 7f584cee0640 1 -- 172.21.15.73:0/694939531 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 14621606 0 0) 0x7f5844001670 con 0x7f584815a9f0 2024-04-26T20:53:37.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.197+0000 7f584cee0640 1 -- 172.21.15.73:0/694939531 >> v1:172.21.15.73:6789/0 conn(0x7f584815a9f0 legacy=0x7f584815d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:37.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.197+0000 7f584cee0640 1 --2- 172.21.15.73:0/694939531 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f58340039b0 0x7f5834003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:37.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.197+0000 7f584cee0640 1 --2- 172.21.15.73:0/694939531 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5834004800 0x7f5834004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:37.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.197+0000 7f584cee0640 1 --2- 172.21.15.73:0/694939531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58340051c0 0x7f583400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:37.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.197+0000 7f584cee0640 1 -- 172.21.15.73:0/694939531 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f58480726e0 con 0x7f58340051c0 2024-04-26T20:53:37.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.197+0000 7f584cee0640 1 -- 172.21.15.73:0/694939531 --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f583400c060 con 0x7f5834004800 2024-04-26T20:53:37.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.197+0000 7f584cee0640 1 -- 172.21.15.73:0/694939531 --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f583400c260 con 0x7f58340039b0 2024-04-26T20:53:37.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.197+0000 7f584e6e3640 1 --2- 172.21.15.73:0/694939531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58340051c0 0x7f583400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:37.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.198+0000 7f584dee2640 1 --2- 172.21.15.73:0/694939531 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f58340039b0 0x7f5834003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:37.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.198+0000 7f584d6e1640 1 --2- 172.21.15.73:0/694939531 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5834004800 0x7f5834004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:37.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.198+0000 7f584e6e3640 1 -- 172.21.15.73:0/694939531 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f58340039b0 msgr2=0x7f5834003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:37.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.198+0000 7f584e6e3640 1 --2- 172.21.15.73:0/694939531 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f58340039b0 0x7f5834003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:37.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.198+0000 7f584e6e3640 1 -- 172.21.15.73:0/694939531 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5834004800 msgr2=0x7f5834004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:53:37.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.198+0000 7f584e6e3640 1 --2- 172.21.15.73:0/694939531 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f5834004800 0x7f5834004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:37.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.198+0000 7f584e6e3640 1 -- 172.21.15.73:0/694939531 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5834003520 con 0x7f58340051c0 2024-04-26T20:53:37.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.198+0000 7f584e6e3640 1 -- 172.21.15.73:0/694939531 reap_dead start 2024-04-26T20:53:37.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.198+0000 7f584e6e3640 1 --2- 172.21.15.73:0/694939531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58340051c0 0x7f583400b9c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f5840009fc0 tx=0x7f5834005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a220e50d32ba9040 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:37.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.199+0000 7f584cee0640 1 -- 172.21.15.73:0/694939531 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5840035020 con 0x7f58340051c0 2024-04-26T20:53:37.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.199+0000 7f584cee0640 1 -- 172.21.15.73:0/694939531 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f5840003bd0 con 0x7f58340051c0 2024-04-26T20:53:37.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.199+0000 7f584cee0640 1 -- 172.21.15.73:0/694939531 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5840009c10 con 0x7f58340051c0 2024-04-26T20:53:37.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.199+0000 7f5850146640 1 -- 172.21.15.73:0/694939531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58340051c0 msgr2=0x7f583400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:37.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.199+0000 7f5850146640 1 --2- 172.21.15.73:0/694939531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58340051c0 0x7f583400b9c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f5840009fc0 tx=0x7f5834005830 comp rx=0 tx=0).stop 2024-04-26T20:53:37.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.199+0000 7f5850146640 1 -- 172.21.15.73:0/694939531 shutdown_connections 2024-04-26T20:53:37.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.199+0000 7f5850146640 1 --2- 172.21.15.73:0/694939531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58340051c0 0x7f583400b9c0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:37.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.199+0000 7f5850146640 1 -- 172.21.15.73:0/694939531 >> 172.21.15.73:0/694939531 conn(0x7f58480b3180 msgr2=0x7f58480b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:37.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.199+0000 7f5850146640 1 -- 172.21.15.73:0/694939531 shutdown_connections 2024-04-26T20:53:37.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.199+0000 7f5850146640 1 -- 172.21.15.73:0/694939531 wait complete. 2024-04-26T20:53:37.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.200+0000 7f5850146640 1 Processor -- start 2024-04-26T20:53:37.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.200+0000 7f5850146640 1 -- start start 2024-04-26T20:53:37.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.201+0000 7f5850146640 1 --2- >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f58340051c0 0x7f5848141040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:37.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.201+0000 7f5850146640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5848144b70 0x7f58480b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:37.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.201+0000 7f5850146640 1 --2- >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58480b1b10 0x7f58481542c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:37.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.201+0000 7f5850146640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f584815af50 con 0x7f5848144b70 2024-04-26T20:53:37.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.201+0000 7f5850146640 1 -- --> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] -- mon_getmap magic: 0 -- 0x7f584815add0 con 0x7f58480b1b10 2024-04-26T20:53:37.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.201+0000 7f5850146640 1 -- --> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] -- mon_getmap magic: 0 -- 0x7f58480b4c20 con 0x7f58340051c0 2024-04-26T20:53:37.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.201+0000 7f584d6e1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5848144b70 0x7f58480b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:37.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.201+0000 7f584d6e1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5848144b70 0x7f58480b15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40614/0 (socket says 172.21.15.73:40614) 2024-04-26T20:53:37.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.201+0000 7f584d6e1640 1 -- 172.21.15.73:0/3540869331 learned_addr learned my addr 172.21.15.73:0/3540869331 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-04-26T20:53:37.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.201+0000 7f584e6e3640 1 --2- 172.21.15.73:0/3540869331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58480b1b10 0x7f58481542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:37.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.201+0000 7f584dee2640 1 --2- 172.21.15.73:0/3540869331 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f58340051c0 0x7f5848141040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:37.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.201+0000 7f584d6e1640 1 -- 172.21.15.73:0/3540869331 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f58340051c0 msgr2=0x7f5848141040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:37.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.201+0000 7f584d6e1640 1 --2- 172.21.15.73:0/3540869331 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f58340051c0 0x7f5848141040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:37.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.201+0000 7f584d6e1640 1 -- 172.21.15.73:0/3540869331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58480b1b10 msgr2=0x7f58481542c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:37.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.201+0000 7f584d6e1640 1 --2- 172.21.15.73:0/3540869331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58480b1b10 0x7f58481542c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:37.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.201+0000 7f584d6e1640 1 -- 172.21.15.73:0/3540869331 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58481562e0 con 0x7f5848144b70 2024-04-26T20:53:37.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.202+0000 7f584dee2640 1 --2- 172.21.15.73:0/3540869331 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f58340051c0 0x7f5848141040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:53:37.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.202+0000 7f584e6e3640 1 --2- 172.21.15.73:0/3540869331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58480b1b10 0x7f58481542c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:53:37.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.202+0000 7f584d6e1640 1 --2- 172.21.15.73:0/3540869331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5848144b70 0x7f58480b15d0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f583800f5f0 tx=0x7f583800fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:37.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.202+0000 7f583e7fc640 1 -- 172.21.15.73:0/3540869331 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58380052e0 con 0x7f5848144b70 2024-04-26T20:53:37.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.202+0000 7f583e7fc640 1 -- 172.21.15.73:0/3540869331 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f5838018070 con 0x7f5848144b70 2024-04-26T20:53:37.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.202+0000 7f5850146640 1 -- 172.21.15.73:0/3540869331 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58481565c0 con 0x7f5848144b70 2024-04-26T20:53:37.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.202+0000 7f583e7fc640 1 -- 172.21.15.73:0/3540869331 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5838003830 con 0x7f5848144b70 2024-04-26T20:53:37.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.202+0000 7f5850146640 1 -- 172.21.15.73:0/3540869331 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5848156b80 con 0x7f5848144b70 2024-04-26T20:53:37.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.203+0000 7f5850146640 1 -- 172.21.15.73:0/3540869331 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5848068fc0 con 0x7f5848144b70 2024-04-26T20:53:37.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.203+0000 7f583e7fc640 1 -- 172.21.15.73:0/3540869331 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f58380039d0 con 0x7f5848144b70 2024-04-26T20:53:37.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.204+0000 7f583e7fc640 1 --2- 172.21.15.73:0/3540869331 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f582c06c920 0x7f582c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:53:37.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.204+0000 7f583e7fc640 1 -- 172.21.15.73:0/3540869331 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5421+0+0 (secure 0 0 0) 0x7f583808b8b0 con 0x7f5848144b70 2024-04-26T20:53:37.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.204+0000 7f584dee2640 1 --2- 172.21.15.73:0/3540869331 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f582c06c920 0x7f582c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:53:37.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.204+0000 7f584dee2640 1 --2- 172.21.15.73:0/3540869331 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f582c06c920 0x7f582c06ede0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f58440015d0 tx=0x7f5844009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:53:37.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.208+0000 7f583e7fc640 1 -- 172.21.15.73:0/3540869331 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5838059d70 con 0x7f5848144b70 2024-04-26T20:53:37.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.419+0000 7f5850146640 1 -- 172.21.15.73:0/3540869331 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58480b1ef0 con 0x7f5848144b70 2024-04-26T20:53:37.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.420+0000 7f583e7fc640 1 -- 172.21.15.73:0/3540869331 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+3869 (secure 0 0 0) 0x7f5838059670 con 0x7f5848144b70 2024-04-26T20:53:37.421 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-26T20:53:37.421 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":36,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5207,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6833/256948100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6832","nonce":256948100},{"type":"v1","addr":"172.21.15.120:6833","nonce":256948100}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5632,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.120:6835/4104096099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6834","nonce":4104096099},{"type":"v1","addr":"172.21.15.120:6835","nonce":4104096099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5641,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/272782804","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":272782804},{"type":"v1","addr":"172.21.15.73:6837","nonce":272782804}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":6264,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/3910535518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3910535518},{"type":"v1","addr":"172.21.15.73:6836","nonce":3910535518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36}],"filesystems":[]} 2024-04-26T20:53:37.421 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 36 2024-04-26T20:53:37.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.422+0000 7f5850146640 1 -- 172.21.15.73:0/3540869331 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f582c06c920 msgr2=0x7f582c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:37.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.422+0000 7f5850146640 1 --2- 172.21.15.73:0/3540869331 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f582c06c920 0x7f582c06ede0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f58440015d0 tx=0x7f5844009040 comp rx=0 tx=0).stop 2024-04-26T20:53:37.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.422+0000 7f5850146640 1 -- 172.21.15.73:0/3540869331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5848144b70 msgr2=0x7f58480b15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:53:37.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.422+0000 7f5850146640 1 --2- 172.21.15.73:0/3540869331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5848144b70 0x7f58480b15d0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f583800f5f0 tx=0x7f583800fab0 comp rx=0 tx=0).stop 2024-04-26T20:53:37.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.422+0000 7f5850146640 1 -- 172.21.15.73:0/3540869331 shutdown_connections 2024-04-26T20:53:37.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.422+0000 7f5850146640 1 --2- 172.21.15.73:0/3540869331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5848144b70 0x7f58480b15d0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:37.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.422+0000 7f5850146640 1 --2- 172.21.15.73:0/3540869331 >> [v2:172.21.15.73:6832/666905549,v1:172.21.15.73:6833/666905549] conn(0x7f582c06c920 0x7f582c06ede0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:37.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.422+0000 7f5850146640 1 --2- 172.21.15.73:0/3540869331 >> [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] conn(0x7f58480b1b10 0x7f58481542c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:37.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.422+0000 7f5850146640 1 --2- 172.21.15.73:0/3540869331 >> [v2:172.21.15.120:3301/0,v1:172.21.15.120:6790/0] conn(0x7f58340051c0 0x7f5848141040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:53:37.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.422+0000 7f5850146640 1 -- 172.21.15.73:0/3540869331 >> 172.21.15.73:0/3540869331 conn(0x7f58480b3180 msgr2=0x7f584815e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:53:37.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.422+0000 7f5850146640 1 -- 172.21.15.73:0/3540869331 shutdown_connections 2024-04-26T20:53:37.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-26T20:53:37.422+0000 7f5850146640 1 -- 172.21.15.73:0/3540869331 wait complete. 2024-04-26T20:53:37.433 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-26T20:53:37.434 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-26T20:53:37.434 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T20:53:37.434 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:53:37.443 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T20:53:37.443 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T20:53:37.443 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:53:37.492 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T20:53:37.492 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T20:53:37.492 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:53:37.504 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T20:53:37.504 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T20:53:37.504 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:53:37.556 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T20:53:37.556 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-26T20:53:37.556 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-26T20:53:37.557 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:53:37.585 INFO:tasks.ceph.osd.0:Stopped 2024-04-26T20:53:37.585 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-26T20:53:37.586 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:53:37.652 INFO:tasks.ceph.osd.1:Stopped 2024-04-26T20:53:37.652 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-26T20:53:37.652 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:53:37.723 INFO:tasks.ceph.osd.2:Stopped 2024-04-26T20:53:37.723 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-26T20:53:37.723 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:53:37.787 INFO:tasks.ceph.osd.3:Stopped 2024-04-26T20:53:37.787 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-26T20:53:37.788 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:53:37.829 INFO:tasks.ceph.osd.4:Stopped 2024-04-26T20:53:37.829 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-26T20:53:37.829 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:53:37.870 INFO:tasks.ceph.osd.5:Stopped 2024-04-26T20:53:37.870 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-26T20:53:37.870 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:53:37.937 INFO:tasks.ceph.osd.6:Stopped 2024-04-26T20:53:37.937 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-26T20:53:37.937 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:53:38.011 INFO:tasks.ceph.osd.7:Stopped 2024-04-26T20:53:38.011 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-26T20:53:38.012 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-26T20:53:38.012 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:53:38.037 INFO:tasks.ceph.mgr.y:Stopped 2024-04-26T20:53:38.037 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-26T20:53:38.037 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:53:38.080 INFO:tasks.ceph.mgr.x:Stopped 2024-04-26T20:53:38.080 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-26T20:53:38.080 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-26T20:53:38.080 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:53:38.091 INFO:tasks.ceph.mon.a:Stopped 2024-04-26T20:53:38.091 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-26T20:53:38.091 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:53:38.133 INFO:tasks.ceph.mon.b:Stopped 2024-04-26T20:53:38.133 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-26T20:53:38.133 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:53:38.185 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2024-04-26T20:53:38.185 INFO:tasks.ceph.mon.c:Stopped 2024-04-26T20:53:38.186 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-26T20:53:38.186 DEBUG:teuthology.orchestra.run.smithi073:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'force file system read-only' | egrep -v 'bad backtrace' | egrep -v 'MDS in read-only mode' | egrep -v '\(MDS_READ_ONLY\)' | head -n 1 2024-04-26T20:53:38.241 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi073.front.sepia.ceph.com 2024-04-26T20:53:38.242 DEBUG:teuthology.orchestra.run.smithi073:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-26T20:53:38.439 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi073.front.sepia.ceph.com 2024-04-26T20:53:38.440 DEBUG:teuthology.orchestra.run.smithi073:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-26T20:53:38.520 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi073.front.sepia.ceph.com 2024-04-26T20:53:38.520 DEBUG:teuthology.orchestra.run.smithi073:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-26T20:53:38.569 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi073.front.sepia.ceph.com 2024-04-26T20:53:38.569 DEBUG:teuthology.orchestra.run.smithi073:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-26T20:53:38.620 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi120.front.sepia.ceph.com 2024-04-26T20:53:38.620 DEBUG:teuthology.orchestra.run.smithi120:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-26T20:53:38.775 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi120.front.sepia.ceph.com 2024-04-26T20:53:38.776 DEBUG:teuthology.orchestra.run.smithi120:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-26T20:53:38.839 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi120.front.sepia.ceph.com 2024-04-26T20:53:38.839 DEBUG:teuthology.orchestra.run.smithi120:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-26T20:53:38.890 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi120.front.sepia.ceph.com 2024-04-26T20:53:38.890 DEBUG:teuthology.orchestra.run.smithi120:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-26T20:53:38.940 INFO:tasks.ceph:Archiving mon data... 2024-04-26T20:53:38.941 DEBUG:teuthology.misc:Transferring archived files from smithi073:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675187/data/mon.a.tgz 2024-04-26T20:53:38.941 DEBUG:teuthology.orchestra.run.smithi073:> mktemp 2024-04-26T20:53:38.957 INFO:teuthology.orchestra.run.smithi073.stdout:/tmp/tmp.Z5iMztLCg0 2024-04-26T20:53:38.957 DEBUG:teuthology.orchestra.run.smithi073:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.Z5iMztLCg0 2024-04-26T20:53:39.101 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 0666 /tmp/tmp.Z5iMztLCg0 2024-04-26T20:53:39.183 DEBUG:teuthology.orchestra.remote:smithi073:/tmp/tmp.Z5iMztLCg0 is 359KB 2024-04-26T20:53:39.283 DEBUG:teuthology.orchestra.run.smithi073:> rm -fr /tmp/tmp.Z5iMztLCg0 2024-04-26T20:53:39.298 DEBUG:teuthology.misc:Transferring archived files from smithi120:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675187/data/mon.b.tgz 2024-04-26T20:53:39.298 DEBUG:teuthology.orchestra.run.smithi120:> mktemp 2024-04-26T20:53:39.315 INFO:teuthology.orchestra.run.smithi120.stdout:/tmp/tmp.Jf3yXan6Sd 2024-04-26T20:53:39.315 DEBUG:teuthology.orchestra.run.smithi120:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.Jf3yXan6Sd 2024-04-26T20:53:39.508 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod 0666 /tmp/tmp.Jf3yXan6Sd 2024-04-26T20:53:39.590 DEBUG:teuthology.orchestra.remote:smithi120:/tmp/tmp.Jf3yXan6Sd is 473KB 2024-04-26T20:53:39.692 DEBUG:teuthology.orchestra.run.smithi120:> rm -fr /tmp/tmp.Jf3yXan6Sd 2024-04-26T20:53:39.707 DEBUG:teuthology.misc:Transferring archived files from smithi120:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675187/data/mon.c.tgz 2024-04-26T20:53:39.707 DEBUG:teuthology.orchestra.run.smithi120:> mktemp 2024-04-26T20:53:39.761 INFO:teuthology.orchestra.run.smithi120.stdout:/tmp/tmp.pykVwkN2PH 2024-04-26T20:53:39.761 DEBUG:teuthology.orchestra.run.smithi120:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.pykVwkN2PH 2024-04-26T20:53:39.914 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod 0666 /tmp/tmp.pykVwkN2PH 2024-04-26T20:53:39.996 DEBUG:teuthology.orchestra.remote:smithi120:/tmp/tmp.pykVwkN2PH is 473KB 2024-04-26T20:53:40.093 DEBUG:teuthology.orchestra.run.smithi120:> rm -fr /tmp/tmp.pykVwkN2PH 2024-04-26T20:53:40.107 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-26T20:53:40.108 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-26T20:53:40.110 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-26T20:53:40.190 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-26T20:53:40.224 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-26T20:53:40.224 INFO:tasks.ceph:Archiving crash dumps... 2024-04-26T20:53:40.226 DEBUG:teuthology.misc:Transferring archived files from smithi073:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675187/remote/smithi073/crash 2024-04-26T20:53:40.227 DEBUG:teuthology.orchestra.run.smithi073:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-26T20:53:40.255 DEBUG:teuthology.misc:Transferring archived files from smithi120:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675187/remote/smithi120/crash 2024-04-26T20:53:40.256 DEBUG:teuthology.orchestra.run.smithi120:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-26T20:53:40.287 INFO:tasks.ceph:Shutting down logrotate 2024-04-26T20:53:40.288 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:53:40.300 DEBUG:teuthology.orchestra.run.smithi120:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:53:40.379 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-26T20:53:40.403 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-26T20:53:40.427 INFO:tasks.ceph:Compressing logs... 2024-04-26T20:53:40.427 DEBUG:teuthology.orchestra.run.smithi073:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T20:53:40.446 DEBUG:teuthology.orchestra.run.smithi120:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T20:53:40.467 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55954.log 2024-04-26T20:53:40.467 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39870.log 2024-04-26T20:53:40.467 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.55954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55912.log 2024-04-26T20:53:40.504 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34671.log 2024-04-26T20:53:40.504 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-26T20:53:40.504 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.34671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34540.log 2024-04-26T20:53:40.504 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34671.log.gz 2024-04-26T20:53:40.504 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-26T20:53:40.504 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.34540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34751.log 2024-04-26T20:53:40.504 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-26T20:53:40.504 INFO:teuthology.orchestra.run.smithi120.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.34540.log.gz 2024-04-26T20:53:40.504 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.34751.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-26T20:53:40.504 INFO:teuthology.orchestra.run.smithi120.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34751.log.gz 2024-04-26T20:53:40.504 INFO:teuthology.orchestra.run.smithi120.stderr: 91.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-26T20:53:40.505 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-26T20:53:40.505 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34711.log 2024-04-26T20:53:40.505 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-mds.b.log: /var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-26T20:53:40.505 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.34711.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-26T20:53:40.505 INFO:teuthology.orchestra.run.smithi120.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34711.log.gz 2024-04-26T20:53:40.505 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-26T20:53:40.505 INFO:teuthology.orchestra.run.smithi120.stderr:gzip/var/log/ceph/ceph-mgr.x.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.34624.log 2024-04-26T20:53:40.505 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-26T20:53:40.505 INFO:teuthology.orchestra.run.smithi120.stderr: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-26T20:53:40.505 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.34624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34791.log 2024-04-26T20:53:40.505 INFO:teuthology.orchestra.run.smithi120.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.34624.log.gz 2024-04-26T20:53:40.505 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.41164.log 2024-04-26T20:53:40.505 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.34791.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34791.log.gz 2024-04-26T20:53:40.505 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34500.log 2024-04-26T20:53:40.505 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.3.41164.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.41164.log.gz 2024-04-26T20:53:40.505 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.40763.log 2024-04-26T20:53:40.506 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.34500.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.34500.log.gz 2024-04-26T20:53:40.506 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.41168.log 2024-04-26T20:53:40.506 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.1.40763.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.40763.log.gz 2024-04-26T20:53:40.506 INFO:teuthology.orchestra.run.smithi120.stderr: 93.0% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-26T20:53:40.506 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-26T20:53:40.506 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-26T20:53:40.506 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-client.3.41168.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.41168.log.gz 2024-04-26T20:53:40.506 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34580.log 2024-04-26T20:53:40.514 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-mon.b.log: 90.5% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-26T20:53:40.514 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.40768.log 2024-04-26T20:53:40.514 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.admin.34580.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34580.log.gz 2024-04-26T20:53:40.514 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-client.1.40768.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.40768.log.gz 2024-04-26T20:53:40.535 INFO:teuthology.orchestra.run.smithi120.stderr: 92.9% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-26T20:53:40.550 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52599.log 2024-04-26T20:53:40.550 INFO:teuthology.orchestra.run.smithi073.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55954.log.gz 2024-04-26T20:53:40.550 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.55912.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39870.log.gz 2024-04-26T20:53:40.551 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47626.log 2024-04-26T20:53:40.551 INFO:teuthology.orchestra.run.smithi073.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55912.log.gz 2024-04-26T20:53:40.551 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.52599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52090.log 2024-04-26T20:53:40.551 INFO:teuthology.orchestra.run.smithi073.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52599.log.gz 2024-04-26T20:53:40.551 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.47626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47338.log 2024-04-26T20:53:40.551 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.52090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44077.log 2024-04-26T20:53:40.551 INFO:teuthology.orchestra.run.smithi073.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47626.log.gz 2024-04-26T20:53:40.551 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.47338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53222.log 2024-04-26T20:53:40.551 INFO:teuthology.orchestra.run.smithi073.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52090.log.gz 2024-04-26T20:53:40.551 INFO:teuthology.orchestra.run.smithi073.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47338.log.gz 2024-04-26T20:53:40.551 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.44077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40681.log 2024-04-26T20:53:40.551 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.53222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43115.log 2024-04-26T20:53:40.551 INFO:teuthology.orchestra.run.smithi073.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44077.log.gz 2024-04-26T20:53:40.552 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40681.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53222.log.gz 2024-04-26T20:53:40.552 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51014.log 2024-04-26T20:53:40.552 INFO:teuthology.orchestra.run.smithi073.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40681.log.gz 2024-04-26T20:53:40.552 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.43115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41643.log 2024-04-26T20:53:40.552 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.0.51014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50348.log 2024-04-26T20:53:40.552 INFO:teuthology.orchestra.run.smithi073.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43115.log.gz 2024-04-26T20:53:40.552 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.51014.log.gz 2024-04-26T20:53:40.552 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.41643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42157.log 2024-04-26T20:53:40.552 INFO:teuthology.orchestra.run.smithi073.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41643.log.gz 2024-04-26T20:53:40.552 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.50348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47878.log 2024-04-26T20:53:40.552 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.42157.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50348.log.gz 2024-04-26T20:53:40.552 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52300.log 2024-04-26T20:53:40.552 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.47878.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42157.log.gz 2024-04-26T20:53:40.552 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47248.log 2024-04-26T20:53:40.553 INFO:teuthology.orchestra.run.smithi073.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47878.log.gz 2024-04-26T20:53:40.553 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.52300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44992.log 2024-04-26T20:53:40.553 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.47248.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52300.log.gz 2024-04-26T20:53:40.553 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54589.log 2024-04-26T20:53:40.553 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.44992.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47248.log.gz 2024-04-26T20:53:40.553 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40368.log 2024-04-26T20:53:40.553 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.54589.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44992.log.gz 2024-04-26T20:53:40.553 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48088.log 2024-04-26T20:53:40.553 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40368.log: gzip -5 89.1% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.54589.log.gz /var/log/ceph/ceph-client.admin.40454.log 2024-04-26T20:53:40.553 INFO:teuthology.orchestra.run.smithi073.stderr: 2024-04-26T20:53:40.553 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.48088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43123.log 2024-04-26T20:53:40.553 INFO:teuthology.orchestra.run.smithi073.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40368.log.gz 2024-04-26T20:53:40.553 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53012.log 2024-04-26T20:53:40.553 INFO:teuthology.orchestra.run.smithi073.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48088.log.gz 2024-04-26T20:53:40.554 INFO:teuthology.orchestra.run.smithi073.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40454.log.gz 2024-04-26T20:53:40.554 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.43123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51325.log 2024-04-26T20:53:40.554 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.53012.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43123.log.gz 2024-04-26T20:53:40.554 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52813.log 2024-04-26T20:53:40.554 INFO:teuthology.orchestra.run.smithi073.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53012.log.gz 2024-04-26T20:53:40.554 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.51325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46147.log 2024-04-26T20:53:40.554 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.52813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45727.log 2024-04-26T20:53:40.554 INFO:teuthology.orchestra.run.smithi073.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51325.log.gz 2024-04-26T20:53:40.554 INFO:teuthology.orchestra.run.smithi073.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52813.log.gz 2024-04-26T20:53:40.554 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.46147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52174.log 2024-04-26T20:53:40.554 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.45727.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46147.log.gz 2024-04-26T20:53:40.554 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43460.log 2024-04-26T20:53:40.554 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.52174.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45727.log.gz 2024-04-26T20:53:40.555 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42783.log 2024-04-26T20:53:40.555 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.43460.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52174.log.gz 2024-04-26T20:53:40.555 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47206.log 2024-04-26T20:53:40.555 INFO:teuthology.orchestra.run.smithi073.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43460.log.gz 2024-04-26T20:53:40.555 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.42783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46740.log 2024-04-26T20:53:40.555 INFO:teuthology.orchestra.run.smithi073.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42783.log.gz 2024-04-26T20:53:40.555 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.47206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34383.log 2024-04-26T20:53:40.555 INFO:teuthology.orchestra.run.smithi073.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47206.log.gz 2024-04-26T20:53:40.555 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.46740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40199.log 2024-04-26T20:53:40.555 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56164.log 2024-04-26T20:53:40.555 INFO:teuthology.orchestra.run.smithi073.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46740.log.gz 2024-04-26T20:53:40.555 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45979.log 2024-04-26T20:53:40.555 INFO:teuthology.orchestra.run.smithi073.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40199.log.gz 2024-04-26T20:53:40.556 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.56164.log: 93.4% -- replaced with /var/log/ceph/ceph-client.admin.34383.log.gz 2024-04-26T20:53:40.556 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54056.log 2024-04-26T20:53:40.556 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.45979.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.56164.log.gz 2024-04-26T20:53:40.556 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50719.log 2024-04-26T20:53:40.556 INFO:teuthology.orchestra.run.smithi073.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45979.log.gz 2024-04-26T20:53:40.556 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.54056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40807.log 2024-04-26T20:53:40.556 INFO:teuthology.orchestra.run.smithi073.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54056.log.gz 2024-04-26T20:53:40.556 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.50719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49323.log 2024-04-26T20:53:40.556 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51544.log 2024-04-26T20:53:40.556 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.49323.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40807.log.gz 2024-04-26T20:53:40.556 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-26T20:53:40.556 INFO:teuthology.orchestra.run.smithi073.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.50719.log.gz 2024-04-26T20:53:40.556 INFO:teuthology.orchestra.run.smithi073.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49323.log.gz 2024-04-26T20:53:40.556 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.51544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40498.log 2024-04-26T20:53:40.557 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56542.log 2024-04-26T20:53:40.557 INFO:teuthology.orchestra.run.smithi073.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51544.log.gz 2024-04-26T20:53:40.557 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53390.log 2024-04-26T20:53:40.557 INFO:teuthology.orchestra.run.smithi073.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40498.log.gz 2024-04-26T20:53:40.557 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.56542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51754.log 2024-04-26T20:53:40.557 INFO:teuthology.orchestra.run.smithi073.stderr: 85.1% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-26T20:53:40.557 INFO:teuthology.orchestra.run.smithi073.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56542.log.gz/var/log/ceph/ceph-client.admin.53390.log: 2024-04-26T20:53:40.557 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50635.log 2024-04-26T20:53:40.557 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.51754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44245.log 2024-04-26T20:53:40.557 INFO:teuthology.orchestra.run.smithi073.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53390.log.gz 2024-04-26T20:53:40.557 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.50635.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.51754.log.gz 2024-04-26T20:53:40.557 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48172.log 2024-04-26T20:53:40.557 INFO:teuthology.orchestra.run.smithi073.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50635.log.gz 2024-04-26T20:53:40.557 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.44245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52216.log 2024-04-26T20:53:40.557 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.48172.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44245.log.gz 2024-04-26T20:53:40.558 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49911.log 2024-04-26T20:53:40.558 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.52216.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48172.log.gz 2024-04-26T20:53:40.558 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42313.log 2024-04-26T20:53:40.558 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.49911.log: 88.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.52216.log.gz 2024-04-26T20:53:40.558 INFO:teuthology.orchestra.run.smithi073.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.46833.log 2024-04-26T20:53:40.558 INFO:teuthology.orchestra.run.smithi073.stderr: 87.6%/var/log/ceph/ceph-client.admin.42313.log: -- replaced with /var/log/ceph/ceph-client.admin.49911.log.gz 2024-04-26T20:53:40.558 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51132.log 2024-04-26T20:53:40.558 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.46833.log: gzip 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42313.log.gz -5 2024-04-26T20:53:40.558 INFO:teuthology.orchestra.run.smithi073.stderr: --verbose -- /var/log/ceph/ceph-client.admin.50264.log 2024-04-26T20:53:40.558 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.51132.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46833.log.gz 2024-04-26T20:53:40.558 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50551.log 2024-04-26T20:53:40.558 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.50264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48400.log 2024-04-26T20:53:40.558 INFO:teuthology.orchestra.run.smithi073.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.51132.log.gz 2024-04-26T20:53:40.558 INFO:teuthology.orchestra.run.smithi073.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50264.log.gz 2024-04-26T20:53:40.558 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.50551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53138.log 2024-04-26T20:53:40.559 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.48400.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50551.log.gz 2024-04-26T20:53:40.559 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-26T20:53:40.559 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.53138.log: gzip 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48400.log.gz -5 2024-04-26T20:53:40.559 INFO:teuthology.orchestra.run.smithi073.stderr: --verbose -- /var/log/ceph/ceph-client.admin.52948.log 2024-04-26T20:53:40.559 INFO:teuthology.orchestra.run.smithi073.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53138.log.gz 2024-04-26T20:53:40.559 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54428.log 2024-04-26T20:53:40.559 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.52948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52683.log 2024-04-26T20:53:40.559 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.54428.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52948.log.gz 2024-04-26T20:53:40.559 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45433.log 2024-04-26T20:53:40.559 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.52683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55374.log 2024-04-26T20:53:40.559 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.45433.log: 88.4% 90.9% -- replaced with /var/log/ceph/ceph-client.admin.52683.log.gz 2024-04-26T20:53:40.559 INFO:teuthology.orchestra.run.smithi073.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54428.log.gz 2024-04-26T20:53:40.559 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40157.log 2024-04-26T20:53:40.559 INFO:teuthology.orchestra.run.smithi073.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45433.log.gz 2024-04-26T20:53:40.559 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.55374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48443.log 2024-04-26T20:53:40.560 INFO:teuthology.orchestra.run.smithi073.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55374.log.gz 2024-04-26T20:53:40.560 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52006.log 2024-04-26T20:53:40.560 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.48443.log: gzip 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40157.log.gz -5 2024-04-26T20:53:40.560 INFO:teuthology.orchestra.run.smithi073.stderr: --verbose -- /var/log/ceph/ceph-client.admin.48004.log 2024-04-26T20:53:40.560 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.52006.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48443.log.gz 2024-04-26T20:53:40.560 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40326.log 2024-04-26T20:53:40.560 INFO:teuthology.orchestra.run.smithi073.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52006.log.gz 2024-04-26T20:53:40.560 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56122.log 2024-04-26T20:53:40.560 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.48004.log: /var/log/ceph/ceph-client.admin.40326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51670.log 2024-04-26T20:53:40.560 INFO:teuthology.orchestra.run.smithi073.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48004.log.gz 2024-04-26T20:53:40.560 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.56122.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40326.log.gz 2024-04-26T20:53:40.560 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40639.log 2024-04-26T20:53:40.560 INFO:teuthology.orchestra.run.smithi073.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56122.log.gz 2024-04-26T20:53:40.560 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.51670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51502.log 2024-04-26T20:53:40.560 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40639.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51670.log.gz 2024-04-26T20:53:40.560 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43455.log 2024-04-26T20:53:40.561 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.51502.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40639.log.gz 2024-04-26T20:53:40.561 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45601.log 2024-04-26T20:53:40.561 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.43455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48815.log 2024-04-26T20:53:40.561 INFO:teuthology.orchestra.run.smithi073.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51502.log.gz 2024-04-26T20:53:40.561 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.45601.log: gzip -5 87.9% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.43455.log.gz 2024-04-26T20:53:40.561 INFO:teuthology.orchestra.run.smithi073.stderr: /var/log/ceph/ceph-client.admin.56684.log 2024-04-26T20:53:40.561 INFO:teuthology.orchestra.run.smithi073.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45601.log.gz 2024-04-26T20:53:40.561 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.48815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34670.log 2024-04-26T20:53:40.561 INFO:teuthology.orchestra.run.smithi073.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48815.log.gz 2024-04-26T20:53:40.561 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.56684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52258.log 2024-04-26T20:53:40.561 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34670.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-26T20:53:40.561 INFO:teuthology.orchestra.run.smithi073.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56684.log.gz 2024-04-26T20:53:40.561 INFO:teuthology.orchestra.run.smithi073.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34670.log.gz 2024-04-26T20:53:40.561 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.52258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49845.log 2024-04-26T20:53:40.561 INFO:teuthology.orchestra.run.smithi073.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52258.log.gz 2024-04-26T20:53:40.561 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53939.log 2024-04-26T20:53:40.562 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.49845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49365.log 2024-04-26T20:53:40.562 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.53939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41685.log 2024-04-26T20:53:40.562 INFO:teuthology.orchestra.run.smithi073.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.49845.log.gz 2024-04-26T20:53:40.562 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.49365.log: gzip -5 --verbose -- 89.6% /var/log/ceph/ceph-client.admin.55103.log 2024-04-26T20:53:40.562 INFO:teuthology.orchestra.run.smithi073.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53939.log.gz 2024-04-26T20:53:40.562 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.41685.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49365.log.gz 2024-04-26T20:53:40.562 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49659.log 2024-04-26T20:53:40.562 INFO:teuthology.orchestra.run.smithi073.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41685.log.gz 2024-04-26T20:53:40.562 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56206.log 2024-04-26T20:53:40.562 INFO:teuthology.orchestra.run.smithi073.stderr: 89.2%/var/log/ceph/ceph-client.admin.55103.log: -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-26T20:53:40.562 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.49659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34751.log 2024-04-26T20:53:40.562 INFO:teuthology.orchestra.run.smithi073.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55103.log.gz 2024-04-26T20:53:40.562 INFO:teuthology.orchestra.run.smithi073.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.49659.log.gz 2024-04-26T20:53:40.562 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.56206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41953.log 2024-04-26T20:53:40.562 INFO:teuthology.orchestra.run.smithi073.stderr: 87.5%/var/log/ceph/ceph-client.admin.34751.log: -- replaced with /var/log/ceph/ceph-client.admin.56206.log.gz 2024-04-26T20:53:40.563 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55786.log 2024-04-26T20:53:40.563 INFO:teuthology.orchestra.run.smithi073.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34751.log.gz 2024-04-26T20:53:40.563 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.41953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43657.log 2024-04-26T20:53:40.563 INFO:teuthology.orchestra.run.smithi073.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41953.log.gz 2024-04-26T20:53:40.563 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.55786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53306.log 2024-04-26T20:53:40.563 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.43657.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55786.log.gz 2024-04-26T20:53:40.563 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54057.log 2024-04-26T20:53:40.563 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.53306.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43657.log.gz 2024-04-26T20:53:40.563 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49449.log 2024-04-26T20:53:40.563 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.54057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44329.log 2024-04-26T20:53:40.563 INFO:teuthology.orchestra.run.smithi073.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53306.log.gz 2024-04-26T20:53:40.563 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.49449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43457.log 2024-04-26T20:53:40.563 INFO:teuthology.orchestra.run.smithi073.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54057.log.gz 2024-04-26T20:53:40.563 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.44329.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.49449.log.gz 2024-04-26T20:53:40.563 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48046.log 2024-04-26T20:53:40.563 INFO:teuthology.orchestra.run.smithi073.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.44329.log.gz 2024-04-26T20:53:40.564 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.43457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53540.log 2024-04-26T20:53:40.564 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.48046.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43457.log.gz 2024-04-26T20:53:40.564 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50509.log 2024-04-26T20:53:40.564 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.53540.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48046.log.gz 2024-04-26T20:53:40.564 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50306.log 2024-04-26T20:53:40.564 INFO:teuthology.orchestra.run.smithi073.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53540.log.gz 2024-04-26T20:53:40.564 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.50509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48730.log 2024-04-26T20:53:40.564 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.50306.log: gzip 89.5% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.50509.log.gz -- 2024-04-26T20:53:40.564 INFO:teuthology.orchestra.run.smithi073.stderr: /var/log/ceph/ceph-osd.3.log 2024-04-26T20:53:40.564 INFO:teuthology.orchestra.run.smithi073.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50306.log.gz 2024-04-26T20:53:40.564 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.48730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46357.log 2024-04-26T20:53:40.564 INFO:teuthology.orchestra.run.smithi073.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48730.log.gz 2024-04-26T20:53:40.564 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55828.log 2024-04-26T20:53:40.564 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.46357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42565.log 2024-04-26T20:53:40.564 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.55828.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46357.log.gz 2024-04-26T20:53:40.565 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56248.log 2024-04-26T20:53:40.565 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46960.log 2024-04-26T20:53:40.565 INFO:teuthology.orchestra.run.smithi073.stderr: 88.2%/var/log/ceph/ceph-client.admin.42565.log: -- replaced with /var/log/ceph/ceph-client.admin.55828.log.gz 2024-04-26T20:53:40.565 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.56248.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42565.log.gz 2024-04-26T20:53:40.565 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55331.log 2024-04-26T20:53:40.565 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54851.log 2024-04-26T20:53:40.565 INFO:teuthology.orchestra.run.smithi073.stderr: 89.2%/var/log/ceph/ceph-client.admin.46960.log: -- replaced with /var/log/ceph/ceph-client.admin.56248.log.gz 2024-04-26T20:53:40.565 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.55331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47836.log 2024-04-26T20:53:40.565 INFO:teuthology.orchestra.run.smithi073.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46960.log.gz 2024-04-26T20:53:40.565 INFO:teuthology.orchestra.run.smithi073.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.55331.log.gz 2024-04-26T20:53:40.565 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.54851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54631.log 2024-04-26T20:53:40.565 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.47836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42037.log 2024-04-26T20:53:40.565 INFO:teuthology.orchestra.run.smithi073.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.54851.log.gz 2024-04-26T20:53:40.565 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.54631.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47836.log.gz 2024-04-26T20:53:40.565 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46063.log 2024-04-26T20:53:40.566 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54298.log 2024-04-26T20:53:40.566 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.42037.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54631.log.gz 2024-04-26T20:53:40.566 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.46063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51880.log 2024-04-26T20:53:40.566 INFO:teuthology.orchestra.run.smithi073.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42037.log.gz 2024-04-26T20:53:40.566 INFO:teuthology.orchestra.run.smithi073.stderr: 88.3%/var/log/ceph/ceph-client.admin.54298.log: -- replaced with /var/log/ceph/ceph-client.admin.46063.log.gz 2024-04-26T20:53:40.566 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42788.log 2024-04-26T20:53:40.566 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.51880.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54298.log.gz 2024-04-26T20:53:40.566 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45769.log 2024-04-26T20:53:40.566 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49407.log 2024-04-26T20:53:40.566 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.42788.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51880.log.gz 2024-04-26T20:53:40.566 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.45769.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42788.log.gz 2024-04-26T20:53:40.566 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54757.log 2024-04-26T20:53:40.566 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48941.log 2024-04-26T20:53:40.566 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.49407.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45769.log.gz 2024-04-26T20:53:40.566 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.54757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48527.log 2024-04-26T20:53:40.566 INFO:teuthology.orchestra.run.smithi073.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49407.log.gz 2024-04-26T20:53:40.567 INFO:teuthology.orchestra.run.smithi073.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54757.log.gz 2024-04-26T20:53:40.567 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.48941.log: -- /var/log/ceph/ceph-client.admin.39980.log 2024-04-26T20:53:40.567 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.48527.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48941.log.gz 2024-04-26T20:53:40.567 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42780.log 2024-04-26T20:53:40.567 INFO:teuthology.orchestra.run.smithi073.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48527.log.gz 2024-04-26T20:53:40.567 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44119.log 2024-04-26T20:53:40.567 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39980.log: /var/log/ceph/ceph-client.admin.42780.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-26T20:53:40.567 INFO:teuthology.orchestra.run.smithi073.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39980.log.gz 2024-04-26T20:53:40.567 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.44119.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42780.log.gz 2024-04-26T20:53:40.567 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54386.log 2024-04-26T20:53:40.567 INFO:teuthology.orchestra.run.smithi073.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.44119.log.gz 2024-04-26T20:53:40.567 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40242.log 2024-04-26T20:53:40.567 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-mds.c.log: /var/log/ceph/ceph-client.admin.54386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42778.log 2024-04-26T20:53:40.567 INFO:teuthology.orchestra.run.smithi073.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54386.log.gz 2024-04-26T20:53:40.567 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42786.log 2024-04-26T20:53:40.567 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54494.log 2024-04-26T20:53:40.568 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.42778.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40242.log.gz 2024-04-26T20:53:40.568 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.42786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44859.log 2024-04-26T20:53:40.568 INFO:teuthology.orchestra.run.smithi073.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42778.log.gz 2024-04-26T20:53:40.568 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55416.log 2024-04-26T20:53:40.568 INFO:teuthology.orchestra.run.smithi073.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42786.log.gz 2024-04-26T20:53:40.568 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.54494.log: /var/log/ceph/ceph-client.0.44859.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.0.44859.log.gz -5 2024-04-26T20:53:40.568 INFO:teuthology.orchestra.run.smithi073.stderr: --verbose -- /var/log/ceph/ceph-client.admin.41427.log 2024-04-26T20:53:40.568 INFO:teuthology.orchestra.run.smithi073.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54494.log.gz 2024-04-26T20:53:40.568 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.55416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46876.log 2024-04-26T20:53:40.568 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43321.log 2024-04-26T20:53:40.568 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.41427.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.55416.log.gz 2024-04-26T20:53:40.568 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-26T20:53:40.568 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.46876.log: /var/log/ceph/ceph-client.admin.43321.log: gzip 88.2% -5 --verbose -- /var/log/ceph/ceph-client.admin.47548.log 2024-04-26T20:53:40.568 INFO:teuthology.orchestra.run.smithi073.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46876.log.gz 2024-04-26T20:53:40.568 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-osd.1.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43321.log.gz 2024-04-26T20:53:40.568 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49067.log 2024-04-26T20:53:40.568 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.47548.log: /var/log/ceph/ceph-client.admin.47794.log 2024-04-26T20:53:40.569 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.49067.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-26T20:53:40.569 INFO:teuthology.orchestra.run.smithi073.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47548.log.gz 2024-04-26T20:53:40.569 INFO:teuthology.orchestra.run.smithi073.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.41427.log.gz 2024-04-26T20:53:40.569 INFO:teuthology.orchestra.run.smithi073.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49067.log.gz 2024-04-26T20:53:40.569 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.47794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42523.log 2024-04-26T20:53:40.569 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-mgr.y.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47794.log.gz 2024-04-26T20:53:40.569 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51964.log 2024-04-26T20:53:40.569 INFO:teuthology.orchestra.run.smithi073.stderr:gzip/var/log/ceph/ceph-client.admin.42523.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.52906.log 2024-04-26T20:53:40.569 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.51964.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42523.log.gz 2024-04-26T20:53:40.569 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47710.log 2024-04-26T20:53:40.569 INFO:teuthology.orchestra.run.smithi073.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51964.log.gz 2024-04-26T20:53:40.569 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50822.log 2024-04-26T20:53:40.569 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.52906.log: /var/log/ceph/ceph-client.admin.47710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56458.log 2024-04-26T20:53:40.569 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.50822.log: 88.7% 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52906.log.gz 2024-04-26T20:53:40.569 INFO:teuthology.orchestra.run.smithi073.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47710.log.gz 2024-04-26T20:53:40.569 INFO:teuthology.orchestra.run.smithi073.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.50822.log.gz 2024-04-26T20:53:40.570 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.53728.log 2024-04-26T20:53:40.570 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.56458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54059.log 2024-04-26T20:53:40.570 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.0.53728.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56458.log.gz 2024-04-26T20:53:40.570 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.53728.log.gz 2024-04-26T20:53:40.570 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50432.log 2024-04-26T20:53:40.570 INFO:teuthology.orchestra.run.smithi073.stderr:gzip/var/log/ceph/ceph-client.admin.54059.log: -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-26T20:53:40.570 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.50432.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54059.log.gz 2024-04-26T20:53:40.570 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55458.log 2024-04-26T20:53:40.570 INFO:teuthology.orchestra.run.smithi073.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50432.log.gz 2024-04-26T20:53:40.570 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55019.log 2024-04-26T20:53:40.570 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-osd.0.log: /var/log/ceph/ceph-client.admin.55458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44161.log 2024-04-26T20:53:40.570 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.55019.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55458.log.gz 2024-04-26T20:53:40.570 INFO:teuthology.orchestra.run.smithi073.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55019.log.gz 2024-04-26T20:53:40.570 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49239.log 2024-04-26T20:53:40.570 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.44161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48983.log 2024-04-26T20:53:40.570 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.49239.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44161.log.gz 2024-04-26T20:53:40.571 INFO:teuthology.orchestra.run.smithi073.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49239.log.gz 2024-04-26T20:53:40.571 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55229.log 2024-04-26T20:53:40.571 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.48983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54935.log 2024-04-26T20:53:40.571 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.55229.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48983.log.gz 2024-04-26T20:53:40.571 INFO:teuthology.orchestra.run.smithi073.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.55229.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51838.log 2024-04-26T20:53:40.571 INFO:teuthology.orchestra.run.smithi073.stderr: 2024-04-26T20:53:40.571 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.54935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53432.log 2024-04-26T20:53:40.571 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.51838.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54935.log.gz 2024-04-26T20:53:40.571 INFO:teuthology.orchestra.run.smithi073.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51838.log.gz 2024-04-26T20:53:40.571 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44535.log 2024-04-26T20:53:40.571 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.53432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45559.log 2024-04-26T20:53:40.571 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.44535.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53432.log.gz 2024-04-26T20:53:40.571 INFO:teuthology.orchestra.run.smithi073.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44535.log.gz 2024-04-26T20:53:40.571 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52557.log 2024-04-26T20:53:40.571 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.45559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56500.log 2024-04-26T20:53:40.571 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.52557.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45559.log.gz 2024-04-26T20:53:40.572 INFO:teuthology.orchestra.run.smithi073.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52557.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56080.log 2024-04-26T20:53:40.572 INFO:teuthology.orchestra.run.smithi073.stderr: 2024-04-26T20:53:40.572 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.56500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46609.log 2024-04-26T20:53:40.572 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.56080.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56500.log.gz 2024-04-26T20:53:40.572 INFO:teuthology.orchestra.run.smithi073.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56080.log.gz 2024-04-26T20:53:40.572 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45685.log 2024-04-26T20:53:40.572 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.46609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51010.log 2024-04-26T20:53:40.572 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.45685.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46609.log.gz 2024-04-26T20:53:40.572 INFO:teuthology.orchestra.run.smithi073.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45685.log.gz 2024-04-26T20:53:40.572 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40076.log 2024-04-26T20:53:40.572 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.0.51010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54715.log 2024-04-26T20:53:40.572 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.51010.log.gz 2024-04-26T20:53:40.572 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53474.log 2024-04-26T20:53:40.572 INFO:teuthology.orchestra.run.smithi073.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40076.log.gz 2024-04-26T20:53:40.572 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.54715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48485.log 2024-04-26T20:53:40.572 INFO:teuthology.orchestra.run.smithi073.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54715.log.gz 2024-04-26T20:53:40.573 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.53474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42439.log 2024-04-26T20:53:40.573 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.48485.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.53474.log.gz 2024-04-26T20:53:40.573 INFO:teuthology.orchestra.run.smithi073.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48485.log.gz 2024-04-26T20:53:40.573 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42260.log 2024-04-26T20:53:40.573 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.42439.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.28366.log 2024-04-26T20:53:40.573 INFO:teuthology.orchestra.run.smithi073.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42439.log.gz 2024-04-26T20:53:40.573 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.42260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42481.log 2024-04-26T20:53:40.573 INFO:teuthology.orchestra.run.smithi073.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42260.log.gz 2024-04-26T20:53:40.573 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph.tmp-client.admin.28366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40723.log 2024-04-26T20:53:40.573 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.28366.log.gz 2024-04-26T20:53:40.573 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.42481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53096.log 2024-04-26T20:53:40.573 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40723.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42481.log.gz 2024-04-26T20:53:40.573 INFO:teuthology.orchestra.run.smithi073.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40723.log.gz 2024-04-26T20:53:40.573 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55271.log 2024-04-26T20:53:40.573 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48214.log 2024-04-26T20:53:40.573 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.53096.log: /var/log/ceph/ceph-client.admin.55271.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53096.log.gz 2024-04-26T20:53:40.574 INFO:teuthology.orchestra.run.smithi073.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55271.log.gz 2024-04-26T20:53:40.574 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49803.log 2024-04-26T20:53:40.574 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46189.log 2024-04-26T20:53:40.574 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.49803.log: /var/log/ceph/ceph-client.admin.48214.log: 88.1% 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48214.log.gz 2024-04-26T20:53:40.574 INFO:teuthology.orchestra.run.smithi073.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49803.log.gz 2024-04-26T20:53:40.574 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43909.log 2024-04-26T20:53:40.574 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.46189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55187.log 2024-04-26T20:53:40.574 INFO:teuthology.orchestra.run.smithi073.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46189.log.gz 2024-04-26T20:53:40.574 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.43909.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43909.log.gz 2024-04-26T20:53:40.574 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41769.log 2024-04-26T20:53:40.574 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.53732.log 2024-04-26T20:53:40.574 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.55187.log: /var/log/ceph/ceph-client.admin.41769.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55187.log.gz 2024-04-26T20:53:40.574 INFO:teuthology.orchestra.run.smithi073.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41769.log.gzgzip 2024-04-26T20:53:40.574 INFO:teuthology.orchestra.run.smithi073.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.55743.log 2024-04-26T20:53:40.574 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.0.53732.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.53732.log.gz 2024-04-26T20:53:40.574 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46021.log 2024-04-26T20:53:40.575 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-26T20:53:40.575 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.55743.log: /var/log/ceph/ceph-client.admin.46021.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55743.log.gz 2024-04-26T20:53:40.575 INFO:teuthology.orchestra.run.smithi073.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46021.log.gz 2024-04-26T20:53:40.575 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45937.log 2024-04-26T20:53:40.617 INFO:teuthology.orchestra.run.smithi120.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-26T20:53:40.628 INFO:teuthology.orchestra.run.smithi120.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-26T20:53:40.642 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40891.log 2024-04-26T20:53:40.642 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.45937.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45937.log.gz 2024-04-26T20:53:40.642 INFO:teuthology.orchestra.run.smithi073.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-26T20:53:40.642 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49575.log 2024-04-26T20:53:40.642 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49025.log 2024-04-26T20:53:40.643 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.49575.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40891.log.gz 2024-04-26T20:53:40.643 INFO:teuthology.orchestra.run.smithi073.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49575.log.gz 2024-04-26T20:53:40.643 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47464.log 2024-04-26T20:53:40.643 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.49025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56642.log 2024-04-26T20:53:40.643 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.47464.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49025.log.gz 2024-04-26T20:53:40.643 INFO:teuthology.orchestra.run.smithi073.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47464.log.gz 2024-04-26T20:53:40.643 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52725.log 2024-04-26T20:53:40.643 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.56642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46231.log 2024-04-26T20:53:40.643 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.52725.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.56642.log.gz 2024-04-26T20:53:40.643 INFO:teuthology.orchestra.run.smithi073.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52725.log.gz 2024-04-26T20:53:40.643 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53264.log 2024-04-26T20:53:40.643 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.46231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55500.log 2024-04-26T20:53:40.643 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.53264.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46231.log.gz 2024-04-26T20:53:40.643 INFO:teuthology.orchestra.run.smithi073.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53264.log.gz 2024-04-26T20:53:40.643 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55996.log 2024-04-26T20:53:40.644 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.55500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48256.log 2024-04-26T20:53:40.644 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.55996.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55500.log.gz 2024-04-26T20:53:40.644 INFO:teuthology.orchestra.run.smithi073.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55996.log.gz 2024-04-26T20:53:40.644 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42787.log 2024-04-26T20:53:40.644 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.48256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42789.log 2024-04-26T20:53:40.644 INFO:teuthology.orchestra.run.smithi073.stderr: 88.8%/var/log/ceph/ceph-client.admin.42787.log: -- replaced with /var/log/ceph/ceph-client.admin.48256.log.gz 2024-04-26T20:53:40.644 INFO:teuthology.orchestra.run.smithi073.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42787.log.gz 2024-04-26T20:53:40.644 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46918.log 2024-04-26T20:53:40.644 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.42789.log: gzip 91.5% -5 --verbose -- replaced with /var/log/ceph/ceph-mgr.y.log.gz -- 2024-04-26T20:53:40.644 INFO:teuthology.orchestra.run.smithi073.stderr: /var/log/ceph/ceph-client.admin.49719.log 2024-04-26T20:53:40.644 INFO:teuthology.orchestra.run.smithi073.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42789.log.gz 2024-04-26T20:53:40.644 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40553.log 2024-04-26T20:53:40.644 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.46918.log: /var/log/ceph/ceph-client.admin.49719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45895.log 2024-04-26T20:53:40.644 INFO:teuthology.orchestra.run.smithi073.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46918.log.gz 2024-04-26T20:53:40.645 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40553.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49719.log.gz 2024-04-26T20:53:40.645 INFO:teuthology.orchestra.run.smithi073.stderr:gzip 88.8% -5 --verbose -- /var/log/ceph/ceph-client.admin.46483.log 2024-04-26T20:53:40.645 INFO:teuthology.orchestra.run.smithi073.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40553.log.gz 2024-04-26T20:53:40.645 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.45895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49281.log 2024-04-26T20:53:40.645 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56416.log 2024-04-26T20:53:40.645 INFO:teuthology.orchestra.run.smithi073.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45895.log.gz 2024-04-26T20:53:40.645 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.46483.log: /var/log/ceph/ceph-client.admin.49281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51628.log 2024-04-26T20:53:40.645 INFO:teuthology.orchestra.run.smithi073.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46483.log.gz 2024-04-26T20:53:40.645 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.56416.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49281.log.gz 2024-04-26T20:53:40.645 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52048.log 2024-04-26T20:53:40.645 INFO:teuthology.orchestra.run.smithi073.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.56416.log.gz 2024-04-26T20:53:40.645 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43699.log 2024-04-26T20:53:40.646 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.51628.log: /var/log/ceph/ceph-client.admin.52048.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51628.log.gz 2024-04-26T20:53:40.646 INFO:teuthology.orchestra.run.smithi073.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52048.log.gz 2024-04-26T20:53:40.646 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41911.log 2024-04-26T20:53:40.646 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43451.log 2024-04-26T20:53:40.646 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.43699.log: /var/log/ceph/ceph-client.admin.41911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49109.log 2024-04-26T20:53:40.646 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.43451.log: 88.3% 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43699.log.gz 2024-04-26T20:53:40.646 INFO:teuthology.orchestra.run.smithi073.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41911.log.gz 2024-04-26T20:53:40.646 INFO:teuthology.orchestra.run.smithi073.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43451.log.gz 2024-04-26T20:53:40.646 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46315.log 2024-04-26T20:53:40.646 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55145.log 2024-04-26T20:53:40.646 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.49109.log: /var/log/ceph/ceph-client.admin.46315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41995.log 2024-04-26T20:53:40.646 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.55145.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49109.log.gz 2024-04-26T20:53:40.646 INFO:teuthology.orchestra.run.smithi073.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.46315.log.gz 2024-04-26T20:53:40.646 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46441.log 2024-04-26T20:53:40.647 INFO:teuthology.orchestra.run.smithi073.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.55145.log.gz 2024-04-26T20:53:40.647 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47422.log 2024-04-26T20:53:40.647 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.41995.log: /var/log/ceph/ceph-client.admin.46441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50103.log 2024-04-26T20:53:40.647 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.47422.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41995.log.gz 2024-04-26T20:53:40.647 INFO:teuthology.orchestra.run.smithi073.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46441.log.gz 2024-04-26T20:53:40.647 INFO:teuthology.orchestra.run.smithi073.stderr: 87.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41727.log 2024-04-26T20:53:40.647 INFO:teuthology.orchestra.run.smithi073.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47422.log.gz 2024-04-26T20:53:40.647 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43783.log 2024-04-26T20:53:40.647 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.0.50103.log: /var/log/ceph/ceph-client.admin.41727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54673.log 2024-04-26T20:53:40.647 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.50103.log.gz 2024-04-26T20:53:40.647 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.43783.log: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56290.log 2024-04-26T20:53:40.647 INFO:teuthology.orchestra.run.smithi073.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41727.log.gz 2024-04-26T20:53:40.647 INFO:teuthology.orchestra.run.smithi073.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43783.log.gz 2024-04-26T20:53:40.647 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46273.log 2024-04-26T20:53:40.648 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.54673.log: /var/log/ceph/ceph-client.admin.56290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47122.log 2024-04-26T20:53:40.648 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.46273.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.54673.log.gz 2024-04-26T20:53:40.648 INFO:teuthology.orchestra.run.smithi073.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56290.log.gz 2024-04-26T20:53:40.648 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50593.log 2024-04-26T20:53:40.648 INFO:teuthology.orchestra.run.smithi073.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46273.log.gz 2024-04-26T20:53:40.648 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52475.log 2024-04-26T20:53:40.648 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.47122.log: /var/log/ceph/ceph-client.admin.50593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49491.log 2024-04-26T20:53:40.648 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.52475.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47122.log.gz 2024-04-26T20:53:40.648 INFO:teuthology.orchestra.run.smithi073.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50593.log.gz 2024-04-26T20:53:40.648 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47962.log 2024-04-26T20:53:40.648 INFO:teuthology.orchestra.run.smithi073.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52475.log.gz 2024-04-26T20:53:40.648 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46789.log 2024-04-26T20:53:40.648 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.49491.log: /var/log/ceph/ceph-client.admin.47962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40933.log 2024-04-26T20:53:40.648 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.46789.log: 87.6% 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49491.log.gz 2024-04-26T20:53:40.649 INFO:teuthology.orchestra.run.smithi073.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47962.log.gz 2024-04-26T20:53:40.649 INFO:teuthology.orchestra.run.smithi073.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46789.log.gz 2024-04-26T20:53:40.649 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51796.log 2024-04-26T20:53:40.649 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42607.log 2024-04-26T20:53:40.649 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40933.log: /var/log/ceph/ceph-client.admin.51796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51712.log 2024-04-26T20:53:40.649 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.42607.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51796.log.gz 2024-04-26T20:53:40.649 INFO:teuthology.orchestra.run.smithi073.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42607.log.gz 91.3% -- replaced with /var/log/ceph/ceph-client.admin.40933.log.gz 2024-04-26T20:53:40.649 INFO:teuthology.orchestra.run.smithi073.stderr: 2024-04-26T20:53:40.649 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47668.log 2024-04-26T20:53:40.649 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.51712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43124.log 2024-04-26T20:53:40.649 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.47668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47380.log 2024-04-26T20:53:40.649 INFO:teuthology.orchestra.run.smithi073.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51712.log.gz 2024-04-26T20:53:40.649 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.43124.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47668.log.gz 2024-04-26T20:53:40.649 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45475.log 2024-04-26T20:53:40.650 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.47380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44287.log 2024-04-26T20:53:40.650 INFO:teuthology.orchestra.run.smithi073.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43124.log.gz 2024-04-26T20:53:40.650 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.45475.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47380.log.gz 2024-04-26T20:53:40.650 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43993.log 2024-04-26T20:53:40.650 INFO:teuthology.orchestra.run.smithi073.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45475.log.gz 2024-04-26T20:53:40.650 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.44287.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44287.log.gz 2024-04-26T20:53:40.650 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42218.log 2024-04-26T20:53:40.650 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42782.log 2024-04-26T20:53:40.650 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.43993.log: /var/log/ceph/ceph-client.admin.42218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43448.log 2024-04-26T20:53:40.650 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.42782.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43993.log.gz 2024-04-26T20:53:40.650 INFO:teuthology.orchestra.run.smithi073.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42218.log.gz 2024-04-26T20:53:40.650 INFO:teuthology.orchestra.run.smithi073.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42782.log.gz 2024-04-26T20:53:40.650 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48607.log 2024-04-26T20:53:40.650 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43459.log 2024-04-26T20:53:40.651 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.43448.log: /var/log/ceph/ceph-client.admin.48607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42397.log 2024-04-26T20:53:40.651 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.43459.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43448.log.gz 2024-04-26T20:53:40.651 INFO:teuthology.orchestra.run.smithi073.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48607.log.gz 2024-04-26T20:53:40.651 INFO:teuthology.orchestra.run.smithi073.stderr: 87.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.43459.log.gz 2024-04-26T20:53:40.651 INFO:teuthology.orchestra.run.smithi073.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47920.log 2024-04-26T20:53:40.651 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56374.log 2024-04-26T20:53:40.651 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.42397.log: /var/log/ceph/ceph-client.admin.47920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55699.log 2024-04-26T20:53:40.651 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.56374.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42397.log.gz 2024-04-26T20:53:40.651 INFO:teuthology.orchestra.run.smithi073.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47920.log.gz 2024-04-26T20:53:40.651 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41811.log 2024-04-26T20:53:40.651 INFO:teuthology.orchestra.run.smithi073.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.56374.log.gz 2024-04-26T20:53:40.651 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47752.log 2024-04-26T20:53:40.651 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.55699.log: /var/log/ceph/ceph-client.admin.41811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51922.log 2024-04-26T20:53:40.651 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.47752.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55699.log.gz 2024-04-26T20:53:40.652 INFO:teuthology.orchestra.run.smithi073.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41811.log.gz 2024-04-26T20:53:40.652 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45853.log 2024-04-26T20:53:40.652 INFO:teuthology.orchestra.run.smithi073.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47752.log.gz 2024-04-26T20:53:40.652 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47080.log 2024-04-26T20:53:40.652 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.51922.log: /var/log/ceph/ceph-client.admin.45853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48857.log 2024-04-26T20:53:40.652 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.47080.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51922.log.gz 2024-04-26T20:53:40.652 INFO:teuthology.orchestra.run.smithi073.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45853.log.gz 2024-04-26T20:53:40.652 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47080.log.gz 2024-04-26T20:53:40.652 INFO:teuthology.orchestra.run.smithi073.stderr: --verbose -- /var/log/ceph/ceph-client.admin.56332.log 2024-04-26T20:53:40.652 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.48857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44371.log 2024-04-26T20:53:40.652 INFO:teuthology.orchestra.run.smithi073.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48857.log.gz 2024-04-26T20:53:40.652 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.56332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47164.log 2024-04-26T20:53:40.652 INFO:teuthology.orchestra.run.smithi073.stderr: 88.9%/var/log/ceph/ceph-client.admin.44371.log: -- replaced with /var/log/ceph/ceph-client.admin.56332.log.gz 2024-04-26T20:53:40.652 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43867.log 2024-04-26T20:53:40.652 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53348.log 2024-04-26T20:53:40.653 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.47164.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.44371.log.gz 2024-04-26T20:53:40.653 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.43867.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47164.log.gz 2024-04-26T20:53:40.653 INFO:teuthology.orchestra.run.smithi073.stderr:gzip 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43867.log.gz 2024-04-26T20:53:40.653 INFO:teuthology.orchestra.run.smithi073.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.55870.log 2024-04-26T20:53:40.653 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.53348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52395.log 2024-04-26T20:53:40.653 INFO:teuthology.orchestra.run.smithi073.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53348.log.gz 2024-04-26T20:53:40.653 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.55870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44854.log 2024-04-26T20:53:40.653 INFO:teuthology.orchestra.run.smithi073.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55870.log.gz 2024-04-26T20:53:40.653 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47506.log 2024-04-26T20:53:40.653 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.52395.log: /var/log/ceph/ceph-client.0.44854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43116.log 2024-04-26T20:53:40.653 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44854.log.gz 2024-04-26T20:53:40.653 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.47506.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52395.log.gz 2024-04-26T20:53:40.653 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48130.log 2024-04-26T20:53:40.653 INFO:teuthology.orchestra.run.smithi073.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47506.log.gz 2024-04-26T20:53:40.653 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.43116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54977.log 2024-04-26T20:53:40.654 INFO:teuthology.orchestra.run.smithi073.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43116.log.gz 2024-04-26T20:53:40.654 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50677.log 2024-04-26T20:53:40.654 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.48130.log: /var/log/ceph/ceph-client.admin.54977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52641.log 2024-04-26T20:53:40.654 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.50677.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48130.log.gz 2024-04-26T20:53:40.654 INFO:teuthology.orchestra.run.smithi073.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54977.log.gz 2024-04-26T20:53:40.654 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40410.log 2024-04-26T20:53:40.654 INFO:teuthology.orchestra.run.smithi073.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50677.log.gz 2024-04-26T20:53:40.654 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46525.log 2024-04-26T20:53:40.654 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.52641.log: /var/log/ceph/ceph-client.admin.40410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51586.log 2024-04-26T20:53:40.654 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.46525.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52641.log.gz 2024-04-26T20:53:40.654 INFO:teuthology.orchestra.run.smithi073.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40410.log.gz 2024-04-26T20:53:40.654 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49533.log 2024-04-26T20:53:40.654 INFO:teuthology.orchestra.run.smithi073.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46525.log.gz 2024-04-26T20:53:40.654 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49761.log 2024-04-26T20:53:40.655 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.51586.log: /var/log/ceph/ceph-client.admin.49533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43741.log 2024-04-26T20:53:40.655 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.49761.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51586.log.gz 2024-04-26T20:53:40.655 INFO:teuthology.orchestra.run.smithi073.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49533.log.gz 2024-04-26T20:53:40.655 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52132.log 2024-04-26T20:53:40.655 INFO:teuthology.orchestra.run.smithi073.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49761.log.gz 2024-04-26T20:53:40.655 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47038.log 2024-04-26T20:53:40.655 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.43741.log: /var/log/ceph/ceph-client.admin.52132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51367.log 2024-04-26T20:53:40.655 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.47038.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43741.log.gz 2024-04-26T20:53:40.655 INFO:teuthology.orchestra.run.smithi073.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52132.log.gz 2024-04-26T20:53:40.655 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51109.log 2024-04-26T20:53:40.655 INFO:teuthology.orchestra.run.smithi073.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47038.log.gz 2024-04-26T20:53:40.655 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49197.log 2024-04-26T20:53:40.655 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.51367.log: /var/log/ceph/ceph-client.admin.51109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49617.log 2024-04-26T20:53:40.655 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.49197.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51367.log.gz 2024-04-26T20:53:40.655 INFO:teuthology.orchestra.run.smithi073.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51109.log.gz 2024-04-26T20:53:40.656 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52769.log 2024-04-26T20:53:40.656 INFO:teuthology.orchestra.run.smithi073.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49197.log.gz 2024-04-26T20:53:40.656 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-26T20:53:40.656 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.49617.log: /var/log/ceph/ceph-client.admin.52769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51409.log 2024-04-26T20:53:40.656 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-mds.a.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.49617.log.gz 2024-04-26T20:53:40.656 INFO:teuthology.orchestra.run.smithi073.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52769.log.gz 2024-04-26T20:53:40.656 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52864.log 2024-04-26T20:53:40.656 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45643.log 2024-04-26T20:53:40.656 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.51409.log: /var/log/ceph/ceph-client.admin.52864.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51409.log.gz 2024-04-26T20:53:40.656 INFO:teuthology.orchestra.run.smithi073.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52864.log.gz 2024-04-26T20:53:40.656 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48899.log 2024-04-26T20:53:40.656 INFO:teuthology.orchestra.run.smithi073.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-26T20:53:40.656 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40999.log 2024-04-26T20:53:40.656 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.45643.log: /var/log/ceph/ceph-client.admin.48899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.45296.log 2024-04-26T20:53:40.657 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40999.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.45643.log.gz 2024-04-26T20:53:40.657 INFO:teuthology.orchestra.run.smithi073.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48899.log.gz 2024-04-26T20:53:40.657 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49153.log 2024-04-26T20:53:40.657 INFO:teuthology.orchestra.run.smithi073.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40999.log.gz 2024-04-26T20:53:40.657 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46567.log 2024-04-26T20:53:40.657 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.2.45296.log: /var/log/ceph/ceph-client.admin.49153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44203.log 2024-04-26T20:53:40.657 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.45296.log.gz 2024-04-26T20:53:40.657 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.46567.log: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46399.log 2024-04-26T20:53:40.657 INFO:teuthology.orchestra.run.smithi073.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49153.log.gz 2024-04-26T20:53:40.657 INFO:teuthology.orchestra.run.smithi073.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46567.log.gz 2024-04-26T20:53:40.657 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.44203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56602.log 2024-04-26T20:53:40.657 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.46399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50390.log 2024-04-26T20:53:40.657 INFO:teuthology.orchestra.run.smithi073.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44203.log.gz 2024-04-26T20:53:40.657 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.56602.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46399.log.gz 2024-04-26T20:53:40.657 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51283.log 2024-04-26T20:53:40.658 INFO:teuthology.orchestra.run.smithi073.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56602.log.gz 2024-04-26T20:53:40.658 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40595.log 2024-04-26T20:53:40.658 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.50390.log: /var/log/ceph/ceph-client.admin.51283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42079.log 2024-04-26T20:53:40.658 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40595.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50390.log.gz 2024-04-26T20:53:40.658 INFO:teuthology.orchestra.run.smithi073.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51283.log.gz 2024-04-26T20:53:40.658 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48772.log 2024-04-26T20:53:40.658 INFO:teuthology.orchestra.run.smithi073.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40595.log.gz 2024-04-26T20:53:40.658 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53180.log 2024-04-26T20:53:40.658 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.42079.log: gzip/var/log/ceph/ceph-client.admin.48772.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.41869.log 2024-04-26T20:53:40.658 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.53180.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48772.log.gz 2024-04-26T20:53:40.658 INFO:teuthology.orchestra.run.smithi073.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42079.log.gz 2024-04-26T20:53:40.658 INFO:teuthology.orchestra.run.smithi073.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53180.log.gz 2024-04-26T20:53:40.658 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40849.log 2024-04-26T20:53:40.658 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.41869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43114.log 2024-04-26T20:53:40.659 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40849.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41869.log.gz 2024-04-26T20:53:40.659 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50099.log 2024-04-26T20:53:40.659 INFO:teuthology.orchestra.run.smithi073.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40849.log.gz 2024-04-26T20:53:40.659 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46105.log 2024-04-26T20:53:40.659 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.43114.log: /var/log/ceph/ceph-client.0.50099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43122.log 2024-04-26T20:53:40.659 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.50099.log.gz 2024-04-26T20:53:40.659 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.46105.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43114.log.gz 2024-04-26T20:53:40.659 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42355.log 2024-04-26T20:53:40.659 INFO:teuthology.orchestra.run.smithi073.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46105.log.gz 2024-04-26T20:53:40.659 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.43122.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43122.log.gz 2024-04-26T20:53:40.659 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43825.log 2024-04-26T20:53:40.659 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.42355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48298.log 2024-04-26T20:53:40.659 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.43825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48340.log 2024-04-26T20:53:40.659 INFO:teuthology.orchestra.run.smithi073.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42355.log.gz 2024-04-26T20:53:40.659 INFO:teuthology.orchestra.run.smithi073.stderr: 88.4%/var/log/ceph/ceph-client.admin.48298.log: -- replaced with /var/log/ceph/ceph-client.admin.43825.log.gz 2024-04-26T20:53:40.659 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40765.log 2024-04-26T20:53:40.660 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.48340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45517.log 2024-04-26T20:53:40.660 INFO:teuthology.orchestra.run.smithi073.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48298.log.gz 2024-04-26T20:53:40.660 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40765.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48340.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54058.log 2024-04-26T20:53:40.660 INFO:teuthology.orchestra.run.smithi073.stderr: 2024-04-26T20:53:40.660 INFO:teuthology.orchestra.run.smithi073.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40765.log.gz 2024-04-26T20:53:40.660 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.45517.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45517.log.gz 2024-04-26T20:53:40.660 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55061.log 2024-04-26T20:53:40.660 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43125.log 2024-04-26T20:53:40.660 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.54058.log: /var/log/ceph/ceph-client.admin.55061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45811.log 2024-04-26T20:53:40.660 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.43125.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55061.log.gz 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54058.log.gz 2024-04-26T20:53:40.660 INFO:teuthology.orchestra.run.smithi073.stderr: 2024-04-26T20:53:40.660 INFO:teuthology.orchestra.run.smithi073.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43125.log.gz 2024-04-26T20:53:40.660 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50766.log 2024-04-26T20:53:40.660 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.45300.log 2024-04-26T20:53:40.661 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.45811.log: /var/log/ceph/ceph-client.admin.50766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44035.log 2024-04-26T20:53:40.661 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.2.45300.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45811.log.gz 2024-04-26T20:53:40.661 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% 90.0% -- replaced with /var/log/ceph/ceph-client.2.45300.log.gz 2024-04-26T20:53:40.661 INFO:teuthology.orchestra.run.smithi073.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50766.log.gz 2024-04-26T20:53:40.661 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54340.log 2024-04-26T20:53:40.661 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.44035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40284.log 2024-04-26T20:53:40.661 INFO:teuthology.orchestra.run.smithi073.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44035.log.gz 2024-04-26T20:53:40.661 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.54340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54893.log 2024-04-26T20:53:40.661 INFO:teuthology.orchestra.run.smithi073.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54340.log.gz 2024-04-26T20:53:40.661 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51241.log 2024-04-26T20:53:40.661 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40284.log: /var/log/ceph/ceph-client.admin.54893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54205.log 2024-04-26T20:53:40.661 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.51241.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54893.log.gz 2024-04-26T20:53:40.661 INFO:teuthology.orchestra.run.smithi073.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40284.log.gz 2024-04-26T20:53:40.661 INFO:teuthology.orchestra.run.smithi073.stderr:gzip 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51241.log.gz 2024-04-26T20:53:40.661 INFO:teuthology.orchestra.run.smithi073.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.41601.log 2024-04-26T20:53:40.661 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.54205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53860.log 2024-04-26T20:53:40.662 INFO:teuthology.orchestra.run.smithi073.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.54205.log.gz 2024-04-26T20:53:40.662 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.41601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48688.log 2024-04-26T20:53:40.662 INFO:teuthology.orchestra.run.smithi073.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41601.log.gz 2024-04-26T20:53:40.662 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.53860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56038.log 2024-04-26T20:53:40.662 INFO:teuthology.orchestra.run.smithi073.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53860.log.gz 2024-04-26T20:53:40.662 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43461.log 2024-04-26T20:53:40.662 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.48688.log: /var/log/ceph/ceph-client.admin.56038.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48688.log.gz 2024-04-26T20:53:40.662 INFO:teuthology.orchestra.run.smithi073.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.56038.log.gz 2024-04-26T20:53:40.662 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43120.log 2024-04-26T20:53:40.662 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43951.log 2024-04-26T20:53:40.662 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.43461.log: gzip/var/log/ceph/ceph-client.admin.43120.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.53054.log 2024-04-26T20:53:40.662 INFO:teuthology.orchestra.run.smithi073.stderr: 88.0% 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43461.log.gz 2024-04-26T20:53:40.662 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.43951.log: -- replaced with /var/log/ceph/ceph-client.admin.43120.log.gz 2024-04-26T20:53:40.663 INFO:teuthology.orchestra.run.smithi073.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43951.log.gz 2024-04-26T20:53:40.663 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.53054.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53054.log.gz 2024-04-26T20:53:40.803 INFO:teuthology.orchestra.run.smithi073.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-26T20:53:41.164 INFO:teuthology.orchestra.run.smithi120.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-26T20:53:41.167 INFO:teuthology.orchestra.run.smithi073.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-26T20:53:41.225 INFO:teuthology.orchestra.run.smithi120.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-26T20:53:41.238 INFO:teuthology.orchestra.run.smithi120.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-26T20:53:41.243 INFO:teuthology.orchestra.run.smithi073.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-26T20:53:41.249 INFO:teuthology.orchestra.run.smithi120.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-26T20:53:41.251 INFO:teuthology.orchestra.run.smithi120.stderr: 2024-04-26T20:53:41.251 INFO:teuthology.orchestra.run.smithi120.stderr:real 0m0.770s 2024-04-26T20:53:41.251 INFO:teuthology.orchestra.run.smithi120.stderr:user 0m3.117s 2024-04-26T20:53:41.251 INFO:teuthology.orchestra.run.smithi120.stderr:sys 0m0.132s 2024-04-26T20:53:41.296 INFO:teuthology.orchestra.run.smithi073.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-26T20:53:41.340 INFO:teuthology.orchestra.run.smithi073.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-26T20:53:41.342 INFO:teuthology.orchestra.run.smithi073.stderr: 2024-04-26T20:53:41.343 INFO:teuthology.orchestra.run.smithi073.stderr:real 0m0.885s 2024-04-26T20:53:41.343 INFO:teuthology.orchestra.run.smithi073.stderr:user 0m3.479s 2024-04-26T20:53:41.343 INFO:teuthology.orchestra.run.smithi073.stderr:sys 0m0.303s 2024-04-26T20:53:41.343 INFO:tasks.ceph:Archiving logs... 2024-04-26T20:53:41.343 DEBUG:teuthology.misc:Transferring archived files from smithi073:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675187/remote/smithi073/log 2024-04-26T20:53:41.344 DEBUG:teuthology.orchestra.run.smithi073:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T20:53:42.130 DEBUG:teuthology.misc:Transferring archived files from smithi120:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675187/remote/smithi120/log 2024-04-26T20:53:42.131 DEBUG:teuthology.orchestra.run.smithi120:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T20:53:42.609 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-26T20:53:42.619 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-26T20:53:42.619 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T20:53:42.622 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T20:53:42.706 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-26T20:53:42.715 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-26T20:53:42.715 DEBUG:teuthology.orchestra.run.smithi073:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T20:53:42.718 DEBUG:teuthology.orchestra.run.smithi120:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T20:53:42.730 INFO:teuthology.orchestra.run.smithi073.stderr:bash: line 1: ntpq: command not found 2024-04-26T20:53:42.734 INFO:teuthology.orchestra.run.smithi073.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T20:53:42.734 INFO:teuthology.orchestra.run.smithi073.stdout:=============================================================================== 2024-04-26T20:53:42.734 INFO:teuthology.orchestra.run.smithi073.stdout:^* hv01.front.sepia.ceph.com 3 6 377 40 +1246us[+1240us] +/- 51ms 2024-04-26T20:53:42.734 INFO:teuthology.orchestra.run.smithi073.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 41 -1058us[-1065us] +/- 64ms 2024-04-26T20:53:42.734 INFO:teuthology.orchestra.run.smithi073.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 42 -645us[ -652us] +/- 76ms 2024-04-26T20:53:42.734 INFO:teuthology.orchestra.run.smithi073.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T20:53:42.734 INFO:teuthology.orchestra.run.smithi120.stderr:bash: line 1: ntpq: command not found 2024-04-26T20:53:42.738 INFO:teuthology.orchestra.run.smithi120.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T20:53:42.738 INFO:teuthology.orchestra.run.smithi120.stdout:=============================================================================== 2024-04-26T20:53:42.738 INFO:teuthology.orchestra.run.smithi120.stdout:^* hv01.front.sepia.ceph.com 3 6 377 39 +1250us[+1253us] +/- 51ms 2024-04-26T20:53:42.738 INFO:teuthology.orchestra.run.smithi120.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 41 -1052us[-1049us] +/- 64ms 2024-04-26T20:53:42.738 INFO:teuthology.orchestra.run.smithi120.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 39 -646us[ -643us] +/- 76ms 2024-04-26T20:53:42.739 INFO:teuthology.orchestra.run.smithi120.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T20:53:42.739 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-26T20:53:42.748 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-26T20:53:42.749 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-26T20:53:42.760 DEBUG:teuthology.orchestra.run.smithi073:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T20:53:42.845 DEBUG:teuthology.orchestra.run.smithi120:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T20:53:42.907 DEBUG:teuthology.orchestra.run.smithi073:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T20:53:42.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:53:42.934 DEBUG:teuthology.orchestra.run.smithi120:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T20:53:42.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:53:42.963 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-26T20:53:42.972 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-26T20:53:42.983 INFO:teuthology.task.internal:Duration was 579.293187 seconds 2024-04-26T20:53:42.983 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-26T20:53:42.992 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-26T20:53:42.992 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T20:53:42.995 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T20:53:43.028 INFO:teuthology.orchestra.run.smithi073.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T20:53:43.040 INFO:teuthology.orchestra.run.smithi120.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T20:53:43.186 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2024-04-26T20:53:43.431 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-26T20:53:43.431 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi073.front.sepia.ceph.com 2024-04-26T20:53:43.431 DEBUG:teuthology.orchestra.run.smithi073:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T20:53:43.491 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi120.front.sepia.ceph.com 2024-04-26T20:53:43.492 DEBUG:teuthology.orchestra.run.smithi120:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T20:53:43.513 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-26T20:53:43.513 DEBUG:teuthology.orchestra.run.smithi073:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T20:53:43.534 DEBUG:teuthology.orchestra.run.smithi120:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T20:53:43.578 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-26T20:53:43.578 DEBUG:teuthology.orchestra.run.smithi073:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T20:53:43.599 DEBUG:teuthology.orchestra.run.smithi120:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T20:53:43.714 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-26T20:53:43.725 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-26T20:53:43.725 DEBUG:teuthology.orchestra.run.smithi073:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T20:53:43.748 DEBUG:teuthology.orchestra.run.smithi120:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T20:53:43.771 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-26T20:53:43.784 DEBUG:teuthology.orchestra.run.smithi073:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T20:53:43.791 DEBUG:teuthology.orchestra.run.smithi120:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T20:53:43.811 INFO:teuthology.orchestra.run.smithi073.stdout:kernel.core_pattern = core 2024-04-26T20:53:43.833 INFO:teuthology.orchestra.run.smithi120.stdout:kernel.core_pattern = core 2024-04-26T20:53:43.845 DEBUG:teuthology.orchestra.run.smithi073:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T20:53:43.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:53:43.877 DEBUG:teuthology.orchestra.run.smithi120:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T20:53:43.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:53:43.900 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-26T20:53:43.910 INFO:teuthology.task.internal:Transferring archived files... 2024-04-26T20:53:43.911 DEBUG:teuthology.misc:Transferring archived files from smithi073:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675187/remote/smithi073 2024-04-26T20:53:43.911 DEBUG:teuthology.orchestra.run.smithi073:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T20:53:43.951 DEBUG:teuthology.misc:Transferring archived files from smithi120:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675187/remote/smithi120 2024-04-26T20:53:43.951 DEBUG:teuthology.orchestra.run.smithi120:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T20:53:43.986 INFO:teuthology.task.internal:Removing archive directory... 2024-04-26T20:53:43.986 DEBUG:teuthology.orchestra.run.smithi073:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T20:53:43.989 DEBUG:teuthology.orchestra.run.smithi120:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T20:53:44.032 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-26T20:53:44.044 INFO:teuthology.task.internal:Not uploading archives. 2024-04-26T20:53:44.044 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-26T20:53:44.054 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-26T20:53:44.055 DEBUG:teuthology.orchestra.run.smithi073:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T20:53:44.057 DEBUG:teuthology.orchestra.run.smithi120:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T20:53:44.069 INFO:teuthology.orchestra.run.smithi073.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 20:53 /home/ubuntu/cephtest 2024-04-26T20:53:44.087 INFO:teuthology.orchestra.run.smithi120.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 20:53 /home/ubuntu/cephtest 2024-04-26T20:53:44.088 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-26T20:53:44.098 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-26T20:53:44.134 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/auto-repair} duration: 579.2931871414185 flavor: default owner: scheduled_rishabh@teuthology success: true 2024-04-26T20:53:44.135 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T20:53:44.218 INFO:teuthology.run:pass